Chapter 5 Collisions: Getting them under control

Time-to-Contact - H. Hecht and G.J.P. Savelsbergh (Editors) © 2004 Elsevier B.V. All rights reserved

CHAPTER 5 Collisions: Getting them under Control

John M. Flach Wright State University, Dayton, OH, USA

Matthew R. H. Smith Delphi Automotive Systems, Kokomo, IN, USA

Terry Stanard Klein Associates, Dayton, OH, USA

Scott M. Dittman Visteon Inc., Detroit, MI, USA

ABSTRACT In a control system, information about a current state is compared to a reference State in order to specify an action that will bring the current state and reference state closer together. This comparison process requires a common currency among the three sources of constraint: intention, action, and information. This chapter considers the possibility that structure in an optic array might be that currency. Performance for several tasks is represented in an optical state space that helps to illustrate the confluence of the multiple sources of constraints. The results suggest that the optical criteria for coUision control vary to reflect the different sources of constraint.

68

John M. Flach, Matthew R. H. Smith, Terry Stanard and Scott M. Dittman

1. Introduction Approach to a solid surface is specified by a centrifugal flow of the texture of the optic array. Approach to an object is specified by a magnification of the closed contour in the array corresponding to the edges of the object. A uniform rate of approach is accompanied by an accelerated rate of magnification. At the theoretical point where the eye touches the object the latter will intercept a visual angle of 180°; the magnification reaches an explosive rate in the last moments before contact. The accelerated expansion in the field of view specifies imminent colUsion, and it is unquestionably an effective stimulus for behavior in animals with well developed visual systems....the fact is that animals need to make contact without colUsion with many sohd objects of their environment: food objects, sex objects, and the landing surfaces on which insects and birds aUght (not to mention heUcopter pilots). Locomotor action must be balanced between approach and aversion. The governing stimulation must be a balance between flow and non-flow of the optic array. The formula is as follows: contact without colUsion is achieved by so moving as to cancel the centrifugal flow of the optic array at the moment when the contour of the object or the texture surface reaches that angular magnification at which contact is made. Gibson (1958/1982, p. 155-156) The opening quote is from an article titled "Visually controlled locomotion and visual orientation in animals" by J.J. Gibson. That paper was one of the earliest and clearest examples where Gibson began to frame the problem of perception as part of a closed-loop system where perception and action were dynamicaUy coupled to support goal directed behavior. The goal of the present chapter is to show the value of a control theoretic perspective for understanding how animals solve collision problems. The following section will introduce the "comparator problem." For us, this is the "heart" of a control systems perspective. It provides afi*ameworkfor conceptually parsing a control problem into three categories of constraint: value, information, and action. This chapter will discuss each of these categories of constraint and will explore some hypotheses about how animals address these constraints in solving the colUsion control problem.

Collisions: Getting them under Control

69

2. The comparator problem In a control system, the comparator is a junction with reference and feedback signals coming in and error signals coming out. In a simple system, such as the servomechanism illustrated in Figure 1, the comparator is analogous to the simple mathematical operation of subtraction. The feedback signal (specifying the current state of the system) is subtracted from the reference signal (specifying the desired state of the system) in order to get an error signal (the deviation from the goal). The error signal then drives action in the direction that will reduce the difference (bring the system state closer to the goal state). In engineering control systems an important step (that is critical in practice, but rarely explicitly acknowledged) is to convert the various signals (reference, feedback, and error) into a common (comparable) medium (e.g., electrical current). Once this is accomplished, the operation of the comparator is directly analogous to the simple mathematical operation of subtraction. Indirect Perception (Information FYocessing)

i^^cv44^ I / \

Direct Perception

Figure 1: How is it possible for a biological system to compare perceptual feedback to intentions in order to specify appropriate corrective actions? Classically, this has been thought to require translation into a symbolic neural representation. The idea of direct perception suggests that lawful relations in perceptual arrays may support an indexical coding in the nervous system that can be fully described in terms of the perceptual referents.

70

John M. Flach, Matthew R. H. Smith, Terry Stanard and Scott M. Dittman

However, for animals, the three signals associated with the comparator rarely come nicely packaged in a common medium or currency that would allow simple subtraction of one from the other to produce the third. For example, the reference or intention may be to get to a meeting across town as quickly as possible without coUision. The information may be patterns in an optical flow field. What does it mean to subtract the patterns from the intention? The difference from this subtraction would have to specify the actions of muscles (perhaps on control devices - steering wheel, accelerator, and brake pedal). The natural units for each of these "signals" converging at the comparator are different - desire not to be late for an important meeting and to avoid collisions, a transforming pattern of texture transduced through a retina, and a force or motion of a Umb perhaps transduced through a vehicle. How does an animal translate from one medium to another in order to behave appropriately - that is, in order to behave in a way so that errors from intentions are kept within acceptable limits. Psychology has conventionally assumed that the comparator problem was solved "in the head." That is, the general notion was that the three signals (intention, feedback/perception, and error/motor command) were converted to some common symboUc neural code (reflected in terms like program, schema, mental map, mental model, gestalt, etc.). Thus, the neural symbols associated with perceptions could be "compared" with the neural symbols associated with intentions in a way that would specify the appropriate neural symbols to guide actions. Gibson, however, suggested an alternative position. The radical notion of "direct perception" suggests that the comparator problem can be solved in the Ught. Gibson (1958/1982) wrote: To begin locomotion, therefore, is to contract the muscles as to make the forward optic array flow outward. To stop locomotion is to make the flow cease. To reverse locomotion is to make it flow inward. To speed up locomotion is to make the rate of flow increase and to slow down is to make it decrease. An animal who is behaving in these ways is optically stimulated in the corresponding ways, or, equally, an animal who so acts as to obtain these kinds of optical stimulation is behaving in the corresponding ways (p. 155). The radical aspect of Gibson's theory relative to more conventional wisdom about perception and cognition is that he was the first to imagine how intentions and actions could be specified in optical terms. For example, the intention of contacting an object can be specified as "the state in which the optical contour associated with that object fills the field of view." The action is specified as "do something to make the optical contour associated with that

Collisions: Getting them under Control

71

object expand." And the current state is specified as the "instantaneous size and location of the optical contour in the field of view". Gibson's ideas are less radical when considered in the context of a control perspective. Perhaps, the most comprehensive application of a control theoretic perspective to psychology is Powers' (1973, 1998) Perceptual Control Theory (PCX). The parallels between how Powers and Gibson describe the locomotion problem are striking: When you learn to drive, the first thing you learn after getting the car into motion is how the road should look relative to the front of the car as you see it from the driver's seat. Somehow this image remains in your head and as you drive along you are continually comparing how the scene does look with how it should look. If the way it does look is shifted to the right of how it should look, you turn the wheel leftward until there is a match. A left shift leads to a rightward turn of the wheel. Once you learn this relationship it becomes automatic; you don't have to think it out any more. You have constructed an automatic control system that will, as long as you're looking, keep the actual perception matching the appearance you know it should have. When you do that, the car settles down into its lane and stays there. The "appearance you know it should have" is called a reference perception, or reference condition, or reference state because it is with reference to this internal information that you judge the perception as too little, too much, or just right; too far left, too far right, or dead on; too hot, too cold, or perfect. If differences exist, we call them "errors" in PCT. Error doesn't mean "blunder" or "mistake"; it means a difference between what is being perceived and what is intended to be perceived (Powers, 1998, p. 8-9). The imphcation of Powers' description of learning to drive and his choice of the term "perceptual" control theory suggests that the currency exchange that allows feedback to inform action relative to intentions is typically negotiated in the medium of perception. An important point of the PCT approach is that the variable that drives a control system (e.g., a thermostat) is not the "output" variable per se (e.g., the actual room temperature), but the measured or perceived temperature. Ideally, in a designed control system there would be a close correspondence between the output variable and the measured variable. However, for the control system, the measured temperature is the only temperature. Analogously, the animal knows nothing of the world except its own

72

John M. Flach, Matthew R. H. Smith, Terry Stanard and Scott M. Dittman

perceptions. Of course, if those perceptions are not fairly well tuned to the actual situations in the world, the animal is not likely to survive for very long. Although Gibson and Powers both focus on perception. Powers treats perception as an "image . . . in your head," where as Gibson focused on the structure in an optical array outside the head. Alternatively, the question of perceptual control of action has been framed as trying to learn "how the Ught gets into the muscles." A problem with this way of framing the problem is that it invites a simple stimulus-response (SR) view of causaUty. In the simple SR view of causality, perception (Ught) causes action (muscles). An important implication of a closed-loop coupUng of perception and action is that neither perception nor action is causally prior to the other. Thus, action and perception are locked in a circular causality. This opens up the possibility that in many cases (not all) that the problem is solved by putting muscles and intentions into the light. In Gibson's words from the eariier citation "an animal so acts as to obtain these kinds of optical stimulation." Muscles can be put into the light by moving to create an optical flow field. Analogously, the dynamics of a vehicle can be put into the Ught by manipulating the controls transforming the optical flow (e.g., jiggUng the steering wheel or tapping the brakes). Further, we believe that by attending to the consequences of motion, animals can learn to associate those consequences with patterns in the optical flow field. This creates the possibility for intentions to be specified in optical terms. This is our understanding of "direct perception" - that the three components of the comparator problem can be specified as signals within a perceptual array (e.g., the optical flow field). Thus, there is no need to translate to or from a symbolic medium in order to close the perception-action loop. In semiotic terms, we interpret "direct perception" to be the claim that there is an "indexical" relation between properties of the optics and properties of any neural code. As opposed to a "symboUc" relation, requiring some form of interpretation or logical inference. We think it would be a mistake to argue that it is always possible to close the loop within a perceptual array, that is, that there is always an indexical relation. However, we beUeve that there are many situations where the relation is indexical and we beUeve that colUsion events typicaUy fall within this category. ColUsion events are typically well specified in the optical flow field - creating the possibiUty for direct perception for an animal that is appropriately "attuned" to the optical structure. Interactions where signals in the ecology have an indexical relation to the action requirements correspond to what Rasmussen (1986) called "skiU-based" processing. It is important to note that the optical states are expected to have correlated structure in the neural medium, (e.g., weights in a neural network). However, the critical point of "direct perception" is that the neural medium does not introduce additional constraints, at least, when we are deaUng with supra-

Collisions: Getting them under Control

73

threshold phenomenon - which is generally the case for control of locomotion. In other words, the claim of direct perception is that the "comparison" of intention with ongoing perception to specify action can be described in terms of lawful relations (e.g., physical laws) that exist independent of any symbolic neural or cognitive process. The neural structure can be tuned to these external constraints, but the constraints exist independently of whether or not they are detected by any cognitive process. Following the lead of Gibson, the goal of this chapter is to explore the possibility that the collision problem can be solved by a control system in which the relevant constraints are specified in optical terms.

3. The intentional constraints: References, values, and consequences Perhaps, the most significant intentional constraint dimension associated with colUsions is whether the goal is to create collisions or to avoid collisions. In many sports, the goal is to create collisions, and in many cases the more violent the coUision the better (e.g., an overhand smash in racquet sports, the home run swing in baseball, or the rocket volley past the goalie in soccer or hockey). In these contexts, the "value" of a collision typically increases with the velocity of the effector (racquet, bat, stick, or foot) at the point of contact. Of course, value depends on many situational factors and it is easy to imagine exceptions where soft contact can have high value (e.g., a drop volley in tennis, a bunt in baseball, or a kiss). In most vehicle control contexts, the goal is generally to avoid collisions all together (e.g., normal driving) or to make collisions that are as soft as possible (e.g., landing an aircraft). In these contexts, the "value" of a collision is typically inversely related to the velocity of the effector at the point of contact. Another dimension of the intentional dynamics of collisions is the cost of different types of errors. For example, the costs associated with responding earlier or later than some normative ideal of the "right time." For creating collisions, as in racquet sports, there tends to be a precise space-time window associated with success. Responses that are "too early" or "too late" are relatively symmetric in terms of the negative consequences (e.g., hitting the ball out of play or missing completely). For avoiding collisions, as in vehicular control, the consequences of timing errors are typically asymmetric. That is, beginning to brake too early in response to an obstacle in the road rarely leads to serious consequences and this "error" is easily corrected once detected. Initiating the braking too late, however, can have catastrophic consequences that might not be easily corrected. The point is that the costs associated with errors are often not uniform. This may have important implications for whether a control strategy is satisfactory or not.

74

John M. Flach, Matthew R. H. Smith, Terry Stanard and Scott M. Dittman

A wide spectrum of tasks can be found in the collision literature - from the forehand drive in table tennis to the left hand (cross-traffic) turn in driving. It is not unusual to find all of these tasks lumped together in discussions of how animals and people control coUisions, creating the implication that these are examples of the same control problem. However, from a control systems perspective - it may make a big difference whether the goal is to create a high velocity collision or to avoid collision. From a control systems perspective it would not be surprising if there were satisfactory solutions to one problem that were not satisfactory for the other. It might be valuable to consider the optical specification of collision in light of some of these different intentional constraints - the information value of an optical pattern may, in part, depend upon the intentional value against which "success" is being scored.

4. The action constraints: The plant The term "plant" is control theoretic jargon to specify the physical processes that are being controlled. In the context of collision, this might be the dynamics of the human motor system or the dynamics of a vehicle such as an automobile or aircraft. These dynamics constrain the action possibilities in ways that have important imphcations for solutions to any control problem. This is clearly illustrated by early research on manual control - which illustrates that the transfer function of the manual controller varies systematically as a function of variations in the plant dynamics (McRuer & Jex, 1967; Flach, 1990; Jagacinski & Flach, 2003). In the literature on the collision problem, a wide range of "plant dynamics" has been studied (from simple head movements, to arm movements, to braking automobiles, to flare maneuvers for landing aircraft). While there are obvious quantitative differences across the range of plants studied, there is one important quality associated with many of the plants studied - inertial dynamics. Inertial dynamics reflect Newton's Second Law of Motion. That is, the acceleration/deceleration of a body in response to a force is directly proportional to the force and inversely proportional to its mass. In the language of control systems - the plant dynamics are second-order. For example, to stop a truck in response to a red light, a braking force is applied which causes the truck to decelerate. In order to bring the truck to zero velocity at a point in front of the red light, the braking force must be initiated well before reaching the target stopping point. How far before depends on the velocity of the truck when the deceleration is initiated and the weight of the truck (whether it is loaded or not). For a given speed, braking would need to be initiated at a farther distance from the target stopping point for a heavier truck. For a given weight, braking would need to be initiated at a farther distance from the target stopping point for faster

Collisions: Getting them under Control

75

speeds. Of course, the state of the brakes and the state of contact with the road (reflecting surface and wheel properties) would also be important variables. The primary implication of inertial dynamics is that feedback based on the single dimension of position is inadequate. That is, there is no functional (one-to-one) relation between position and action. In other words, the control (or problem space) for inertial systems requires consideration of variables in addition to position (or distance). Typically, control engineers add velocity as a second dimension of this state or problem space. However, other dimensions might also be considered in place of or in addition to velocity (e.g., time, or higher derivatives of the motion). It is also important to understand how the inertial constraints shape the response of a system. One way to represent this is in terms of the action system's open-loop response to a fixed (constant input). This is typically called the "step response." The logic here is that since the input is fixed [not modulated to reflect goals (value constraints) or feedback (information constraints)] the pattern of response allows inferences about the action constraints, independent of the other two sources of constraint (i.e., value or information). For example, the step response of the braking system of a car would be the response to a constant deflection of the brake. Or the step response of a bird braking would be the effects of it extending its wings and holding them in a fixed position. In both cases, the pattern of slowing down that resulted would reflect the physical laws of motion. For the same step input given the same parameters (e.g., weight) and the same initial conditions (e.g., speed) the response would be similar, independent of what the driver or bird was intending and independent of whether the driver or bird's eyes were open or closed. In control theoretic terms this response is open-loop. Figure 2 illustrates the response produced by a step braking input to a simple second order process. The step input creates a constant deceleration. In terms of the analytical problem of identifying the logic of a control system it is important that the analyst know the step response for the physical plant being investigated, so that the action constraints are not confused with properties of the control logic. So, for a simple second order process, a constant deceleration may not have to be explained in terms of a continuous adjustment of a brake in reference to some information feedback (e.g., x dot). The constant deceleration may come for "free" as a result of the inertial dynamics.

76

John M. Flach, Matthew R. H. Smith, Terry Stanard and Scott M. Dittman

Figure 2: The step response of a simple second-order system.

People interact with many different types of dynamic processes. Inertial dynamics are just one common example. The major point to this section is that the nature of the physical processes being controlled constrains actions and creates demands for the type of information that might provide useful guidance for directing those actions. For researchers who are trying to understand the logic of the comparator process, it will often be useful to know the step response of the physical process. This will help the researchers to differentiate the purely physical (open-loop) constraints from constraints of the closed-loop, control logic. Also, knowledge of the dynamics can be useful for generating hypotheses about the information requirements.

Collisions: Getting them under Control

11

5. The information constraints: Optical flow flelds The action dynamics of a system creates information requirements for a control system. For example, inertial dynamics mean that information about position will generally be inadequate for solving the collision problem. That is, there is no one-to-one mapping of position (i.e., distance to an obstacle) and action. In other words, the answer to the question: "At what distance should I initiate braking in order to stop at the threshold of the intersection?" is "It depends, among other things, on how fast you are going!" So, the fact of inertial dynamic means that information about position is inadequate for solving the collision problem. The inertial dynamic also means that in principle, for a given "plant" (e.g., a given weight, given state of brakes, given road conditions, etc.) information about position and velocity is typically adequate for solving the collision problem. If the comparator problem were solved in the head, then a logical approach to the information/perception problem would be to study the psychophysical problems of distance and velocity perception. And many have taken this route. However, we will assume that the comparator problem is solved in the light. Thus, the question becomes how are distance and velocity specified optically? A reasonable hypothesis is that distance is specified by the optical angle (0) associated with relevant textures. For example, the distance to a lead car when driving would be specified by the optical angle associated with the taillights of that car. When the distance is great, the optical angle will be small. As the distance becomes less, the optical angle will grow. Similarly, when approaching an intersection the relevant texture elements would be the margins of the intersection - the nearer the intersection the larger would be the optical angle associated with the margin textures. Note that the angular size is not oneto-one with distance - it also varies with the size of the reference object. However, in ecologies where size is relatively constant (e.g., the size of the lead car or the intersection does not change during an approaching event), it may provide a reasonably robust indexical referent for relative distance. The speed or closure rate to a lead car might be specified as the rate of change of the optical angle or the angular expansion rate (6'). High angular expansion rates would be associated with high rates of closure. And low angular expansion rates would be associated with low rates of closure. Again, as with optical angle, angular expansion rate is not one-to-one with closure rate. It varies

78

John M. Flach, Matthew R. H. Smith, Terry Stanard and Scott M. Dittman

as a function of distance, size, and velocity relative to the reference texture. Thus, for example, a constant velocity approach results in an increasing expansion rate - (i.e., looming). Although, the two optical variables are not perfectly correlated with position and velocity, they do span the two dimensions (position and velocity) required by the action constraints of a second-order process. Thus, it is plausible that they might provide a satisfactory indexical reference for solving the collision control problem. Whether, the "noise" associated with the mapping (e.g., variability due to size) is a problem or not may depend on the degree of consistency of that dimension in the control ecology and the nature of control errors associated with this noise.

6. Some hypotheses about the control logic Lee (1974, 1976, 1980) was one of the first to demonstrate how the collision problem might be solved by closing the loop in the optical array. Lee (1976) suggested that braking performance might be contingent on the optical parameter x and its derivative x dot. Tau is the inverse fractional rate of expansion of the optical contour associated with a coUision obstacle. That is, it is the ratio of the angular projection of an object and the first derivative of this projection (angular expansion rate). Thus, although x is a single optical invariant, it integrates the two dimensions (optical angle/distance, expansion rate/closure velocity) required by the inertial dynamics. Tau has units of time and is thus often referred to as time-to-contact. Lee's work was ground braking in that it provided, for the first time, a plausible control algorithm based completely on optical constraints and it provided clear predictions that could be evaluated empirically. Unfortunately, although there has been a plethora of research papers showing patterns of performance in which control actions are initiated at the right time (as predicted by the x hypothesis), other predictions of the X hypothesis have not been consistent with empirical evidence. In particular, if X and its derivatives are the optical primitives, then performance should be independent of both speed of approach and size of the approaching object. The reason for this is that the dimensions of speed and size cancel in the ratio of angle to expansion rate, leaving only time as the critical variable. Thus, for example, the x hypothesis suggests that actions based on x should occur at the same time-to-arrival independent of the approach speeds to the collision obstacle. To date, every pubUshed study that we are aware of that has manipulated speed as an independent variable has found significant performance differences. In general, people respond earlier (at longer time-to-contacts) for slow speeds, than for faster speeds. Likewise, every pubUshed study that we are aware of that has manipulated size of the coUision obstacle has found significant

Collisions: Getting them under Control

79

performance differences associated with that variable. In general, people respond earlier (at longer time-to-contacts) for large objects. In order to account for the consistent effects of speed and size on human performance Smith, Flach, Dittman, and Stanard (2001) proposed an alternative to the T hypothesis. They proposed that optical angle and its derivative, expansion rate, were independent optical primitives that provided the "direct" indexes to the distance and closure rate associated with a coUision event. They proposed that control of collision could be accomplished by weighting these two sources of information to reflect the action constraints of an inertial control system. There are three significant advantages of the Smith et al. Model over the T hypothesis: 1. It can account for both a high correlation with time-to-contact and the significant effects of both speed and size. 2. It is consistent with the dynamic demands associated with control of inertial systems. It has long been known that optimal control of an inertial system can be accomplished using weighted feedback of position and velocity estimates (e.g., see Kirk, 1970). With angle providing an estimate of position and expansion rate providing an estimate of velocity, the Smith et al. Model provides a nice bridge between standard control models and the perceptual analysis. 3. The weighting of the two optical primitives can provide a model for both individual differences and skill development. Both can be modeled as differential weightings of the two primitives. For example, Smith et al. showed that practice, negative transfer, and positive transfer could be predicted as a function of differential weightings of the optical primitives. Similarly, different styles of driving can be modeled as different weightings of angle and expansion rate. Figure 3 illustrates the logic of a simple control system tuned to angle and expansion rate. Note that in this simple control system, angle and expansion rate are input to a comparison process. Typically, the other input to the comparator is called a reference or simply input. In Figure 3 the term "criterion" is used to emphasize that the reference input is a criterion for a decision process. The output of the comparator is typically called the error signal. This makes sense when the criterion function is a "target" value or state and the output of the comparator process is the difference between a current value and a target value. However, as noted in the earUer quote from Powers, the term error has a connotation that can be misleading. The term "command" is used here to emphasize that the output of the comparator process is a call to action (i.e., act in a way to make the optical feedback congruent with some criterion).

80

John M. Flach, Matthew R. H. Smith, Terry Stanard and Scott M. Dittman

Position (Separation) Velocity (Closure Rate)

>

Figure 3: A simple control system in which optical angle (9) and expansion rate (0') are fed back through a comparator which in turn modifies actions in order to satisfy some task criterion.

A useful way to visualize the comparison process is a state space diagram as illustrated in Figure 4. The two feedback variables (angle and expansion rate) are considered the states of this system. They are the coordinates for the state space graph. Any behavior of the system can be illustrated as an event Une through this state space. For example, the gray dashed hnes show the optical state trajectories for balls approaching at seven different constant speeds (actually radial travel times, see next section for explanation). An event begins near the origin (the object projects a very small optical angle and expansion rate is also small). As the object approaches, its projected angle will grow at an increasing expansion rate. The projected angles for faster balls will grow at higher rates than slower balls. Time is not exphcitly represented in the state space. However, a diagonal line with positive slope and zero intercept represents a constant time-to-contact. Time-to-contact would be the inverse of the slope. Shallow slopes represent longer time-to-contacts and steeper slopes represent shorter time-to-contacts. For the constant speed trajectories shown by the dotted Une, points closest to the origin represent larger time to contacts and time-tocontact becomes shorter as the event unfolds (moving up and to the right in state space).

Collisions: Getting them under Control

81

Imfem^ 0-10.5

1.0

15

ZO

Figure 4: An optical state space. Data are from Smith et al. (2001).

The control task illustrated in the state space diagram was a ball-hitting task used by Smith et al. In this task, a ball approached at a constant velocity and the participant had to release a pendulum at the "right time" to hit the oncoming ball Velocity varied unpredictably from trial to trial. The white region running diagonally through the center of the space shows the "hit zone." If the pendulum was released when the optical variables were in this region of state space, then a collision will result. This region represents the "right time" to release the pendulum. If the pendulum is released at states corresponding to points below this region, then the swing will be too early. If the pendulum is released at states corresponding to points above this region, then the swing will be too late. The circles in Figure 4 represent average data from human participants attempting to create collisions by releasing the pendulum in order to make contact with the on coming ball. The open circles represent performance early in practice and the soUd circles represent performance later in practice. Note that the circles fall on the event trajectories for the different ball speeds. The circles represent the "states" where the pendulum was released for the different speed events. As you can see, early in practice there was a tendency for participants to release the pendulum "too early" for the slower balls. With practice, however, the participants learned to release the pendulum at states that fell within the hit zone for balls at all speeds. The soUd and dashed black Unes in Figure 4 represent functions fit to the performance data. The dashed lines represent simple linear functions (K =

82

John M. Flach, Matthew R. H. Smith, Terry Stanard and Scott M. Dittman

kG+9') which we will call the A,-function. These functions may reflect the criterion that participants were using for this task. For the performance obtained early in training, the data were fit using a horizontal Une. This suggests that the criterion that was being used in the comparator process was a constant value of expansion rate. That is, when the expansion rate reached a criterion value, the pendulum was released. Of course, this criterion did not completely satisfy the goal to hit all balls. Participants using this criterion would miss the slower balls. The comparator process can be described logically as follows: If e'(t) X, then too late. Where X is a critical expansion rate value. Later in practice, the data tended to fall along a diagonal Une with a positive slope and nonzero intercept. Again, this linear function may represent the criterion for pendulum release - when the state is below this Une wait, when the state reaches the Une, release the pendulum: If e'(t) < ?H- ke(t), then wait. If e'(t) = X, + ke(t), then swing. If e'(t) > X + ke(t), then too late. Where X and k are constants. This criterion is more successful than the criterion adopted early in training - as this criterion results in hits at aU speeds. Thus, it appears that practice led to a change in which the criterion for the comparison process became better tuned to the objective demands of hitting the baU. Note that a simple T criterion would be represented as a diagonal line with positive slope and zero intercept (A.=0):

Collisions: Getting them under Control

83

If e'(t) ke(t), then too late. Where k would be the inverse of time-to-contact. This would be an excellent strategy for this task since such a criterion could be nicely matched to the objective criterion reflected by the white region. The fact that participants used an expansion rate criterion early in practice and then adopted a criterion that was a function of both angle and expansion rate later in practice led Smith et al. (2001) to propose that angle and expansion rate are independent optical primitives. Note that this does not preclude a i-hke strategy. That is, with practice it seems possible that participants might continue to adjust their criterion until the criterion was a boundary with zero intercept that bisected the hit region (closely matching the objective for a perfect hit reflected by the grey line in the center of the hit region). Note that the optical state space provides a powerful tool for visuahzing the different constraints contributing to the control problem. The dimensions of the space represent hypotheses about the relevant perceptual variables (reflecting information and action constraints). The criterion functions in this space represent the "reference" in terms of the perceptual variables (reflecting value and information constraints). Commands for action (e.g., wait or release the pendulum) can be logically associated with regions or functions in this state space. Also, the normative criteria for success can be represented as functions or regions in this space (in Figure 4, the hit window). This reflects all three sources of constraint intrinsic to the comparator problem and allows those constraints to be evaluated relative to extrinsic norms for success. Note that for the control system to be successful, the constraints intrinsic to the comparator process must correspond in a functional way to the extrinsic factors that determine success (e.g., whether the ball is actually hit). The fact that the dimensions of the state space are "perceptual" variables is consistent with the spirit of Gibson's direct perception and Powers' PCT. Figure 5 shows an optical state space with a criterion that has a negative slope. In the context of the current literature, which assumes that control of coUision means responding at a "right time," this function will seem somewhat odd. However, what is the "right time" to contact when the goal is to avoid contact all together as is typically the case in driving? Instead of representing a

84

John M. Flach, Matthew R. H. Smith, Terry Stanard and Scott M. Dittman

fixed time to contact, the negative sloped function represents how imminent a coUision is relative to the rate of approach (reflected in the expansion rate). For example, when the angular separation between taiUights on the leading car is small and expansion rate is low to moderate, there is plenty of time to brake. As the angular separation grows larger, the range of acceptable expansion rates diminishes. The x-intercept, when expansion rate is zero, represents the angular size of the obstacle where you would like to be completely stopped (zero expansion means no relative motion). Thus, the criterion function in Figure 5 reflects a different value system than the criterion function in Figure 4. A control system using this criterion would initiate braking when the optical state approached the criterion function. Pressure on the brake might be adjusted so that the current state stayed at or below the critical margin. For a stationary obstacle, the x-intercept would be the angular size of the object at which the car will be stopped. For a moving object, the x-intercept would be the target following distance when the following car is traveling at the same speed as the lead car.

0'

0 Figure 5: Hypothetical optical criterion for braking to avoid collisions when driving.

Collisions: Getting them under Control

85

The criterion function in Figure 5 could be used to parse the collision space into different regions requiring quahtatively different modes of action. In the region well below and to the left of the criterion function no braking is required: If 0'(t) «X'

k6(t), then no braking required.

In the region surrounding the criterion function controlled (proportional) braking might be required to keep the state close to the criterion function. If e'(t) « X - ke(t), then brake in a way to make e'(t) = X - ke(t), If 6'(t) < ^ - kG(t), then reduce brake pressure. If 0'(t) > X - kG(t), then increase brake pressure. If 0'(t) = A, - ke(t), then maintain pressure at current level. In the region above and to the right of the criterion function, full (hard) braking might be required. If 6'(t) » X. - k0(t), then apply maximum pressure to brakes. At the extreme of this region, it will be impossible to avoid coUision through braking and perhaps the best response would be to try an evasive maneuver or to prepare for the inevitable collision. If G ' ( t ) » » X - ke(t), then pray. When using the state space diagrams in Figures 4 & 5, it is easy to fall back into classical ways of thinking about causaUty. That is, the states and criterion functions can be easily thought of as external stimuli that cause the actions identified with the different regions (specified by the if-then statements). This is a mistake! It is important to keep in mind the circular coupling illustrated in Figure 3. The state values aren't imposed on the system from without. The states are functions of relative motion of an observer and an obstacle - they are dynamic variables (functions of time). Motion through the state space is the consequence of action (e.g., motion of the car), so that the current state (what region of the state space the system is in) is a result of previous actions (e.g., braking or not). Again, causality is circular ~ so the states are simultaneously the input to the action and the product of that action. To give causal priority to either the perceptual or action system is to miss the whole point of the control theoretic perspective.

86

John M. Flach, Matthew R. H. Smith, Terry Stanard and Scott M. Dittman

6. A look inside the head In the previous section, the control system was designed using properties of an optical array as the primitives (state variables) for describing the control problem. This was in part motivated by our beUef in the possibiUty of direct perception and our confidence in this approach has been bolstered by empirical studies of coUision control (e.g., Smith, et al., 2001). By direct perception, we simply mean that the control problem is dominated by constraints that are outside the nervous system. This does not deny the participation of the nervous system; but simply means that it should be possible to describe the contribution of the nervous system in purely optical terms. In other words, the behavior of any neural networks involved in collision control should be highly correlated with properties of the optic array, so that a complete functional representation of the control problem can be made without reference to uniquely neural constraints. Thus, we tend to look to constraints external to the nervous system when developing models of the control task. However, that does not mean that research on neural processing is irrelevant. The discovery of neural components that are tuned to specific optical relations can be an important convergent operation in the search to identify the appropriate optical state variables for framing the collision control problem. Sun and Frost (1998) recently reported the discovery of three types of looming-selective neurons in the nucleus rotundus of pigeons. One class of neurons (x) showed response patterns that were independent of size and speed variation. These neurons showed firing patterns (onset and peak rates) that were an invariant function of the time-to-collision. This pattern of firing suggests that the neurons were tuned to a critical ratio between the visual angle (0) and the expansion rate (8') of the approaching object: e(t)/e'(t) = T(t) or e'(t) = [i/x(t)]e(t) A second class of neurons (p) showed firing patterns that changed with both the size and speed of the approaching object. These neurons showed onsets and peak firing rates that were earlier (longer time-to-contacts) for larger objects and for slower approaches. This firing pattern suggests that these neurons were tuned to expansion rate: e'(t) = p(t)

Collisions: Getting them under Control

87

A third class of neurons (r|) also showed firing rates that changed with size and speed. Again, onset and peak firing rates were earlier for larger objects and for slower approaches. However, this class showed a decline in firing rate immediately prior to contact (when the visual angle becomes very large), suggesting that firing rate resulted from a competition between an excitatory input associated with expansion rate (6') and an inhibitory input associated with visual angle (9). This combination can be described using the following function: Cxe'(t)/e'^®^'^ = r|(t) This equation is not as intuitive as Equations 1 and 2. However, there is a fairly simple logic to this function. The numerator reflects the excitatory contribution associated with expansion rate (G') and the denominator reflects the inhibitory contribution associated with visual angle (6). The exponential function in the denominator reflects the difference in time course for the excitation and inhibition. The excitation associated with expansion rate grows linearly with a gain equal to C. However, the inhibition associated with angle grows exponentially with a gain equal to a. The result is that expansion rate dominates early but that it is eventually overtaken by the exponentially increasing inhibition associated with visual angle. While there appear to be at least three different classes of neurons, note that there are only two "optical" primitives involved - visual angle (6) and visual expansion rate (6'). Thus, the different neural mechanisms may not reflect different "optical variables," but different "control laws." That is, the different neural mechanisms might represent different solutions to the comparator problem that reflect constraints in addition to the optics (i.e., intentional and dynamic constraints). The pigeons in Sun and Frost's (1998) experiment were being stimulated passively (i.e., no action was possible). A virtual soccer ball was launched at the pigeons that were constrained by a stereotaxic device. In control theoretic terms, the loop had been cut, so in the experimental context the response had no impact on the state of the system. In this context, it is difficult to know what the pigeon were trying to do. However, one guess might be that there was a spreading activation so that several different control circuits related to collisions were stimulated. In other words, since the control task was ambiguous, all possible control circuits were activated. Figure 6 shows the peak response data reported by Sun and Frost plotted in the optical state space. The curved gray lines extending from near the origin and extending with increasing slopes upward to the right represent the event lines for "balls" approaching at constant velocities. At the origin of each trajectory [near (0, 0)] the balls are small and not expanding (large time to contact) as the balls approach (moving up the trajectory) they grow larger and expand at a greater rate as time-to-contact

88

John M. Flach, Matthew R. H. Smith, Terry Stanard and Scott M. Dittman

gets smaller and smaller. Each of the gray curves represents a different radial travel time, where radial travel time refers to the ratio of ball radius to ball speed (radius/velocity). The optical path is uniquely determined by this ratio. Thus, a ball of radius 15 cm traveling at 250 cm/s will have a trajectory in optical state space that is identical to that for a ball of 30 cm traveling at 500 cm/s. The five curves reflect trajectories for a ball of 30 cm diameter approaching at five different speeds (750 cm/s, 500 cm/s, 300 cm/s, 250 cm/s, 150 cm/s). These were chosen to reflect a subset of the stimulus conditions used by Sun and Frost. The steeper (left most) curve represents the fastest approach (smallest radial travel time) and the shallower (right most) curve represents the slowest approach (largest radial travel time). The data points in Figure 6 are estimates from the data reported by Sun and Frost (1998) for the response onsets for the three classes of neurons [T (open squares), p (filled triangles), & r| (open circles)] that they found. The response criteria corresponding to these three mechanisms are represented by soHd lines. The criterion for T is a diagonal line with zero intercept. Thus, it is a constant time from contact. Wagner (1982) plotted the point of initiation of deceleration in approach-to-landing for house flies in an optical state space and found that the points fell along a diagonal line with a zero intercept and a slope with an inverse of 60 ms. This data is consistent with the simple i control system. This might reflect a neural circuit tuned to intercept a moving object. The criterion corresponding to p is a horizontal line that reflects a constant expansion rate (6'). Note that a critical expansion rate criterion will result in responding earlier in time to slower and/or larger objects (larger radial travel times). This trend has been observed repeatedly in studies of human performance in collision judgment and control tasks. This might reflect a control circuit tuned to evade an approaching predator. Perhaps, responding early, particularly to larger predators, might have an adaptive advantage. In this case, the consequences may not be symmetric - responding sooner, may be a lot better than responding later. The criterion corresponding to T| is an exponential curve in optical state space. However, the exponential curve can be well approximated as a line with nonzero intercept (dashed-line). Note that the line fit to this data has a negative slope as in Figure 5. Perhaps, this response might reflect a neural circuit tuned for braking with the goal of soft contact as in landing. The sohd black curves fit to the response data from Smith et al. (Figure 4) are T|-functions. Note that for the range of events used in both Figures 4 and 5, it is difficult to differentiate between the linear QC) and exponential (T|) fits.

Collisions: Getting them under Control

89

'/'''/''' Tx 6

r

/^

^X

H

5

Ol A

T3

CD

2

^

•^"^il^ • V ^ ^ ^ A

-

*

1 -

/

p

H

-1 -^ 1

, 0

1

2

3

4

5

6

7

8

e(deg) Figure 6: Neural response data from Sun and Frost (1998) plotted in an optical state space.

7. Summary and conclusions Organismic thought ... is simply the reliance on analogy, something that every physicist has done from Newton forward and back. Every significant thinker in science has drawn upon useful analogies for simplifying certain stages of thought. What is important is not to stop with rough analogy when the occasion demands that we go on, but to render the analogy into a precise, explicit, and predictive model. Weinberg (1975, p. 31). In this chapter, we have tried to illustrate how control theory can be used when studying the phenomena of collision control. In this regard, control theory is considered a language for building precise models, and for making predictions and experimental data more explicit. As with any other modeling languages or analogies, the language of control theory may illuminate some aspects of the phenomena and obscure other aspects. The analogy is not right or wrong. Rather, it is simply useful or not. Thus, we do not go so far as to claim that animals are control systems or to offer control theory a"^ an absolute truth relative to the collision problem. We simply offer control theory, as one tool for exploring the phenomena of collision control. We have found it to be a useful tool and hope that others will also find it useful. At a theoretical level, control theory has been useful to us, as we struggle to understand the fundamental nature of causality in biological and

90

John M. Flach, Matthew R. H. Smith, Terry Stanard and Scott M. Dittman

cognitive systems. Control theory provides a rich language for thinking about how perception and action might be Unked through circular causal relations. We also beUeve this language offers unique insights into Gibson's curious notion of direct perception and how this notion might be realized in concrete models. The control diagrams (e.g., Figure 3) and if-then logical statements describing the comparator process may be useful in building models and simulations of collision control. Simulations and mathematical models can be important tools for making precise predictions to be evaluated against performance data. Finally, the state space diagrams (Figures 4, 5, & 6) provide a useful framework for visualizing data in relation to the comparator problem. These diagrams allow the relations among information, action, and value constraints to be visuahzed. If the dimensions for this space are chosen correctly, then performance data plotted in this space should help to make the functional relations involved in a comparator process visible. Currently, we have found the dimensions of angle and expansion rate to be useful. However, it might also be interesting to examine other optical (e.g., x x x dot) or non-optical (e.g., distance X velocity) state spaces to see whether these spaces add additional insights into the process. At the end of the day, it is important to remember that in this case the phenomenon of natural colUsion behavior is king (how animals act to create and avoid collisions). No analogy should usurp this throne. The language of control theory is simply offered as a tool to help researchers to better serve this king. It is offered not as an "answer" to the coUision problem, but as a productive way to frame interesting questions about these phenomena.

Collisions: Getting them under Control

91

REFERENCES Flach, J. M. (1990). Control with an eye for perception: Precursors to an active psychophysics. Ecological Psychology^ 2, 83-111. Gibson, J. J. (1958/1982). Visually controlled locomotion and visual orientation in animals. Jagacinski, R. J. & Flach, J. M. (2003). Control theory for humans. Mahwah, NJ: Erlbaum. Kirk, D. E. (1970). Optimal control theory: An introduction. Englewood Cliffs, NJ: Prentice-Hall. Lee, D. N. (1974). Visual information during locomotion. In R.B. McLeod & H. Pick (Eds.), Perception: Essays in honor of J.J. Gibson (pp. 250-267). Ithaca, NY: Cornell University Press. Lee, D. N. (1976). A theory of visual control of braking based on information about time-tocollision. Perception, 5,437-459. Lee, D. N. (1980). Visuo-motor coordination in space-time. In G.E. Stelmach & Requin (Eds.). Tutorials in motor behavior {^^, 281-293). Amsterdam: North-Holland. McRuer, D. T. & Jex, H. R. (1967). A review of quasi-linear pilot models. IEEE Transactions on Human Factors, 8, 231-249. Powers, W. (1973). Behavior: The control ofperception. New York, NY: Aldine. Powers, W. (1998). Making sense of behavior. New Canaan, CT: Benchmark. Smith, M. R. H., Flach, J. M., Dittman, S. M., & Stanard, T. (2001). Monocular optical constraints on collision control. Journal of Experimental Psychology: Human Perception and Performance, 27, 395-410. Sun, H. & Frost, B. J. (1998). Computation of differential optical variables of looming objects in pigeon nucleus rotundus neurons. Nature Neuroscience, 1, 296-303. Rasmussen, J. (1986). Information processing and human-machine interaction: An approach to cognitive engineering. New York, NY: North-Holland. Wagner, H. (1982). Flow-field variables trigger landing in flies. Nature, 296, 147-148. Weinberg, G. M. (1975). An introduction to general systems thinking. New York, NY: Wiley.