- Email: [email protected]
A one step motion detection circuitry
BioSystems 40 (1997) 141-148
ELSEVIER
A one step motion detection circuitry Robert Pallbo* Lund University Cognitive Science, Kungshuset, Lundagiird, S-222 22 Lund, Sweden
Abstract A model of motion and direction detection is presented as well as results from a computer simulation of the model. The model is based on cooperative computations which allows an extremely simple architecture. Furthermore, it involves only one logical step in its computations even though Hubel and others have argued that motion detection requires at least two. This is made possible by not including the initial detection of movement. The model simply updates prior detections which is a simpler problem. New movement can be captured by the system by means of spontaneous activity in the movement selective cells. The cost is that some iterations are required to capture new movement. On the other hand, the model handles noisy input very well. The reason for this is that movement is viewed as a stable phenomenon rather than change. Keyrvords: Motion detection;
Spontaneous
activity; Computer simulation;
1. Introduction Since Hubel and Wiesel’s publication 1962, models of directional selective cells have involved more than one logical step. Usually, these models also involve time delays of some kind (e.g., Barlow and Levick’s model of the rabbit retina, 1965). In fact, Hubel (1982) writes that direction selectivity probably cannot be explained without such delays in the circuitry (see also Borst and Egelhaaf (1989) and Ullman (1983) for a general discussion on the problems of motion detection). This paper wishes to challenge this view by presenting a model of directional selectivity which
*Correspondingauthor, e-mail: [email protected] 0303-2647/96/$15.00 0 1996 Elsevier PII SO303-2647(96) 01640-2
Noisy stimuli
includes only one logical step in the circuit and a novel position of the time delay. This is made possible as the model does not include the detection of initial movement. The circuit propagates movement that has already been detected. Initial movement can still be captured by means of spontaneous activity within a few logical cycles of activity. Below, the model will be outlined, and then, the results of a computer simulation of the model will be discussed. 2. A one step motion detector In the creation of a motion detecting circuit, it is natural to start with the initial detection of the movement. Once this is solved, the process of
Science Ireland Ltd. Ah rights reserved
142
I
0
R. Pallbo / BioSystems 40 (1997) 141-148
t
0
I
0
Fig. 1. Schematic outline of the network architecture. The target cell is illustrated by a filled circle. The other fields illustrate sets of ceils. (A) The afferent input connection from the LGN. (B) The lateral excitatory connections that are used to propagate information of a forthcoming stimulus. (C) Lateral inhibition that increases acuity of the detection. (D) All connections to a cell selective for right directed movement. (E) Same as D but for left directed movement.
initial detection can be repeated in a cycle to accomplish continuous detection. However, the problem of initial detection is a problem much harder than the problem of maintaining an accomplished detection. To update the previous detection, all that needs to be done is to make sure that the stimulus is displaced in the direction of movement - a task simple enough to be carried out in one logical step. The circuitry in the model has been designed to perform this single step. This is accomplished by making each cell selective to only one direction of movement. Cells selective to one direction may then be connected in such a way that neighbouring cells will excite the cell when stimuli approach. The topology of excitatory lateral connections determine the direction that the cell is selective to (see Fig. 1). Cells selective to the same direction all have the same relative topology. For a cell that is selective to rightward movement, the topology of these lateral connections should be such that the cell receives excitation from neighbouring cells on its left-hand side. These cells will respond to a stimulus moving to the right before it reaches the position of the target cell. Likewise, a cell selective to leftward movement should be connected to the cells selective to lefhvard movement on its right-hand side. Lateral excitation is of course not sufficient in itself. A retinotopic projection is necessary to allow the cells to be exposed to the retinal stimuli. This projection is excitatory and simply projects the input onto the circuit in a way that preserves the retinal topology. In contrast to traditional Barlow and Levick style models, there is
only one set of connections involved in such a projection. Time delayed input, in the current model, is performed by the lateral connections mentioned above. The lateral input indicates a presence of a stimulus moving in a certain direction. This offers a more refined input than the traditional time delay and is also more informative. Therefore, the computational burden is decreased for the cells. These two connections, the lateral and the input projections, completes a functional circuit. If the threshold and the weights are such that, under normal conditions, a cell needs stimulation from both its lateral connections as well as the input projection, the circuitry will propagate detected movement. However, in order to initiate a detection some spontaneously active cells must be present. That way, a spontaneously active cell will impose upon its neighbours that a moving stimulus is approaching. If this is nof the case, the neighbouring cells will not receive any projectional input and therefore nof propagate the declared detection any further. If there in fact is a moving stimulus, on the other hand, the neighbours will receive the additional projectional input and the detection will thereby spread in the circuit. From computer simulations, to be discussed below, an improvement of the outlined model was discovered. It was found that additional inhibitory connections improved the direction acuity considerably. This inhibition is lateral and is made between cells of different direction preferences and is also located approximately at the same topological position. For a cell selective to rightward movement, this means that it receives lateral inhibition from the neighbouring cells of all other direction preferences but rightward - and this time without any displacement (see Fig. 10 3. Results from computer simulations To put the model to a test, it was simulated using a computer (Pallbo, 1994). The implemented model comprises eight matrixes of cells of different direction preferences as well as four matrixes of cells selective for line orientations. In
R. Pallbo / BioSystems 40 (1997) 141-148
total, over 300 000 cells and more than 6 000000 connections were simulated. This large scale was needed as the model is based on cooperative computations. A smaller scale simulation would not be able to demonstrate such cooperative effects. The model of the cell, on the other hand, was reduced to a simple integrating threshold device. This also kept the simulation within a manageable time scale. The stimuli used in this simulation consisted of a video sequence of a man walking across a vestibule. The sequence was shot at the department without any special preparations regarding light or background conditions. The obtained image was 180 x 143 pixels. The same size was used for the eight matrixes of direction selective cells. Before the stimuli was applied to the simulation, it was filtered with a Mexican hat style filter (cf. Marr, 1982). This operation mimics the processing of the ganglion cells in the retina to present the contrast changes in the original image. In the simplified filter used here, this resulted in an image with only one bit of depth (Fig. 2). The filtered image was then projected as excitatory input to the direction selective cells. Each target cell received input from one source cell at the corresponding matrix position to achieve a retinotopic projection. The lateral excitatory connection was imple-
143
mented by connecting the neighbouring cells in a 5 X 3 matrix displaced in the proper direction, e.g. left of the target cell for a rightward selective cell (Fig. 3). The lateral inhibitory connections were implemented by connecting one cell from each directions but the one of the direction of the target cell. Each of these connections was made from a cell displaced one position in the direction opposite to the preferred direction of the source cell. The strength of the connections were 800 for the retinotopic projection, 110 from the lateral excitatory connections, and 200 for the inhibitory connections. The threshold of the cells were set to 1000. In addition to this, a spontaneous activity was imposed on the direction selective cells. At each iteration, each cell had a 2% probability of being activated regardless of its current input. The result achieved from the above settings was satisfactory. The afferent connection was not able to activate a direction selective node by itself, but would activate if supported by at least two laterally connected cells. With the chosen level of spontaneous activity, detection was initiated after only a few iterations. When the stimulus ceased to move, or changed direction, the target cells lost the strong afferent input and the propagation of the movement halted. In theory, the lateral connections could by themselves maintain a propagation if ten or more cells where
Fig. 2. The image as it appears before and after filtering. The filter used is a simple rectangular version of the Mexican hat function. The result of this operation is that only contrast crossings are preserved in the filtered image. Furthermore, the filtered image is only one bit deep, i.e. each pixel is either on or off, in contrast to the 127 gray levels of the original image.
144
R Pallbo /BioSystems
Excitation 000000000 clEERBBn!Jn
Inhibition 000000000 ? o? c!oclmD23[;3n
000000000
? 0?000[21uI3140 ?? c!oooooo[7
?? IEEEEEOCID ?? IEHEEECIOO
000000080
Fig. 3. The network architecture for detecting rightward movement. The target cell is circular and the surrounding cells are square. Connected ceils are shown with their direction specificity. Left: Lateral excitation from nodes of the same specificity. Right: Lateral inhibition from nodes of different specificity.
activated in the same neighbourhood. This never occurred in practice since the filtered input image did not contain such large homogeneous areas. Areas of self-maintained activity arose when the strength of the lateral connections were experimentally increased (cf. Pallbo, 1994). A phenomenon in the detection that was not satisfactory was illusions of movement created by static extended lines. A propagation of, e.g. leftward movement would get support not only from a moving pattern, but also from a stationary horizontal line. Such lines would namely supply the movement selective cells with afferent input along an illusionary path of movement. To avoid this behaviour, the model was extended to include cells selective for line orientations. The idea was to use the detection of static lines to inhibit the corresponding direction selective cells, something which turned out to be successful. Line orientation selective cells were modelled in the same spirit as the direction selective cells, that is, by simply updating a previous detection and leave the initial detection to the spontaneous activity. For these cells, the lateral excitatory connections come from both sides of the target cell. The inhibitory lateral connections were made from a single cell from each of the other orientations and without displacement. Totally, four different orientations were implemented in the simulation. The lateral excitatory connection matrixes was of size 1 x 5 (two for each cell) with the strength 80. The inhibitory connections had the strength 100 and the retinotopic connection 512. The threshold and the arousal of the spontaneous activity was the same as for the direction selective cells, but a temporal decay in the model of the neuron was necessary to obtain a satisfactory
40 (1997) 141-148
result. This decay was implemented by preserving half of the cells arousal from one iteration to the next. The inhibition from the line orientation cells to the direction selective cells was made from the orientation matching the direction and had the strength 512. The inhibition in the other direction had the strength 500. The reason that these values differs are historical rather than functional. With the above settings, the network displayed a satisfactory behaviour (Fig. 4). The arousal of the spontaneous activity can be changed according to preferences. A higher level of arousal causes movements to be captured more rapidly but at a higher risk of mistakes. A lower level, in turn, reduces the number of mistakes but requires more iterations until movement is captured. 4. Discussion 4.1. Noise insensitivity One possible objection to the proposed model is that it creates false detections of movement. Traditionally, one strives toward an as good signal to noise ratio as possible, and viewed this way, the model appears inferior to prior models. Such criticism is not fair. The criticism is based on the idea that from a given input a resulting output should be produced that is as pure as possible. Even though this is a fair requirement for an end result, it is not as essential when the output is continuously recreated. Any false detection that is made by the circuit quickly vanishes since such activity is unable to survive without actual movement in the source image to support it. Thereby, these short appearances of false detections will most probably not have any impact on other circuits connected to the direction selective network. To support this argument, we studied how noise in the source image affected the results of the model. In these experiments, the pixels in the filtered source image were set to a random value (on or off) with a probability of lo%, 25% or 50% (this was repeated for each frame in the sequence). For 10% and 25% of additional noise, the result was satisfactory as the detection of
R Pallbo /BioSystems
: . .;.- .. :
‘;‘...’
. . ,
-2f
.*
.. .
. :
,
. ? ? ??
. ????? ?
‘..i
I
* .,: .
: .
c .
:
.
: :
:’ f”
-I
.. .
..
‘2
.
I
..
m
. . . .: . :
<
.
:;. -.,
.I. I
145
40 (1997) 141-148
‘!’ ..7.
.
. .
..
..
??
?? ?
Fig. 4. A snapshot from the computer simulation. The filtered source image appears in the center and eight different direction detections are located around it. The man is walking toward the left side. The ‘noise’ in the images are false detections of movement. The imposed spontaneous activity has been removed before producing these snapshots.
movement still was evident, but at 50% the model was put to its limit (as is a human observer) (Fig. 5). These results show that if not only the direction selective circuit, but also other circuits that are connected to it, operate along similar principles, noise is not a problem. Note that it is the noisy spontaneous activity that makes the operation possible (cf. the model of olfaction by Skarda and Freeman, 1987). The noise insensibility of the model stems from the fact that movement is treated as a stable phenomenon (cf. Lamontage, 1973). This contrasts to traditional models where movement is treated as change. In those model one starts out
with what has changed in the image and tries to find out which changes were caused by movement. In such a model noise imposes a problem as noise causes changes from image to image. The model proposed here does not share this problem. Since it updates the previous detection of movement - a movement that is a steady phenomenon for a prolonged period of time noise has a too short duration to seriously affect the system. 4.2. Simple vs. complex ceh When Hubel and Wiesel made their discover-
146
R. Pallbo /BioSystems 40 (1997) 141-148
Original
10% Noise
25% Noise
50% Noise
Fig. 5. A sequence of simulations with noise added to the stimuli. On the top row are the stimuli with 0, 10, 25, and 50% of added noise. On the bottom row are the resulting detection of lefhvard motion for respective case.
ies, they labelled the cells that exhibited orientation selectivity ‘simple cells’ as their properties could be derived from one logical step from the lateral geniculate cells (Hubel, 1982). The cells exhibiting motion selectivity, on the other hand, required more than one logical step so they were labeled ‘complex cells’. The model proposed here opposes this view. Not only is the direction selective property derived in one logical step (once the movement has been captured), the model of direction selectivity is also simpler than the one for orientation selectivity. The latter required, in the computer simulation, a temporal decay to be included in the model of the neuron, something that was not necessary in the former. The main reason for direction selectivity to be simpler than orientation selectivity is that whenever some property in the observed image deceives the circuit to react as if a proper stimuli is present, the problem is likely to disappear in the case of direction selectivity. The reason for this is that such a circuit can expect the stimulus to move in a certain direction. Improper detection will fade away because it will not be supported by the expectation of the structure. For orientation selectivity, on the other hand, the stimuli are expected to be stationary. Therefore, a visual illusion created from observing a stationary image
is more likely to persist in causing the illusion. For a direction selective circuit it is even more easy to avoid illusions if transient cells are employed, because no stationary stimuli will be represented by these cells including the one causing deceptive detections. 4.3. Lateral connections The lateral connections in the model of direction selectivity (as well as orientation selectivity) are motivated, not only by their functional attribute, but also through the existence of such connections in the cortex (Gilbert et al., 1990). Interactions, which are excitatory in nature, are made between cells with matching receptive cell properties (Gilbert and Wiesel, 1992). Furthermore, such horizontal connections have a relatively long range in comparison to thalamic afferents, and may connect cells with nonoverlapping receptive fields. This is consistent with the model proposed in this paper where the lateral excitatory connections extend wider than the afferent input. 4.4. Limitations of the model The anatomy of the lateral connections imposes limitations on the functionality of the model.
R. Pallbo /BioSystems 40 (1997) 141-148
A moving stimulus must move slow enough to ensure that the laterally connected cells will have time to respond before the stimulus enters the receptive field of the target cell. If this does not occur, the target cell will not activate. To compensate for this disadvantage, one may extend the lateral connections to cover a larger distance. Thereby, the risk of failing to capture a moving stimulus is reduced. Lateral connections that extends over large distances have their own disadvantages. It increases the probability that two stimuli will appear within the receptive fields of the connected cells. This situation may cause an illusionary movement if the first stimulus is correlated with the second by the circuitry. That is, it may appear that there is a movement from the position of the first stimulus to the position of the second. By reducing the extension of the lateral connections, this phenomenon is less likely to occur. 5. Conclusion A model of directional motion detection has been created that involves only one logical step in its computations. It does not correlate two successive images by means of time delays and parallel input afferents (as in the traditional correlational scheme) but simply updates a previous detection of motion. Thereby, the hard problem of initial detection is avoided. Initial movement is captured by the system by means of spontaneous activity which creates false indications of movements. These false indications are propagated in the circuit only when there is an actual movement in the source image that can support it. When this support is absent, false detections disappear in the next iteration. By this means, movement in this model is viewed as a stable phenomenon rather than change. As a result, the network is extremely resistant to noise which has been shown in the computer simulations. These simulations also shows that the model is capable of producing a good acuity under normal circumstances. To reduce illusions of movement along stationary lines, it was beneficial to extend the model to include detection of such lines. The detections made by
147
these line detectors were used to inhibit the motion detectors which suppressed illusionary detections. Another solution to this problem would be to use transient cells to supply the afferent connections from the LGN. Even though this solution is more biologically accurate, it was not implemented here. The cells selective to static line orientations were implemented along the same line of principles as the cells selective to movement. That is, no initial detection was directly implemented, but was covered for by means of spontaneous activity. Somewhat to our surprise, line orientation selectivity turned out to be a more difficult area to implement and required a more complex model of the neuron than was needed for motion detection. The main cause for this is that the detection of motion is constantly moving to track the moving stimuli. This offers a possibility to leave behind the (cause of) problems and constantly renew the detection. This is not possible in the case of line detection where a potential problem may grow to severe proportions and seriously affect the system. To avoid this, more precautions are required than were needed for motion detection. Acknowledgements This work has been supported by the Swedish Council for Research in the Humanities and Social Sciences. References Barlow, H.B. and Levick, W.R., 1965, The mechanism of directionally selective units in rabbit’s retina. J. Physiol. 178,477-5&t. Borst, A. and Egelhaaf, M., 1989, Principles of visual motion detection. Trends Neurosci. 12, 297-306. Gilbert, C.D., Hirsch, J.A. and Wiesel, T.N., 1990, Lateral interactions in visual cortex. Cold Spring Harbor Symposia on Quantitative Biology 50, 663-677. Gilbert, CD. and Wiesel, T.N., 1992, Receptive field dynamics in adult primary visual cortex. Nature 356, 150-152. Hubel, D.H., 1982, Exploration of the primary visual cortex, 1955-78. Nature 299, 515-524. Hubel, D.H. and Wiesel, T.N., 1962, Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. 160, 106-154.
148
R Pallbo /BioSystems
Lamontage, C., 1973, A new experimental paradigm for the investigation of the secondary system of human visual motion perception. Perception 2, 167-180. Marr, D., 1982, Vision (W. H. Freeman and Company). Pallbo, R., 1994, Motion detection - a neural model and its implementation. LUCS Minor 1. ISSN 11041609.
40 (1997) 141-148
Skarda, C.A. and Freeman, W.J., 1987, How brains make chaos in order to make sense of the world. Behav. Brain Sci. 10, 161-195. Ullman, S., 1983, The measurement of visual motion: Computational considerations and some neurophysiological implications. Trends Neurosci. 6, 177-179.
ELSEVIER
A one step motion detection circuitry Robert Pallbo* Lund University Cognitive Science, Kungshuset, Lundagiird, S-222 22 Lund, Sweden
Abstract A model of motion and direction detection is presented as well as results from a computer simulation of the model. The model is based on cooperative computations which allows an extremely simple architecture. Furthermore, it involves only one logical step in its computations even though Hubel and others have argued that motion detection requires at least two. This is made possible by not including the initial detection of movement. The model simply updates prior detections which is a simpler problem. New movement can be captured by the system by means of spontaneous activity in the movement selective cells. The cost is that some iterations are required to capture new movement. On the other hand, the model handles noisy input very well. The reason for this is that movement is viewed as a stable phenomenon rather than change. Keyrvords: Motion detection;
Spontaneous
activity; Computer simulation;
1. Introduction Since Hubel and Wiesel’s publication 1962, models of directional selective cells have involved more than one logical step. Usually, these models also involve time delays of some kind (e.g., Barlow and Levick’s model of the rabbit retina, 1965). In fact, Hubel (1982) writes that direction selectivity probably cannot be explained without such delays in the circuitry (see also Borst and Egelhaaf (1989) and Ullman (1983) for a general discussion on the problems of motion detection). This paper wishes to challenge this view by presenting a model of directional selectivity which
*Correspondingauthor, e-mail: [email protected] 0303-2647/96/$15.00 0 1996 Elsevier PII SO303-2647(96) 01640-2
Noisy stimuli
includes only one logical step in the circuit and a novel position of the time delay. This is made possible as the model does not include the detection of initial movement. The circuit propagates movement that has already been detected. Initial movement can still be captured by means of spontaneous activity within a few logical cycles of activity. Below, the model will be outlined, and then, the results of a computer simulation of the model will be discussed. 2. A one step motion detector In the creation of a motion detecting circuit, it is natural to start with the initial detection of the movement. Once this is solved, the process of
Science Ireland Ltd. Ah rights reserved
142
I
0
R. Pallbo / BioSystems 40 (1997) 141-148
t
0
I
0
Fig. 1. Schematic outline of the network architecture. The target cell is illustrated by a filled circle. The other fields illustrate sets of ceils. (A) The afferent input connection from the LGN. (B) The lateral excitatory connections that are used to propagate information of a forthcoming stimulus. (C) Lateral inhibition that increases acuity of the detection. (D) All connections to a cell selective for right directed movement. (E) Same as D but for left directed movement.
initial detection can be repeated in a cycle to accomplish continuous detection. However, the problem of initial detection is a problem much harder than the problem of maintaining an accomplished detection. To update the previous detection, all that needs to be done is to make sure that the stimulus is displaced in the direction of movement - a task simple enough to be carried out in one logical step. The circuitry in the model has been designed to perform this single step. This is accomplished by making each cell selective to only one direction of movement. Cells selective to one direction may then be connected in such a way that neighbouring cells will excite the cell when stimuli approach. The topology of excitatory lateral connections determine the direction that the cell is selective to (see Fig. 1). Cells selective to the same direction all have the same relative topology. For a cell that is selective to rightward movement, the topology of these lateral connections should be such that the cell receives excitation from neighbouring cells on its left-hand side. These cells will respond to a stimulus moving to the right before it reaches the position of the target cell. Likewise, a cell selective to leftward movement should be connected to the cells selective to lefhvard movement on its right-hand side. Lateral excitation is of course not sufficient in itself. A retinotopic projection is necessary to allow the cells to be exposed to the retinal stimuli. This projection is excitatory and simply projects the input onto the circuit in a way that preserves the retinal topology. In contrast to traditional Barlow and Levick style models, there is
only one set of connections involved in such a projection. Time delayed input, in the current model, is performed by the lateral connections mentioned above. The lateral input indicates a presence of a stimulus moving in a certain direction. This offers a more refined input than the traditional time delay and is also more informative. Therefore, the computational burden is decreased for the cells. These two connections, the lateral and the input projections, completes a functional circuit. If the threshold and the weights are such that, under normal conditions, a cell needs stimulation from both its lateral connections as well as the input projection, the circuitry will propagate detected movement. However, in order to initiate a detection some spontaneously active cells must be present. That way, a spontaneously active cell will impose upon its neighbours that a moving stimulus is approaching. If this is nof the case, the neighbouring cells will not receive any projectional input and therefore nof propagate the declared detection any further. If there in fact is a moving stimulus, on the other hand, the neighbours will receive the additional projectional input and the detection will thereby spread in the circuit. From computer simulations, to be discussed below, an improvement of the outlined model was discovered. It was found that additional inhibitory connections improved the direction acuity considerably. This inhibition is lateral and is made between cells of different direction preferences and is also located approximately at the same topological position. For a cell selective to rightward movement, this means that it receives lateral inhibition from the neighbouring cells of all other direction preferences but rightward - and this time without any displacement (see Fig. 10 3. Results from computer simulations To put the model to a test, it was simulated using a computer (Pallbo, 1994). The implemented model comprises eight matrixes of cells of different direction preferences as well as four matrixes of cells selective for line orientations. In
R. Pallbo / BioSystems 40 (1997) 141-148
total, over 300 000 cells and more than 6 000000 connections were simulated. This large scale was needed as the model is based on cooperative computations. A smaller scale simulation would not be able to demonstrate such cooperative effects. The model of the cell, on the other hand, was reduced to a simple integrating threshold device. This also kept the simulation within a manageable time scale. The stimuli used in this simulation consisted of a video sequence of a man walking across a vestibule. The sequence was shot at the department without any special preparations regarding light or background conditions. The obtained image was 180 x 143 pixels. The same size was used for the eight matrixes of direction selective cells. Before the stimuli was applied to the simulation, it was filtered with a Mexican hat style filter (cf. Marr, 1982). This operation mimics the processing of the ganglion cells in the retina to present the contrast changes in the original image. In the simplified filter used here, this resulted in an image with only one bit of depth (Fig. 2). The filtered image was then projected as excitatory input to the direction selective cells. Each target cell received input from one source cell at the corresponding matrix position to achieve a retinotopic projection. The lateral excitatory connection was imple-
143
mented by connecting the neighbouring cells in a 5 X 3 matrix displaced in the proper direction, e.g. left of the target cell for a rightward selective cell (Fig. 3). The lateral inhibitory connections were implemented by connecting one cell from each directions but the one of the direction of the target cell. Each of these connections was made from a cell displaced one position in the direction opposite to the preferred direction of the source cell. The strength of the connections were 800 for the retinotopic projection, 110 from the lateral excitatory connections, and 200 for the inhibitory connections. The threshold of the cells were set to 1000. In addition to this, a spontaneous activity was imposed on the direction selective cells. At each iteration, each cell had a 2% probability of being activated regardless of its current input. The result achieved from the above settings was satisfactory. The afferent connection was not able to activate a direction selective node by itself, but would activate if supported by at least two laterally connected cells. With the chosen level of spontaneous activity, detection was initiated after only a few iterations. When the stimulus ceased to move, or changed direction, the target cells lost the strong afferent input and the propagation of the movement halted. In theory, the lateral connections could by themselves maintain a propagation if ten or more cells where
Fig. 2. The image as it appears before and after filtering. The filter used is a simple rectangular version of the Mexican hat function. The result of this operation is that only contrast crossings are preserved in the filtered image. Furthermore, the filtered image is only one bit deep, i.e. each pixel is either on or off, in contrast to the 127 gray levels of the original image.
144
R Pallbo /BioSystems
Excitation 000000000 clEERBBn!Jn
Inhibition 000000000 ? o? c!oclmD23[;3n
000000000
? 0?000[21uI3140 ?? c!oooooo[7
?? IEEEEEOCID ?? IEHEEECIOO
000000080
Fig. 3. The network architecture for detecting rightward movement. The target cell is circular and the surrounding cells are square. Connected ceils are shown with their direction specificity. Left: Lateral excitation from nodes of the same specificity. Right: Lateral inhibition from nodes of different specificity.
activated in the same neighbourhood. This never occurred in practice since the filtered input image did not contain such large homogeneous areas. Areas of self-maintained activity arose when the strength of the lateral connections were experimentally increased (cf. Pallbo, 1994). A phenomenon in the detection that was not satisfactory was illusions of movement created by static extended lines. A propagation of, e.g. leftward movement would get support not only from a moving pattern, but also from a stationary horizontal line. Such lines would namely supply the movement selective cells with afferent input along an illusionary path of movement. To avoid this behaviour, the model was extended to include cells selective for line orientations. The idea was to use the detection of static lines to inhibit the corresponding direction selective cells, something which turned out to be successful. Line orientation selective cells were modelled in the same spirit as the direction selective cells, that is, by simply updating a previous detection and leave the initial detection to the spontaneous activity. For these cells, the lateral excitatory connections come from both sides of the target cell. The inhibitory lateral connections were made from a single cell from each of the other orientations and without displacement. Totally, four different orientations were implemented in the simulation. The lateral excitatory connection matrixes was of size 1 x 5 (two for each cell) with the strength 80. The inhibitory connections had the strength 100 and the retinotopic connection 512. The threshold and the arousal of the spontaneous activity was the same as for the direction selective cells, but a temporal decay in the model of the neuron was necessary to obtain a satisfactory
40 (1997) 141-148
result. This decay was implemented by preserving half of the cells arousal from one iteration to the next. The inhibition from the line orientation cells to the direction selective cells was made from the orientation matching the direction and had the strength 512. The inhibition in the other direction had the strength 500. The reason that these values differs are historical rather than functional. With the above settings, the network displayed a satisfactory behaviour (Fig. 4). The arousal of the spontaneous activity can be changed according to preferences. A higher level of arousal causes movements to be captured more rapidly but at a higher risk of mistakes. A lower level, in turn, reduces the number of mistakes but requires more iterations until movement is captured. 4. Discussion 4.1. Noise insensitivity One possible objection to the proposed model is that it creates false detections of movement. Traditionally, one strives toward an as good signal to noise ratio as possible, and viewed this way, the model appears inferior to prior models. Such criticism is not fair. The criticism is based on the idea that from a given input a resulting output should be produced that is as pure as possible. Even though this is a fair requirement for an end result, it is not as essential when the output is continuously recreated. Any false detection that is made by the circuit quickly vanishes since such activity is unable to survive without actual movement in the source image to support it. Thereby, these short appearances of false detections will most probably not have any impact on other circuits connected to the direction selective network. To support this argument, we studied how noise in the source image affected the results of the model. In these experiments, the pixels in the filtered source image were set to a random value (on or off) with a probability of lo%, 25% or 50% (this was repeated for each frame in the sequence). For 10% and 25% of additional noise, the result was satisfactory as the detection of
R Pallbo /BioSystems
: . .;.- .. :
‘;‘...’
. . ,
-2f
.*
.. .
. :
,
. ? ? ??
. ????? ?
‘..i
I
* .,: .
: .
c .
:
.
: :
:’ f”
-I
.. .
..
‘2
.
I
..
m
. . . .: . :
<
.
:;. -.,
.I. I
145
40 (1997) 141-148
‘!’ ..7.
.
. .
..
..
??
?? ?
Fig. 4. A snapshot from the computer simulation. The filtered source image appears in the center and eight different direction detections are located around it. The man is walking toward the left side. The ‘noise’ in the images are false detections of movement. The imposed spontaneous activity has been removed before producing these snapshots.
movement still was evident, but at 50% the model was put to its limit (as is a human observer) (Fig. 5). These results show that if not only the direction selective circuit, but also other circuits that are connected to it, operate along similar principles, noise is not a problem. Note that it is the noisy spontaneous activity that makes the operation possible (cf. the model of olfaction by Skarda and Freeman, 1987). The noise insensibility of the model stems from the fact that movement is treated as a stable phenomenon (cf. Lamontage, 1973). This contrasts to traditional models where movement is treated as change. In those model one starts out
with what has changed in the image and tries to find out which changes were caused by movement. In such a model noise imposes a problem as noise causes changes from image to image. The model proposed here does not share this problem. Since it updates the previous detection of movement - a movement that is a steady phenomenon for a prolonged period of time noise has a too short duration to seriously affect the system. 4.2. Simple vs. complex ceh When Hubel and Wiesel made their discover-
146
R. Pallbo /BioSystems 40 (1997) 141-148
Original
10% Noise
25% Noise
50% Noise
Fig. 5. A sequence of simulations with noise added to the stimuli. On the top row are the stimuli with 0, 10, 25, and 50% of added noise. On the bottom row are the resulting detection of lefhvard motion for respective case.
ies, they labelled the cells that exhibited orientation selectivity ‘simple cells’ as their properties could be derived from one logical step from the lateral geniculate cells (Hubel, 1982). The cells exhibiting motion selectivity, on the other hand, required more than one logical step so they were labeled ‘complex cells’. The model proposed here opposes this view. Not only is the direction selective property derived in one logical step (once the movement has been captured), the model of direction selectivity is also simpler than the one for orientation selectivity. The latter required, in the computer simulation, a temporal decay to be included in the model of the neuron, something that was not necessary in the former. The main reason for direction selectivity to be simpler than orientation selectivity is that whenever some property in the observed image deceives the circuit to react as if a proper stimuli is present, the problem is likely to disappear in the case of direction selectivity. The reason for this is that such a circuit can expect the stimulus to move in a certain direction. Improper detection will fade away because it will not be supported by the expectation of the structure. For orientation selectivity, on the other hand, the stimuli are expected to be stationary. Therefore, a visual illusion created from observing a stationary image
is more likely to persist in causing the illusion. For a direction selective circuit it is even more easy to avoid illusions if transient cells are employed, because no stationary stimuli will be represented by these cells including the one causing deceptive detections. 4.3. Lateral connections The lateral connections in the model of direction selectivity (as well as orientation selectivity) are motivated, not only by their functional attribute, but also through the existence of such connections in the cortex (Gilbert et al., 1990). Interactions, which are excitatory in nature, are made between cells with matching receptive cell properties (Gilbert and Wiesel, 1992). Furthermore, such horizontal connections have a relatively long range in comparison to thalamic afferents, and may connect cells with nonoverlapping receptive fields. This is consistent with the model proposed in this paper where the lateral excitatory connections extend wider than the afferent input. 4.4. Limitations of the model The anatomy of the lateral connections imposes limitations on the functionality of the model.
R. Pallbo /BioSystems 40 (1997) 141-148
A moving stimulus must move slow enough to ensure that the laterally connected cells will have time to respond before the stimulus enters the receptive field of the target cell. If this does not occur, the target cell will not activate. To compensate for this disadvantage, one may extend the lateral connections to cover a larger distance. Thereby, the risk of failing to capture a moving stimulus is reduced. Lateral connections that extends over large distances have their own disadvantages. It increases the probability that two stimuli will appear within the receptive fields of the connected cells. This situation may cause an illusionary movement if the first stimulus is correlated with the second by the circuitry. That is, it may appear that there is a movement from the position of the first stimulus to the position of the second. By reducing the extension of the lateral connections, this phenomenon is less likely to occur. 5. Conclusion A model of directional motion detection has been created that involves only one logical step in its computations. It does not correlate two successive images by means of time delays and parallel input afferents (as in the traditional correlational scheme) but simply updates a previous detection of motion. Thereby, the hard problem of initial detection is avoided. Initial movement is captured by the system by means of spontaneous activity which creates false indications of movements. These false indications are propagated in the circuit only when there is an actual movement in the source image that can support it. When this support is absent, false detections disappear in the next iteration. By this means, movement in this model is viewed as a stable phenomenon rather than change. As a result, the network is extremely resistant to noise which has been shown in the computer simulations. These simulations also shows that the model is capable of producing a good acuity under normal circumstances. To reduce illusions of movement along stationary lines, it was beneficial to extend the model to include detection of such lines. The detections made by
147
these line detectors were used to inhibit the motion detectors which suppressed illusionary detections. Another solution to this problem would be to use transient cells to supply the afferent connections from the LGN. Even though this solution is more biologically accurate, it was not implemented here. The cells selective to static line orientations were implemented along the same line of principles as the cells selective to movement. That is, no initial detection was directly implemented, but was covered for by means of spontaneous activity. Somewhat to our surprise, line orientation selectivity turned out to be a more difficult area to implement and required a more complex model of the neuron than was needed for motion detection. The main cause for this is that the detection of motion is constantly moving to track the moving stimuli. This offers a possibility to leave behind the (cause of) problems and constantly renew the detection. This is not possible in the case of line detection where a potential problem may grow to severe proportions and seriously affect the system. To avoid this, more precautions are required than were needed for motion detection. Acknowledgements This work has been supported by the Swedish Council for Research in the Humanities and Social Sciences. References Barlow, H.B. and Levick, W.R., 1965, The mechanism of directionally selective units in rabbit’s retina. J. Physiol. 178,477-5&t. Borst, A. and Egelhaaf, M., 1989, Principles of visual motion detection. Trends Neurosci. 12, 297-306. Gilbert, C.D., Hirsch, J.A. and Wiesel, T.N., 1990, Lateral interactions in visual cortex. Cold Spring Harbor Symposia on Quantitative Biology 50, 663-677. Gilbert, CD. and Wiesel, T.N., 1992, Receptive field dynamics in adult primary visual cortex. Nature 356, 150-152. Hubel, D.H., 1982, Exploration of the primary visual cortex, 1955-78. Nature 299, 515-524. Hubel, D.H. and Wiesel, T.N., 1962, Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. 160, 106-154.
148
R Pallbo /BioSystems
Lamontage, C., 1973, A new experimental paradigm for the investigation of the secondary system of human visual motion perception. Perception 2, 167-180. Marr, D., 1982, Vision (W. H. Freeman and Company). Pallbo, R., 1994, Motion detection - a neural model and its implementation. LUCS Minor 1. ISSN 11041609.
40 (1997) 141-148
Skarda, C.A. and Freeman, W.J., 1987, How brains make chaos in order to make sense of the world. Behav. Brain Sci. 10, 161-195. Ullman, S., 1983, The measurement of visual motion: Computational considerations and some neurophysiological implications. Trends Neurosci. 6, 177-179.