- Email: [email protected]
Eidetic images in insects: their role in navigation
TINS - March 1983
ment in fly photoreceptors:propertiesand candidates. Biophys. Struct. Mech. Vol. 9 (in press) 7 Kirschfeld, K., Feiler, R. and Minke, B. (1978) Z. Naturforsch. 33c, 1009-1010 8 Kirschfeld, K., Franceschini, N. and Minke, B. (1977) Nature (London) 269, 386-390 9 Kirschfeld, K. and Wenk, P. (1976) Z. Naturforsch. 31c, 764-765 10 Mclntyre, P. D. and Snyder, A. W. (1973)J. Opt. Soc. Am. 63, 1518-1527 II Minke, B. and Kirschfeld, K. (1980)J. Gen.
101 PhysioL 75,381--402 12 Pak, W. L. and Lidington, K. J. (1974) J. Gen. PhysioL 63,740-756 13 Pick, B. (1977) B/ol. Cybern. 26, 215-224 14 Snyder, A. W. and Menzel, R. (1975) Photoreceptor Optics (Snyder, A. W. and Menzel, R., eds), Springer-Verlag,Berlin 15 Thurm, U. (1982) Biophysik (Hoppe, W., Lohmann, W., Markl, H. and Ziegler, H., eds), 2rid edn, p. 691, Springer-Verlag,Berlin 16 Wijngaard, W. (1974) J. Opt. Soc. Am. 64,
1136-1144
17 Wijnganrd,W. (1981) in Optical Sciences - Vertebrate Photoreceptor Optics (Enoch, J. M. and Tobey, F. L., eds), pp. 301-323, SpringerVerlag, Berlin 18 Wijngaard,W. and Stavenga, D. G. (1975) Biol. Cybern. 18, 61--67 K. Kirschfeld is at the Max-Planck-lnstitut f~r Biologische Kybernetik, Spemannstrasse 38, D- 7400 Tiibingen, F.R.G.
Eidetic images in insects: their role in navigation
ney by keeping a running tally of its direction and distance from its starting point. This information can then be used at any time to specify a direct path back. Many animals, both invertebrates 7 and vertebrates 8 use dead reckoning of this kind. Indeed, when worker bees inside a hive are recruited to a food source, the dance tells them no more than the distance of the source and its direction with respect to gravity 11. Outside the hive they refer the directional signal to the sun or pattern of polarized light in the sky and fly to the approximate position of the source. However, such direction and distance signals are unlikely to be sufficiently precise to allow a bee on a subsequent trip to fly back to a specific, small patch of flowers. For this it needs more detailed knowledge of the terrain. There is much evidence suggesting that
T. S. Collett and B. A. Cartwright In o r d e r to return to a particular place in their e n v i r o n m e n t s o m e insects store what is essentially a p a n o r a m i c snapshot o f their surroundings taken f r o m the spot they wish to retrieve. They can then guide their return with the s n a p s h o t by m o v i n g so that their current retinal i m a g e c o m e s to m a t c h the image they have stored.
A somewhat neglected phenomenon in the study of human memory is that of photographic memory - the ability to store and recall in detail images which have at one time fallen upon the retina. Some individuals evidently have much richer and more persistent visual imageD, than others, but it is unknown to what extent people in general are capable of retaining arbitrary details of an image. One reason why this topic seems to remain almost a curiosity is that little attempt has been made to identify everyday tasks for which such an ability is essential. In this article we argue that it is probably common for insects to memorize precise details of their retinal images, and that they do so in particular circumstances and for particular reasons. Many insects are able to return to a given spot. Wasps dig burrows in sand in which they lay eggs and which they later need to relocate in order to provide the growing larvae with food. It is remarkable with what ease they can find an inconspicuous hole sited in a uniform plain. Foraging bees are equally good at returning repeatedly to the same clump of flowers to exploit a fruitful source of nectar or pollen. Similarly, longlegged ants of the species Cataglyphis bicolor found in sandy areas of the Near and Middle East, forage individually far from their nest seeking dead insects and are able to return and unerringly locate their nest which is a barely noticeable hole in the sand. Behaviour of this kind is not limited to hymenoptera. Male hoverflies hang motionless in the air, occasionally darting out to chase what they hope may be a female and then returning to the same spot. Dragonflies
also have perches to which they return after brief flights, and heliconiid butterflies roost every night on the sanle part o f a particular tree or vine. The extensive literature on this intriguing topic is fully reviewed by Wehner TM. One way an animal can regain a location in space is for it to monitor its outward jourT
0.5 brr
0-4
wW
f~ o.3
~ 0-2
\
q~
0.1
0-0 I
1
I
I
0
2O
40
6O
I
I
1
I~
12O
L
1~
I
I
16O
I~
it)
°Aa
o •
Fig. 1. Frequencies of bees' choices between test and training patterns as the test pattern is rotated clock wise ((3) or anti-clockwise (0) from the training orientation. Patterns were presented on a vertical surface. Bees chose the training pattern more frequently as the difference between the orientations of the two patterns increased. From Ref. 12. ~
Elsevier B i o m e d i c a l Press 1983
0378
5912/83/0CO0 - 0000/$0100
102
! I.'Y'~
nearby landmarks are an important cue Ior guiding insects over the last stages of their return. One of the f'n'st and clearest demonstrations was that of Tinbergen ~° who marked the nest hole of a sphecid wasp by a ring of pine cones. When the pine cones were moved to a new position the returning wasp searched for its nest in the spot defined by the displaced ring. More recently we have begun to learn in more detail how the landmarks are used. Insects seem to store something similar to a photographic image of the immediate surroundings of the spot they wish to retrieve. Then, equipped with this picture or template they can fred the place again by moving until the image on their retina matches their template. The test piece of evidence that insects can remember relatively raw, unprocessed retinal images comes from visual discrimination experiments conducted by WehnePL He trained bees to associate food with a single two-dimensional pattern presented on a vertical surface and then tested them with a choice of the trained pattern and a second pattern which was altered to some degree from the first. If bees selected indiscriminately between the two patterns, then in a sense they saw patterns as the same, whereas the more the training pattern was preferred, the greater the perceived difference between the two. The conclusion from a number of experiments of the type illustrated in Fig. 1 is that discriminability is based on a very simple form of template matching. Imagine a fine grid superimposed over each pattern, dividing it into cells which are either black or white, then similarity is defined by how many cells in one grid match the brightness of their partners in the other. By painting over parts of the eye Wehner and Flatt ~4 found that particular regions of the pattern were learnt and inspected using particular regions of the eye - the stored image was fixed with respect to the retina.
(o)
,
This suggests that a bee looking at these patterns will always adopt the same posture and orientation in space, and films of bees performing this task confirmed that this was SO.
This work implies that patterns are learnt before they have been extensively analysed or categorized. For instance, in Fig. 1 the bee has been trained to a disk divided by a straight line through the middle into equal white and black areas. When the disk is rotated through 180° from the training orientation, the bee treats the pattern as quite different, even though the orientation of the dividing edge between the black and white sections is now the same as it was during training. The bee does not seem to have abstracted and learnt the orientation of the pattern, a feature which strikes us very vividly. For many purposes it seems far from ideal that information about the world should be categorized in such a rigid format. Individual objects often need to be recognized from a variety of points of view and different objects of the same type need to be seen as sharing certain properties, even though in terms of a fixed template they may have very little in common. However, a photographic image may be appropriate for storing the minutiae of a locale as seen from a particular vantage point. The position of a nest, for example, is defined by an arbitrary arrangement of sticks, bushes and stones. The identity of these objects is of no significance, the insect only has to learn their appearance and layout. Furthermore, the information is of no use in any other context; it can only specify a single place. An elegant example of an insect using stored images for navigation is provided by the African stink ant, P a l t o t h y r e u s tarsatusL This ant scavenges and hunts on the forest floor, so that it must view the sky through the forest canopy and rarely has an unobstructed view of the sky. There is
~:[ar~-h I ,)~'.:"
moreover often thick cloud cover so that neither the sun nor skylight cues are avail able as directional guides to help lead ants to their nest after foraging. HolldobleP has shown that instead they use the pattern of the foliage high above them seen silhouetted against the sky. To demonstrate this he took transparencies of the canopy from a camera on the ground and used them to make a patterned ceiling for ants kept in an arena in the laboratory. Worker ants accu~ tomed to the arena would lorage and then return directly to their nest. The direction of these homing runs depended on the orientation of the ceiling patterns, and when the pattern was turned through 180° the ants were thrown off course by the same amount. For the ant to obtain compass bearings from an arbitrary' canopy pattern, it must be able to remember specific details of the pattern. Explicit tests have now been made of the suggestion that a stored image or snapshot of nearby landmarks is also used to relocate a nest or foraging spot. The problem falls into two parts. First, what information do insects pick up and learn about the spatial layout of landmarks near a spot they wish to remember? Second, how do they use this information to guide their return there? To discover the way insects represent their immediate surroundings similar experiments have been done by Anderson t and ourselves, on honey bees ~.~, and by Wehner and R~ibeP 5, on desert ants. The strategy in all these studies was to persuade bees or ants to associate a food source or nest site with a given constellation of landmarks and then to investigate the insect's behaviour when the array was altered. The method we used with bees was as follows. Single marked bees from a nearby hive were trained to forage at a food source placed on the floor of a large, empty room. The location of the food source was defined by one or more matt-black cylinders positioned vertically on the floor. After
(b)
Fig. 2. (a) Distribution o f a single bee's flight time within the floor area surveyed by the camera when tested after training to a cylindrical landmark 50 cm from a food source. Base o f landmark is shown by fdled c~rcle. Po,fftion o f food source during t r a k ~ g marked by Fon x and y axes. Height shows relative times spent in different regions o f the test area. Lines on grid are 8. 7 cm apart. Co)Distribution o f bee's flight time during training after it had learnt to associate the position o f the food source with an arrangement o f three landmarks. From Ref. 3.
TINS - March 1983
103 a
0
b jo
•
I
~
o
'1
100 cm Fig. 3. Bee's search area during tests and aJ~er training to the arrangement o f three cylindrical landmarks shown in plan view in (a). The hatching represents the search area defined as those squares within the test area (as in Fig. 2) which are at least 80 % as high as the peak value. (a) Search area with training configuration. (b) Distance between landmarks double that o f the training distance. (c) Distance halved. (d) Landmarks arranged so bee could search either where the landmarks had the same bearings on the retina as the bee had experienced at the food source during training, or where the distances o f the landmarks were the same as those seen from the food source. In all cases the bee chooses to search where the bearings o f the landmarks on its retina were what it had been accustomed to seeing. From Ref. 3.
each of the bee's visits to the food source, landmarks and food source were moved together as a group to a new position. These shifts were purely translational, and the orientation of the landmark array was kept constant with respect to external coordinates. Once the bee was trained to the landmarks, the food source could be removed and the insect's search path was then recorded through a video camera on the ceiling. The way the bee distributes its time within the test area (Fig. 2) implies that it does indeed use the landmark array to decide where to search. The interesting part of the experiment is then to change the landmark array, either by altering the size or shape of individual landmarks or by changing their number and spatial arrangement, and to plot where the insect searches when the food source is absent. The insect's expectation of where food is to be found when the array is distorted can tell us what it has learnt about the landmarks. We began with the position of the food source defined by a single landmark. During a'aining the food source was kept at a constant orientation and distance from the landmark. When the food source was removed the bee searched in approximately the right direction. This indicates that the bees have directional information from a source external to the landmark (possibly from the diffused light which filtered through the curtained windows which covered most of one wall). If the landmark
had been the only available cue, bees would have searched in an annulus centered on it. More significantly, the search area is also situated in approximately the right distance from the landmark. To ensure that bees were not just flying at a preferred distance from the landmark which happened to coincide with that to which they had been trained, different bees were trained to food at different distances from the landmark and the search area was always located at the appropriate spot. This result does not mean that bees know how far they are from the landmark in terms of the distance they would have to fly in order to reach it, but merely that they search where the landmark subtends the same angle on their retina that it did when the bees were at the food source. When bees were tested with a landmark larger than the training size, they searched further away. Conversely, when the test landmark was smaller, they searched closer. Such data suggest that the bee learns the apparent size of the landmark as seen from the food source and moves so that its current retinal image matches its memory. Experiments using three landmarks gave results in accordance with this general hypothesis. Bees were trained to the configuration shown in plan view in Fig. 3a. When the landmark array during the test differed from the training situation, bees consistently searched where the compass bearings of the landmarks on their retina
matched those they had experienced at the food source during training. This can be seen in Fig. 3 which shows the search area when the distances between landmarks were either halved or doubled, or when the shape oftbe array was changed. The deduction from such results is that bees do not learn a plan view or map of the array, but only the bearings and sizes of the landmarks as seen from the food source. With these distorted arrays there cannot be a perfect match between the bee's snapshot and its retinal image. When the bee searches where landmarks are in the correct location on the retina to match the snapshot, then their size must be wrong. Their choice of search area does, however, seem to be the best compromise. Suppose for example the bee were to position itself so that the retinal image of one landmark was congruent with its partner in the snapshot, then the images of the other landmarks would differ from their partners in both size and position. Desert ants employ landmarks in a similar way, but with some intriguing variations. In Wehner and R~iber's experiments ~5, a nest opening of a colony of ants was marked by two small black cylinders placed either side of the nest (Fig. 4). After a period to allow the ants to become accustomed to the presence of the landmarks, both landmarks and ants were displaced to a new area and the ant's search path was recorded. When the constellation of landmarks was the same as that used for training, ants searched midway between the two (Fig. 4a), i.e. where the nest ought to have been. They did likewise when distance and size were both altered together so that the view of the landmarks from halfway between them remained what the ants had been accustomed to see from the nest site (Fig. 4c). However, if either the distance between the landmarks or their size was increased or decreased without any alteration in the other parameter, the ants then searched very close to one or other landmark (Fig. 4b). There is now no place where the retinal image can match the usual picture the ant has from its nest site, and it is as though the ants then give up and adopt a new strategy. It is difficult to compare the results of experiments performed in the open desert with those undertaken in a restricted laboratory environment, and it is unwise at this stage of our knowledge to conclude that there are genuine differences in the landmark guidance systems of the two animals. It is safer to stress the overall similarities. The best explanation of these results is the same as that proposed for bees and earlier for hoverflies when they return to preferred hovering locations 4. The ant takes a snapshot of its surroundings from the nest site, which it can later compare with its current
t I\,~
104
.
.
,
.
.
.
.
,
.
.
.
.
,
.
.
.
.
•
.
.
.
.
landmarks almost inevitably leads to que~ tions with a photographic slant, and it will be interesting to see how hard the analogy can be pressed. One of the first things t~ appreciate when holding a camera is that the pictures will be blurred if the camera is unsteady. However, insects are not at all still when learning landmark arrays. They perform characteristic orientation flights in which their departure from the location they wish to learn is in a series of zigzags during which they continually face the hive or food source to which they hope to return. Why' should they do so? Are they taking a succession of snapshots from different points of view? Or are they in some way filtering out of their shapshot parts of the image which move slowly across their retina, or which shrink to an insignificant size once the insect has moved away ? The snapshot may be an edited one with very distant and with very small and close objects ignored. For instance, a distant landmark can only provide compass information. As the bee moves within the vicinity of the food source it will not detect any change in the size of the landmark's image on its retina. Indeed, in our model the presence of a distant landmark is potentially confusing: when areas of retinal image and snapshot are correctly paired, distant landmarks provide no useful information; whereas with mistakes in pairing, distant landmarks are likely to mislead the bee. The insect would benefit from having a procedure for filtering out the effects of distant landmarks from both its snapshot
and demands no global pattern recognition. Landmarks do not have to be identified or even separated from the background. All that is needed is a partition of the image into areas of different brightness. The next step is to use the discrepancies between the paired angles to control the direction of the bee's movements. This is done by making the result of every local comparison generate a vector which will tend to move the bee in a particular direction. Take, for example, the bee positioned to the left of Fig. 5. To reduce the clutter in the illustration just the three landmarks and one of the gaps in the snapshot have been paired with the appropriate angles on the retina. For each of these pairs a unit vector is shown by an arrow. This tends to move the bee towards the object or gap imaged on the retina if the retinal angle is smaller than its partner in the snapshot, and away if it is larger. This rule will normally drive the bee towards a landmark which is further away than it should be, and away from one which is too close. The bee's overall direction represented by the arrow in the centre of the circle is then obtained by summation of all the unit vectors. In this way the snapshot guides the bee along a path which leads to the food source. A model of this general type though slightly more complicated in detail successfully reproduces the results of all oar experimental manipulations on bees, and with some modification it works for desert ants as well. This hypothesis of the way insects use
image in order to reach home. To explore how insects might make use of snapshots to guide their way we turned to computer modellings. The task we set ourselves was to devise a procedure by which the discrepancy between the retinal image and snapshot could enable a model bee to reach the food source. The problem and model are illustrated in Fig. 5 in which the landmarks are indicated by the filled circles. For simplicity, snapshot and retina are restricted to a single horizontal plane. They are represented by two concentric circles on which the images of the dark landmarks are shown by thick lines and the spaces between them by thin lines, the whole panorama covering 360 ° . At the food source retinal image and snapshot are congruent, but away from the food source they are not. Rules must be found which will make the bee move to transform its current retinal image to its snapshot. Here we can give no more than the flavour of the model which is described more fully elsewhere 3. The first problem is to match the retinal image to the snapshot. To do this both retina and snapshot are divided into dark and light regions, corresponding in this case to the angles subtended by the images of the black landmarks and the white gaps between them. Each angle in the snapshot is then paired with the closest angle in the retina of the same sign (light with light and dark with dark) and a comparison made between the members of each linked pair of angles. This matching process is a local one ,
.
.
.
.
8
,
.
.
.
.
,
.
.
.
.
.
.
100"1 1.Ore, 108":
.
.
i
.
.
.
.
I
.
.
.
~1arck 19&~
.
1
.
.
.
C
.
.
•
.
.
.
i
.
.
.
.
;
.
.
.
.
i
.
.
.
.
mO'l 2Ore,
•
* t
:,: "'o--"''" •
•
e e
•
~
-..
• %
• I
..~
•
T
P
~
~
.
.
.
.
i
.
•
•
.
.
.
i
.
.
.
.
i
-'.
.
.
.
.
w
.
.
.
I
t
....
i
.t~.t.,¢...
.
•
•
•
i
.
.
.
.
""
'
o •
--"~-::'~. . . . .
"' ".
"
. I •
.
.
.
.
.
I
.
.
.
.
I
.
.
.
.
t
.
.
.
.
.
.
.
.
I
.
.
.
.
I
•
•
.
.
.
.
.
.
.
•
0 0
•. " .
...
O
.
o 0
.•.
.~,•.
~ . 4. The searchpattern of desert ants when looking for their nest site. (a) Cylindrical landmarks in the training arranfement. ~) Mrkers separated by tw/ce the ~ ~ e . (c) Markers twicetraining ~ and separated by twicem~in8 ~ . Points show pogations of ardspetio~atly during their search. Eight ants were each tested for 5 minutes. Ants search at the expected nest site (the centre of the rectangle) provided that the apparent size of the landmarks is what they were accustomed to see. PromRef. 15,
105
TINS-March 1983
at food
/
~g. 5. Simplified model to show how insects might use snapshots to relocate a nest or food source. Cross indicates spot to which insect wishes to return, where snapshot and retinal image are dongruent, Retina and snapshot are indicated by concentric circles with landmarks shown by thickenings o f the line. A way from the food source there are discrepancies between image and snapshot, Rules described in the text move the insect so as to reduce these, so bringing it towards the food source. The full model, which mimics the experimental data on bees, demands that the snapshot is defined with respect to earth-based co-ordinates.
and its retinal image. Information in a photograph is coded spatially, and models which work by local comparisons of retinal image and snapshot encourage the assumption that the insect's metaphorical snapshot is also stored in a spatially extended form, where it can easily interact with the retinal image. An obvious guess is that the bee keeps its snapshot somewhere in the optic lobe or mushroom bodies, both of which contain topographical maps of the retina. One complication with such a hypothesis, however, is that for our model to fit the bee's behaviour, the orientation of the snapshot must be kept constant with respect to external coordinates. In principle this could be achieved in several ways. The simplest is for the snapshot to be fixed to the retina and for the bee always to have its head pointing along the same compass bearing, whatever its direction of flight. This, however, does not happen - bees tend to face the way in which they are flying, and in our experiments they are able to reach their destination from any point in the room. A second possibility is for the bee to store several ver-
sions of its snapshot, each learnt at the food source with the bee facing in a different direction. On the return these would be called into service according to the direction in which the bee was pointing. Lastly, the snapshot might spin inside the bee's head, counter-rotating as the bee turns. One might suspect that a rotating snapshot is difficult to implement in an insect brain, indeed in any brain, but multiple snapshots seem easier and are in any case needed for a variety of other reasons. For example, Lindauer 6 has shown that bees will learn to exploit different sources of food at different times of day. These sources may have different scents or colours and be in different locations, in which case a separate snapshot will be required for each. In our experiments bees had to be taught the journey from the hive to the experimental room in several stages; first to find the window and then to find the food source inside. For this they must certainly have taken at least two snapshots, since their surroundings outside have nothing in common with those inside. One could perhaps view the insect's brain as containing a box of slides. How the bee
selects which ones to use is just one of many intriguing questions which remain unanswered. We can build up a three-dimensional map of our immediate surroundings, and we are able to do so from a single spot, just by visual exploration. We have no need to move our bodies while assembling it. We can then use this precise internal picture of the layout of nearby objects to plan routes between many different points in our surroundings. Moreover, as Thomson 9 has recently shown, our internal map is sufficiently accurate that we can execute a route in the absence of visual feedback without bumping into obstacles. The experiments described in this article imply that insects do not represent their environment in the same way at all. They rely instead upon twodimensional snapshots which are taken from a particular viewpoint. Since these snapshots do not contain any information about the distances between objects they serve a very limited function, that of guiding the insect's return to the spot from which the snapshot was taken. A new snapshot is needed should the insect wish to find its way to a different destination.
Reading list 1 Anderson, A. M. (1977)J. Comp. Physiol. 114, 335-355 2 Cartwright, B. A. and Collett, T. S. (1979) J. Exp. Biol. 82,367-372 3 Cartwright, B. A. and Collett, T. S. (1982) Nature (London) 295,503-564 4 CoUett, T. S. and Land, M. E. (1975)J. Comp. Physiol. 100, 59-84 5 Holldobler, B. (1980)Science 210, 86-88 6 Lindauer, M. (1961) Communication Among Social Bees, Harvard University Press 7 Mittelstaedt, H. Fortschr. Zool. (in press) 8 Mittelsteadt, H. (1980) Naturwissenschafien 67, 566 9 Thomson, J. A. (1980) Trends NeuroSci. 3, 247-250 10 Tinbergen, N (1974)Curious Naturalists, Penguin Education, Harmondsworth 11 Von Frisch. K. (1967 ) The Dance Language and Orientat~ o f Bees, Harvard University Press, Camb~dg~l[ 12 Wehner, R. (1972)J. Comp Physiol. 77, 256-277 13 Wehner, R. (1981) in Handbook o f Sensoo, Physiology (Autrum, H.-J., ed,), Vol. VII c, pp. 287--616, Springer, Berlin 14 Wehner, R. and Flatt, I. (1977) Z. Naturforsch. 32c, 469-471 15 Wehner, R. and Riiber, F. (1979) Experientia 35, 1569-1571
T. S. Collett and B. A. Cartwright are at the School o f Biological Sciences, The University o f Sussex, Falmer, Brighton, Sussex BN1 9QG, U.K.
ment in fly photoreceptors:propertiesand candidates. Biophys. Struct. Mech. Vol. 9 (in press) 7 Kirschfeld, K., Feiler, R. and Minke, B. (1978) Z. Naturforsch. 33c, 1009-1010 8 Kirschfeld, K., Franceschini, N. and Minke, B. (1977) Nature (London) 269, 386-390 9 Kirschfeld, K. and Wenk, P. (1976) Z. Naturforsch. 31c, 764-765 10 Mclntyre, P. D. and Snyder, A. W. (1973)J. Opt. Soc. Am. 63, 1518-1527 II Minke, B. and Kirschfeld, K. (1980)J. Gen.
101 PhysioL 75,381--402 12 Pak, W. L. and Lidington, K. J. (1974) J. Gen. PhysioL 63,740-756 13 Pick, B. (1977) B/ol. Cybern. 26, 215-224 14 Snyder, A. W. and Menzel, R. (1975) Photoreceptor Optics (Snyder, A. W. and Menzel, R., eds), Springer-Verlag,Berlin 15 Thurm, U. (1982) Biophysik (Hoppe, W., Lohmann, W., Markl, H. and Ziegler, H., eds), 2rid edn, p. 691, Springer-Verlag,Berlin 16 Wijngaard, W. (1974) J. Opt. Soc. Am. 64,
1136-1144
17 Wijnganrd,W. (1981) in Optical Sciences - Vertebrate Photoreceptor Optics (Enoch, J. M. and Tobey, F. L., eds), pp. 301-323, SpringerVerlag, Berlin 18 Wijngaard,W. and Stavenga, D. G. (1975) Biol. Cybern. 18, 61--67 K. Kirschfeld is at the Max-Planck-lnstitut f~r Biologische Kybernetik, Spemannstrasse 38, D- 7400 Tiibingen, F.R.G.
Eidetic images in insects: their role in navigation
ney by keeping a running tally of its direction and distance from its starting point. This information can then be used at any time to specify a direct path back. Many animals, both invertebrates 7 and vertebrates 8 use dead reckoning of this kind. Indeed, when worker bees inside a hive are recruited to a food source, the dance tells them no more than the distance of the source and its direction with respect to gravity 11. Outside the hive they refer the directional signal to the sun or pattern of polarized light in the sky and fly to the approximate position of the source. However, such direction and distance signals are unlikely to be sufficiently precise to allow a bee on a subsequent trip to fly back to a specific, small patch of flowers. For this it needs more detailed knowledge of the terrain. There is much evidence suggesting that
T. S. Collett and B. A. Cartwright In o r d e r to return to a particular place in their e n v i r o n m e n t s o m e insects store what is essentially a p a n o r a m i c snapshot o f their surroundings taken f r o m the spot they wish to retrieve. They can then guide their return with the s n a p s h o t by m o v i n g so that their current retinal i m a g e c o m e s to m a t c h the image they have stored.
A somewhat neglected phenomenon in the study of human memory is that of photographic memory - the ability to store and recall in detail images which have at one time fallen upon the retina. Some individuals evidently have much richer and more persistent visual imageD, than others, but it is unknown to what extent people in general are capable of retaining arbitrary details of an image. One reason why this topic seems to remain almost a curiosity is that little attempt has been made to identify everyday tasks for which such an ability is essential. In this article we argue that it is probably common for insects to memorize precise details of their retinal images, and that they do so in particular circumstances and for particular reasons. Many insects are able to return to a given spot. Wasps dig burrows in sand in which they lay eggs and which they later need to relocate in order to provide the growing larvae with food. It is remarkable with what ease they can find an inconspicuous hole sited in a uniform plain. Foraging bees are equally good at returning repeatedly to the same clump of flowers to exploit a fruitful source of nectar or pollen. Similarly, longlegged ants of the species Cataglyphis bicolor found in sandy areas of the Near and Middle East, forage individually far from their nest seeking dead insects and are able to return and unerringly locate their nest which is a barely noticeable hole in the sand. Behaviour of this kind is not limited to hymenoptera. Male hoverflies hang motionless in the air, occasionally darting out to chase what they hope may be a female and then returning to the same spot. Dragonflies
also have perches to which they return after brief flights, and heliconiid butterflies roost every night on the sanle part o f a particular tree or vine. The extensive literature on this intriguing topic is fully reviewed by Wehner TM. One way an animal can regain a location in space is for it to monitor its outward jourT
0.5 brr
0-4
wW
f~ o.3
~ 0-2
\
q~
0.1
0-0 I
1
I
I
0
2O
40
6O
I
I
1
I~
12O
L
1~
I
I
16O
I~
it)
°Aa
o •
Fig. 1. Frequencies of bees' choices between test and training patterns as the test pattern is rotated clock wise ((3) or anti-clockwise (0) from the training orientation. Patterns were presented on a vertical surface. Bees chose the training pattern more frequently as the difference between the orientations of the two patterns increased. From Ref. 12. ~
Elsevier B i o m e d i c a l Press 1983
0378
5912/83/0CO0 - 0000/$0100
102
! I.'Y'~
nearby landmarks are an important cue Ior guiding insects over the last stages of their return. One of the f'n'st and clearest demonstrations was that of Tinbergen ~° who marked the nest hole of a sphecid wasp by a ring of pine cones. When the pine cones were moved to a new position the returning wasp searched for its nest in the spot defined by the displaced ring. More recently we have begun to learn in more detail how the landmarks are used. Insects seem to store something similar to a photographic image of the immediate surroundings of the spot they wish to retrieve. Then, equipped with this picture or template they can fred the place again by moving until the image on their retina matches their template. The test piece of evidence that insects can remember relatively raw, unprocessed retinal images comes from visual discrimination experiments conducted by WehnePL He trained bees to associate food with a single two-dimensional pattern presented on a vertical surface and then tested them with a choice of the trained pattern and a second pattern which was altered to some degree from the first. If bees selected indiscriminately between the two patterns, then in a sense they saw patterns as the same, whereas the more the training pattern was preferred, the greater the perceived difference between the two. The conclusion from a number of experiments of the type illustrated in Fig. 1 is that discriminability is based on a very simple form of template matching. Imagine a fine grid superimposed over each pattern, dividing it into cells which are either black or white, then similarity is defined by how many cells in one grid match the brightness of their partners in the other. By painting over parts of the eye Wehner and Flatt ~4 found that particular regions of the pattern were learnt and inspected using particular regions of the eye - the stored image was fixed with respect to the retina.
(o)
,
This suggests that a bee looking at these patterns will always adopt the same posture and orientation in space, and films of bees performing this task confirmed that this was SO.
This work implies that patterns are learnt before they have been extensively analysed or categorized. For instance, in Fig. 1 the bee has been trained to a disk divided by a straight line through the middle into equal white and black areas. When the disk is rotated through 180° from the training orientation, the bee treats the pattern as quite different, even though the orientation of the dividing edge between the black and white sections is now the same as it was during training. The bee does not seem to have abstracted and learnt the orientation of the pattern, a feature which strikes us very vividly. For many purposes it seems far from ideal that information about the world should be categorized in such a rigid format. Individual objects often need to be recognized from a variety of points of view and different objects of the same type need to be seen as sharing certain properties, even though in terms of a fixed template they may have very little in common. However, a photographic image may be appropriate for storing the minutiae of a locale as seen from a particular vantage point. The position of a nest, for example, is defined by an arbitrary arrangement of sticks, bushes and stones. The identity of these objects is of no significance, the insect only has to learn their appearance and layout. Furthermore, the information is of no use in any other context; it can only specify a single place. An elegant example of an insect using stored images for navigation is provided by the African stink ant, P a l t o t h y r e u s tarsatusL This ant scavenges and hunts on the forest floor, so that it must view the sky through the forest canopy and rarely has an unobstructed view of the sky. There is
~:[ar~-h I ,)~'.:"
moreover often thick cloud cover so that neither the sun nor skylight cues are avail able as directional guides to help lead ants to their nest after foraging. HolldobleP has shown that instead they use the pattern of the foliage high above them seen silhouetted against the sky. To demonstrate this he took transparencies of the canopy from a camera on the ground and used them to make a patterned ceiling for ants kept in an arena in the laboratory. Worker ants accu~ tomed to the arena would lorage and then return directly to their nest. The direction of these homing runs depended on the orientation of the ceiling patterns, and when the pattern was turned through 180° the ants were thrown off course by the same amount. For the ant to obtain compass bearings from an arbitrary' canopy pattern, it must be able to remember specific details of the pattern. Explicit tests have now been made of the suggestion that a stored image or snapshot of nearby landmarks is also used to relocate a nest or foraging spot. The problem falls into two parts. First, what information do insects pick up and learn about the spatial layout of landmarks near a spot they wish to remember? Second, how do they use this information to guide their return there? To discover the way insects represent their immediate surroundings similar experiments have been done by Anderson t and ourselves, on honey bees ~.~, and by Wehner and R~ibeP 5, on desert ants. The strategy in all these studies was to persuade bees or ants to associate a food source or nest site with a given constellation of landmarks and then to investigate the insect's behaviour when the array was altered. The method we used with bees was as follows. Single marked bees from a nearby hive were trained to forage at a food source placed on the floor of a large, empty room. The location of the food source was defined by one or more matt-black cylinders positioned vertically on the floor. After
(b)
Fig. 2. (a) Distribution o f a single bee's flight time within the floor area surveyed by the camera when tested after training to a cylindrical landmark 50 cm from a food source. Base o f landmark is shown by fdled c~rcle. Po,fftion o f food source during t r a k ~ g marked by Fon x and y axes. Height shows relative times spent in different regions o f the test area. Lines on grid are 8. 7 cm apart. Co)Distribution o f bee's flight time during training after it had learnt to associate the position o f the food source with an arrangement o f three landmarks. From Ref. 3.
TINS - March 1983
103 a
0
b jo
•
I
~
o
'1
100 cm Fig. 3. Bee's search area during tests and aJ~er training to the arrangement o f three cylindrical landmarks shown in plan view in (a). The hatching represents the search area defined as those squares within the test area (as in Fig. 2) which are at least 80 % as high as the peak value. (a) Search area with training configuration. (b) Distance between landmarks double that o f the training distance. (c) Distance halved. (d) Landmarks arranged so bee could search either where the landmarks had the same bearings on the retina as the bee had experienced at the food source during training, or where the distances o f the landmarks were the same as those seen from the food source. In all cases the bee chooses to search where the bearings o f the landmarks on its retina were what it had been accustomed to seeing. From Ref. 3.
each of the bee's visits to the food source, landmarks and food source were moved together as a group to a new position. These shifts were purely translational, and the orientation of the landmark array was kept constant with respect to external coordinates. Once the bee was trained to the landmarks, the food source could be removed and the insect's search path was then recorded through a video camera on the ceiling. The way the bee distributes its time within the test area (Fig. 2) implies that it does indeed use the landmark array to decide where to search. The interesting part of the experiment is then to change the landmark array, either by altering the size or shape of individual landmarks or by changing their number and spatial arrangement, and to plot where the insect searches when the food source is absent. The insect's expectation of where food is to be found when the array is distorted can tell us what it has learnt about the landmarks. We began with the position of the food source defined by a single landmark. During a'aining the food source was kept at a constant orientation and distance from the landmark. When the food source was removed the bee searched in approximately the right direction. This indicates that the bees have directional information from a source external to the landmark (possibly from the diffused light which filtered through the curtained windows which covered most of one wall). If the landmark
had been the only available cue, bees would have searched in an annulus centered on it. More significantly, the search area is also situated in approximately the right distance from the landmark. To ensure that bees were not just flying at a preferred distance from the landmark which happened to coincide with that to which they had been trained, different bees were trained to food at different distances from the landmark and the search area was always located at the appropriate spot. This result does not mean that bees know how far they are from the landmark in terms of the distance they would have to fly in order to reach it, but merely that they search where the landmark subtends the same angle on their retina that it did when the bees were at the food source. When bees were tested with a landmark larger than the training size, they searched further away. Conversely, when the test landmark was smaller, they searched closer. Such data suggest that the bee learns the apparent size of the landmark as seen from the food source and moves so that its current retinal image matches its memory. Experiments using three landmarks gave results in accordance with this general hypothesis. Bees were trained to the configuration shown in plan view in Fig. 3a. When the landmark array during the test differed from the training situation, bees consistently searched where the compass bearings of the landmarks on their retina
matched those they had experienced at the food source during training. This can be seen in Fig. 3 which shows the search area when the distances between landmarks were either halved or doubled, or when the shape oftbe array was changed. The deduction from such results is that bees do not learn a plan view or map of the array, but only the bearings and sizes of the landmarks as seen from the food source. With these distorted arrays there cannot be a perfect match between the bee's snapshot and its retinal image. When the bee searches where landmarks are in the correct location on the retina to match the snapshot, then their size must be wrong. Their choice of search area does, however, seem to be the best compromise. Suppose for example the bee were to position itself so that the retinal image of one landmark was congruent with its partner in the snapshot, then the images of the other landmarks would differ from their partners in both size and position. Desert ants employ landmarks in a similar way, but with some intriguing variations. In Wehner and R~iber's experiments ~5, a nest opening of a colony of ants was marked by two small black cylinders placed either side of the nest (Fig. 4). After a period to allow the ants to become accustomed to the presence of the landmarks, both landmarks and ants were displaced to a new area and the ant's search path was recorded. When the constellation of landmarks was the same as that used for training, ants searched midway between the two (Fig. 4a), i.e. where the nest ought to have been. They did likewise when distance and size were both altered together so that the view of the landmarks from halfway between them remained what the ants had been accustomed to see from the nest site (Fig. 4c). However, if either the distance between the landmarks or their size was increased or decreased without any alteration in the other parameter, the ants then searched very close to one or other landmark (Fig. 4b). There is now no place where the retinal image can match the usual picture the ant has from its nest site, and it is as though the ants then give up and adopt a new strategy. It is difficult to compare the results of experiments performed in the open desert with those undertaken in a restricted laboratory environment, and it is unwise at this stage of our knowledge to conclude that there are genuine differences in the landmark guidance systems of the two animals. It is safer to stress the overall similarities. The best explanation of these results is the same as that proposed for bees and earlier for hoverflies when they return to preferred hovering locations 4. The ant takes a snapshot of its surroundings from the nest site, which it can later compare with its current
t I\,~
104
.
.
,
.
.
.
.
,
.
.
.
.
,
.
.
.
.
•
.
.
.
.
landmarks almost inevitably leads to que~ tions with a photographic slant, and it will be interesting to see how hard the analogy can be pressed. One of the first things t~ appreciate when holding a camera is that the pictures will be blurred if the camera is unsteady. However, insects are not at all still when learning landmark arrays. They perform characteristic orientation flights in which their departure from the location they wish to learn is in a series of zigzags during which they continually face the hive or food source to which they hope to return. Why' should they do so? Are they taking a succession of snapshots from different points of view? Or are they in some way filtering out of their shapshot parts of the image which move slowly across their retina, or which shrink to an insignificant size once the insect has moved away ? The snapshot may be an edited one with very distant and with very small and close objects ignored. For instance, a distant landmark can only provide compass information. As the bee moves within the vicinity of the food source it will not detect any change in the size of the landmark's image on its retina. Indeed, in our model the presence of a distant landmark is potentially confusing: when areas of retinal image and snapshot are correctly paired, distant landmarks provide no useful information; whereas with mistakes in pairing, distant landmarks are likely to mislead the bee. The insect would benefit from having a procedure for filtering out the effects of distant landmarks from both its snapshot
and demands no global pattern recognition. Landmarks do not have to be identified or even separated from the background. All that is needed is a partition of the image into areas of different brightness. The next step is to use the discrepancies between the paired angles to control the direction of the bee's movements. This is done by making the result of every local comparison generate a vector which will tend to move the bee in a particular direction. Take, for example, the bee positioned to the left of Fig. 5. To reduce the clutter in the illustration just the three landmarks and one of the gaps in the snapshot have been paired with the appropriate angles on the retina. For each of these pairs a unit vector is shown by an arrow. This tends to move the bee towards the object or gap imaged on the retina if the retinal angle is smaller than its partner in the snapshot, and away if it is larger. This rule will normally drive the bee towards a landmark which is further away than it should be, and away from one which is too close. The bee's overall direction represented by the arrow in the centre of the circle is then obtained by summation of all the unit vectors. In this way the snapshot guides the bee along a path which leads to the food source. A model of this general type though slightly more complicated in detail successfully reproduces the results of all oar experimental manipulations on bees, and with some modification it works for desert ants as well. This hypothesis of the way insects use
image in order to reach home. To explore how insects might make use of snapshots to guide their way we turned to computer modellings. The task we set ourselves was to devise a procedure by which the discrepancy between the retinal image and snapshot could enable a model bee to reach the food source. The problem and model are illustrated in Fig. 5 in which the landmarks are indicated by the filled circles. For simplicity, snapshot and retina are restricted to a single horizontal plane. They are represented by two concentric circles on which the images of the dark landmarks are shown by thick lines and the spaces between them by thin lines, the whole panorama covering 360 ° . At the food source retinal image and snapshot are congruent, but away from the food source they are not. Rules must be found which will make the bee move to transform its current retinal image to its snapshot. Here we can give no more than the flavour of the model which is described more fully elsewhere 3. The first problem is to match the retinal image to the snapshot. To do this both retina and snapshot are divided into dark and light regions, corresponding in this case to the angles subtended by the images of the black landmarks and the white gaps between them. Each angle in the snapshot is then paired with the closest angle in the retina of the same sign (light with light and dark with dark) and a comparison made between the members of each linked pair of angles. This matching process is a local one ,
.
.
.
.
8
,
.
.
.
.
,
.
.
.
.
.
.
100"1 1.Ore, 108":
.
.
i
.
.
.
.
I
.
.
.
~1arck 19&~
.
1
.
.
.
C
.
.
•
.
.
.
i
.
.
.
.
;
.
.
.
.
i
.
.
.
.
mO'l 2Ore,
•
* t
:,: "'o--"''" •
•
e e
•
~
-..
• %
• I
..~
•
T
P
~
~
.
.
.
.
i
.
•
•
.
.
.
i
.
.
.
.
i
-'.
.
.
.
.
w
.
.
.
I
t
....
i
.t~.t.,¢...
.
•
•
•
i
.
.
.
.
""
'
o •
--"~-::'~. . . . .
"' ".
"
. I •
.
.
.
.
.
I
.
.
.
.
I
.
.
.
.
t
.
.
.
.
.
.
.
.
I
.
.
.
.
I
•
•
.
.
.
.
.
.
.
•
0 0
•. " .
...
O
.
o 0
.•.
.~,•.
~ . 4. The searchpattern of desert ants when looking for their nest site. (a) Cylindrical landmarks in the training arranfement. ~) Mrkers separated by tw/ce the ~ ~ e . (c) Markers twicetraining ~ and separated by twicem~in8 ~ . Points show pogations of ardspetio~atly during their search. Eight ants were each tested for 5 minutes. Ants search at the expected nest site (the centre of the rectangle) provided that the apparent size of the landmarks is what they were accustomed to see. PromRef. 15,
105
TINS-March 1983
at food
/
~g. 5. Simplified model to show how insects might use snapshots to relocate a nest or food source. Cross indicates spot to which insect wishes to return, where snapshot and retinal image are dongruent, Retina and snapshot are indicated by concentric circles with landmarks shown by thickenings o f the line. A way from the food source there are discrepancies between image and snapshot, Rules described in the text move the insect so as to reduce these, so bringing it towards the food source. The full model, which mimics the experimental data on bees, demands that the snapshot is defined with respect to earth-based co-ordinates.
and its retinal image. Information in a photograph is coded spatially, and models which work by local comparisons of retinal image and snapshot encourage the assumption that the insect's metaphorical snapshot is also stored in a spatially extended form, where it can easily interact with the retinal image. An obvious guess is that the bee keeps its snapshot somewhere in the optic lobe or mushroom bodies, both of which contain topographical maps of the retina. One complication with such a hypothesis, however, is that for our model to fit the bee's behaviour, the orientation of the snapshot must be kept constant with respect to external coordinates. In principle this could be achieved in several ways. The simplest is for the snapshot to be fixed to the retina and for the bee always to have its head pointing along the same compass bearing, whatever its direction of flight. This, however, does not happen - bees tend to face the way in which they are flying, and in our experiments they are able to reach their destination from any point in the room. A second possibility is for the bee to store several ver-
sions of its snapshot, each learnt at the food source with the bee facing in a different direction. On the return these would be called into service according to the direction in which the bee was pointing. Lastly, the snapshot might spin inside the bee's head, counter-rotating as the bee turns. One might suspect that a rotating snapshot is difficult to implement in an insect brain, indeed in any brain, but multiple snapshots seem easier and are in any case needed for a variety of other reasons. For example, Lindauer 6 has shown that bees will learn to exploit different sources of food at different times of day. These sources may have different scents or colours and be in different locations, in which case a separate snapshot will be required for each. In our experiments bees had to be taught the journey from the hive to the experimental room in several stages; first to find the window and then to find the food source inside. For this they must certainly have taken at least two snapshots, since their surroundings outside have nothing in common with those inside. One could perhaps view the insect's brain as containing a box of slides. How the bee
selects which ones to use is just one of many intriguing questions which remain unanswered. We can build up a three-dimensional map of our immediate surroundings, and we are able to do so from a single spot, just by visual exploration. We have no need to move our bodies while assembling it. We can then use this precise internal picture of the layout of nearby objects to plan routes between many different points in our surroundings. Moreover, as Thomson 9 has recently shown, our internal map is sufficiently accurate that we can execute a route in the absence of visual feedback without bumping into obstacles. The experiments described in this article imply that insects do not represent their environment in the same way at all. They rely instead upon twodimensional snapshots which are taken from a particular viewpoint. Since these snapshots do not contain any information about the distances between objects they serve a very limited function, that of guiding the insect's return to the spot from which the snapshot was taken. A new snapshot is needed should the insect wish to find its way to a different destination.
Reading list 1 Anderson, A. M. (1977)J. Comp. Physiol. 114, 335-355 2 Cartwright, B. A. and Collett, T. S. (1979) J. Exp. Biol. 82,367-372 3 Cartwright, B. A. and Collett, T. S. (1982) Nature (London) 295,503-564 4 CoUett, T. S. and Land, M. E. (1975)J. Comp. Physiol. 100, 59-84 5 Holldobler, B. (1980)Science 210, 86-88 6 Lindauer, M. (1961) Communication Among Social Bees, Harvard University Press 7 Mittelstaedt, H. Fortschr. Zool. (in press) 8 Mittelsteadt, H. (1980) Naturwissenschafien 67, 566 9 Thomson, J. A. (1980) Trends NeuroSci. 3, 247-250 10 Tinbergen, N (1974)Curious Naturalists, Penguin Education, Harmondsworth 11 Von Frisch. K. (1967 ) The Dance Language and Orientat~ o f Bees, Harvard University Press, Camb~dg~l[ 12 Wehner, R. (1972)J. Comp Physiol. 77, 256-277 13 Wehner, R. (1981) in Handbook o f Sensoo, Physiology (Autrum, H.-J., ed,), Vol. VII c, pp. 287--616, Springer, Berlin 14 Wehner, R. and Flatt, I. (1977) Z. Naturforsch. 32c, 469-471 15 Wehner, R. and Riiber, F. (1979) Experientia 35, 1569-1571
T. S. Collett and B. A. Cartwright are at the School o f Biological Sciences, The University o f Sussex, Falmer, Brighton, Sussex BN1 9QG, U.K.