- Email: [email protected]
Visuomotor sensitivity for shape and orientation in a patient with visual form agnosia
~
Pergamon
0028-3932(95)00169
7
Neuropsvcholoyia, Vol. 34, No. 5, pp. 329 337, 1996 Copyright ~) 1996 Elsevier Science Ltd. All rights reserved Printed in Great Britain 0028 3932/96 $15.00+0.00
Visuomotor sensitivity for shape and orientation in a patient with visual form agnosia D . P. C A R E Y , * t
M. HARVEY~
and A. D. MILNERt
tSchool of Psychology, University of St. Andrews, St. Andrews, Fife, KY16 9JU, U.K. (Received 31 July 1995; accepted 3 November 1995)
Abstract--We have previously demonstrated that a patient with visual form agnosia (DF), who is unable to report the orientation
or size of visual targets, can nevertheless use these same visual attributes to control motor acts. In the first of three new experiments, we found that DF is able to grasp everyday tools and utensils proficiently (i.e. with a well-formed hand posture) but has difficulty in visually selecting the correct part of the object to grasp (e.g. the handle) for subsequent use of that object. A second experiment revealed that DF's visuomotor system is able to adjust concurrently to variations in both the size and orientation of target objects; when these visual attributes were both varied, she adjusted both her grip aperture and the orientation of her hand well in advance of target contact. These spared visuomotor abilities do not seem to extend to shape processing per se, however. In the final experiment we found that DF was insensitive to changes in the orientation of a cross-shaped object, where no single principal axis could be extracted to control orientation of the grasp. These observations extend our knowledge of DF's residual visuomotor abilities, and suggest limitations on the visual processing capacities of the human dorsal stream. Key Words: visuomotor sensitivity; visuomotor abilities; visual form agnosia.
stream) is crucially implicated in the perceptual analysis of shape and other features of objects [2, 25]. We have suggested that DF's profound shape recognition difficulty can possibly be attributed to a disabling lesion within this ventral system. This would be consistent with the lesion evidence available on other patients with visual object agnosia [for review, see Ref. 9] and with the magnetic resonance imaging (MRI) evidence on DF's lesion [17]. We have accordingly argued [8, 18, 19] that DF's intact visuomotor skills may have as their neural substrate the dorsal stream, which projects from striate to posterior parietal cortex (with associated projections into premotor regions of the frontal lobe). This proposal of a visuomotor function for the dorsal stream echoes similar suggestions by others [4, 15]. The argument is bolstered by physiological recording studies that have revealed a range of neurones in the primate posterior parietal cortex which are associated with visually guided acts such as reaching and grasping, and saccadic and pursuit eye movements [for reviews see Refs. 11, 20, 22] as well as studies of the effects of parietal lesions in humans and monkeys [14, 16]. Furthermore, recent research shows that some of these cells are sensitive to object parameters such as size and orientation, and yet may be insensitive to target location [22, 24]. Neurones of this kind within the human homo-
Introduction
In a series of recent papers, we have described the residual visuomotor abilities of a patient with visual form agnosia (DF) [5, 6, 17]. These investigations have shown that despite her severe perceptual difficulties, which prevent her from recognizing or discriminating between even simple geometrical shapes, D F has a remarkably intact ability to act upon visually presented objects. This ability includes the anticipatory guidance of the hand in visuomotor acts such as inserting a card into an oriented slot [5,17] and grasping blocks of different widths [5]. The latter skill has been found to maintain its accuracy even when the blocks were placed at variable distances [Carey and Milner, unpublished]. We have argued that DF's residual visual abilities are explicable in terms of the current knowledge of visual processing subsystems in the primate brain. It is now accepted that one of the two major streams of cortical visual processing (the occipitotemporal or 'ventral'
* Address for correspondence: Department of Psychology, The University of Aberdeen, King's College, Aberdeen AB9 2UB, U.K. $ Now at: Department of Psychology, University of Bristol, Bristol BS8 1TN, U.K. 329
330
D.P. Carey et al./Visuomotor sensitivity for shape and orientation
logue of the monkey's dorsal stream could underlie DF's successful grasping of visually presented target stimuli varying in either size or orientation. Despite these developments, however, there has been no convincing demonstration thus far of true f o r m processing in posterior parietal neurones, i.e. over and above size and orientation processing. We would, however, argue that an unequivocal demonstration of form-dependent visuomotor skills in a severely form-agnosic patient like D F would provide good presumptive evidence on which to base predictions as to the existence of such shape analysis in the dorsal stream. It might also allow predictions to be made as to the physiological characteristics of that analysis. In other words, if a visuomotor interpretation of dorsal stream function is correct, then an analysis of the visual requirements of the range of sensorimotor behaviours available after destruction or disconnection of the ventral stream should help to delineate the limits of visual processing in the primate dorsal stream. At present, our knowledge of DF's ability to act upon complex visual configurations--in which more than one relevant feature is present--is rather limited. In one experiment, it was found that she made many errors when attempting to 'post' a T-shaped object into a T-shaped aperture [6]. In that task, most of DF's errors consisted of her attempting to insert the object at 90 ° to the correct orientation, suggesting that she was only able to take account of one of the two oriented segments at once. These authors suggested that wrist orientation, having only one degree of freedom, would be sensitive only to a single orientation present in a target; whereas object grasping, involving coordinated movements of the digits, might reveal a true sensitivity to object shape. Indeed, in a recent paper, Goodale et al. [7] reported that D F was adept at picking up a range of irregular smooth shapes, pairs of which she was unable to tell apart. In fact, D F was indistinguishable from controls in the finger-thumb 'grasp points' at which she contacted the shapes. The authors argued that DF's ability in grasping these shapes demonstrated a capacity to process form, over and above her capacity to process simple orientation and/or size. They proposed that DF's intact visuomotor processing allows for the computation of optimal grasp points, in accordance with recent theoretical models [1, 12]. However, the majority of the shapes used in the Goodale et al. [7] study were elongated to some degree, generally having one clear major axis. It is therefore arguable that orientation and width cues alone could provide enough information to guide many of DF's successful grasps, which in general were made either along these axes or (more frequently) at a position midway along and rotated 90" with respect to them [see Fig. 5 in Ref. 7]. If so, it is possible that shape processing in the true sense is not needed to explain the observed pattern of results in DF's grasping performance. In the present experiments, we revisited the question of DF's shape processing skills in three ways. First, we used a procedure similar to one recently described by
Sirigu et al. [23], in which we presented a series of everyday objects to D F and asked her to pick them up and then mime their use. Although in many ways D F seems to be able to interact with objects in a normal fashion, we wanted to examine the extent to which this ability was dependent upon compensatory strategies like the use of subtle tactile exploration. The study was therefore intended to explore the extent of DF's ability, using visual information only, to grasp objects appropriately, as well as her ability to grasp them skilfully and to demonstrate their use. In the second experiment, we examined whether D F retained the ability to grasp different rectangular blocks adeptly even when they were presented at a range of different orientations: that is, could she simultaneously combine size and orientation in her prehensive behaviour? And in the third and final experiment, we presented D F with a doubly symmetrical X-shape mounted above a table and varying in orientation from trial to trial. We asked whether in the absence of a single major axis, she would nevertheless orient her hand to match the orientation of the cross when asked to reach out to grasp and rotate it. The intention in using each of these different approaches was to help define more precisely the visual information that D F is able to use (by virtue, ex hypothesi, of her dorsal stream of cortical processing) in controlling her actions. Exploring these limits should contribute to a fuller specification of our model of primate cortical visual processing [8, 18, 19].
Subjects DF, who suffered from carbon monoxide poisoning in 1988, was tested in the present experiments at the age of 40 (August-September, 1994). Her visual impairments have been described in detail elsewhere, along with relevant clinical details [17]. The inability to discriminate perceptually between equal-area rectangles has become a defining characteristic of visual form agnosia [3], and this was retested just prior to the present experiments. The shapes used were: (a) 50 × 50 mm; (b) 45 × 56 ram; (c) 40 × 62.5 mm; (d) 35 x 71.5 mm; (e) 30 x 83 mm; and (f) 20 × 125 mm. As shown in Fig. 1, a severe disorder is still present in DF. Although her performance is slightly better overall than we found in earlier tests [17], she is still unable to judge at above chance levels whether rectangle D, for example, with a side ratio of 0.49, is the same as or different from a square (rectangle A). Furthermore, we have found chance performance on other perceptual tests recently carried out, such as the recognition of famous faces and making same/different judgements of unfamiliar faces (29/48; 54%, chance = 50%). Yet in spite of all these indications of a lack of significant recovery, D F has adapted to her visual disorder remarkably well, managing, for example, to carry out basic culinary tasks and other complex visually guided activities around the house. A recent eye test using
D. P. Carey et al./Visuomotor sensitivity for shape and orientation 100
90
O o
8O
~
7o
n
60
50 1.0
i
,
,
,
0.8
0.6
0.4
0.2
0.0
Side Ratio Fig. 1. The 'same~lifferent' matching performance of DF on the Efron shapes test. She was given 20 trials at each of the five points plotted; these consisted of 10 'different' presentations (a square with an oblong), and 10 'same' trials (two squares or two oblongs).
a u t o m a t e d refractometry indicated insufficient departure from normal refraction to justify correction. A total o f six healthy female subjects approximately matched with D F for age and education, were used as controls in experiment 1. In each o f experiments 2 and 3, two (similarly matched) controls were used.
Experiment 1 Methods Apparatus and tasks. Subjects were seated at a table and asked to pick up complex three-dimensional objects, with one hand, and to demonstrate their use. A total of 35 everyday objects were presented (see Appendix), all of a grey or metallic colour in order to minimize the use of colour cues for object recognition (which have been demonstrated to aid DF's performance in object recognition tasks; [10]). Objects were presented one at a time and all other stimuli were hidden from view. The subjects were asked to close their eyes during each placement of an individual object. In two separate sessions, the objects were presented either in a standard view (i.e. in the normal orientation for use, typically with the handle pointing towards the subject) or in a view rotated by 180° with respect to this standard view. All subjects were tested on the standard view first. Object manipulation was recorded on super-VHS videotape, using two rotary shutter cameras (JVC BY 10E) which provided clear images at 25 frames/sec. One camera was attached to the ceiling with the lens at a distance of 110 cm above the table surface, providing a top view of the objects and the subject's hand and arm. The second was placed on a tripod at a distance of 150 cm just above the level of the table and oriented perpendicularly to the subject's mid-sagittal axis, thus providing a side view. Both views were recorded together on videotape in a split screen format using a video mixer. Scorin 9 procedure. DF's performance and that of the six control subjects was rated by three judges with respect to how the objects were grasped and manipulated. The following aspects of behaviour were measured: (a) the appropriateness of the point at which the object was grasped; (b) the skill with which it was picked up (i.e. smoothly versus clumsily); (c) the extent of tactile exploration; and (d) the appropriateness of the
331
mime by which the use of the object was demonstrated. Each of these behaviours was ranked on a 7-point scale from 1 (highly inappropriate) to 7 (highly appropriate). Two of the three raters had no knowledge as to the identity of the subjects. Analyses. Spearman rank correlations were calculated pairwise between the raters, separately for each subject and for the four scales. The inter-rater reliability proved very high, with all correlations exceeding +0.97, and most of them exceeding +0.99. No difference was found between the ratings of the informed judge and those of the na~'vejudges, nor did the ratings differ in their reliability among the different subjects tested. In order to compare the performance of DF with that of the six controls, the ratings of the three judges were therefore averaged (for each of the 35 objects, and separately for each of the four scales, the two views and the seven subjects). These scores were then averaged across the 35 objects, and DF's data compared with those of the six controls on each of the four scales. This was done separately for each of the two object views.
Results D F ' s performance did not differ from that o f the control subjects in regard to the skill o f reaching and grasping, or in the mimed demonstration of each object's use. Thus her grip formation was judged to be of comparable skill to that shown by the controls, particularly in the standard view condition; and although her score in the rotated view condition fell slightly below the mean o f the six c o n t r o l s - - 6 . 9 versus 6.6--it was still within 3 S.D. o f it. Similarly, D F was able to demonstrate the use o f the objects as proficiently as the controls, scoring less than 2 S.D. away from the control mean in both view conditions. There were, however, large differences between the subjects with regard to the points at which the objects were grasped, and in the a m o u n t o f tactile exploration. D F generally grasped the objects at the appropriate point (typically the handle) when they were presented in the standard view, scoring within 3 S.D. o f the mean o f the six controls; but when the objects were rotated by 180 r', her performance deteriorated markedly, falling more than 23 S.D. below the average o f the controls (5.0 versus 6.9). This result is evident from Fig. 2, in which the frequency distributions are plotted: whereas all control subjects grasped the objects at highly appropriate points, with 99% of objects being judged between 6-7, D F ' s distribution covered the whole range from highly inappropriate to appropriate. The observation that, despite her inappropriate grasp points, D F was nevertheless able to mime the use o f objects successfully, can be explained by the fact that she was found to engage in m u c h more tactile exploration than any o f the controls, in both view conditions. Whereas controls showed even less tactile exploration in the second session than in the first, D F showed a much greater a m o u n t in the rotated view condition (differing from the controls by 24 S . D . - - 5 . 4 versus 6.9) than in the standard condition (differing by only four S.D.--6.2 versus 6.7). In summary, D F pre-formed the grasp o f her hand to
D.P. Carey et al./Visuomotor sensitivity for shape and orientation
332
250
35
Controls: Normal Orientation
DF: Normal Orientation 30 200
25
I
e.~' 150
20 -I
cr •~ 100 LL
g is 10
50
%.
"5
t~
to
ca
"X
4" ~5" "55" ~>" ,~5" ~5" 250
35
DF: 180 o Rotation
Controls: 1800 Rotation
30
200
25 c
~' 150 E
20
g is 10
,
i_.li ,
,
,
o,m 1oo LL
50
,
,
,
o
4" ~k" "55" ~5" ~5" ~>"
,
,
,
4" ~,5" "5.~" ~5" ,,5" ,o.~"
Fig. 2. Experiment 1: The appropriateness with which household objects were grasped by DF. The judged appropriateness (on a scale of 1-7) is plotted separately for objects presented in standard view or non-standard view, for D F and for six control subjects.
various objects as proficiently as did c o n t r o l subjects. But when objects were presented in an unusual orientation, she often g r a s p e d t h e m at i n a p p r o p r i a t e points a n d palp a t e d them in e x p l o r a t o r y fashion before d e m o n s t r a t i n g their use.
Experiment 2 Methods Apparatus and tasks. Subjects were seated at a table and asked to pick up rectangular wooden blocks (Efron shapes) with one hand. The shapes consisted of six different stimuli varying in length and width, but all were of 5 mm thickness and had a surface area of 25 sq cm. The shapes corresponded precisely to the 2-D patterns used in testing DF's shape matching (see above), varying from a square (shape A) to an oblong with an aspect ratio of 0.16 (shape F). They were presented one at a time, 40 cm in front of the subject's hand, with all other stimuli hidden from view. The subjects were also asked to close their eyes during each placement of an individual shape. The shapes were presented in a pseudorandom sequence at six different orientations (0 °, 30', 6ff~, 90 °, 120 °, 150 °) relative to the sagittal midline. Each shape was presented four times each at 0 ° and 90 '~ and twice at each of the remaining orientations, giving a total of 96 trials. Subjects were asked on each trial to open their eyes, and reach out and grasp the object, using only their index finger and thumb, and pick up the object and raise it above the table surface. Reaching and grasping actions were recorded on superVHS videotape as in experiment 1, with one camera giving a side view (i.e. the parasagittal plane) and the second camera giving an overhead view of the grasping movements. The two camera views were recorded together on tape in split-screen format.
Scoring procedure. Two measurements were derived from the videotapes. The first was the orientation of a vector which joined the tips of the thumb and index finger, relative to the orientation of the Efron shape, at a point four frames before contact with the object. This 'grip orientation' was measured relative to the narrower dimension of each shape. The second measure was an estimate of grip size, i.e. the distance between the thumb and index finger, again four frames before contact with the object. (Note that this measure is not an estimate of maximum grip aperture, which tends to occur when approximately 65% of a movement has been completed [13].) Because deviations from the mid-sagittal axis (usually of the index finger) produced foreshortening in the parasagittal camera view, grip apertures were estimated from the overhead view, with the side view facilitating estimation of the finger-thumb distance whenever necessary. Grip apertures estimated for objects with long axes placed perpendicular to the subject's mid-sagittal plane were very similar to those estimated for other reaches to the same object, where some foreshortening in one camera view was present.
Results Orientation. A n a l y s i s o f the video records o f each g r a s p i n g m o v e m e n t showed t h a t D F ' s p e r f o r m a n c e was indistinguishable f r o m that o f the two controls. Like the controls, she oriented her h a n d correctly with respect to the shape on all trials. As illustrated in Fig. 3 (top), when r o t a t e d by 90 °, some shapes were p i c k e d u p along the thicker r a t h e r than the t h i n n e r extent, b u t this b e h a v i o u r was sometimes shown by the c o n t r o l s as well as b y D F . F o r all three subjects this type o f b e h a v i o u r was m o s t frequent with the m o r e ' c l o s e d ' shapes, b e c o m i n g p r o gressively less c o m m o n with the m o r e e l o n g a t e d ones.
D. P. Carey et al./Visuomotor sensitivity for shape and orientation
333
DF
k,t b
a
lil}7 ~Ii1't
lq/ /
d ML
C
I~li,~ ILl e
\d
i~/lill /~ / / '11lit f
/j
a
-'1
//I\l l \
d VM
f
e
~3 I Vllk
a
11'\~
b
I
C
I
d
e
f
i~\ \ i
J
Fig. 3. Experiment 2: Orientation of finger-thumb grip at a point four frames before contact with a rectangular Efron block placed at different orientations from trial to trial. The lines are superimposed on a normalized representation of each block. The data are from patient DF and two healthy controls (ML and VM).
D.P. Carey et al./Visuomotor sensitivity for shape and orientation
334
D F was more variable in the exact position of her hand four frames before contact (i.e. in Fig. 3 the line segments intersect the objects at a slightly wider range o f positions). However, the orientation of each segment is clearly related to the principal axis o f the target shape, as it is in the control subjects. Like the controls, D F is sensitive to object width, in that the more elongated objects result in grips which consistently target the narrower extent of the shape. Grip size. Analysis of the video records showed that the D F ' s performance was again indistinguishable from that o f the two controls (see Fig. 4). D F ' s grip size varied linearly with the width of the object, becoming progressively smaller from shape A (the square) to F (the narrowest rectangle), closely paralleling the performance of the controls. Because on some trials the shapes were grasped along the long axis (by all three subjects), data points were available on more than just the six nominal widths presented. These, however, were based on small numbers o f observations, and therefore are not plotted in Fig. 4.
Experiment 3
the visibility of the target. The cross was presented randomly at three different orientations (0 °, 22.5 ° and 45 ", anti-clockwise from the subject's point of view relative to the vertical). There were 14 trials at each of the three orientations, giving a total of 42 trials. The subjects were asked to close their eyes between trials. Three practice trials were given before testing began. All of the experimental trials were recorded on super VHS videotape using the two rotary shutter cameras. The first camera was pointed perpendicularly to the subject's mid-sagittal axis, as in experiments 1 and 2. The second camera was placed along the mid-sagittal axis, behind the object and facing the subject. The crosses were mounted in a plexiglas screen fixed vertically, so that the second camera provided an excellent view of the hand as it closed in on the object. Subjects were instructed to open their eyes, and reach out and grasp the cross and then give it a 45 ':' clockwise turn. Scoring procedure. The position of the digits of the hand was examined at a point four frames before contact with the cross. Three separate orientation measures were taken from the videotape, corresponding to lines joining the thumb with the index finger, the thumb with the middle finger, and the thumb with the ring finger, respectively (see Fig. 5). Hand orientation was scored in all cases relative to the gravitational vertical. Consequently if the subject adjusted her hand to the different orientations of the cross, the measured angles would each be expected to increase linearly with increasing stimulus orientation. Analyses. One-way analyses of variance (ANOVAs) (with the three stimulus orientations as the 'between' factor) were performed separately for each subject and each finger combination (thumb-index, thumb-middle, thumb-ring finger).
Methods Apparatus and tasks. Subjects were seated at a table and asked to reach out, grasp, and rotate a cross placed centrally in front of them in the frontoparallel plane, at a height of 50 cm from the table surface (approximately at chin height) and a distance of 40 cm. The two bars constituting the cross were 7.5 cm in length and 1.6 in width, and were 1.1 cm thick. The cross was made from an opaque white plastic. A black background cloth was placed on the wall behind the cross in order to increase
,,--. E
6
~
5
--A---c}-
DF ML - Control VM - Control
< Q. ,r-
~
4
E
a
. m
W
2
i
20
2.2
i
i
i
i
30
3.2
4.0
4.5
Results The main finding was that the two control subjects showed highly significant main effects of orientation over all three finger combinations [MH: F ( 2 , 3 9 ) = 72.30, 58.69, 54.91, respectively, all P < 0.001; BP: F(2,39) = 8.43, 16.29, 8.11; all P < 0.002], whereas D F showed no effect of orientation on any o f the measures [F(2,39) = 0.55, 0.74, 1.23; all n.s.]. As shown in Fig. 5, D F held her hand at the same orientation when approaching the cross, irrespective o f the orientation o f the stimulus, whereas the control subjects rotated their hands in linear fashion to follow the orientation o f the cross. Post-hoc analyses o f the performance of the two controls showed that for both o f them there were highly significant differences between each orientation and the others, for all finger combinations. Furthermore, when the positions o f all o f the fingers are considered, the fingers o f the two controls were spreading to engage with the shape of the cross (the t h u m b - i n d e x and t h u m b - r i n g axes separating by a mean o f 90 ° 57' in M H and 61 ° 15' in BP), whereas D F ' s fingers remained relatively closed (mean 41r~) as her h a n d a p p r o a c h e d the cross.
50
Object Width (cm) Fig. 4. Experiment 2: Grip scaling when reaching to grasp a rectangular Efron block. The size of the finger-thumb grip aperture at a point four frames before contact is plotted as a function of object width, for DF and the two healthy control subjects. Error bars represent standard errors.
Discussion The experiments reported here provide further information on the input coding limitations o f patient D F ' s residual visuomotor control system. In the first experiment, we confirmed our anecdotal impressions that D F
D. P. Carey et al./Visuomotor sensitivity for shape and orientation
0.0 °
22.5 °
335
45.0 °
DF
BP J
MH
Fig. 5. Experiment 3: Grip orientation as defined by the thumb and middle finger positions at a point four frames before contact with a cross-shaped stimulus, presented at three different orientations. The plots are aligned with the centre of the object for clarity.
was able to grasp and use everyday implements with skill and precision. However, she was often only able to grasp the implement in the appropriate place or demonstrate its function, after an exploratory palpation had allowed her to establish its identity. In other words, her initial grasp was always guided by visual information about axis orientation and size, but often not guided by functional knowledge of the object. In the second experiment, we were able to extend our previous observations that D F could use a single stimulus feature such as contour orientation or width to control her hand and fingers, by showing that she could synthesize both orientation and width in guiding her prehension. Both dimensions were evidently coded with normal precision. In the third and final experiment, we combined two orientations in the form of a cross which D F was asked to grasp and rotate: in this case her actions betrayed no evidence of visual guidance by orientation, contrasting markedly with the actions of our normal controls. That is, D F demonstrated no ability to rotate her grip to match a target object which did not possess a unique principal axis. The results of the first experiment are entirely consistent with previous work. They indicate that despite an intelligent adaptation to her perceptual deficit, D F remains unable to use functional information ('affordances') to guide her prehensive behaviour, except in cases where her guesswork (presumably using cues like reflectance and gross size) allows her to identify the object and thereby programme her grasp correctly. Although DF's errors resemble those made by the patient of Sirigu et al. [23], their cause is somewhat different: D F can arrive at an appropriately placed grasp response on the basis of object knowledge when she has that knowledge, but her
acquisition of such knowledge from contour information is grossly impaired. In contrast, the patient of Sirigu et al. [23] can acquire the information for perceptual purposes but seems unable to use that knowledge to instruct the visuomotor system. Experiment 2 is also consistent with previous studies. It extends them, however, in showing that information about independently varying features of two different kinds (orientation and size) can be successfully combined by DF's visuomotor system to mediate accurate and quick grasping behaviours. Indeed D F adjusted her grasp, based on the dimensions and disposition of each Efron shape on each trial, in a manner indistinguishable from neurologically-intact control subjects. Furthermore, in informal testing, we have found that D F can go much further than this. Having observed that she can catch a sponge-rubber ball unimanually without difficulty, we tried throwing a light wooden dowel (35 cm long × 15 mm diameter) towards her. She successfully caught it on 11 trials out of 12. This score was typical of the performance of six matched control subjects, who scored, respectively, 12, 11, 11, 11, 11 and 7. In other words, DF's visuomotor system is not only able to combine orientation with size, it can deal with continually changing orientation simultaneously with information about motion direction. (In contrast to this effective use of motion information, recent observations collected with M. W. Oram and D. I. Perrett confirm earlier data [17] that D F still has difficulties in the perception of motion, e.g. when matching two movements in a same/different paradigm). Although we did not analyse reaction time data in any of these studies, the speed and accuracy with which D F performed these grasping and catching tasks
336
D.P. Carey et al./Visuomotor sensitivity for shape and orientation
is such that we would hypothesize that orientation, size and motion are probably processed in parallel by dorsal stream mechanisms. Such a hypothesis seems plausible on functional grounds, given that these visual attributes are likely to vary independently (and concurrently) in any natural environment. These experiments all add to the list of remarkably well-preserved visuomotor skills that we have been able to identify in D F and that have survived in spite of her severe object recognition difficulties. In contrast, in experiment 3 we appear to have identified a clear limit in those abilities. Unlike normal controls, she did not prerotate her hand to match the orientation of a cross that she had been asked to grasp, although on contact she grasped and rotated the cross with the same ease as did the control subjects. Whereas this finding is at first sight consistent with the finding of Goodale et al. [6] that DF frequently made errors when attempting to 'post' a Tshaped form, it appears to contradict the later finding that she could pick up irregular smooth shapes, using grasp points very much like those of healthy controls [7]. There are two possible explanations of these discrepant results. The first (favoured by Goodale et al. [6]) is that DF can demonstrate form processing only through actions that have multiple degrees of f r e e d o m - s u c h as the combined use of different digits when picking up small shapes. According to this argument, actions that involve only one degree of freedom, such as wrist rotation, can only be controlled by one oriented contour (e.g. one of the bars of the T-shape--frequently the wrong one). But if this account is correct, it is difficult to see why D F did not behave similarly in the present experiment 3: that is, why did she not, on half of the trials, orient her hand to match one arm of the cross, and on the other half orient it to match the other arm? The second way to explain the results is to argue for the 'null hypothesis' that D F cannot use true form at all for guiding her movements; that is, she is not able to integrate more than one orientation together in exercising visuomotor control. If this view is correct, then DF's successful grasping of smooth irregular shapes has to be attributed wholly to her computation of the orientation of the major axis (or axes, where ambiguous) of the shapes, along with her ability to combine this with size processing. Inspection of the data provided by Goodale et al. [7] suggest that such a strategy, with some random variation, could generate most of the grasp patterns displayed by DF. In our view, therefore, it remains uncertain whether DF has any ability to combine visual contour orientations into a 'shape', whether for the purposes of perception or for the control of a visuomotor act. It should be borne in mind, of course, that posterior parts of DF's dorsal stream (as well as her ventral stream) may be partially damaged and/or disconnected from V l. Consequently any inferences derived from studying D F could underestimate the visual capacities of the normal dorsal stream. In conclusion, we have found no compelling evidence
that the (relatively) undamaged regions of visual processing in DF's brain can handle form. On the other hand we have found good evidence that these structures can integrate orientation with other attributes such as size and motion. We would therefore predict that neurones should be identifiable in monkey posterior parietal cortex that are activated in association with moving contours and conjointly sensitive to both orientation and size. We would not, however, feel able to predict with any confidence the existence of neurones selective for shape per se. In recent reviews, Sakata et al. has recently suggested that shape sensitivity might be found in some neurones in the posterior parietal cortex [16, 22]. However, no systematic study appears to have been done to substantiate this claim. The cells described to date show selectivity for the orientation of three-dimensional objects such as bars, plates or cylinders or the overall size of the target object [21, 22], but their properties would not require coding of shape in the true sense. We look forward to seeing more detailed studies of the activities of these neurones. For example, it would be instructive to compare neural responses to objects which vary in shape but not size or three-dimensional orientation. On present data, our prediction would be that if the motor requirements for grasping two stimuli do not differ, then parietal neurones will be unable to distinguish between them.
Acknowledgements--The authors are grateful to the Wellcome Trust and McDonnell-Pew Program for financial support. They also thank A. Burnley, A. Harvey, P. Wilcox and D. Rowland for expert technical assistance; P. Hennessy, C. Lucas and L. Murray for their help with pilot testing and some of the data collection; and the control subjects and the raters for experiment 1 for their kind participation. Finally we owe a profound debt of gratitude to DF and her family, for all of their time, effort and patience.
References
1. Blake, A. Computational modelling of hand-eye coordination. Philosophical Transactions of the Royal Society of London B337, 351-360, 1992. 2. Desimone, R. and Ungerleider, L. G. Neural mechanisms of visual processing in monkeys. In Handbook ofNeuropsvchology, Vol. 2, F. Boiler and J. Grafman (Editors), pp. 267-299. Elsevier, Amsterdam, 1989. 3. Efron, R. What is perception? Boston Studies in the Philosophy of Science 4, 137-173, 1969. 4. Glickstein, M., May, J. G. and Mercier, B. E. Corticopontine projection in the macaque: The distribution of labelled cortical cells after large injections of horseradish peroxidase in the pontine nuclei. Journal of Comparative Neurology 235, 343 359, 1985. 5. Goodale, M. A., Milner, A. D., Jakobson, L. S. and Carey, D. P. A neurological dissociation between perceiving objects and grasping them. Nature 349, 154-156, 1991.
D. P. Carey et al./Visuomotor sensitivity for shape and orientation 6. Goodale, M. A., Jakobson, L. S., Milner, A. D., Perrett, D. I., Benson, P. J. and Hietanen, J. K. The nature and limits of orientation and pattern processing supporting visuomotor control in a visual form agnosic. Journal of Cognitive Neuroscience 6, 46-56, 1994. 7. Goodale, M. A., Meenan, J. P., Bfilthoff, H. H., Nicolle, D. A., Murphy, K. J. and Racicot, C. I. Separate neural pathways for the visual analysis of object shape in perception and prehension. Current Biology 4, 604-610, 1994. 8. Goodale, M. A. and Milner, A. D. Separate visual pathways for perception and action. Trends in Neuroscience 15, 20-25, 1992. 9. Grfisser, O.-J. and Landis, T. Visual Agnosias and Other Disturbances of Visual Perception and Cognition. Macmillan, London, 1991. 10. Humphrey, G. K., Goodale, M. A., Jakobson, L. S. and Servos, P. The role of surface information in object recognition: Studies of a visual form agnosic and normal subjects. Perception 23, 1457-1481, 1994. 11. Husain, M. Visuospatial and visuomotor functions of the posterior parietal lobe. In Vision and Visual Dyslexia ( Vision and Visual Dysfunction, Vol. 13), J. F. Stein (Editor), pp. 12-4Y Macmillan, London, 1991. 12. Iberall, T., Bingham, G. and Arbib, M. A. Opposition space as a structuring concept for the analysis of skilled hand movements. In Generation and Modulation of Action Patterns. Experimental Brain Research Series, Vol. 15, H. Heuer and C. Fromm (Editors), pp. 158-173. Springer-Verlag, Berlin, 1986. 13. Jakobson, L. S. and Goodale, M. A. Factors affecting higher-order movement planning: A kinematic analysis of human prehension. Experimental Brain Research 86, 199-208, 1991. 14. Jeannerod, M. (1988) The Neural and Behavioural Oryanization of Goal-Directed Movements. Oxford University Press, Oxford. 15. Jeannerod, M. and Rossetti, Y. Visuomotor coordination as a dissociable visual function: Experimental and clinical evidence. In Visual Perceptual Defects (Baillikre's Clinical Neurology, Vol. 2, No. 2), C. Kennard (Editor), pp. 439-460. Balli6re Tindall, London, 1993. 16. Jeannerod, M., Arbib, M. A., Rizzolatti, G. and Sakata, H. Grasping objects: The cortical mechanisms of visuomotor transformation. Trends" in Neuroscience 18, 314-320, 1995. 17. Milner, A. D., Perrett, D. I., Johnston, R. S., Benson, P. J., Jordan, T. R., Heeley, D. W., Bettucci, D., Mortara, F., Mutani, R., Terazzi, E. and Davidson, D. L. W. Perception and action in 'visual form agnosia'. Brain 114, 405-428, 1991. 18. Milner, A. D. and Goodale, M. A. Visual pathways to perception and action. In Progress in Brain
19. 20.
21.
22. 23.
24.
25.
337
Research, Vol. 95, T. P. Hicks, S. Molotchnikoff and T. Ono (Editors), pp. 317-337. Elsevier, Amsterdam, 1993. Milner, A. D. and Goodale, M. A. The Visual Brain in Action. Oxford University Press, Oxford, 1995. Motter, B. C. Beyond extrastriate cortex: The parietal visual system. In The Neural Basis of Visual Function (Vision and Visual Dysfunction, Vol. 4), A. G. Leventhal (Editor), pp. 371-387. Macmillan, London, 1991. Sakata, H. Parietal visual neurons coding 3-D characteristics of objects and their relation to hand action. Paper presented at Workshop on Parietal Lobe Contributions to Orientation in 3D Space, Ttibingen, May 1995. Sakata, H. and Taira, M. Parietal control of hand action. Current Opinion in Neurobiology 4, 847 856, 1994. Sirigu, A., Cohen, L., Duhamel, J.-R., Pillon, B., Dubois, B. and Agid, Y. A selective impairment of hand posture for object utilization in apraxia. Cortex 31, 41 55, 1995. Taira, M., Mine, S., Georgopoulos, A. P., Mutara, A. and Sakata, H. Parietal cortex neurons of the monkey related to the visual guidance of hand movements. Experimental Brain Research 83, 29-36, 1990. Ungerleider, L. G. and Mishkin, M. Two cortical visual systems. In: Analysis of Visual Behavior, D. J. Ingle, M. A. Goodale and R. J. W. Mansfield (Editors), pp. 549-58. MIT Press, Cambridge, MA, 1982.
Appendix The 35 objects used in experiment 3
small hacksaw toy pistol comb nail-clippers ladle potato peeler knife fork key adjustable spanner nail file biscuit tongs leather punch hair brush scissors meat fork jug
screw tweezers (2 different pairs) cake tongs (2 different pairs) tankard T-spanner spanner pencil sharpener fish slice tea strainer pen pan spoon pliers cheese grater screwdriver corkscrew
Pergamon
0028-3932(95)00169
7
Neuropsvcholoyia, Vol. 34, No. 5, pp. 329 337, 1996 Copyright ~) 1996 Elsevier Science Ltd. All rights reserved Printed in Great Britain 0028 3932/96 $15.00+0.00
Visuomotor sensitivity for shape and orientation in a patient with visual form agnosia D . P. C A R E Y , * t
M. HARVEY~
and A. D. MILNERt
tSchool of Psychology, University of St. Andrews, St. Andrews, Fife, KY16 9JU, U.K. (Received 31 July 1995; accepted 3 November 1995)
Abstract--We have previously demonstrated that a patient with visual form agnosia (DF), who is unable to report the orientation
or size of visual targets, can nevertheless use these same visual attributes to control motor acts. In the first of three new experiments, we found that DF is able to grasp everyday tools and utensils proficiently (i.e. with a well-formed hand posture) but has difficulty in visually selecting the correct part of the object to grasp (e.g. the handle) for subsequent use of that object. A second experiment revealed that DF's visuomotor system is able to adjust concurrently to variations in both the size and orientation of target objects; when these visual attributes were both varied, she adjusted both her grip aperture and the orientation of her hand well in advance of target contact. These spared visuomotor abilities do not seem to extend to shape processing per se, however. In the final experiment we found that DF was insensitive to changes in the orientation of a cross-shaped object, where no single principal axis could be extracted to control orientation of the grasp. These observations extend our knowledge of DF's residual visuomotor abilities, and suggest limitations on the visual processing capacities of the human dorsal stream. Key Words: visuomotor sensitivity; visuomotor abilities; visual form agnosia.
stream) is crucially implicated in the perceptual analysis of shape and other features of objects [2, 25]. We have suggested that DF's profound shape recognition difficulty can possibly be attributed to a disabling lesion within this ventral system. This would be consistent with the lesion evidence available on other patients with visual object agnosia [for review, see Ref. 9] and with the magnetic resonance imaging (MRI) evidence on DF's lesion [17]. We have accordingly argued [8, 18, 19] that DF's intact visuomotor skills may have as their neural substrate the dorsal stream, which projects from striate to posterior parietal cortex (with associated projections into premotor regions of the frontal lobe). This proposal of a visuomotor function for the dorsal stream echoes similar suggestions by others [4, 15]. The argument is bolstered by physiological recording studies that have revealed a range of neurones in the primate posterior parietal cortex which are associated with visually guided acts such as reaching and grasping, and saccadic and pursuit eye movements [for reviews see Refs. 11, 20, 22] as well as studies of the effects of parietal lesions in humans and monkeys [14, 16]. Furthermore, recent research shows that some of these cells are sensitive to object parameters such as size and orientation, and yet may be insensitive to target location [22, 24]. Neurones of this kind within the human homo-
Introduction
In a series of recent papers, we have described the residual visuomotor abilities of a patient with visual form agnosia (DF) [5, 6, 17]. These investigations have shown that despite her severe perceptual difficulties, which prevent her from recognizing or discriminating between even simple geometrical shapes, D F has a remarkably intact ability to act upon visually presented objects. This ability includes the anticipatory guidance of the hand in visuomotor acts such as inserting a card into an oriented slot [5,17] and grasping blocks of different widths [5]. The latter skill has been found to maintain its accuracy even when the blocks were placed at variable distances [Carey and Milner, unpublished]. We have argued that DF's residual visual abilities are explicable in terms of the current knowledge of visual processing subsystems in the primate brain. It is now accepted that one of the two major streams of cortical visual processing (the occipitotemporal or 'ventral'
* Address for correspondence: Department of Psychology, The University of Aberdeen, King's College, Aberdeen AB9 2UB, U.K. $ Now at: Department of Psychology, University of Bristol, Bristol BS8 1TN, U.K. 329
330
D.P. Carey et al./Visuomotor sensitivity for shape and orientation
logue of the monkey's dorsal stream could underlie DF's successful grasping of visually presented target stimuli varying in either size or orientation. Despite these developments, however, there has been no convincing demonstration thus far of true f o r m processing in posterior parietal neurones, i.e. over and above size and orientation processing. We would, however, argue that an unequivocal demonstration of form-dependent visuomotor skills in a severely form-agnosic patient like D F would provide good presumptive evidence on which to base predictions as to the existence of such shape analysis in the dorsal stream. It might also allow predictions to be made as to the physiological characteristics of that analysis. In other words, if a visuomotor interpretation of dorsal stream function is correct, then an analysis of the visual requirements of the range of sensorimotor behaviours available after destruction or disconnection of the ventral stream should help to delineate the limits of visual processing in the primate dorsal stream. At present, our knowledge of DF's ability to act upon complex visual configurations--in which more than one relevant feature is present--is rather limited. In one experiment, it was found that she made many errors when attempting to 'post' a T-shaped object into a T-shaped aperture [6]. In that task, most of DF's errors consisted of her attempting to insert the object at 90 ° to the correct orientation, suggesting that she was only able to take account of one of the two oriented segments at once. These authors suggested that wrist orientation, having only one degree of freedom, would be sensitive only to a single orientation present in a target; whereas object grasping, involving coordinated movements of the digits, might reveal a true sensitivity to object shape. Indeed, in a recent paper, Goodale et al. [7] reported that D F was adept at picking up a range of irregular smooth shapes, pairs of which she was unable to tell apart. In fact, D F was indistinguishable from controls in the finger-thumb 'grasp points' at which she contacted the shapes. The authors argued that DF's ability in grasping these shapes demonstrated a capacity to process form, over and above her capacity to process simple orientation and/or size. They proposed that DF's intact visuomotor processing allows for the computation of optimal grasp points, in accordance with recent theoretical models [1, 12]. However, the majority of the shapes used in the Goodale et al. [7] study were elongated to some degree, generally having one clear major axis. It is therefore arguable that orientation and width cues alone could provide enough information to guide many of DF's successful grasps, which in general were made either along these axes or (more frequently) at a position midway along and rotated 90" with respect to them [see Fig. 5 in Ref. 7]. If so, it is possible that shape processing in the true sense is not needed to explain the observed pattern of results in DF's grasping performance. In the present experiments, we revisited the question of DF's shape processing skills in three ways. First, we used a procedure similar to one recently described by
Sirigu et al. [23], in which we presented a series of everyday objects to D F and asked her to pick them up and then mime their use. Although in many ways D F seems to be able to interact with objects in a normal fashion, we wanted to examine the extent to which this ability was dependent upon compensatory strategies like the use of subtle tactile exploration. The study was therefore intended to explore the extent of DF's ability, using visual information only, to grasp objects appropriately, as well as her ability to grasp them skilfully and to demonstrate their use. In the second experiment, we examined whether D F retained the ability to grasp different rectangular blocks adeptly even when they were presented at a range of different orientations: that is, could she simultaneously combine size and orientation in her prehensive behaviour? And in the third and final experiment, we presented D F with a doubly symmetrical X-shape mounted above a table and varying in orientation from trial to trial. We asked whether in the absence of a single major axis, she would nevertheless orient her hand to match the orientation of the cross when asked to reach out to grasp and rotate it. The intention in using each of these different approaches was to help define more precisely the visual information that D F is able to use (by virtue, ex hypothesi, of her dorsal stream of cortical processing) in controlling her actions. Exploring these limits should contribute to a fuller specification of our model of primate cortical visual processing [8, 18, 19].
Subjects DF, who suffered from carbon monoxide poisoning in 1988, was tested in the present experiments at the age of 40 (August-September, 1994). Her visual impairments have been described in detail elsewhere, along with relevant clinical details [17]. The inability to discriminate perceptually between equal-area rectangles has become a defining characteristic of visual form agnosia [3], and this was retested just prior to the present experiments. The shapes used were: (a) 50 × 50 mm; (b) 45 × 56 ram; (c) 40 × 62.5 mm; (d) 35 x 71.5 mm; (e) 30 x 83 mm; and (f) 20 × 125 mm. As shown in Fig. 1, a severe disorder is still present in DF. Although her performance is slightly better overall than we found in earlier tests [17], she is still unable to judge at above chance levels whether rectangle D, for example, with a side ratio of 0.49, is the same as or different from a square (rectangle A). Furthermore, we have found chance performance on other perceptual tests recently carried out, such as the recognition of famous faces and making same/different judgements of unfamiliar faces (29/48; 54%, chance = 50%). Yet in spite of all these indications of a lack of significant recovery, D F has adapted to her visual disorder remarkably well, managing, for example, to carry out basic culinary tasks and other complex visually guided activities around the house. A recent eye test using
D. P. Carey et al./Visuomotor sensitivity for shape and orientation 100
90
O o
8O
~
7o
n
60
50 1.0
i
,
,
,
0.8
0.6
0.4
0.2
0.0
Side Ratio Fig. 1. The 'same~lifferent' matching performance of DF on the Efron shapes test. She was given 20 trials at each of the five points plotted; these consisted of 10 'different' presentations (a square with an oblong), and 10 'same' trials (two squares or two oblongs).
a u t o m a t e d refractometry indicated insufficient departure from normal refraction to justify correction. A total o f six healthy female subjects approximately matched with D F for age and education, were used as controls in experiment 1. In each o f experiments 2 and 3, two (similarly matched) controls were used.
Experiment 1 Methods Apparatus and tasks. Subjects were seated at a table and asked to pick up complex three-dimensional objects, with one hand, and to demonstrate their use. A total of 35 everyday objects were presented (see Appendix), all of a grey or metallic colour in order to minimize the use of colour cues for object recognition (which have been demonstrated to aid DF's performance in object recognition tasks; [10]). Objects were presented one at a time and all other stimuli were hidden from view. The subjects were asked to close their eyes during each placement of an individual object. In two separate sessions, the objects were presented either in a standard view (i.e. in the normal orientation for use, typically with the handle pointing towards the subject) or in a view rotated by 180° with respect to this standard view. All subjects were tested on the standard view first. Object manipulation was recorded on super-VHS videotape, using two rotary shutter cameras (JVC BY 10E) which provided clear images at 25 frames/sec. One camera was attached to the ceiling with the lens at a distance of 110 cm above the table surface, providing a top view of the objects and the subject's hand and arm. The second was placed on a tripod at a distance of 150 cm just above the level of the table and oriented perpendicularly to the subject's mid-sagittal axis, thus providing a side view. Both views were recorded together on videotape in a split screen format using a video mixer. Scorin 9 procedure. DF's performance and that of the six control subjects was rated by three judges with respect to how the objects were grasped and manipulated. The following aspects of behaviour were measured: (a) the appropriateness of the point at which the object was grasped; (b) the skill with which it was picked up (i.e. smoothly versus clumsily); (c) the extent of tactile exploration; and (d) the appropriateness of the
331
mime by which the use of the object was demonstrated. Each of these behaviours was ranked on a 7-point scale from 1 (highly inappropriate) to 7 (highly appropriate). Two of the three raters had no knowledge as to the identity of the subjects. Analyses. Spearman rank correlations were calculated pairwise between the raters, separately for each subject and for the four scales. The inter-rater reliability proved very high, with all correlations exceeding +0.97, and most of them exceeding +0.99. No difference was found between the ratings of the informed judge and those of the na~'vejudges, nor did the ratings differ in their reliability among the different subjects tested. In order to compare the performance of DF with that of the six controls, the ratings of the three judges were therefore averaged (for each of the 35 objects, and separately for each of the four scales, the two views and the seven subjects). These scores were then averaged across the 35 objects, and DF's data compared with those of the six controls on each of the four scales. This was done separately for each of the two object views.
Results D F ' s performance did not differ from that o f the control subjects in regard to the skill o f reaching and grasping, or in the mimed demonstration of each object's use. Thus her grip formation was judged to be of comparable skill to that shown by the controls, particularly in the standard view condition; and although her score in the rotated view condition fell slightly below the mean o f the six c o n t r o l s - - 6 . 9 versus 6.6--it was still within 3 S.D. o f it. Similarly, D F was able to demonstrate the use o f the objects as proficiently as the controls, scoring less than 2 S.D. away from the control mean in both view conditions. There were, however, large differences between the subjects with regard to the points at which the objects were grasped, and in the a m o u n t o f tactile exploration. D F generally grasped the objects at the appropriate point (typically the handle) when they were presented in the standard view, scoring within 3 S.D. o f the mean o f the six controls; but when the objects were rotated by 180 r', her performance deteriorated markedly, falling more than 23 S.D. below the average o f the controls (5.0 versus 6.9). This result is evident from Fig. 2, in which the frequency distributions are plotted: whereas all control subjects grasped the objects at highly appropriate points, with 99% of objects being judged between 6-7, D F ' s distribution covered the whole range from highly inappropriate to appropriate. The observation that, despite her inappropriate grasp points, D F was nevertheless able to mime the use o f objects successfully, can be explained by the fact that she was found to engage in m u c h more tactile exploration than any o f the controls, in both view conditions. Whereas controls showed even less tactile exploration in the second session than in the first, D F showed a much greater a m o u n t in the rotated view condition (differing from the controls by 24 S . D . - - 5 . 4 versus 6.9) than in the standard condition (differing by only four S.D.--6.2 versus 6.7). In summary, D F pre-formed the grasp o f her hand to
D.P. Carey et al./Visuomotor sensitivity for shape and orientation
332
250
35
Controls: Normal Orientation
DF: Normal Orientation 30 200
25
I
e.~' 150
20 -I
cr •~ 100 LL
g is 10
50
%.
"5
t~
to
ca
"X
4" ~5" "55" ~>" ,~5" ~5" 250
35
DF: 180 o Rotation
Controls: 1800 Rotation
30
200
25 c
~' 150 E
20
g is 10
,
i_.li ,
,
,
o,m 1oo LL
50
,
,
,
o
4" ~k" "55" ~5" ~5" ~>"
,
,
,
4" ~,5" "5.~" ~5" ,,5" ,o.~"
Fig. 2. Experiment 1: The appropriateness with which household objects were grasped by DF. The judged appropriateness (on a scale of 1-7) is plotted separately for objects presented in standard view or non-standard view, for D F and for six control subjects.
various objects as proficiently as did c o n t r o l subjects. But when objects were presented in an unusual orientation, she often g r a s p e d t h e m at i n a p p r o p r i a t e points a n d palp a t e d them in e x p l o r a t o r y fashion before d e m o n s t r a t i n g their use.
Experiment 2 Methods Apparatus and tasks. Subjects were seated at a table and asked to pick up rectangular wooden blocks (Efron shapes) with one hand. The shapes consisted of six different stimuli varying in length and width, but all were of 5 mm thickness and had a surface area of 25 sq cm. The shapes corresponded precisely to the 2-D patterns used in testing DF's shape matching (see above), varying from a square (shape A) to an oblong with an aspect ratio of 0.16 (shape F). They were presented one at a time, 40 cm in front of the subject's hand, with all other stimuli hidden from view. The subjects were also asked to close their eyes during each placement of an individual shape. The shapes were presented in a pseudorandom sequence at six different orientations (0 °, 30', 6ff~, 90 °, 120 °, 150 °) relative to the sagittal midline. Each shape was presented four times each at 0 ° and 90 '~ and twice at each of the remaining orientations, giving a total of 96 trials. Subjects were asked on each trial to open their eyes, and reach out and grasp the object, using only their index finger and thumb, and pick up the object and raise it above the table surface. Reaching and grasping actions were recorded on superVHS videotape as in experiment 1, with one camera giving a side view (i.e. the parasagittal plane) and the second camera giving an overhead view of the grasping movements. The two camera views were recorded together on tape in split-screen format.
Scoring procedure. Two measurements were derived from the videotapes. The first was the orientation of a vector which joined the tips of the thumb and index finger, relative to the orientation of the Efron shape, at a point four frames before contact with the object. This 'grip orientation' was measured relative to the narrower dimension of each shape. The second measure was an estimate of grip size, i.e. the distance between the thumb and index finger, again four frames before contact with the object. (Note that this measure is not an estimate of maximum grip aperture, which tends to occur when approximately 65% of a movement has been completed [13].) Because deviations from the mid-sagittal axis (usually of the index finger) produced foreshortening in the parasagittal camera view, grip apertures were estimated from the overhead view, with the side view facilitating estimation of the finger-thumb distance whenever necessary. Grip apertures estimated for objects with long axes placed perpendicular to the subject's mid-sagittal plane were very similar to those estimated for other reaches to the same object, where some foreshortening in one camera view was present.
Results Orientation. A n a l y s i s o f the video records o f each g r a s p i n g m o v e m e n t showed t h a t D F ' s p e r f o r m a n c e was indistinguishable f r o m that o f the two controls. Like the controls, she oriented her h a n d correctly with respect to the shape on all trials. As illustrated in Fig. 3 (top), when r o t a t e d by 90 °, some shapes were p i c k e d u p along the thicker r a t h e r than the t h i n n e r extent, b u t this b e h a v i o u r was sometimes shown by the c o n t r o l s as well as b y D F . F o r all three subjects this type o f b e h a v i o u r was m o s t frequent with the m o r e ' c l o s e d ' shapes, b e c o m i n g p r o gressively less c o m m o n with the m o r e e l o n g a t e d ones.
D. P. Carey et al./Visuomotor sensitivity for shape and orientation
333
DF
k,t b
a
lil}7 ~Ii1't
lq/ /
d ML
C
I~li,~ ILl e
\d
i~/lill /~ / / '11lit f
/j
a
-'1
//I\l l \
d VM
f
e
~3 I Vllk
a
11'\~
b
I
C
I
d
e
f
i~\ \ i
J
Fig. 3. Experiment 2: Orientation of finger-thumb grip at a point four frames before contact with a rectangular Efron block placed at different orientations from trial to trial. The lines are superimposed on a normalized representation of each block. The data are from patient DF and two healthy controls (ML and VM).
D.P. Carey et al./Visuomotor sensitivity for shape and orientation
334
D F was more variable in the exact position of her hand four frames before contact (i.e. in Fig. 3 the line segments intersect the objects at a slightly wider range o f positions). However, the orientation of each segment is clearly related to the principal axis o f the target shape, as it is in the control subjects. Like the controls, D F is sensitive to object width, in that the more elongated objects result in grips which consistently target the narrower extent of the shape. Grip size. Analysis of the video records showed that the D F ' s performance was again indistinguishable from that o f the two controls (see Fig. 4). D F ' s grip size varied linearly with the width of the object, becoming progressively smaller from shape A (the square) to F (the narrowest rectangle), closely paralleling the performance of the controls. Because on some trials the shapes were grasped along the long axis (by all three subjects), data points were available on more than just the six nominal widths presented. These, however, were based on small numbers o f observations, and therefore are not plotted in Fig. 4.
Experiment 3
the visibility of the target. The cross was presented randomly at three different orientations (0 °, 22.5 ° and 45 ", anti-clockwise from the subject's point of view relative to the vertical). There were 14 trials at each of the three orientations, giving a total of 42 trials. The subjects were asked to close their eyes between trials. Three practice trials were given before testing began. All of the experimental trials were recorded on super VHS videotape using the two rotary shutter cameras. The first camera was pointed perpendicularly to the subject's mid-sagittal axis, as in experiments 1 and 2. The second camera was placed along the mid-sagittal axis, behind the object and facing the subject. The crosses were mounted in a plexiglas screen fixed vertically, so that the second camera provided an excellent view of the hand as it closed in on the object. Subjects were instructed to open their eyes, and reach out and grasp the cross and then give it a 45 ':' clockwise turn. Scoring procedure. The position of the digits of the hand was examined at a point four frames before contact with the cross. Three separate orientation measures were taken from the videotape, corresponding to lines joining the thumb with the index finger, the thumb with the middle finger, and the thumb with the ring finger, respectively (see Fig. 5). Hand orientation was scored in all cases relative to the gravitational vertical. Consequently if the subject adjusted her hand to the different orientations of the cross, the measured angles would each be expected to increase linearly with increasing stimulus orientation. Analyses. One-way analyses of variance (ANOVAs) (with the three stimulus orientations as the 'between' factor) were performed separately for each subject and each finger combination (thumb-index, thumb-middle, thumb-ring finger).
Methods Apparatus and tasks. Subjects were seated at a table and asked to reach out, grasp, and rotate a cross placed centrally in front of them in the frontoparallel plane, at a height of 50 cm from the table surface (approximately at chin height) and a distance of 40 cm. The two bars constituting the cross were 7.5 cm in length and 1.6 in width, and were 1.1 cm thick. The cross was made from an opaque white plastic. A black background cloth was placed on the wall behind the cross in order to increase
,,--. E
6
~
5
--A---c}-
DF ML - Control VM - Control
< Q. ,r-
~
4
E
a
. m
W
2
i
20
2.2
i
i
i
i
30
3.2
4.0
4.5
Results The main finding was that the two control subjects showed highly significant main effects of orientation over all three finger combinations [MH: F ( 2 , 3 9 ) = 72.30, 58.69, 54.91, respectively, all P < 0.001; BP: F(2,39) = 8.43, 16.29, 8.11; all P < 0.002], whereas D F showed no effect of orientation on any o f the measures [F(2,39) = 0.55, 0.74, 1.23; all n.s.]. As shown in Fig. 5, D F held her hand at the same orientation when approaching the cross, irrespective o f the orientation o f the stimulus, whereas the control subjects rotated their hands in linear fashion to follow the orientation o f the cross. Post-hoc analyses o f the performance of the two controls showed that for both o f them there were highly significant differences between each orientation and the others, for all finger combinations. Furthermore, when the positions o f all o f the fingers are considered, the fingers o f the two controls were spreading to engage with the shape of the cross (the t h u m b - i n d e x and t h u m b - r i n g axes separating by a mean o f 90 ° 57' in M H and 61 ° 15' in BP), whereas D F ' s fingers remained relatively closed (mean 41r~) as her h a n d a p p r o a c h e d the cross.
50
Object Width (cm) Fig. 4. Experiment 2: Grip scaling when reaching to grasp a rectangular Efron block. The size of the finger-thumb grip aperture at a point four frames before contact is plotted as a function of object width, for DF and the two healthy control subjects. Error bars represent standard errors.
Discussion The experiments reported here provide further information on the input coding limitations o f patient D F ' s residual visuomotor control system. In the first experiment, we confirmed our anecdotal impressions that D F
D. P. Carey et al./Visuomotor sensitivity for shape and orientation
0.0 °
22.5 °
335
45.0 °
DF
BP J
MH
Fig. 5. Experiment 3: Grip orientation as defined by the thumb and middle finger positions at a point four frames before contact with a cross-shaped stimulus, presented at three different orientations. The plots are aligned with the centre of the object for clarity.
was able to grasp and use everyday implements with skill and precision. However, she was often only able to grasp the implement in the appropriate place or demonstrate its function, after an exploratory palpation had allowed her to establish its identity. In other words, her initial grasp was always guided by visual information about axis orientation and size, but often not guided by functional knowledge of the object. In the second experiment, we were able to extend our previous observations that D F could use a single stimulus feature such as contour orientation or width to control her hand and fingers, by showing that she could synthesize both orientation and width in guiding her prehension. Both dimensions were evidently coded with normal precision. In the third and final experiment, we combined two orientations in the form of a cross which D F was asked to grasp and rotate: in this case her actions betrayed no evidence of visual guidance by orientation, contrasting markedly with the actions of our normal controls. That is, D F demonstrated no ability to rotate her grip to match a target object which did not possess a unique principal axis. The results of the first experiment are entirely consistent with previous work. They indicate that despite an intelligent adaptation to her perceptual deficit, D F remains unable to use functional information ('affordances') to guide her prehensive behaviour, except in cases where her guesswork (presumably using cues like reflectance and gross size) allows her to identify the object and thereby programme her grasp correctly. Although DF's errors resemble those made by the patient of Sirigu et al. [23], their cause is somewhat different: D F can arrive at an appropriately placed grasp response on the basis of object knowledge when she has that knowledge, but her
acquisition of such knowledge from contour information is grossly impaired. In contrast, the patient of Sirigu et al. [23] can acquire the information for perceptual purposes but seems unable to use that knowledge to instruct the visuomotor system. Experiment 2 is also consistent with previous studies. It extends them, however, in showing that information about independently varying features of two different kinds (orientation and size) can be successfully combined by DF's visuomotor system to mediate accurate and quick grasping behaviours. Indeed D F adjusted her grasp, based on the dimensions and disposition of each Efron shape on each trial, in a manner indistinguishable from neurologically-intact control subjects. Furthermore, in informal testing, we have found that D F can go much further than this. Having observed that she can catch a sponge-rubber ball unimanually without difficulty, we tried throwing a light wooden dowel (35 cm long × 15 mm diameter) towards her. She successfully caught it on 11 trials out of 12. This score was typical of the performance of six matched control subjects, who scored, respectively, 12, 11, 11, 11, 11 and 7. In other words, DF's visuomotor system is not only able to combine orientation with size, it can deal with continually changing orientation simultaneously with information about motion direction. (In contrast to this effective use of motion information, recent observations collected with M. W. Oram and D. I. Perrett confirm earlier data [17] that D F still has difficulties in the perception of motion, e.g. when matching two movements in a same/different paradigm). Although we did not analyse reaction time data in any of these studies, the speed and accuracy with which D F performed these grasping and catching tasks
336
D.P. Carey et al./Visuomotor sensitivity for shape and orientation
is such that we would hypothesize that orientation, size and motion are probably processed in parallel by dorsal stream mechanisms. Such a hypothesis seems plausible on functional grounds, given that these visual attributes are likely to vary independently (and concurrently) in any natural environment. These experiments all add to the list of remarkably well-preserved visuomotor skills that we have been able to identify in D F and that have survived in spite of her severe object recognition difficulties. In contrast, in experiment 3 we appear to have identified a clear limit in those abilities. Unlike normal controls, she did not prerotate her hand to match the orientation of a cross that she had been asked to grasp, although on contact she grasped and rotated the cross with the same ease as did the control subjects. Whereas this finding is at first sight consistent with the finding of Goodale et al. [6] that DF frequently made errors when attempting to 'post' a Tshaped form, it appears to contradict the later finding that she could pick up irregular smooth shapes, using grasp points very much like those of healthy controls [7]. There are two possible explanations of these discrepant results. The first (favoured by Goodale et al. [6]) is that DF can demonstrate form processing only through actions that have multiple degrees of f r e e d o m - s u c h as the combined use of different digits when picking up small shapes. According to this argument, actions that involve only one degree of freedom, such as wrist rotation, can only be controlled by one oriented contour (e.g. one of the bars of the T-shape--frequently the wrong one). But if this account is correct, it is difficult to see why D F did not behave similarly in the present experiment 3: that is, why did she not, on half of the trials, orient her hand to match one arm of the cross, and on the other half orient it to match the other arm? The second way to explain the results is to argue for the 'null hypothesis' that D F cannot use true form at all for guiding her movements; that is, she is not able to integrate more than one orientation together in exercising visuomotor control. If this view is correct, then DF's successful grasping of smooth irregular shapes has to be attributed wholly to her computation of the orientation of the major axis (or axes, where ambiguous) of the shapes, along with her ability to combine this with size processing. Inspection of the data provided by Goodale et al. [7] suggest that such a strategy, with some random variation, could generate most of the grasp patterns displayed by DF. In our view, therefore, it remains uncertain whether DF has any ability to combine visual contour orientations into a 'shape', whether for the purposes of perception or for the control of a visuomotor act. It should be borne in mind, of course, that posterior parts of DF's dorsal stream (as well as her ventral stream) may be partially damaged and/or disconnected from V l. Consequently any inferences derived from studying D F could underestimate the visual capacities of the normal dorsal stream. In conclusion, we have found no compelling evidence
that the (relatively) undamaged regions of visual processing in DF's brain can handle form. On the other hand we have found good evidence that these structures can integrate orientation with other attributes such as size and motion. We would therefore predict that neurones should be identifiable in monkey posterior parietal cortex that are activated in association with moving contours and conjointly sensitive to both orientation and size. We would not, however, feel able to predict with any confidence the existence of neurones selective for shape per se. In recent reviews, Sakata et al. has recently suggested that shape sensitivity might be found in some neurones in the posterior parietal cortex [16, 22]. However, no systematic study appears to have been done to substantiate this claim. The cells described to date show selectivity for the orientation of three-dimensional objects such as bars, plates or cylinders or the overall size of the target object [21, 22], but their properties would not require coding of shape in the true sense. We look forward to seeing more detailed studies of the activities of these neurones. For example, it would be instructive to compare neural responses to objects which vary in shape but not size or three-dimensional orientation. On present data, our prediction would be that if the motor requirements for grasping two stimuli do not differ, then parietal neurones will be unable to distinguish between them.
Acknowledgements--The authors are grateful to the Wellcome Trust and McDonnell-Pew Program for financial support. They also thank A. Burnley, A. Harvey, P. Wilcox and D. Rowland for expert technical assistance; P. Hennessy, C. Lucas and L. Murray for their help with pilot testing and some of the data collection; and the control subjects and the raters for experiment 1 for their kind participation. Finally we owe a profound debt of gratitude to DF and her family, for all of their time, effort and patience.
References
1. Blake, A. Computational modelling of hand-eye coordination. Philosophical Transactions of the Royal Society of London B337, 351-360, 1992. 2. Desimone, R. and Ungerleider, L. G. Neural mechanisms of visual processing in monkeys. In Handbook ofNeuropsvchology, Vol. 2, F. Boiler and J. Grafman (Editors), pp. 267-299. Elsevier, Amsterdam, 1989. 3. Efron, R. What is perception? Boston Studies in the Philosophy of Science 4, 137-173, 1969. 4. Glickstein, M., May, J. G. and Mercier, B. E. Corticopontine projection in the macaque: The distribution of labelled cortical cells after large injections of horseradish peroxidase in the pontine nuclei. Journal of Comparative Neurology 235, 343 359, 1985. 5. Goodale, M. A., Milner, A. D., Jakobson, L. S. and Carey, D. P. A neurological dissociation between perceiving objects and grasping them. Nature 349, 154-156, 1991.
D. P. Carey et al./Visuomotor sensitivity for shape and orientation 6. Goodale, M. A., Jakobson, L. S., Milner, A. D., Perrett, D. I., Benson, P. J. and Hietanen, J. K. The nature and limits of orientation and pattern processing supporting visuomotor control in a visual form agnosic. Journal of Cognitive Neuroscience 6, 46-56, 1994. 7. Goodale, M. A., Meenan, J. P., Bfilthoff, H. H., Nicolle, D. A., Murphy, K. J. and Racicot, C. I. Separate neural pathways for the visual analysis of object shape in perception and prehension. Current Biology 4, 604-610, 1994. 8. Goodale, M. A. and Milner, A. D. Separate visual pathways for perception and action. Trends in Neuroscience 15, 20-25, 1992. 9. Grfisser, O.-J. and Landis, T. Visual Agnosias and Other Disturbances of Visual Perception and Cognition. Macmillan, London, 1991. 10. Humphrey, G. K., Goodale, M. A., Jakobson, L. S. and Servos, P. The role of surface information in object recognition: Studies of a visual form agnosic and normal subjects. Perception 23, 1457-1481, 1994. 11. Husain, M. Visuospatial and visuomotor functions of the posterior parietal lobe. In Vision and Visual Dyslexia ( Vision and Visual Dysfunction, Vol. 13), J. F. Stein (Editor), pp. 12-4Y Macmillan, London, 1991. 12. Iberall, T., Bingham, G. and Arbib, M. A. Opposition space as a structuring concept for the analysis of skilled hand movements. In Generation and Modulation of Action Patterns. Experimental Brain Research Series, Vol. 15, H. Heuer and C. Fromm (Editors), pp. 158-173. Springer-Verlag, Berlin, 1986. 13. Jakobson, L. S. and Goodale, M. A. Factors affecting higher-order movement planning: A kinematic analysis of human prehension. Experimental Brain Research 86, 199-208, 1991. 14. Jeannerod, M. (1988) The Neural and Behavioural Oryanization of Goal-Directed Movements. Oxford University Press, Oxford. 15. Jeannerod, M. and Rossetti, Y. Visuomotor coordination as a dissociable visual function: Experimental and clinical evidence. In Visual Perceptual Defects (Baillikre's Clinical Neurology, Vol. 2, No. 2), C. Kennard (Editor), pp. 439-460. Balli6re Tindall, London, 1993. 16. Jeannerod, M., Arbib, M. A., Rizzolatti, G. and Sakata, H. Grasping objects: The cortical mechanisms of visuomotor transformation. Trends" in Neuroscience 18, 314-320, 1995. 17. Milner, A. D., Perrett, D. I., Johnston, R. S., Benson, P. J., Jordan, T. R., Heeley, D. W., Bettucci, D., Mortara, F., Mutani, R., Terazzi, E. and Davidson, D. L. W. Perception and action in 'visual form agnosia'. Brain 114, 405-428, 1991. 18. Milner, A. D. and Goodale, M. A. Visual pathways to perception and action. In Progress in Brain
19. 20.
21.
22. 23.
24.
25.
337
Research, Vol. 95, T. P. Hicks, S. Molotchnikoff and T. Ono (Editors), pp. 317-337. Elsevier, Amsterdam, 1993. Milner, A. D. and Goodale, M. A. The Visual Brain in Action. Oxford University Press, Oxford, 1995. Motter, B. C. Beyond extrastriate cortex: The parietal visual system. In The Neural Basis of Visual Function (Vision and Visual Dysfunction, Vol. 4), A. G. Leventhal (Editor), pp. 371-387. Macmillan, London, 1991. Sakata, H. Parietal visual neurons coding 3-D characteristics of objects and their relation to hand action. Paper presented at Workshop on Parietal Lobe Contributions to Orientation in 3D Space, Ttibingen, May 1995. Sakata, H. and Taira, M. Parietal control of hand action. Current Opinion in Neurobiology 4, 847 856, 1994. Sirigu, A., Cohen, L., Duhamel, J.-R., Pillon, B., Dubois, B. and Agid, Y. A selective impairment of hand posture for object utilization in apraxia. Cortex 31, 41 55, 1995. Taira, M., Mine, S., Georgopoulos, A. P., Mutara, A. and Sakata, H. Parietal cortex neurons of the monkey related to the visual guidance of hand movements. Experimental Brain Research 83, 29-36, 1990. Ungerleider, L. G. and Mishkin, M. Two cortical visual systems. In: Analysis of Visual Behavior, D. J. Ingle, M. A. Goodale and R. J. W. Mansfield (Editors), pp. 549-58. MIT Press, Cambridge, MA, 1982.
Appendix The 35 objects used in experiment 3
small hacksaw toy pistol comb nail-clippers ladle potato peeler knife fork key adjustable spanner nail file biscuit tongs leather punch hair brush scissors meat fork jug
screw tweezers (2 different pairs) cake tongs (2 different pairs) tankard T-spanner spanner pencil sharpener fish slice tea strainer pen pan spoon pliers cheese grater screwdriver corkscrew