Impaired Discrimination of Familiar from Unfamiliar Faces

IMPAIRED DISCRIMINATION OF FAMILIAR FROM UNFAMILIAR FACES Andrew W. Youngl, Brenda M. Flude2, Dennis C. Hay2 and Andrew W. Ellis3 CDepartment of Psychology, University of Durham; 2Department of Psychology, University of Lancaster; 3Department of Psychology, University of York)

INTRODUCTION

Face recognition impairments are often found in the context of brain injury involving the right cerebral hemisphere, and investigations of these impairments have revealed a number of important features of the human face processing system (for reviews see Benton, 1980; H. Ellis, 1983; Rhodes, 1985). One of the central issues highlighted by this work concerns whether it is possible to fractionate the face processing system into isolable component parts, and map out their relation to each other. This approach involves carefully contrasting the effects of brain injury across people who show different patterns of impairment, and relating them to an explicit functional model intended to account for both normal and impaired performance (Young and Bruce, 1991; Young, 1992). For this purpose, Hay and Young (1982) introduced a simple 'functional model', which was further developed by Bruce and Young (1986). This (Bruce and Young, 1986) model claims that recognition proceeds in parallel with other aspects of face processing, such as the analysis of expressions. Recognition itself involves sequential stages of perceptual classification (by domain-specific face recognition units), semantic classification (involving domain-independent person identity nodes which can access previously learnt semantic information from the person's face, voice, or name), and name retrieval. Existing studies of face recognition impairments are consistent with Bruce and Young's (1986) idea that familiarity, semantic information, and name retrieval involve sequential stages, in that order. Cases have been described in which there is no sense of a face's familiarity (as in the neurological condition prosopagnosia; Bodamer, 1947), the face seems familiar only (de Haan, Young and Newcombe, 1991), or semantic information but not the name can be retrieved (Flude, Ellis and Kay, 1989). Hence there are types of breakdown corresponding to each level of Bruce and Young's (1986) model. However, as the model would predict, impairments at the earlier levels affect later stages: without a sense of the face's familiarity, occupation and name cannot be retrieved (as in prosopagnosia); if the face is familiar but the occupation cannot be retrieved, then it can't be named either (de Haan et aI., 1991); and name retrieval impairments can exist even when familiarity and occupation are available (Flude et aI., 1989). The same patterns of error also arise as transitory phenomena for normal people in everyday life (Young, Hay and Ellis, 1985). Also important to the Bruce and Young (1986) model are patterns of impairment which do not occur. For example, there is no well-documented report Cortex, (1993) 29, 65-75

66

A. W. Young and Others

of a brain-injured person for whom name retrieval was normal from seen faces but access to occupations was impaired. Such a case would clearly violate the proposed hierarchy, and therefore be of considerable theoretical importance. A model such as Bruce and Young's does not solve the problem of how we recognise faces. But it does provide a convenient way of summarising what is known. More importantly, it can predict hitherto unreported types of deficit, and allow new findings to act as tests of the model by the extent to which they can be readily accommodated or force revision. In this paper we report investigation of a man with a form of face recognition impairment which involved problems in discriminating familiar faces from unfamiliar faces. This deficit has not been investigated in detail before. CASE DESCRIPTION

When first seen by us in 1984, J.T. was a 72 year old right-handed retired engineer, who had suffered a stroke affecting the right cerebral hemisphere some 18 months ago. He had left-sided motor weakness and impaired Performance IQ (WAIS Verbal IQ 113, Performance IQ 93), but no obvious problems in language comprehension, speech, reading or writing. Verbal memory was within normal limits (WMS Logical Memory, story I: immediate recall 10, delayed [1 hour] recall 7), but memory for nonverbal material was poor (Benton Visual Retention Test: 17; Rey-Osterrieth complex figure: immedite copy 35, delayed [45 minutes] 8), There was no evidence of neglect in cancellation tasks (e.g., Albert, 1973) and no impairment on tasks involving the recognition of everyday objects ('canonical' views 10/10, 'unusual' views 10/10).

FACE PROCESSING ABILITIES

Tests were given to assess J.T.'s ability to identify familiar faces, to match and remember unfamiliar faces, and to determine facial expression. The results of initial testing sessions in 1984 and a long-term follow-up in 1991 are summarised in Table I. We begin by reporting the results of testing in 1984. Identification of Familiar Faces ('Faces Line-up')

J.T. was shown a 'faces line-up' involving 20 highly familiar faces, 20 moderately familiar, and 20 unfamiliar faces, presented one at a time in random order. For each face he was asked to rate its familiarity on a 1-7 scale 1 = unknown, 7 = very familiar), and if it was a familiar person to give his or her occupation and name. Control data were taken from 28 men of comparable age (60-75 years) from a series of normal men tested at the Radcliffe Infimary, Oxford (de Haan, Young and Newcombe, 1991). The data of interest concern ability to give correct occupations and names to the 20 highly familiar faces, and the rate at which the 20 unfamiliar faces were misidentified. Data for the moderately familiar faces are not included in Table I, because the control subjects showed high variance in their responses to these. Results showed no problems in recognising familiar faces (see Table I), but severe problems with the faces of unfamiliar people. On 16/20 of the trials involving unfamiliar faces, J.T. misidentified them as being familiar (only 4/20

Familiar and unfamiliar fa ces

67

TABLE I

IT's Performance of Face Processing Tasks in 1984 and 1991, and Means and Standard Deviations for Control Subjects J.T.

Identification of familiar faces High familiarity faces Mean familiarity rating Occupation Name Unfamiliar faces Mean familiarity rating Correct rejections Identification of familiar names High familiarity names Mean familiarity rating Occupation Unfamiliar names Mean familiarity rating Correct rejections Unfamiliar face matching Benton test Facial expressions Matching Labelling Recognition memory Warrington RMT: Faces Warrington RMT: Words

Controls 1991

Mean

S.D.

6.25 18/20 10120*

5.98 18.86 16.25

0.51 1.15 2.81

1.25 18120

1.36 17.80

0.45 2.35

7.00 20/20

7.00 20/20

6.27 19.66

0.63 0.84

1.25 20/20

1.00 20/20

1.10

0.10

38/54*

36/54**

44.22

3.07

17/24**

29.79 21.10

1.67 1.70

31150*** 39/50

42.26 42.63

3.44 5.21

1984

6.60 18/20 l7I20 3.70** 4/20***

27/32* 34/50** 44/50

Asterisked scores are significantly impaired in comparison to the performance of controls: * z> 1.65, p 2.33, p3.1O, p< O.OOI. Control data are taken from de Haan, Young and Newcombe (1991), Warrington (1984), and unpublished data kindly provided by Dr. Freda Newcombe.

correct rejections, z=5.87, p
A parallel test to the familiar face recognition test was given, using written names of the 20 highly familiar people and 20 moderately familiar people from

68

A. W. Young and Others.

the face recognition test, interspersed with 20 unfamiliar names. For each name J.T. was asked to rate its familiarity, and if it belonged to a familiar person, to give his or her occupation. His performance was unimpaired, both in terms of recognition of highly familiar names and rejection of unfamiliar names. Unfamiliar Face Matching The Benton Test of Facial Recognition was used (Benton, Hamsher, Varney and Spreen, 1983). In this test, subjects have to choose which of six photographs of unfamiliar faces are pictures of the same person as a simultaneously presented target face photograph. The test includes items involving choice of identical photographs, and transformations of orientation or lighting, which are pooled into an overall composite score. J.T.'s overall score (38/54 correct) was mildly impaired both in terms of the test's norms and in comparison to a group of 32 men of comparable age tested at the Radcliffe Infirmary (see Table I). Facial Expressions J.T. was shown pairs of photographs of faces of two different people (presented simultaneously), and asked to decide whether or not they had same or different facial expressions. Sixteen of the pairs had the same expressions, and 16 different expressions. His performance (27/32 correct) was slightly impaired in comparison with that of 34 men of comparable age from the Radcliffe Infirmary series (z = 1.67, I-tailed p<0.05). Recognition Memory The Warrington Recognition Memory Test (Warrington, 1984) was given. In this test, Recognition Memory is tested separately for Faces and Words. In the Faces part of the test, 50 faces are shown at the rate of one every three seconds for a "pleasant or unpleasant" decision, and then recognition memory is tested by presenting each of the faces paired with a distractor, with the subject having to choose which has been seen before. A similar procedure is used with Words. Warrington's (1984) norms do not extend as far as age 72 years, so we have compared J.T.'s performance to her 65-70 year old controls. His score of 34/50 correct for the Faces part of the test was significantly impaired, whereas his score of 44/50 correct on recognition memory for Words was no different from Warrington's (1984) 65-70 year old controls (see Table I). Further Investigation of Impaired Discrimination of Familiar from Unfamiliar Faces Our initial (1984) assessment had shown that in addition to being poor at unfamiliar face matching tasks, poor at remembering unfamiliar faces, and poor at matching facial expressions, J.T. showed a marked tendency to misi-

Familiar and unfamiliar faces

69

dentify unfamiliar faces as familiar. In contrast, he showed no sign of any corresponding problem with unfamiliar names. Further investigations of this impaired discrimination of familiar from unfamiliar faces were carried out during 1984. Retest on Faces Line-up A retest of the faces line-up 6 months after the initial test session produced an almost identical result, with no problems in recognising highly familiar faces (mean familiarity rating = 6.20; 18120 given correct occupations; 14/20 correctly named) and severe impairments to unfamiliar faces (mean familiarity rating=3.70, z=5.20, p
70

A. W. Young and Others

and unfamiliar to him. In addition, a parallel task using written names was given. Trials with faces and names were arranged into blocks of 24 trials each (12 familiar, 12 unfamiliar stimuli in each block), given in a counterbalanced order. The time taken to classify each set of stimuli was recorded using a stopwatch, and errors noted. J.T. made no errors in classifying the names (96/96 correct), but produced a substantial number of errors to the faces (76/96 correct). He was also faster at classifying the names than the faces (total time: names = 237 seconds, faces = 284 seconds). Of the errors to faces, 14 involved failing to recognise familiar faces (which were erroneously classified as unfamiliar) and 6 involved erroneously classifying unfamiliar faces as familiar. The 14 familiar faces J.T. had misclassified were shown to him again at the end of each session, with unlimited presentation time, and he recognised 11114 correctly, showing that he did know most of these people. Like the line-up tasks, this speeded familiarity decision task showed a problem in discriminating between familiar and unfamiliar faces. However, whereas in the un speeded line-up tasks this problem was characteristically one in which unfamiliar faces were misrecognised as familiar, in the speeded task J.T. misclassified both familiar and unfamiliar faces. Half the faces used in the speeded familiarity decision task were male, and half female. When J.T. was asked to classify the faces as male or female, he was much faster (total time = 183 seconds) and more accurate (95/96 correct) than when classifying them as familiar or unfamiliar. Recognition Memory Our initial assessment of J.T. had shown poor recognition memory for faces on Warrington's (1984) Recognition Memory Test. A further investigation of his recognition memory was carried out, to see whether it would be affected by perceived familiarity. A large number of familiar und unfamiliar faces were shown to J.T., who was asked to rate the familiarity of each on a 1-7 scale 1 = unknown, 7 = very familiar). From these ratings, we assembled sets of 20 familiar faces (mean rated familiarity=7.00), 20 unfamiliar faces J.T. had misidentified as familiar (mean rated familiarity = 5.50), and 20 unfamiliar faces J.T. had correctly rated as being unfamiliar (mean rated familiarity = lAO). Three weeks later, these faces were used in a recognition memory test. Half the faces from each set were designated as targets (Le., 10 familiar faces, 10 unfamiliar faces J.T. had misidentified as familiar, and 10 unfamiliar faces he had correctly classified as unfamiliar), and shown to J.T. one at a time in random order, for a few seconds each. J.T. was told to look at each face carefully, because he would be tested on his memory for them. An hour later, these target faces were mixed with the remaining faces from each set, and J.T. was asked whether each was from the series he had seen an hour previously. Results of this recognition memory task are presented in Table II, which gives target hit rates, distractor rejection rates, and a nonparametric estimate of the area under the ROC curve (McNicol, 1972) for each set of faces. J.T.

Familiar and unfamiliar faces

71

TABLE II

JT's performance in a recognition memory task using familiar faces, unfamiliar faces he had misidentified as familiar, and urifamiliar faces he had correctly classified as unfamiliar to him

Familiar faces Unfamiliar faces mistaken for familiar Unfamiliar faces recognised as unfamiliar

Targets (hits)

Distractors (correct rejections)

Area under ROC curve

10/10 7110 3/10

9/10 9/10 9/10

97% 88% 72%

showed better recognition memory for unfamiliar faces he had rated as being familiar to him than for unfamiliar faces he considered unfamiliar. Resemblance Task

One of the properties of faces we know well is that we tend to think they don't look like anyone else (A. Ellis, Young and Hay, 1987). A test was undertaken to determine whether this would also hold for the unfamiliar faces J.T. had mistakenly identified as familiar. He was shown a series of 30 photographs of unfamiliar faces, and asked whether each of these was a familiar person, and whether it bore any resemblance to anyone he could think of. Of these 30 faces, J.T. considered 11 to be familiar people, and noted that 2 of these resembled other people. For the 19 faces he considered unfamiliar, he was able to generate resemblances in 13 cases. There is a statistiCally significant difference between the proportions of resemblances to familiar and unfamiliar faces (2/11 versus 13/19; X2 = 7.03, dJ. = 1, 2-tailed p<0.02). Long-Term Follow-up Investigation

A long-term follow-up was carried out in 1991, 7 years after the other tests reported here (i.e., 9 years post stroke). This involved repetition of all the face processing tests used for initial assessment in 1984, except than an expression labelling task was substituted in 1991 for the expression matching task used in 1984. This used 24 photographs of emotional facial expressions from the Ekman and Friesen (1976) series, presented one ata time with a list of six possible emotion labels printed below each face (see de Haan et aI., 1991). The task was to choose the correct label to describe the :fflcial expression. Control data . are taken from de Haan et a1. (1991). Results of this long-term follow-up are su~~sed in Table I. Of course, J.T. was now aged 79, whereas the control data·are generally for somewhat younger people. However, it is clear that the impairments on the Benton unfamiliar face matching test, in analysing faciaFexpressions, and on recognition memory for faces were still present after this 7 years interval, whereas the problem with discriminating unfarililiar faces from familiar faces had resolved. There was a slight problem with name retrieval (10/20 highly familiar faces

72

A. W. Young and Others

named correctly in 1991, z=2.22, p<0.05), though this may reflect the use of norms relating to 60-75 year-olds, since this is a problem which is not uncommon in elderly people (Cohen and Faulkner, 1986; Maylor, 1990). DISCUSSION

J.T. showed a marked problem in discriminating between familiar and unfamiliar faces, with no corresponding problem in discriminating familiar from unfamiliar names. The problem with faces was documented in several tasks, and persisted across a considerable period of time (18-24 months post stroke), though it had resolved at long-term follow-up (7 years later). When given unlimited time to respond (as in our line-up tasks), J.T. did not show any problem in recognising familiar faces, but tended to think that he recognised unfamiliar faces, leading him to misc1assify unfamiliar faces as familiar and often to provide incorrect semantic information. Occasionally, he misidentified unfamiliar faces as particular (named) familiar people. When under time pressure, however, J.T. also failed to recognise a number of familiar faces which he could readily identify when the time pressure was removed. His response times in our speeded task were also longer to faces than names, which is the reverse of the pattern usually found for normal subjects (Young, McWeeny, Ellis and Hay, 1986). There have been no previous reports of detailed investigation of this type of face recognition impairment. However, we note with interest that A. Ellis, HiIlam, Cardno and Kay (1991) found a similar pattern in a group study of patients with temporal lobe epilepsy. Patients with a right temporal lobe focus were able to recognise familiar faces as well as controls, but made more misrecognitions of unfamiliar faces. Bartlett and Fulton (1991) and Bartlett, Strater and Fulton (1991) have found that, in general, old people show more false recognitions of unfamiliar faces than are found for younger people. Whilst this is not in itself sufficient to account for J.T.'s impairment, because his performance was compared to control subjects of comparable age (and also because the impairment in discriminating unfamiliar from familiar faces later resolved), it is of interest that his problems might take the form of an exaggeration of difficulties experienced by many elderly people. In terms of the Bruce and Young (1986) model, recognition of familiar faces involves determining whether the structural description of a seen face matches one of the previously stored structural descriptions describing the appearance of familiar faces (face recognition units). J.T.'s problem could therefore arise in one of three ways; degraded input to the face recognition units, malfunction of the recognition units themselves, or incorrect evaluation of the degree of match by impaired decision mechanisms. For normal people, everyday errors involving the misidentification of unfamiliar faces are often a result of poor viewing conditions (Young et aI., 1985). The possibility of degraded input to the face recognition units caused by impaired ability to form an adequately detailed representation thus forms a plau-

Familiar and unfamiliar faces

73

sible contributory factor in J.T.'s case, and it is one that we cannot rule out since his performance of the Benton unfamiliar face matching test (Benton et aI., 1983) was not entirely normal. It will be important to know whether other patients with equivalent problems in discriminating familiar from unfamiliar faces are also impaired on the Benton test. However, we are not inclined to view any problem of degraded input as in itself a sufficient explanation of J.T.'s problems, because the impairment on the Benton test was still present in 1991, when the problem in discriminating familiar from unfamiliar faces had resolved. In addition, many patients with equal or more severe impairments on the Benton test do not seem to show this problem in discriminating familiar from unfamiliar faces. We therefore think it more likely that there must have been another deficit. The alternatives which remain to be considered are malfunction of the face recognition units themselves, or incorrect evaluation by decision mechanisms involved in deciding whether or not the resemblance between a seen face and one of the stored representations of familiar faces is sufficiently close to qualify as a genuine (rather than superficial) match. Of these alternatives, we find the idea of impaired decision mechanisms the less plausible because J.T.'s ability to remember a face and to think of other people who might be similar in appearance was affected by whether or not it seemed familiar or unfamiliar to him. These findings imply that whatever underlay the spurious sense of familiarity was sufficient to produce real differences between the way in which genuinely unfamiliar and spuriously familiar faces were seen. We thus favour the hypothesis of malfunction of face recognition units. A proper sense of the familiarity of known faces is important to our ability to live in a world where we must rapidly pick out people we know in busy environments, and may have more fundamental significance in colouring our emotional reactions. The importance of familiarity has been emphasised in neurological, neurophysiological, and psychological work (Gaffan, 1976; Mandler, 1980; Critchley, 1989). Moreover, studies of psychiatric conditions in which loss of familiarity or jamais vu forms a pronounced component have linked these to right hemisphere disease (e.g., Feinberg and Shapiro, 1989; Young, Robertson, Hellawell et aI., 1992). Our findings with J.T. underline the importance of examining the breakdown of familiarity in cases of neuropsychological impairment.

ABSTRACT

We report investigations of the face processing abilities of J.T., a man who had suffered a right hemisphere stroke. J.T. showed a marked problem in discriminating between familiar and unfamiliar faces, with no corresponding problem in discriminating familiar from unfamiliar names. The problem with faces was still found 2 years after the stroke, but had resolved at long-term follow-up (9 years post stroke). When given unlimited time to respond, J.T. did not show any problem in recognising familiar faces, but tended to think that he recognised unfamiliar faces. When under time

74

A. w. Young and Others

pressure, however, J.T. also failed to recognise a number of familiar faces which he could readily identify when the time pressure was removed. J.T.'s ability to remember a face and to think of other people who might be similar in appearance was affected by whether or not the face seemed familiar or unfamiliar to him. Hence, whatever underlay the spurious sense of familiarity was sufficient to produce real differences between the way in which genuinely unfamiliar and spuriously familiar faces were seen, leading us to suggest that his impaired dis.;:rimination of unfamiliar from familiar faces reflected a malfunction of face recognition units. Acknowledgements. This research was supported by a grant from Lancaster University's Research Grant Fund to Andy Ellis, Andy Young and Dennis Hay, and by ESRC project grant ROOO233091 to Andy Ellis, Andy Young, Mike Burton and Brenda Flude. We are grateful to the Press Association and the Lancashire Evening Post for help in finding suitable photographs for use as stimuli. We thank Dr. Freda Newcombe (Radcliffe Infirmary, Oxford) for allowing us to use control data collected for her own studies. REFERENCES

ALBERT, M.L. A simple test of visual neglect. Neurology, 23: 658-664, 1973. BARTLETT, J., and FULTON, A. Familiarity and recognition of faces in old age. Memory and Cognition, 19: 229-238, 1991. BARTLETT, J., STRATER, L., and FULTON, A. False recency and false fame of faces in young adulthood and old age. Memory and Cognition, 19: 177-188, 1991. BENTON, A.L. The neuropsychology of facial recognition. American Psychologist, 35: 176-186, 1980. BENTON, A.L., HAMSHER, K. de S., VARNEY, N., and SPREEN, O. Contributions to Neuropsychological Assessment: A Clinical Manual. Oxford: Oxford University Press, 1983. BODAMER, J. Die Prosop-Agnosie. Archiv fur Psychiatrie und Nervenkrankheiten, 179: 6-53, 1947. BRUCE, V. , and YOUNG, A. Understanding face recognition. British Journal of Psychology, 77: 305327, 1986. COHEN, G., and FAULKNER, D. Memory for proper names: age differences in retrieval. British Journal of Developmental Psychology, 4: 187-197, 1986. CRITCHLEY, E.M.R. The neurology of familiarity. Behavioural Neurology, 2: 195-200, 1989. DE HAAN, E.H.F., YOUNG A.W., and NEWCOMBE, F. A dissociation between the sense of familiarity and access to semantic information concerning familiar people. European Journal of Cognitive Psychology, 3: 51-67, 1991. EKMAN, P., and FRIESEN, W.V. Pictures of Facial Affect. Palo Alto, California: Consulting Psychologists Press, 1976. ELLIS, A.W., HILLAM, J.C., CARDNO, A., and KAY, J. Processing of words and faces by patients with left and right temporal lobe epilepsy. Behavioural Neurology, 4: 121-128, 1991. ELLIS, A.W., YOUNG, A.W., and HAY, D.C. Modelling the recognition of faces and words. In P.E. Morris (Ed.), Modelling Cognition. Chichester: Wiley, 1987, pp. 269-297. ELLIS, H.D. The role of the right hemisphere in face perception. In A.W. Young (Ed.), Functions of the Right Cerebral Hemisphere. London: Academic Press, 1983, pp. 33-64. FEINBERG, T.E., and SHAPIRO, R.M. Misidentification-reduplication and the right hemisphere. Neuropsychiatry, Neuropsychology, and Behavioral Neurology, 2: 39-48, 1989. PLUDE, B.M., ELLIS, A.W., and KAY, J. Face processing and name retrieval in an anomic aphasic: names are stored separately from semantic information about familiar people. Brain and Cognition, 11:60-72,1989. GAFFAN, D. Recognition memory in animals. In J. Brown (Ed.), Recall and Recognition. London: Wiley, 1976, pp. 229-242. HAY, D.C., and YOUNG, A.W. The human face. In A.W. Ellis (Ed.), Normality and Pathology in Cognitive Functions. London: Academic Press, 1982, pp. 173-202. MANDLER, G. Recognizing: the judgment of previous occurrence. Psychological Review, 87: 252-271, 1980. MA YLOR, E.A. Recognizing and naming faces: aging, memory retrieval, and the tip of the tongue state. Journal of Gerontology: Psychological Sciences, 45: 215-226, 1990. McNICOL, D. A Primer of Signal Detection Theory. London: George Allen and Unwin, 1972. RHODES, G. Lateralized processes in face recognition. British Journal of Psychology, 76: 249-271, 1985. WARRINGTON, E.K. Recognition Memory Test. Windsor: NFER-Nelson, 1984. YOUNG A.W. Face recognition impairments. Philosophical Transactions of the Royal Society, B335: 47-54, 1992.

Familiar and unfamiliar faces

75

YOUNG, A.W., and BRUCE, V. Perceptual categories and the computation of 'grandmother'. European Journal of Cognitive Psychology, 3: 5-49, 1991. YOUNG, A.W., HAY, D.C., and ELLIS, A.W. The faces that launched a thousand slips: everyday difficulties and errors in recognizing people. British Journal of Psychology, 76: 495-523, 1985. YOUNG, A.W., MCWEENY, K.H., ELLIS, A.W., and HAY, D.C. Naming and categorising faces and written names. Quarterly Journal of Experimental Psychology, 38A: 297-318, 1986. YOUNG, A.W., ROBERTSON, l.H., HELLAWELL, DJ., DE PAVW, K.W., and PENTLAND, B. Cotard delusion after brain injury. Psychological Medicine, 22. 799-804, 1992.

Andrew W. Young, Department of Psychology, University of Durham, Durham DHI 3LE, England, U.K.