- Email: [email protected]
Automatic guidance of attention from working memory
Review
Automatic guidance of attention from working memory David Soto1,2, John Hodsoll2, Pia Rotshtein2 and Glyn W. Humphreys2 1
Imperial College London, Faculty of Medicine, Division of Neuroscience and Mental Health, Charing Cross Campus, St. Dunstan’s Road, London, W6 8RP, UK 2 Behavioural Brain Sciences Centre, School of Psychology, University of Birmingham, Birmingham, B15 2TT, UK
Recent research has shown interactions between the process of keeping information ‘online’ in working memory, and the processes that select relevant information for a response. In particular, our ability to select stimuli in the environment can be modulated by whether the stimuli match the current contents of working memory. Guidance of selection from working memory occurs automatically, even when it is detrimental to performance. Neurophysiological data, from functional brain imaging, indicate that the interaction between working memory and attention is based on neuronal mechanisms distinct from the processes mediating ‘bottom-up’ priming effects from implicit memory. We discuss the importance of ‘top-down’ influences from working memory on the ‘early’ deployment of attention and on the processes that gate visual information into awareness. Introduction To survive in a complex and dynamically changing world, our attention needs to be guided to select stimuli that are relevant to our behavioural goals. It has been known for some time that stimulus-driven factors can guide attention in a relatively automatic way [1] such as when the sudden appearance of an unexpected object draws attention without any intention on the part of the observer. However, if we are not at the mercy of such changes, attention needs to be directed in a ‘top-down’ manner. In many cases, the topdown directing of attention will be based on information held in working memory (WM) [2–4]. For example, our search for a favourite shirt in the wardrobe might be guided by an on-line memory of the features of the relevant object (e.g. its texture and colour). This linkage between WM and attention might also sometimes have unwanted consequences – for example, if it turns out that the information we are holding in WM is irrelevant or even misleading for search. Although perhaps disruptive in everyday life, these unwanted consequences can be scientifically useful for studying the automaticity of any coupling between WM and attention. In this article we review emerging evidence on this automatic coupling, documenting both the behavioural characteristics of the interaction and the underlying neural circuits. Our discussion is confined to visual selection, although it is an open question as to whether a similar coupling exists in other sensory modalities.
Corresponding author: Soto, D. ([email protected]).
342
Common resources for WM and visual selection One reason for arguing for a close coupling between WM and visual selection is that a common pool of resources seems to subserve both processes. For example, Lavie and colleagues [5] have demonstrated that our ability to filter out irrelevant stimuli during selection depends on the processing load in WM. As the WM load increases, so fewer resources seem to be available to support efficient target selection and distractor rejection. The net result of this is that interference from distractors increases under conditions of high WM load [5]. This contrasts with conditions in which the perceptual load (e.g. the complexity of a visual display) is increased and distractor interference can decrease [6], indicating the selective linkage of WM resources with a stage of selecting visual information for response. Effects of the contents of WM on the deployment of attention in search The biased competition model of attention proposes that stimuli compete for selection at multiple levels of representation, with the ‘winner’ gaining control of both perceptual and response systems [7] (Box 1). Within this framework, WM acts to bias the competition for attention to favour objects that fit the goals of the task. Behavioural studies are consistent with this, with visual selection being strongly modulated by information in WM set up for a particular search target. For example, if we are looking for a red target, inadvertent capture of attention by unexpected salient stimuli can be enhanced when the irrelevant salient item shares the defining properties of the target [8–11]. The effects of WM seem to modulate relatively early visual processing. In visual search, targets can be detected efficiently if their feature values are categorically different from those of the distractors (e.g. the target is steep and the distractors are shallow in orientation) [12], or if the target is defined at the end of a continuum of feature values (e.g. if there is a large target and small and medium distractors). This evidence has also been used to define the nature of the basic features subserving visual processing (i.e. colour) [13]. Interestingly, these effects are contingent on participants holding the feature values in WM. Hodsoll and Humphreys [14] showed that the effects of categorical differences in orientation search were dependent on holding a memory ‘template’ of the target feature (Figure 1). This
1364-6613/$ – see front matter ß 2008 Elsevier Ltd. All rights reserved. doi:10.1016/j.tics.2008.05.007 Available online 5 August 2008
Review Box 1. The biased competition model of visual selection According to the model conceived by Desimone and Duncan [7], the neural representations of different objects in a visual scene compete against each other in a mutually inhibitory fashion to gain access to higher level processing and awareness. Visual attention is conceived of as an emergent property that results from distributed activity in different brain systems working on the same stimulus. The activation of relevant object features in WM biases neural activity in specific brain regions that encode those particular features so that the object in question ‘wins’ to become the ‘focus’ of attention. Different mechanisms can help to resolve competition and bias selection, including purely ‘bottom-up’ stimulus-driven influences and also topdown sources (i.e. active memory) that identify objects of particular behavioural significance. Initial neurophysiological evidence for the model came from single-cell studies in the monkey. Chelazzi et al. [50] recorded the activity of neurons in the inferior temporal cortex (IT) while a monkey performed a visual search task. At the beginning of each trial, a memory cue was displayed at fixation. After a delay period a search array with several objects appeared, one of them matching the target. The task was to make an eye movement towards the location of the target. The firing rate of IT neurons selective to a particular feature increased when that feature appeared as the memory cue and it was sustained during the delay before the onset of the search array, indicating a neural basis for top-down expectancies associated with the crucial target for selection. With the onset of the search array, the pre-activated cells increased their firing rate to stimuli in the array for which they were selective (relative to the cells that were not pre-activated) and before the execution of the saccade to the target object, the neural responses for irrelevant objects were suppressed. IT neurons then responded as if the target object was alone in the search array. According to the biased competition model of visual selection, top-down control signals from the contents of WM biased selection in favour of the object whose features were preactivated from WM.
result also holds for search along the size dimension [15,16]. These findings are consonant with the idea of a target template in WM ‘tuning’ feature values in early vision.
Trends in Cognitive Sciences
Vol.12 No.9
The automatic influence of WM on search In most studies of WM and attention, the WM content overlaps with the crucial target of attention so that a match between a stimulus and the contents of WM facilitates performance. Under these conditions there is a strategic incentive for participants to use WM to guide visual selection. However, as we have noted, WM might also influence selection automatically, even if a match between a stimulus and the contents of WM is disruptive for the task. Pashler and Shiu [17] provided initial evidence that mental images seem to be involuntarily detected when they reappear within a rapid serial visual presentation. The automatic influence of WM on visual selection in search can be demonstrated when the contents of WM and the target for subsequent selection are varied independently (Figure 2). Downing [18] had participants hold an object in WM before the appearance of a two-object search display. The task was to discriminate a target that fell on one of the two objects in the search display, one of which matched the item in WM. Discrimination responses were faster when targets fell on an object matching the stimulus in WM. Because the WM stimulus was not predictive of the location of the target, the data indicate that WM can guide attention automatically. Subsequent work by Soto and colleagues [19,20] has demonstrated that stimuli held in WM can affect the direction of the first saccade in search, the fastest reaction times (RT) and the detection of salient ‘pop-out’ targets that generate a flat search slope, even when the WM stimulus never matches the search target. The WM effect does not seem to depend on the time interval between memory and search displays, arising with asynchronies ranging from 200 ms to 4000 ms [21]. This last finding indicates that the effect is not limited to cases in which information is being consolidated in WM, but occurs even when consolidation in WM has taken place.
Figure 1. Example of search displays for categorically steep and non-categorically steep orientation target displays. (a) In the categorically steep condition the target was the only steep item tilted 108 left with shallow 708 left and 508 right oriented distractors. (b) For the non-categorical condition the target was a steep 208 right, and the distractors were shallow lines 808 right and also steep lines 408 left. (c) A word cue before each trial indicated the identity of the target in the known condition. For the unknown condition, participants identified the target as the odd-item out in the display. There was only a reaction time (RT) advantage for the categorical targets over noncategorical orientation targets in the known condition. This advantage was eliminated for the unknown condition. Reprinted, with permission, from Ref. [14].
343
Review
Trends in Cognitive Sciences Vol.12 No.9
Figure 2. Example of the display sequences used in studies of inadvertent WM effects on selection. (a) The observers are presented with a cue (to hold in WM or to merely attend) and, subsequently, the task is to search for a tilted line target. In the valid condition, the target line is surrounded by an object matching the cue. In the invalid condition, the pre-cued object again re-appears but this time it contains a distracter. In the neutral condition, the memory item does not re-appear in the search display. (b) A typical pattern of search performance (the data here are taken from a condition in which the search display contained just two items). Search is faster for targets displayed at the location of the object matching the contents of WM relative to targets presented at the location of a non-matching object. Reproduced, with permission, from Ref. [45]. Copyright ß (2007) National Academy of Sciences (www.nasonline.org). (c) Data showing that the WM effect on selection is present even when the cue always contains a distracter when it re-appears in the search display (search with a display size of four items is illustrated (18). Note that effects are greatly reduced when the cue is merely attended but not held in WM for a later report. Reproduced, with permission, from Experiments 1 and 3 in Ref. [26]. Copyright ß (2007) American Psychological Association (www.apa.org).
Importantly, these inadvertent effects of WM on search are absent when observers are merely exposed to the cue but do not have to commit it to memory for later report [18–21]. Therefore, the effects cannot be because of an implicit memory trace arising from the mere repetition of the stimulus, a conclusion supported by imaging studies (see later). These results indicate that WM exerts an involuntary influence on visual attention.
have also been established. The effect of a WM distractor on search (Figure 2) is effective even when the WM stimulus is encoded verbally [26]. This effect can also be moderated by the semantic relationship between an item held in WM and a distractor in a subsequent search display [27,28]. These studies demonstrate that selection can be captured through automatic links between visual stimuli and different forms of representation in WM.
Visual or conceptual guidance? Interestingly, attentional guidance does not seem contingent on a visual representation being held in WM. A seminal study by Potter [22] demonstrated that the recognition of a visual stimulus in a rapid visual stream presentation can be facilitated just as much by advance verbal cueing of its meaning as by visual cueing of its appearance. Additional work showed how spatial orienting and selective processing of targets can be influenced by semantic relationships between central and peripheral word stimuli [23–25]. The involuntary effects of verbal cueing from WM
Improving visual awareness through the contents of WM One by-product of influencing where visual attention is allocated is that WM can modulate the entry of information into awareness. Parietal patients can suffer visual extinction, in which they are impaired at detecting the presence of a contralesional stimulus when an ipsilesional items appears simultaneously [29]. Such patients can show enhanced awareness for contralesional targets when they match the contents of WM [30]. Moreover, research on ‘inattentional blindness’ (the failure to notice the presence
344
Review
Trends in Cognitive Sciences
Vol.12 No.9
Figure 3. Illustration of the effects of cognitive load on the guidance of attention by WM. The paradigm used was identical to that depicted in Figure 2, except that, in this study [37], an articulatory suppression task was required while the WM stimulus was presented (i.e. verbalising two numbers through the trial). Two aspects of the results are noteworthy. First, when articulary suppression was required, attentional capture from the memory item was reduced at the shorter inter stimulus intervals (ISIs) (Figure 2c versus Figure 3a,b). Second, the WM effect even at the longest ISI was abolished when the WM load was increased from one (a) to two (b) items. Reprinted, with permission, from Ref. [41]. Copyright ß (2008) Psychomonics Society (http://www.nasonline.org/).
of unexpected stimuli) has shown that awareness of unexpected information improves when an unexpected stimulus is either visually or semantically related to the behaviourally relevant stimulus in WM [31,32]. Whether such effects are best conceptualized in terms of a change in perceptual sensitivity or in the criterion adopted for a response is a question that remains open (cf. Refs. [33,34] versus Ref. [35]). The boundaries of WM guidance The evidence reviewed so far demonstrates a strong influence of the contents of WM on selection. However, there also seem to be boundary conditions on this effect. For example, Downing and Dodds [36] failed to find WM effects when two items were held in WM. Others have found only weak [37] or no effects [38] when the target for search changes across trials. When the target for search varies across trials, more resources might be devoted to the template for this item than when the target is constant across trials, highlighting this item over other information held in WM. Woodman et al. [39] examined general effects of WM load on search with non-overlapping WM and search items. They found minimal effect of WM load when the search target was constant across trials, but that high WM load interfered when the search target varied. This fits with the idea that, to maintain good search performance with a varying target, WM resources have to be devoted to highlighting the template of the target; search performance suffers when these resources decrease in most cases. Oberauer [40], for example, has argued that although WM might hold up to four or so items for direct access from the outside environment, only one can be actively attended at a time. The consequence of actively attending to the search template might be to ‘compartmentalize’ WM, so that information in WM that is irrelevant to the immediate task might have less effect over the deployment of visual attention. Soto and Humphreys [41] also found that attentional guidance from irrelevant information in WM decreased as the WM load increased (Figure 3), a manip-
ulation which might also have the effect of reducing the resources devoted to each irrelevant item in WM. The evidence that memory guidance decreases when the capacity of WM is stressed is interesting because it also goes against any ‘strategic’ account of the data when attentional guidance does occur. According to this strategic account, the WM effects on visual selection are because of participants attending strategically to any re-appearance of the WM item in the search display to improve their WM. However, such a strategy should come into play particularly in cases in which WM is loaded and a higher decay of the WM representations is more likely. This is clearly not the case. Instead the data indicate that WM has an automatic influence on selection, but one that remains dependent on the availability of resources. Strikingly, some studies reported that with a heavy load in WM (i.e. articulatory suppression plus several memory items) observers seem to ignore a WM matching item that re-appears as a distractor during search [36,42] because, in these studies, search was more efficient in the presence of a WM matching distracter compared to a neutral baseline. We argue that high cognitive load might automatically lead to a degradation of the items held in WM, as a result of increased inter-item competition for limited resources. When representations are impoverished by competition in WM (under high load conditions), any strategy to ignore items that match stimuli in WM might be helped because degraded representations might be more easily inhibited. The net result would be that search might be guided away from stimuli that match the WM contents. Future research needs to explore the role of cognitive load on attentional guidance from WM in more detail (Box 2). There is evidence that the ability to compartmentalize WM, to keep separate irrelevant information from any template for search, is dependent on the prefrontal cortex (Box 3). Patients with prefrontal lesions are impaired particularly when the item they select turns out to be the memory item rather than the search target [43], indicating a failure in prioritizing information matching the 345
Review
Trends in Cognitive Sciences Vol.12 No.9
Box 2. Questions for future research To date, the effects of the contents of WM on attention have been studied within the visual modality. Future research should investigate whether similar WM effects happen in different modalities. Does the maintenance of a sound in WM attract our attention in auditory space? Additionally, are there cross-modal links between WM and selection. Does the maintenance of an auditory stimulus (’the sound of a sea gull’) attract attention to the equivalent visual stimulus (an image of a gull) or even that of a related item (i.e. a ship or a fish)? Moreover, are any crossmodal effects based on a single neuronal mechanism (i.e. neuronal enhancement to WM matching items), or are any effects based on different brain systems that are specific to each sensory modality? Although there is some evidence that the WM contents can impinge early stages of perceptual processing [35], more studies are needed to assess whether, when and how WM influences perceptual processing (e.g. measuring contrast sensitivity for targets, orientation sensitivity, spatial frequency discriminationthresholds, and temporal order judgments).
main task goal (search) over other representations in WM. Neurophysiological studies also show that cells in the prefrontal cortex show selective responses to a stimulus matching an item in WM under delayed match-to-sample conditions, but, in contrast to cells in more posterior regions, prefrontal neurons are insensitive to distractors appearing during the delay [44]. Thus, prefrontal cells are involved in prioritizing the relevant goals for tasks. Neural substrates of attentional guidance by WM Neurons in prefrontal regions, along with cells in more posterior neural regions tuned to particular features of
Most research on the interaction between WM and attention has been carried out with stimuli presented at a supra-threshold level. Future research will assess whether matches between the contents of WM and the stimulus array can operate at levels below the perceptual threshold. Can the WM content enhance the detection of visual signals which, by themselves, cannot be consciously identified? More research is needed to understand the effect of cognitive load on the interaction between WM and attention. For example, it remains to be explained why the effects of WM guidance are reduced or even reversed as the level of cognitive load increases [41,42]? Are there different effects of items in WM depending on the time available to consolidate that information in memory? Also, does altering the perceptual load in a display have the same effect as varying the cognitive load (cf. Ref. [5])? Future studies also need to examine the ecological constraints of the influence of WM on selection, for example, assessing effects in more real life environments. Do effects of WM guidance emerge even as the complexity of the environment increases? Can the automatic capture of attention by WM lead to some of the action errors that can occur in everyday situations?
stimuli, can both show sustained activity when stimuli are held in WM [44]. The sustained enhancement of cells tuned to particular features might provide the neural correlate of expectancies that influence subsequent selection (Box 1), leading to enhanced responding when the item in memory is re-presented in a search display. In the neurophysiological studies, the content of WM has typically overlapped with the crucial target for the subsequent selection task. It is possible, then, that enhanced neuronal responses for the memorized item reflect the behavioural relevance of the stimulus rather than the match between the contents of WM and any
Box 3. A neuropsychological study on the role of the frontal cortex Soto, Humphreys and Heinke [43] tested neuropsychological patients with lesions to the prefrontal cortex in the combined WM and attention paradigm depicted in Figure 1 (see main text). The task required participants to maintain an object in WM for a later report and to make an eye movement to a tilted line target in the subsequent search array. Overall, it took longer to make a saccade to the tilted target when the WM item matched a distracter (on invalid trials) rather than the tilted target (on valid trials). The prefrontal patients were particularly impaired at rejecting distracters that matched items held in WM. Figure I illustrates these findings. The
results indicate that, normally, prefrontal brain regions help to keep separate (at least to some degree) information in WM and information about the target for search. The prefrontal patients seemed to have problems in doing this. As a consequence, the patients had difficulties in rejecting WM items that were selected. This problem, in partitioning WM into task relevant and irrelevant components, could also explain why prefrontal patients might often have problems under dual-task conditions because such conditions often require that relevant and irrelevant representations must be kept separate in WM.
Figure I. Mean reaction times (RTs) for a fixation to be made to the target for each group as a function of the validity of the WM item. Although the normals were slower at fixating the target on invalid than on valid trials, this validity effect was exacerbated for the prefrontal patients. Reprinted, with permission, from Ref. [43].
346
Review
Trends in Cognitive Sciences
Vol.12 No.9
Figure 4. Illustration of the two networks involved in WM guidance of visual selection (see Figure 2 for an example procedure). (a) A first network was sensitive to the reappearance of the WM cue in the search display (activated regions: superior frontal gyrus, parahippocampal gyrus and lingual gyrus). Importantly, there were different neural responses in the WM condition and in a condition in which the cue was ‘merely repeated’ in the search display. When the task required active maintenance of the cue in WM, the re-appearance of the cue in the search display enhanced activity in the crucial brain areas. By contrast, when the initial cue had to be attended but not actively memorized, the same regions showed decreased responses to the re-appearance of the cue in the search display (measured relative to the neutral condition, when the cue did not re-appear in the search display). (b) A second network involved frontal (BA10, dorsolateral prefrontal cortex), thalamic and earlier visual regions (fusiform gyrus). Regions in this network show a differential response exclusively linked to WM. Relative to the neutral conditions (cue did not re-appear in the search display), activity increased when the WM and search goals coincided (valid trials) and decreased when the WM and search target were at different locations (invalid trials). Reproduced, with permission, from Ref. [45]. Copyright ß (2007) National Academy of Sciences (www.nasonline.org).
search display. To assess automatic attentional guidance from WM, a recent functional magnetic resonance imaging investigation [45] used independent WM and search stimuli (Figure 1). When the cue held in WM re-appeared in the search display, there was enhanced activity in a variety of brain regions known to be sensitive to the prior history of events (the superior frontal gyrus, mid-temporal and occipital areas) [46]. By contrast, when the cue had only to be identified but not held in WM, repetition of the cue elicited a suppressed response in the same regions. This repetition suppression effect might reflect enhanced perceptual processing of repeated items. We argue that the enhanced response, under WM conditions, might signal increased attentional interest to a stimulus that matches the content of WM. In addition, a second fronto-thalamic network was uniquely sensitive to the match between the contents of WM and the goal of the search task, showing increased activity on valid and decreased activity on invalid trials (Figure 4). This network might provide a strong signal to shift attention when the search goal and the WM stimulus coincide, whereas any attention shift might be inhibited when the WM stimulus and the search target are non-coincident and compete for selection.
The existence of a fronto-thalamic network involved in selection from WM might complement the role of a frontoparietal system involved in the control of spatial attention and spatial working memory [47,48], and might account for the effects of WM on visual selection in patients showing extinction after parietal lesions [30]. For example, the fronto-thalamic system might provide top-down support for earlier visual processes, by-passing any detrimental effects of the parietal lesion in these patients. Concluding remarks The research reviewed in this article stresses the importance of top-down processes from WM in modulating the deployment of attention and the gating of information into awareness. Our proposals are consonant with a reversed hierarchy model of visual processing, in which perceptual selection and visual awareness are achieved by re-entrant activation from higher-order areas (i.e. prefrontal cortex) to early visual areas [49]. Our evidence indicates that this re-entrant activation occurs automatically but is dependent on the availability of processing resources in crucial brain regions. Although the evidence indicates that WM guidance can sometimes be detrimental to task perform347
Review ance under most circumstances, this will not always be the case. Selection of visual information relevant to our immediate behavioural goals will benefit from matches to the contents of WM. The inadvertent effects that arise, when WM mis-cues attention, might reflect the way the system is ‘wired’ to gain benefits in the more usual cases in which the target for attention is represented in WM. Acknowledgements This work was supported by grants from the British Academy, the BBSRC, the MRC and the ESRC/MRC (UK). We thank Todd Horowitz, Geoff Woodman and one anonymous referee for comments on a prior draft of this paper.
References 1 Theeuwes, J. et al. (2003) Attentional and oculomotor capture with static singletons. Percept. Psychophys. 65, 735–746 2 Duncan, J. and Humphreys, G.W. (1989) Visual search and stimulus similarity. Psychol. Rev. 96, 433–458 3 Wolfe, J.M. (1994) Guided search 2.0: a revised model of visual search. Psychon. Bull. Rev. 1, 202–238 4 Bundesen, C. (1990) A theory of visual attention. Psychol. Rev. 97, 523– 547 5 Lavie, N. (2005) Distracted and confused?: selective attention under load. Trends Cogn. Sci. 9, 75–82 6 Forster, S. and Lavie, N. (2007) High perceptual load makes everybody equal: eliminating individual differences in distractibility with load. Psychol. Sci. 18, 377–382 7 Desimone, R. and Duncan, J. (1995) Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18, 193–222 8 Folk, C.L. et al. (1992) Involuntary covert orienting is contingent on attentional control settings. J. Exp. Psychol. Hum. Percept. Perform. 18, 1030–1044 9 Bacon, W.F. and Egeth, H.E. (1994) Overriding stimulus driven attentional capture. Percept. Psychophys. 55, 485–498 10 Leber, A.B. and Egeth, H.E. (2006) It’s under control: top-down search strategies can override attentional capture. Psychon. Bull. Rev. 13, 132–138 11 Lamy, D. et al. (2004) Effects of stimulus-driven salience within feature search mode. J. Exp. Psychol. Hum Percept. Perform. 30, 1019–1031 12 Wolfe, J.M. et al. (1992) The role of categorization in visual search for orientation. J. Exp. Psychol. Hum. Percept. Perform. 18, 34–49 13 Bauer, B. et al. (1996) Visual search for colour targets that are or are not linearly separable from distractors. Vision Res. 36, 1439–1465 14 Hodsoll, J.P. and Humphreys, G.W. (2005) The effect of target foreknowledge on visual search for categorically separable orientation targets. Vision Res. 45, 2346–2351 15 Hodsoll, J.P. et al. (2006) Dissociating the effects of similarity, salience, and top-down processes in search for linearly separable size targets. Percept. Psychophys. 68, 558–570 16 Hodsoll, J. and Humphreys, G.W. (2001) Driving attention with the top down: the relative contribution of target templates to the linear separability effect in the size dimension. Percept. Psychophys. 63, 918–926 17 Pashler, H. and Shiu, L.P. (1999) Do images trigger search? A test of Pillsbury’s hypothesis. Psychon. Bull. Rev. 6, 445–448 18 Downing, P.E. (2000) Interactions between visual working memory and selective attention. Psychol. Sci. 11, 467–473 19 Soto, D. et al. (2005) Early, involuntary top-down guidance of attention from working memory. J. Exp. Psychol. Hum. Percept. Perform. 31, 248–261 20 Soto, D. et al. (2006) Working memory can guide pop-out search. Vision Res. 46, 1010–1018 21 Olivers, C.N. et al. (2006) Feature-based memory-driven attentional capture: visual working memory content affects visual attention. J. Exp. Psychol. Hum. Percept. Perform. 32, 1243–1265 22 Potter, M.C. (1975) Meaning in visual search. Science 187, 965–966 23 Dark, V.J. et al. (1996) Semantic and spatial components of selective attention. J. Exp. Psychol. Hum. Percept. Perform. 22, 63–81
348
Trends in Cognitive Sciences Vol.12 No.9 24 Stolz, J.A. (1996) Exogenous orienting does not reflect an encapsulated set of processes. J. Exp. Psychol. Hum. Percept. Perform. 22, 187–201 25 Stolz, J.A. (1999) Word recognition and temporal order judgments: semantics turns back the clock. Can. J. Exp. Psychol. 53, 316–322 26 Soto, D. and Humphreys, G.W. (2007) Automatic guidance of visual attention from verbal working memory. J. Exp. Psychol. Hum. Percept. Perform. 33, 730–737 27 Huang, L. and Pashler, H. (2007) Working memory and the guidance of visual attention: consonance-driven orienting. Psychon. Bull. Rev. 14, 148–153 28 Moores, E. et al. (2003) Associative knowledge controls deployment of visual selective attention. Nat. Neurosci. 6, 182–189 29 Karnath, H-O. (1988) Deficits of attention in acute and recovered visual hemi-neglect. Neuropsychologia 26, 27–43 30 Soto, D. and Humphreys, G.W. (2006) Seeing the content of the mind: enhanced awareness through working memory in patients with visual extinction. Proc. Natl. Acad. Sci. U. S. A. 103, 4789–4792 31 Koivisto, M. and Revonsuo, A. (2007) How meaning shapes seeing. Psychol. Sci. 18, 845–849 32 Most, S.B. et al. (2005) What you see is what you set: sustained inattentional blindness and the capture of awareness. Psychol. Rev. 112, 217–242 33 Farah, M.J. (1989) Mechanisms of imagery-perception interaction. J. Exp. Psychol. Hum. Percept. Perform. 15, 203–211 34 Theeuwes, J. and Van der Burg, E. (2007) The role of spatial and nonspatial information in visual selection. J. Exp. Psychol. Hum. Percept. Perform. 33, 1335–1351 35 Vierck, E. and Miller, J. (2008) Precuing benefits for color and location in a visual search task. Percept. Psychophys. 70, 365–373 36 Downing, P.E. and Dodds, C.M. (2004) Competition in visual working memory for control of search. Vis. Cogn. 11, 689–703 37 Houtkamp, R. and Roelfsema, P.R. (2006) The effect of Items in working memory on the deployment of attention and the eyes during visual search. J. Exp. Psychol. Hum. Percept. Perform. 32, 423–442 38 Olivers, C.N. What drives memory-driven attentional capture? The effects of memory type, display type, and search type. J. Exp. Psychol. Hum. Percept. Perform. (in press) 39 Woodman, G.F. et al. (2007) The role of working memory representations in the control of attention. Cereb. Cortex 17, 118–124 40 Oberauer, K. (2002) Access to information in working memory: exploring the focus of attention. J. Exp. Psychol. Learn. Mem. Cogn. 28, 411–421 41 Soto, D. and Humphreys, G.W. (2008) Stressing the mind: the effect of verbal suppression and cognitive load on the time course of attentional guidance from working memory. Percept. Psychophys. 70, 924–934 42 Woodman, G.F. and Luck, S.J. (2007) Do the contents of visual working memory automatically influence attentional selection during visual search? J. Exp. Psychol. Hum. Percept. Perform. 33, 363–377 43 Soto, D. et al. (2006) Dividing the mind: the necessary role of the frontal lobes in separating memory from search. Neuropsychologia 44, 1282– 1289 44 Desimone, R. (1996) Neural mechanisms for visual memory and their role in attention. Proc. Natl. Acad. Sci. U. S. A. 93, 13494–13499 45 Soto, D. et al. (2007) Dissociating the neural mechanisms of memorybased guidance of visual selection. Proc. Natl. Acad. Sci. U. S. A. 104, 17186–17191 46 Brown, M.W. and Xiang, J.Z. (1998) Recognition memory: neuronal substrates of the judgement of prior occurrence. Prog. Neurobiol. 55, 149–189 47 Thiebaut de Schotten, M. et al. (2005) Direct evidence for a parietalfrontal pathway subserving spatial awareness in humans. Science 309, 2226–2228 48 Corbetta, M. and Shulman, G.L. (2002) Control of goal-directed and stimulus- driven attention in the brain. Nat. Rev. Neurosci. 3, 201–215 49 Ahissar, M. and Hochstein, S. (2004) The reverse hierarchy theory of visual perceptual learning. Trends Cogn. Sci. 8, 457–464 50 Chelazzi, L. et al. (1993) A neural basis for visual search in inferior temporal cortex. Nature 363, 345–347
Automatic guidance of attention from working memory David Soto1,2, John Hodsoll2, Pia Rotshtein2 and Glyn W. Humphreys2 1
Imperial College London, Faculty of Medicine, Division of Neuroscience and Mental Health, Charing Cross Campus, St. Dunstan’s Road, London, W6 8RP, UK 2 Behavioural Brain Sciences Centre, School of Psychology, University of Birmingham, Birmingham, B15 2TT, UK
Recent research has shown interactions between the process of keeping information ‘online’ in working memory, and the processes that select relevant information for a response. In particular, our ability to select stimuli in the environment can be modulated by whether the stimuli match the current contents of working memory. Guidance of selection from working memory occurs automatically, even when it is detrimental to performance. Neurophysiological data, from functional brain imaging, indicate that the interaction between working memory and attention is based on neuronal mechanisms distinct from the processes mediating ‘bottom-up’ priming effects from implicit memory. We discuss the importance of ‘top-down’ influences from working memory on the ‘early’ deployment of attention and on the processes that gate visual information into awareness. Introduction To survive in a complex and dynamically changing world, our attention needs to be guided to select stimuli that are relevant to our behavioural goals. It has been known for some time that stimulus-driven factors can guide attention in a relatively automatic way [1] such as when the sudden appearance of an unexpected object draws attention without any intention on the part of the observer. However, if we are not at the mercy of such changes, attention needs to be directed in a ‘top-down’ manner. In many cases, the topdown directing of attention will be based on information held in working memory (WM) [2–4]. For example, our search for a favourite shirt in the wardrobe might be guided by an on-line memory of the features of the relevant object (e.g. its texture and colour). This linkage between WM and attention might also sometimes have unwanted consequences – for example, if it turns out that the information we are holding in WM is irrelevant or even misleading for search. Although perhaps disruptive in everyday life, these unwanted consequences can be scientifically useful for studying the automaticity of any coupling between WM and attention. In this article we review emerging evidence on this automatic coupling, documenting both the behavioural characteristics of the interaction and the underlying neural circuits. Our discussion is confined to visual selection, although it is an open question as to whether a similar coupling exists in other sensory modalities.
Corresponding author: Soto, D. ([email protected]).
342
Common resources for WM and visual selection One reason for arguing for a close coupling between WM and visual selection is that a common pool of resources seems to subserve both processes. For example, Lavie and colleagues [5] have demonstrated that our ability to filter out irrelevant stimuli during selection depends on the processing load in WM. As the WM load increases, so fewer resources seem to be available to support efficient target selection and distractor rejection. The net result of this is that interference from distractors increases under conditions of high WM load [5]. This contrasts with conditions in which the perceptual load (e.g. the complexity of a visual display) is increased and distractor interference can decrease [6], indicating the selective linkage of WM resources with a stage of selecting visual information for response. Effects of the contents of WM on the deployment of attention in search The biased competition model of attention proposes that stimuli compete for selection at multiple levels of representation, with the ‘winner’ gaining control of both perceptual and response systems [7] (Box 1). Within this framework, WM acts to bias the competition for attention to favour objects that fit the goals of the task. Behavioural studies are consistent with this, with visual selection being strongly modulated by information in WM set up for a particular search target. For example, if we are looking for a red target, inadvertent capture of attention by unexpected salient stimuli can be enhanced when the irrelevant salient item shares the defining properties of the target [8–11]. The effects of WM seem to modulate relatively early visual processing. In visual search, targets can be detected efficiently if their feature values are categorically different from those of the distractors (e.g. the target is steep and the distractors are shallow in orientation) [12], or if the target is defined at the end of a continuum of feature values (e.g. if there is a large target and small and medium distractors). This evidence has also been used to define the nature of the basic features subserving visual processing (i.e. colour) [13]. Interestingly, these effects are contingent on participants holding the feature values in WM. Hodsoll and Humphreys [14] showed that the effects of categorical differences in orientation search were dependent on holding a memory ‘template’ of the target feature (Figure 1). This
1364-6613/$ – see front matter ß 2008 Elsevier Ltd. All rights reserved. doi:10.1016/j.tics.2008.05.007 Available online 5 August 2008
Review Box 1. The biased competition model of visual selection According to the model conceived by Desimone and Duncan [7], the neural representations of different objects in a visual scene compete against each other in a mutually inhibitory fashion to gain access to higher level processing and awareness. Visual attention is conceived of as an emergent property that results from distributed activity in different brain systems working on the same stimulus. The activation of relevant object features in WM biases neural activity in specific brain regions that encode those particular features so that the object in question ‘wins’ to become the ‘focus’ of attention. Different mechanisms can help to resolve competition and bias selection, including purely ‘bottom-up’ stimulus-driven influences and also topdown sources (i.e. active memory) that identify objects of particular behavioural significance. Initial neurophysiological evidence for the model came from single-cell studies in the monkey. Chelazzi et al. [50] recorded the activity of neurons in the inferior temporal cortex (IT) while a monkey performed a visual search task. At the beginning of each trial, a memory cue was displayed at fixation. After a delay period a search array with several objects appeared, one of them matching the target. The task was to make an eye movement towards the location of the target. The firing rate of IT neurons selective to a particular feature increased when that feature appeared as the memory cue and it was sustained during the delay before the onset of the search array, indicating a neural basis for top-down expectancies associated with the crucial target for selection. With the onset of the search array, the pre-activated cells increased their firing rate to stimuli in the array for which they were selective (relative to the cells that were not pre-activated) and before the execution of the saccade to the target object, the neural responses for irrelevant objects were suppressed. IT neurons then responded as if the target object was alone in the search array. According to the biased competition model of visual selection, top-down control signals from the contents of WM biased selection in favour of the object whose features were preactivated from WM.
result also holds for search along the size dimension [15,16]. These findings are consonant with the idea of a target template in WM ‘tuning’ feature values in early vision.
Trends in Cognitive Sciences
Vol.12 No.9
The automatic influence of WM on search In most studies of WM and attention, the WM content overlaps with the crucial target of attention so that a match between a stimulus and the contents of WM facilitates performance. Under these conditions there is a strategic incentive for participants to use WM to guide visual selection. However, as we have noted, WM might also influence selection automatically, even if a match between a stimulus and the contents of WM is disruptive for the task. Pashler and Shiu [17] provided initial evidence that mental images seem to be involuntarily detected when they reappear within a rapid serial visual presentation. The automatic influence of WM on visual selection in search can be demonstrated when the contents of WM and the target for subsequent selection are varied independently (Figure 2). Downing [18] had participants hold an object in WM before the appearance of a two-object search display. The task was to discriminate a target that fell on one of the two objects in the search display, one of which matched the item in WM. Discrimination responses were faster when targets fell on an object matching the stimulus in WM. Because the WM stimulus was not predictive of the location of the target, the data indicate that WM can guide attention automatically. Subsequent work by Soto and colleagues [19,20] has demonstrated that stimuli held in WM can affect the direction of the first saccade in search, the fastest reaction times (RT) and the detection of salient ‘pop-out’ targets that generate a flat search slope, even when the WM stimulus never matches the search target. The WM effect does not seem to depend on the time interval between memory and search displays, arising with asynchronies ranging from 200 ms to 4000 ms [21]. This last finding indicates that the effect is not limited to cases in which information is being consolidated in WM, but occurs even when consolidation in WM has taken place.
Figure 1. Example of search displays for categorically steep and non-categorically steep orientation target displays. (a) In the categorically steep condition the target was the only steep item tilted 108 left with shallow 708 left and 508 right oriented distractors. (b) For the non-categorical condition the target was a steep 208 right, and the distractors were shallow lines 808 right and also steep lines 408 left. (c) A word cue before each trial indicated the identity of the target in the known condition. For the unknown condition, participants identified the target as the odd-item out in the display. There was only a reaction time (RT) advantage for the categorical targets over noncategorical orientation targets in the known condition. This advantage was eliminated for the unknown condition. Reprinted, with permission, from Ref. [14].
343
Review
Trends in Cognitive Sciences Vol.12 No.9
Figure 2. Example of the display sequences used in studies of inadvertent WM effects on selection. (a) The observers are presented with a cue (to hold in WM or to merely attend) and, subsequently, the task is to search for a tilted line target. In the valid condition, the target line is surrounded by an object matching the cue. In the invalid condition, the pre-cued object again re-appears but this time it contains a distracter. In the neutral condition, the memory item does not re-appear in the search display. (b) A typical pattern of search performance (the data here are taken from a condition in which the search display contained just two items). Search is faster for targets displayed at the location of the object matching the contents of WM relative to targets presented at the location of a non-matching object. Reproduced, with permission, from Ref. [45]. Copyright ß (2007) National Academy of Sciences (www.nasonline.org). (c) Data showing that the WM effect on selection is present even when the cue always contains a distracter when it re-appears in the search display (search with a display size of four items is illustrated (18). Note that effects are greatly reduced when the cue is merely attended but not held in WM for a later report. Reproduced, with permission, from Experiments 1 and 3 in Ref. [26]. Copyright ß (2007) American Psychological Association (www.apa.org).
Importantly, these inadvertent effects of WM on search are absent when observers are merely exposed to the cue but do not have to commit it to memory for later report [18–21]. Therefore, the effects cannot be because of an implicit memory trace arising from the mere repetition of the stimulus, a conclusion supported by imaging studies (see later). These results indicate that WM exerts an involuntary influence on visual attention.
have also been established. The effect of a WM distractor on search (Figure 2) is effective even when the WM stimulus is encoded verbally [26]. This effect can also be moderated by the semantic relationship between an item held in WM and a distractor in a subsequent search display [27,28]. These studies demonstrate that selection can be captured through automatic links between visual stimuli and different forms of representation in WM.
Visual or conceptual guidance? Interestingly, attentional guidance does not seem contingent on a visual representation being held in WM. A seminal study by Potter [22] demonstrated that the recognition of a visual stimulus in a rapid visual stream presentation can be facilitated just as much by advance verbal cueing of its meaning as by visual cueing of its appearance. Additional work showed how spatial orienting and selective processing of targets can be influenced by semantic relationships between central and peripheral word stimuli [23–25]. The involuntary effects of verbal cueing from WM
Improving visual awareness through the contents of WM One by-product of influencing where visual attention is allocated is that WM can modulate the entry of information into awareness. Parietal patients can suffer visual extinction, in which they are impaired at detecting the presence of a contralesional stimulus when an ipsilesional items appears simultaneously [29]. Such patients can show enhanced awareness for contralesional targets when they match the contents of WM [30]. Moreover, research on ‘inattentional blindness’ (the failure to notice the presence
344
Review
Trends in Cognitive Sciences
Vol.12 No.9
Figure 3. Illustration of the effects of cognitive load on the guidance of attention by WM. The paradigm used was identical to that depicted in Figure 2, except that, in this study [37], an articulatory suppression task was required while the WM stimulus was presented (i.e. verbalising two numbers through the trial). Two aspects of the results are noteworthy. First, when articulary suppression was required, attentional capture from the memory item was reduced at the shorter inter stimulus intervals (ISIs) (Figure 2c versus Figure 3a,b). Second, the WM effect even at the longest ISI was abolished when the WM load was increased from one (a) to two (b) items. Reprinted, with permission, from Ref. [41]. Copyright ß (2008) Psychomonics Society (http://www.nasonline.org/).
of unexpected stimuli) has shown that awareness of unexpected information improves when an unexpected stimulus is either visually or semantically related to the behaviourally relevant stimulus in WM [31,32]. Whether such effects are best conceptualized in terms of a change in perceptual sensitivity or in the criterion adopted for a response is a question that remains open (cf. Refs. [33,34] versus Ref. [35]). The boundaries of WM guidance The evidence reviewed so far demonstrates a strong influence of the contents of WM on selection. However, there also seem to be boundary conditions on this effect. For example, Downing and Dodds [36] failed to find WM effects when two items were held in WM. Others have found only weak [37] or no effects [38] when the target for search changes across trials. When the target for search varies across trials, more resources might be devoted to the template for this item than when the target is constant across trials, highlighting this item over other information held in WM. Woodman et al. [39] examined general effects of WM load on search with non-overlapping WM and search items. They found minimal effect of WM load when the search target was constant across trials, but that high WM load interfered when the search target varied. This fits with the idea that, to maintain good search performance with a varying target, WM resources have to be devoted to highlighting the template of the target; search performance suffers when these resources decrease in most cases. Oberauer [40], for example, has argued that although WM might hold up to four or so items for direct access from the outside environment, only one can be actively attended at a time. The consequence of actively attending to the search template might be to ‘compartmentalize’ WM, so that information in WM that is irrelevant to the immediate task might have less effect over the deployment of visual attention. Soto and Humphreys [41] also found that attentional guidance from irrelevant information in WM decreased as the WM load increased (Figure 3), a manip-
ulation which might also have the effect of reducing the resources devoted to each irrelevant item in WM. The evidence that memory guidance decreases when the capacity of WM is stressed is interesting because it also goes against any ‘strategic’ account of the data when attentional guidance does occur. According to this strategic account, the WM effects on visual selection are because of participants attending strategically to any re-appearance of the WM item in the search display to improve their WM. However, such a strategy should come into play particularly in cases in which WM is loaded and a higher decay of the WM representations is more likely. This is clearly not the case. Instead the data indicate that WM has an automatic influence on selection, but one that remains dependent on the availability of resources. Strikingly, some studies reported that with a heavy load in WM (i.e. articulatory suppression plus several memory items) observers seem to ignore a WM matching item that re-appears as a distractor during search [36,42] because, in these studies, search was more efficient in the presence of a WM matching distracter compared to a neutral baseline. We argue that high cognitive load might automatically lead to a degradation of the items held in WM, as a result of increased inter-item competition for limited resources. When representations are impoverished by competition in WM (under high load conditions), any strategy to ignore items that match stimuli in WM might be helped because degraded representations might be more easily inhibited. The net result would be that search might be guided away from stimuli that match the WM contents. Future research needs to explore the role of cognitive load on attentional guidance from WM in more detail (Box 2). There is evidence that the ability to compartmentalize WM, to keep separate irrelevant information from any template for search, is dependent on the prefrontal cortex (Box 3). Patients with prefrontal lesions are impaired particularly when the item they select turns out to be the memory item rather than the search target [43], indicating a failure in prioritizing information matching the 345
Review
Trends in Cognitive Sciences Vol.12 No.9
Box 2. Questions for future research To date, the effects of the contents of WM on attention have been studied within the visual modality. Future research should investigate whether similar WM effects happen in different modalities. Does the maintenance of a sound in WM attract our attention in auditory space? Additionally, are there cross-modal links between WM and selection. Does the maintenance of an auditory stimulus (’the sound of a sea gull’) attract attention to the equivalent visual stimulus (an image of a gull) or even that of a related item (i.e. a ship or a fish)? Moreover, are any crossmodal effects based on a single neuronal mechanism (i.e. neuronal enhancement to WM matching items), or are any effects based on different brain systems that are specific to each sensory modality? Although there is some evidence that the WM contents can impinge early stages of perceptual processing [35], more studies are needed to assess whether, when and how WM influences perceptual processing (e.g. measuring contrast sensitivity for targets, orientation sensitivity, spatial frequency discriminationthresholds, and temporal order judgments).
main task goal (search) over other representations in WM. Neurophysiological studies also show that cells in the prefrontal cortex show selective responses to a stimulus matching an item in WM under delayed match-to-sample conditions, but, in contrast to cells in more posterior regions, prefrontal neurons are insensitive to distractors appearing during the delay [44]. Thus, prefrontal cells are involved in prioritizing the relevant goals for tasks. Neural substrates of attentional guidance by WM Neurons in prefrontal regions, along with cells in more posterior neural regions tuned to particular features of
Most research on the interaction between WM and attention has been carried out with stimuli presented at a supra-threshold level. Future research will assess whether matches between the contents of WM and the stimulus array can operate at levels below the perceptual threshold. Can the WM content enhance the detection of visual signals which, by themselves, cannot be consciously identified? More research is needed to understand the effect of cognitive load on the interaction between WM and attention. For example, it remains to be explained why the effects of WM guidance are reduced or even reversed as the level of cognitive load increases [41,42]? Are there different effects of items in WM depending on the time available to consolidate that information in memory? Also, does altering the perceptual load in a display have the same effect as varying the cognitive load (cf. Ref. [5])? Future studies also need to examine the ecological constraints of the influence of WM on selection, for example, assessing effects in more real life environments. Do effects of WM guidance emerge even as the complexity of the environment increases? Can the automatic capture of attention by WM lead to some of the action errors that can occur in everyday situations?
stimuli, can both show sustained activity when stimuli are held in WM [44]. The sustained enhancement of cells tuned to particular features might provide the neural correlate of expectancies that influence subsequent selection (Box 1), leading to enhanced responding when the item in memory is re-presented in a search display. In the neurophysiological studies, the content of WM has typically overlapped with the crucial target for the subsequent selection task. It is possible, then, that enhanced neuronal responses for the memorized item reflect the behavioural relevance of the stimulus rather than the match between the contents of WM and any
Box 3. A neuropsychological study on the role of the frontal cortex Soto, Humphreys and Heinke [43] tested neuropsychological patients with lesions to the prefrontal cortex in the combined WM and attention paradigm depicted in Figure 1 (see main text). The task required participants to maintain an object in WM for a later report and to make an eye movement to a tilted line target in the subsequent search array. Overall, it took longer to make a saccade to the tilted target when the WM item matched a distracter (on invalid trials) rather than the tilted target (on valid trials). The prefrontal patients were particularly impaired at rejecting distracters that matched items held in WM. Figure I illustrates these findings. The
results indicate that, normally, prefrontal brain regions help to keep separate (at least to some degree) information in WM and information about the target for search. The prefrontal patients seemed to have problems in doing this. As a consequence, the patients had difficulties in rejecting WM items that were selected. This problem, in partitioning WM into task relevant and irrelevant components, could also explain why prefrontal patients might often have problems under dual-task conditions because such conditions often require that relevant and irrelevant representations must be kept separate in WM.
Figure I. Mean reaction times (RTs) for a fixation to be made to the target for each group as a function of the validity of the WM item. Although the normals were slower at fixating the target on invalid than on valid trials, this validity effect was exacerbated for the prefrontal patients. Reprinted, with permission, from Ref. [43].
346
Review
Trends in Cognitive Sciences
Vol.12 No.9
Figure 4. Illustration of the two networks involved in WM guidance of visual selection (see Figure 2 for an example procedure). (a) A first network was sensitive to the reappearance of the WM cue in the search display (activated regions: superior frontal gyrus, parahippocampal gyrus and lingual gyrus). Importantly, there were different neural responses in the WM condition and in a condition in which the cue was ‘merely repeated’ in the search display. When the task required active maintenance of the cue in WM, the re-appearance of the cue in the search display enhanced activity in the crucial brain areas. By contrast, when the initial cue had to be attended but not actively memorized, the same regions showed decreased responses to the re-appearance of the cue in the search display (measured relative to the neutral condition, when the cue did not re-appear in the search display). (b) A second network involved frontal (BA10, dorsolateral prefrontal cortex), thalamic and earlier visual regions (fusiform gyrus). Regions in this network show a differential response exclusively linked to WM. Relative to the neutral conditions (cue did not re-appear in the search display), activity increased when the WM and search goals coincided (valid trials) and decreased when the WM and search target were at different locations (invalid trials). Reproduced, with permission, from Ref. [45]. Copyright ß (2007) National Academy of Sciences (www.nasonline.org).
search display. To assess automatic attentional guidance from WM, a recent functional magnetic resonance imaging investigation [45] used independent WM and search stimuli (Figure 1). When the cue held in WM re-appeared in the search display, there was enhanced activity in a variety of brain regions known to be sensitive to the prior history of events (the superior frontal gyrus, mid-temporal and occipital areas) [46]. By contrast, when the cue had only to be identified but not held in WM, repetition of the cue elicited a suppressed response in the same regions. This repetition suppression effect might reflect enhanced perceptual processing of repeated items. We argue that the enhanced response, under WM conditions, might signal increased attentional interest to a stimulus that matches the content of WM. In addition, a second fronto-thalamic network was uniquely sensitive to the match between the contents of WM and the goal of the search task, showing increased activity on valid and decreased activity on invalid trials (Figure 4). This network might provide a strong signal to shift attention when the search goal and the WM stimulus coincide, whereas any attention shift might be inhibited when the WM stimulus and the search target are non-coincident and compete for selection.
The existence of a fronto-thalamic network involved in selection from WM might complement the role of a frontoparietal system involved in the control of spatial attention and spatial working memory [47,48], and might account for the effects of WM on visual selection in patients showing extinction after parietal lesions [30]. For example, the fronto-thalamic system might provide top-down support for earlier visual processes, by-passing any detrimental effects of the parietal lesion in these patients. Concluding remarks The research reviewed in this article stresses the importance of top-down processes from WM in modulating the deployment of attention and the gating of information into awareness. Our proposals are consonant with a reversed hierarchy model of visual processing, in which perceptual selection and visual awareness are achieved by re-entrant activation from higher-order areas (i.e. prefrontal cortex) to early visual areas [49]. Our evidence indicates that this re-entrant activation occurs automatically but is dependent on the availability of processing resources in crucial brain regions. Although the evidence indicates that WM guidance can sometimes be detrimental to task perform347
Review ance under most circumstances, this will not always be the case. Selection of visual information relevant to our immediate behavioural goals will benefit from matches to the contents of WM. The inadvertent effects that arise, when WM mis-cues attention, might reflect the way the system is ‘wired’ to gain benefits in the more usual cases in which the target for attention is represented in WM. Acknowledgements This work was supported by grants from the British Academy, the BBSRC, the MRC and the ESRC/MRC (UK). We thank Todd Horowitz, Geoff Woodman and one anonymous referee for comments on a prior draft of this paper.
References 1 Theeuwes, J. et al. (2003) Attentional and oculomotor capture with static singletons. Percept. Psychophys. 65, 735–746 2 Duncan, J. and Humphreys, G.W. (1989) Visual search and stimulus similarity. Psychol. Rev. 96, 433–458 3 Wolfe, J.M. (1994) Guided search 2.0: a revised model of visual search. Psychon. Bull. Rev. 1, 202–238 4 Bundesen, C. (1990) A theory of visual attention. Psychol. Rev. 97, 523– 547 5 Lavie, N. (2005) Distracted and confused?: selective attention under load. Trends Cogn. Sci. 9, 75–82 6 Forster, S. and Lavie, N. (2007) High perceptual load makes everybody equal: eliminating individual differences in distractibility with load. Psychol. Sci. 18, 377–382 7 Desimone, R. and Duncan, J. (1995) Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18, 193–222 8 Folk, C.L. et al. (1992) Involuntary covert orienting is contingent on attentional control settings. J. Exp. Psychol. Hum. Percept. Perform. 18, 1030–1044 9 Bacon, W.F. and Egeth, H.E. (1994) Overriding stimulus driven attentional capture. Percept. Psychophys. 55, 485–498 10 Leber, A.B. and Egeth, H.E. (2006) It’s under control: top-down search strategies can override attentional capture. Psychon. Bull. Rev. 13, 132–138 11 Lamy, D. et al. (2004) Effects of stimulus-driven salience within feature search mode. J. Exp. Psychol. Hum Percept. Perform. 30, 1019–1031 12 Wolfe, J.M. et al. (1992) The role of categorization in visual search for orientation. J. Exp. Psychol. Hum. Percept. Perform. 18, 34–49 13 Bauer, B. et al. (1996) Visual search for colour targets that are or are not linearly separable from distractors. Vision Res. 36, 1439–1465 14 Hodsoll, J.P. and Humphreys, G.W. (2005) The effect of target foreknowledge on visual search for categorically separable orientation targets. Vision Res. 45, 2346–2351 15 Hodsoll, J.P. et al. (2006) Dissociating the effects of similarity, salience, and top-down processes in search for linearly separable size targets. Percept. Psychophys. 68, 558–570 16 Hodsoll, J. and Humphreys, G.W. (2001) Driving attention with the top down: the relative contribution of target templates to the linear separability effect in the size dimension. Percept. Psychophys. 63, 918–926 17 Pashler, H. and Shiu, L.P. (1999) Do images trigger search? A test of Pillsbury’s hypothesis. Psychon. Bull. Rev. 6, 445–448 18 Downing, P.E. (2000) Interactions between visual working memory and selective attention. Psychol. Sci. 11, 467–473 19 Soto, D. et al. (2005) Early, involuntary top-down guidance of attention from working memory. J. Exp. Psychol. Hum. Percept. Perform. 31, 248–261 20 Soto, D. et al. (2006) Working memory can guide pop-out search. Vision Res. 46, 1010–1018 21 Olivers, C.N. et al. (2006) Feature-based memory-driven attentional capture: visual working memory content affects visual attention. J. Exp. Psychol. Hum. Percept. Perform. 32, 1243–1265 22 Potter, M.C. (1975) Meaning in visual search. Science 187, 965–966 23 Dark, V.J. et al. (1996) Semantic and spatial components of selective attention. J. Exp. Psychol. Hum. Percept. Perform. 22, 63–81
348
Trends in Cognitive Sciences Vol.12 No.9 24 Stolz, J.A. (1996) Exogenous orienting does not reflect an encapsulated set of processes. J. Exp. Psychol. Hum. Percept. Perform. 22, 187–201 25 Stolz, J.A. (1999) Word recognition and temporal order judgments: semantics turns back the clock. Can. J. Exp. Psychol. 53, 316–322 26 Soto, D. and Humphreys, G.W. (2007) Automatic guidance of visual attention from verbal working memory. J. Exp. Psychol. Hum. Percept. Perform. 33, 730–737 27 Huang, L. and Pashler, H. (2007) Working memory and the guidance of visual attention: consonance-driven orienting. Psychon. Bull. Rev. 14, 148–153 28 Moores, E. et al. (2003) Associative knowledge controls deployment of visual selective attention. Nat. Neurosci. 6, 182–189 29 Karnath, H-O. (1988) Deficits of attention in acute and recovered visual hemi-neglect. Neuropsychologia 26, 27–43 30 Soto, D. and Humphreys, G.W. (2006) Seeing the content of the mind: enhanced awareness through working memory in patients with visual extinction. Proc. Natl. Acad. Sci. U. S. A. 103, 4789–4792 31 Koivisto, M. and Revonsuo, A. (2007) How meaning shapes seeing. Psychol. Sci. 18, 845–849 32 Most, S.B. et al. (2005) What you see is what you set: sustained inattentional blindness and the capture of awareness. Psychol. Rev. 112, 217–242 33 Farah, M.J. (1989) Mechanisms of imagery-perception interaction. J. Exp. Psychol. Hum. Percept. Perform. 15, 203–211 34 Theeuwes, J. and Van der Burg, E. (2007) The role of spatial and nonspatial information in visual selection. J. Exp. Psychol. Hum. Percept. Perform. 33, 1335–1351 35 Vierck, E. and Miller, J. (2008) Precuing benefits for color and location in a visual search task. Percept. Psychophys. 70, 365–373 36 Downing, P.E. and Dodds, C.M. (2004) Competition in visual working memory for control of search. Vis. Cogn. 11, 689–703 37 Houtkamp, R. and Roelfsema, P.R. (2006) The effect of Items in working memory on the deployment of attention and the eyes during visual search. J. Exp. Psychol. Hum. Percept. Perform. 32, 423–442 38 Olivers, C.N. What drives memory-driven attentional capture? The effects of memory type, display type, and search type. J. Exp. Psychol. Hum. Percept. Perform. (in press) 39 Woodman, G.F. et al. (2007) The role of working memory representations in the control of attention. Cereb. Cortex 17, 118–124 40 Oberauer, K. (2002) Access to information in working memory: exploring the focus of attention. J. Exp. Psychol. Learn. Mem. Cogn. 28, 411–421 41 Soto, D. and Humphreys, G.W. (2008) Stressing the mind: the effect of verbal suppression and cognitive load on the time course of attentional guidance from working memory. Percept. Psychophys. 70, 924–934 42 Woodman, G.F. and Luck, S.J. (2007) Do the contents of visual working memory automatically influence attentional selection during visual search? J. Exp. Psychol. Hum. Percept. Perform. 33, 363–377 43 Soto, D. et al. (2006) Dividing the mind: the necessary role of the frontal lobes in separating memory from search. Neuropsychologia 44, 1282– 1289 44 Desimone, R. (1996) Neural mechanisms for visual memory and their role in attention. Proc. Natl. Acad. Sci. U. S. A. 93, 13494–13499 45 Soto, D. et al. (2007) Dissociating the neural mechanisms of memorybased guidance of visual selection. Proc. Natl. Acad. Sci. U. S. A. 104, 17186–17191 46 Brown, M.W. and Xiang, J.Z. (1998) Recognition memory: neuronal substrates of the judgement of prior occurrence. Prog. Neurobiol. 55, 149–189 47 Thiebaut de Schotten, M. et al. (2005) Direct evidence for a parietalfrontal pathway subserving spatial awareness in humans. Science 309, 2226–2228 48 Corbetta, M. and Shulman, G.L. (2002) Control of goal-directed and stimulus- driven attention in the brain. Nat. Rev. Neurosci. 3, 201–215 49 Ahissar, M. and Hochstein, S. (2004) The reverse hierarchy theory of visual perceptual learning. Trends Cogn. Sci. 8, 457–464 50 Chelazzi, L. et al. (1993) A neural basis for visual search in inferior temporal cortex. Nature 363, 345–347