Prior-entry: A review

Consciousness and Cognition 19 (2010) 364–379

Contents lists available at ScienceDirect

Consciousness and Cognition journal homepage: www.elsevier.com/locate/concog

Review

Prior-entry: A review Charles Spence a,*, Cesare Parise a,b a b

Crossmodal Research Laboratory, Department of Experimental Psychology, University of Oxford, United Kingdom Department of Cognitive and Education Science, University of Trento, Italy

a r t i c l e

i n f o

Article history: Received 13 September 2009 Available online 28 December 2009 Keywords: Prior entry Attention TOJ SJ Crossmodal Spatial

a b s t r a c t The law of prior entry was one of E.B. Titchener’s seven fundamental laws of attention. According to Titchener (1908, p. 251): ‘‘the object of attention comes to consciousness more quickly than the objects which we are not attending to.” Although researchers have been studying prior entry for more than a century now, progress in understanding the effect has been hindered by the many methodological confounds present in early research. As a consequence, it is unclear whether the behavioral effects reported in the majority of published studies in this area should be attributed to attention, decisional response biases, and/or, in the case of exogenous spatial cuing studies of the prior-entry effect, to sensory facilitation effects instead. In this article, the literature on the prior-entry effect is reviewed, the confounds present in previous research highlighted, current consensus summarized, and some of the key questions for future research outlined. In particular, recent research has now provided compelling psychophysical and electrophysiological evidence to support the claim that attending to a sensory modality, spatial location, or stimulus feature/attribute can all give rise to a relative speeding-up of the time of arrival of attended, as compared to relatively less attended (or unattended) stimuli. Prior-entry effects have now been demonstrated following both the endogenous and exogenous orienting of attention, though prior-entry effects tend to be smaller in magnitude when assessed by means of participants’ performance on SJ tasks than when assessed by means of their performance on TOJ tasks. Ó 2009 Elsevier Inc. All rights reserved.

1. Introduction Does attention speed-up perceptual processing? Or, in other words, does attending to (or expecting) a particular stimulus (or event) mean that it will be perceived earlier in time than if attention had been directed elsewhere? This seemingly simple question is in fact one of the oldest in the field of experimental psychology (Mollon & Perkins, 1996; Scharlau, 2007; see Spence, Shore, & Klein, 2001, for a review). However, while researchers have been investigating the topic of temporal perception in humans for more than two centuries, it is only in the last decade or so that convincing psychophysical evidence in support of the ‘prior-entry’ effect (as the phenomenon is known) has finally been obtained (see Shore & Spence, 2005). That said, there has been a recent resurgence of research interest in the prior-entry effect (e.g., Lester, Hecht, & Vecera, 2009; Weiss & Scharlau, 2009; West, Anderson, & Pratt, in press; Yates & Nicholls, 2009; Zhuang & Papathomas, 2009). What is more, the latest research utilizing event-related potentials (ERPs) has now started to demonstrate just how early in human information processing the effects of attention can be observed (McDonald, Teder-Sälejärvi, Di Russo, & Hillyard, 2005; Vibell, Klinge, Zampini, Spence, & Nobre, 2007, submitted for publication). * Corresponding author. Address: Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford, OX1 3UD, United Kingdom. Fax: +44 1865 310447. E-mail address: [email protected] (C. Spence). 1053-8100/$ - see front matter Ó 2009 Elsevier Inc. All rights reserved. doi:10.1016/j.concog.2009.12.001

C. Spence, C. Parise / Consciousness and Cognition 19 (2010) 364–379

365

The findings of research on the prior-entry effect are not only of interest to psychologists, psychophysicists, and cognitive neuroscientists, but are also of relevance to philosophers interested in the question of how time (at least the fine ‘millisecond’ timescale captured by studies of prior entry; see Buonomano & Karmarkar, 2002; Eagleman, 2008; Eagleman et al., 2005) is represented neurally (e.g., see Dennett & Kinsbourne, 1992; Durgin & Sternberg, 2002; Kelly, 2005; Mellor, 1985; Roache, 1999). Indeed, the latest research on the prior-entry effect has shown that information concerning temporal order is, to some extent, represented temporally in the brain (at least for the case of crossmodal temporal judgments; Köhler, 1947; Vibell et al., submitted for publication, 2007). In this article, we review the extensive empirical literature that has investigated the effects of attention (to a sensory modality or to a spatial location) on temporal perception (in both unimodal and multisensory settings) in humans.

2. Measuring the effect of attention on temporal perception The problem when investigating the effects of attention on temporal perception is that it is impossible for a person to index when exactly a given stimulus or event was perceived as occurring. Instead, researchers have had to rely on a person’s judgments of the relative timing of an event of interest with respect to another (comparison or marker) stimulus (see also Schneider & Bavelier, 2003). The two tasks that have been used most frequently to study the effects of attention on temporal perception in humans are the temporal order judgment (TOJ) task and the simultaneity judgment (SJ) task. For both tasks, the stimulus onset asynchrony (SOA) between the two to-be-judged stimuli on each trial is normally varied using the method of constant stimuli (e.g., Spence, Shore, & Klein, 2001), or else some form of adaptive procedure (e.g., Stelmach & Herdman, 1991; Sternberg, Knoll, & Gates, 1971; Zampini, Guest, Shore, & Spence, 2005). In a typical TOJ study, participants have to report which stimulus was presented first (or, on occasion, second; see Frey, 1990; Parise & Spence, 2008, 2009; Shore, Spence, & Klein, 2001), while in the SJ task, they have to report whether the two stimuli were presented simultaneously or not. Occasionally, these two tasks are combined into a so-called ternary-response task (Jas´kowski, 1993; Stelmach & Herdman, 1991; Stone, 1926; Ulrich, 1987; Zampini et al., 2007), in which the participants either report which stimulus was presented first, or else that the two stimuli appeared to have been presented simultaneously. While the ternary-response task might, at first, seem preferable in terms of providing a more nuanced measure of people’s perceptual experience, it should be noted that the task suffers from the problem that participants vary markedly in the criterion they set to choose the simultaneous response option when given three (rather than just two) response alternatives (e.g., see Schneider & Bavelier, 2003). This can make it rather difficult to compare the results of different studies (e.g., see Jas´kowski, 1993; and Stelmach & Herdman, 1991, on this point). Hence, the majority of prior entry studies published to date have tended to use TOJ or SJ tasks instead, although it should be noted that all three methods are subject to criterion effects of one sort or another. Two key performance indicators can be estimated by fitting psychometric (e.g., cumulative Gaussian) functions to the data from TOJ tasks: the first is the amount of time by which one stimulus has to precede (or follow) the other in order for the two stimuli to be perceived as simultaneous, known as the point of subjective simultaneity (PSS; note that this value actually reflects an estimate of the SOA at which participants would be likely to make each response equally often). The second is a participant’s sensitivity to the temporal order in which two stimuli were presented, known as the just noticeable difference (JND). The JND is defined as the delay between two stimuli needed for participants to perceive the correct order of presentation of the two stimuli at a specified level. In studies of the prior-entry effect, the JND has conventionally been calculated as half the temporal interval between the 25% and 75% points on the psychometric function. Similarly, when analyzing the data from a SJ task, researchers typically fit a simple Gaussian function to the simultaneous response data. The spread (or standard deviation, SD) of this distribution provides a measure of a participant’s sensitivity to asynchrony (equivalent to the JND) while the center of the distribution provides an estimate of the PSS. The data from the ternary-response task has been analyzed in different ways in different studies, but often involves some combination of the above techniques (e.g., Zampini et al., 2007). No matter what the task (i.e., TOJ, SJ, or ternary-response task), evidence supporting the existence of the prior-entry effect comes from the observation of a significant difference in the PSS between those conditions in which one of the target stimuli is attended as compared to when the other stimulus is attended instead. According to Titchener’s (1908) claim regarding prior entry, the stimulus that is attended should be presented later in time relative to the unattended stimulus in order for the two stimuli to be perceived as simultaneous (i.e., at the PSS). It is, however, important to note that the TOJ, SJ, and ternary-response tasks are all subject to the potential influence of a variety of response biases. Crucially, certain kinds of response bias that can affect performance in the TOJ task may be expected to lead to a shift in the PSS that can easily be confused with an attentional (i.e., perceptual) effect on the PSS. So, for example, participants may simply report the modality to which they had been instructed to attend (rather than the stimulus that was perceived as having been presented first). The shift in the PSS documented in such studies is, however, decisional, rather than attentional, in nature. To date, no evidence has been forthcoming regarding potential biases that might influence the measure of the PSS derived from fitting a psychometric function to the data from the SJ task. Consequently, many researchers now argue that it is more appropriate to use the SJ task than to use TOJs when trying to assess the effects of attention on the perceived time of arrival of an event (i.e., when trying to measure the effects of prior entry on temporal perception; see Schneider and Bavelier (2003), Zampini, Shore, and Spence (2005); though see below).

366

C. Spence, C. Parise / Consciousness and Cognition 19 (2010) 364–379

It is worth noting here that researchers have yet to come to a consensus with regard to the question of whether SJs and TOJs actually measure the same thing or not: the fact that various experimental manipulations have been shown to have different effects on SJs and TOJs (see Guerrini, Berlucchi, Bricolo, & Aglioti, 2003; Shore, Gray, Spry, & Spence, 2005; Vatakis, Navarra, Soto-Faraco, & Spence, 2008), has led some researchers to argue that they may measure somewhat different aspects of temporal perception, and even that they perhaps reflect the operation of different underlying neural mechanisms (e.g., Allan, 1975; Jas´kowski, 1991; Mitrani, Shekerdjiiski, & Yakimoff, 1986; Stelmach & Herdman, 1991; Van Eijk, Kohlrausch, Juola, & van de Par, 2008). Theoretically-speaking, attention might be expected to have at least two distinct effects on temporal perception: first, attending to a stimulus (i.e., to its location, modality, etc.) might lead to the prior entry (or earlier arrival) of the attended stimulus relative to the same stimulus when a participant’s attention happens to have been directed elsewhere.1 Additionally, however, attention might also be expected to improve the precision of participants’ judgments for attended stimuli in both the TOJ and SJ task (analogous to the enhancement of spatial resolution documented in studies of visual spatial attention; e.g., Carver & Brown, 1997; Nicol, Watter, Gray, & Shore, 2009; Pestilli, Viera, & Carrasco, 2007; Rolke, Dinkelbach, Hein, & Ulrich, 2008; Shore et al., 2001; Stelmach & Herdman, 1991; Yeshurun & Carrasco, 1998; Yeshurun & Levy, 2003). The latest evidence has shown that both spatial and temporal attentional manipulations can lead to improved temporal-discrimination performance (e.g., Chica & Christie, 2009; Correa, Sanabria, Spence, Tudela, & Lupiáñez, 2006), though it should be noted that the story here is complicated by the fact that attention can both increase and decrease precision (see Nicol et al., 2009; Yeshurun & Levy, 2003). However, the shift in the PSS (rather than any reduction in the JND) is the performance measure that is most directly relevant to assessing Titchener’s (1908) claims regarding the existence of the prior-entry effect, and it is on that we will focus. It is also worth noting that while the participants in studies of the prior-entry effect are normally encouraged to make unspeeded perceptual judgments (though see Gibson and Egeth (1994), Prinzmetal, McCool and Park (2005), Prinzmetal, Park and Garrett (2005), Santangelo and Spence (2009), for exceptions), analysis of the distribution of reaction times (RTs) from such studies can also prove to be informative (Cardoso-Leite, Gorea, & Mamassian, 2007; Heath, 1984): so, for example, if a wide enough range of SOAs are tested, one may find that participants respond more slowly at SOAs close to the PSS and more rapidly at SOAs that are further from the PSS (see Shore et al., 2001). Results such as these are consistent with participants responding most slowly when they are least certain with regard to the appropriate response, although, of course, it should be noted that the causes of differences in RT are often difficult to ascertain unequivocally (e.g., Blake, Land, & Mollon, 2008; McDonald, Teder-Sälejärvi, & Hillyard, 2000; Watt, 1991). Shore et al. (2001) highlighted the existence of just such a shift of the peak of the RT distribution in their study of visual spatial prior entry. However, while there are a number of ways of measuring the effects of attention on temporal perception, the focus in this review (in line with the majority of studies that have been published on the prior-entry effect) will be on any shift in the PSS derived from participants’ temporal judgments. 3. Prior-entry effects resulting from attention being directed to a sensory modality: experimental evidence Psychologists distinguish between the endogenous and exogenous orienting of spatial attention (e.g., Corbetta & Shulman, 2002; Klein, 2004; Klein & Shore, 2000; Prinzmetal, McCool et al., 2005; Prinzmetal, Zvinyatskovskiy, Gutierrez, & Dilem, 2009). Exogenous shifts of attention can be elicited by the peripheral presentation of a non-predictive cue stimulus, whereas endogenous shifts of attention are voluntarily induced by the provision of prior information about the likely identity or location of the target. Several early studies provided evidence that is consistent with the claim that endogenously (i.e., voluntarily) attending to a particular sensory modality results in the prior entry into awareness of stimuli subsequently presented in that modality (Frey, 1990; Sternberg et al., 1971; Stone, 1926; Wilberg & Frey, 1977, 1990). In these studies, a pair of stimuli was presented on each trial and an attempt made to direct the participant’s attention to the modality of one or other target stimulus (by default, the other stimulus in such studies is considered as being relatively ‘unattended’). For example, Stone observed a 46 ms prior-entry effect (see Sternberg & Knoll, 1973) when participants’ attention was directed to either the auditory or tactile modality; on each trial, the participants judged whether a tactile or an auditory stimulus had been presented first (see Table 1). Sternberg et al. in an oft-cited (though never published) conference presentation given at a meeting of the Psychonomics Society in 1971, reported a 55 ms prior-entry effect when participants’ attention was directed to either audition or touch (once again, only stimuli in these two modalities were presented to participants). In a second experiment, Sternberg and his colleagues documented a 30 ms prior-entry effect when participants’ attention was directed to the auditory or visual modalities instead (note that in this latter study, the participants had to judge on each trial whether an auditory or visual stimulus had been presented first). Importantly, however, many other studies have failed to show any such prior-entry effect (see Cairney, 1975a; Drew, 1896; Frey & Wilberg, 1975; Hamlin, 1895; Vanderhaeghen & Bertelson, 1974). What is more, these null results would appear to reflect more than simply a lack of statistical power (cf. Frick, 1995), since the numbers of participants/trials etc. tested were fairly similar across the various studies that either have, or have not, demonstrated a prior-entry effect. In fact, the inconsistent pattern of results reported in the early research led Pashler (1998, p. 260) to conclude that the empirical

1 While it is often claimed that attention speeds up the perceptual processing of attended (or expected) stimuli, it is important to note that in addition or even instead of this facilitation, the perceptual processing of unattended stimuli might also be delayed (see Spence, Shore, & Klein, 2001, Fig. 14, on this point).

367

C. Spence, C. Parise / Consciousness and Cognition 19 (2010) 364–379

Table 1 Summary of published studies that have shown a significant prior-entry effect resulting from the endogenous focusing of a participants’ attention on a particular sensory modality. The studies have been separated into four groups as a function of the participants’ task: non-orthogonal TOJ, Orthogonal TOJ, SJ, or ternary-response task (A = Auditory; T = Tactile; V = Visual; P = Pain; the modality here referring both to the stimulus modalities presented and the sensory modalities attended). Note that in Zampini et al.’s (2007) study the prior-entry effect was calculated on the basis of the average of the prior-entry effect reported in the SJ data (40 ms) and that reported by looking at the TOJ responses (22 ms). Task

Study

Magnitude of prior-entry effect (in ms)

Target modalities

TOJ (non-orthogonal) Stone (1926) Sternberg et al. (1971)

46 55 30

AT AT AV

Spence et al. (2001; E2) Vibell et al. (2007)

121 38

VT VT

Zampini et al. (2005b)

17

AV

Zampini et al. (2007)

29

VP

TOJ (orthogonal)

SJ Ternary-response task

evidence in support of the phenomenon of prior entry was unconvincing. Spence, Shore, & Klein, 2001 were, however, able to highlight a number of methodological and interpretational problems with all previous studies of the prior-entry effect – both the significant prior-entry effects reported by certain researchers and the null effects reported by others. For instance, Spence, Shore, and Klein (2001); see also Cairney (1975a) pointed out that the apparent prior-entry effects reported in many early studies might simply have reflected some form of response bias, given that participants’ attention was manipulated along the same dimension that they had to use when responding (i.e., participants were instructed to attend to modality X or to modality Y, and they had to judge whether the stimulus in modality X or Y had been presented first): that is, given the sometimes-difficult perceptual task of judging which of two near-simultaneous events had actually been presented first, participants might simply have chosen to respond on the basis of the stimulus that they had been instructed to attend to when uncertain of the correct response. Clearly, this form of decisional bias needs to be distinguished from the perceptual change that Titchener (1908) had in mind when he first outlined the concept of prior entry (see also Jas´kowski, 1993).2 On the other hand, Spence et al. highlighted the possibility that the null results reported in many other studies of the prior-entry effect (Cairney, 1975a; Drew, 1896; Frey & Wilberg, 1975; Hamlin, 1895; Vanderhaeghen & Bertelson, 1974) might have reflected the failure by the experimenters concerned to manipulate their participants’ attention successfully (see also below). 3.1. Orthogonal-response paradigm Spence, Shore, and Klein (2001) introduced the orthogonal-responding version of the TOJ task in order to reduce the likelihood that decisional response bias would affect participants’ TOJ performance. Specifically, in their studies, attention was manipulated in one dimension (i.e., the participants were instructed to ‘attend vision’ or to ‘attend touch’) while the participants had to judge the temporal order of the stimuli along another (orthogonal) dimension (e.g., ‘Was the first stimulus presented on the left versus on the right?’). The visual and tactile stimuli in Spence et al.’s studies were always presented from the same set of spatial locations (with one possible target location situated on either side of fixation). One target was always presented from the left and the other stimulus from the right. In order to ensure that their participants’ attention was indeed endogenously focused on either the visual or tactile modality, Spence et al. manipulated the probability of target stimuli being presented in each modality. In particular, they varied the relative proportions of unimodal tactile and visual TOJ trials in the different blocks of experimental trials. So, for example, the majority of stimuli (75%) in the ‘attend touch’ blocks were tactile (as compared to just 25% visual stimuli), while these stimulus probabilities were reversed in the ‘attend vision’ blocks (see Fig. 1). What is more, the participants were informed verbally of the stimulus probabilities at the start of each block of experimental trials, and they were explicitly instructed to direct their attention to the more probable target modality. Spence, Shore, and Klein (2001) results demonstrated a significant prior-entry effect: that is, the visual target had to be presented further in advance of the tactile stimulus in their study in order for the two stimuli to be perceived as

2 Scientists first became interested in the topic of temporal perception when they realized that there were individual differences in observers’ measurements of the time of stellar transits (i.e., measuring the time at which a star crossed the hairline of the telescope’s eye-piece; Cairney, 1975b). Traditionally, stellar transits were measured by ‘the eye and ear method’; That is, an observer would judge the time of the visual event (the transit) with respect to the sound of the second hand of the clock ticking in the background. Early on, differences in the focus of an observer’s attention were posited as a contributory factor to these individual differences in arrival time, captured by the notion of the personal equation (see Mollon & Perkins, 1996; Scharlau, 2007). While, at first, researchers attempted to investigate the factors modulating multisensory temporal perception using variants of ‘the complication situation’ (e.g., Dunlap, 1910, 1917), problems of interpretation (associated with participants’ eye movements to the moving stimulus etc.; Cairney, 1975b) soon led to the development of the TOJ task, and thence to studies of prior entry.

368

C. Spence, C. Parise / Consciousness and Cognition 19 (2010) 364–379

Fig. 1. Graph highlighting the prior-entry effect reported by Spence, Shore, and Klein (2001; Experiment 2) in their study of the effects of endogenously attending to a sensory modality (either vision or touch). The mean PSS is presented for the various attention conditions: Attend touch, divided attention, and attend vision. The proportions of different trial types presented in the various blocks of trials are shown to the left of each bar on the bar graph: VT – crossmodal visual-tactile TOJ trial; VV – unimodal visual TOJ trial; TT – unimodal tactile TOJ trial. The results highlight the shift in the PSS resulting from participants’ endogenously shifting the focus of their attention between the tactile and visual modalities, or else dividing their attention between the two modalities. Error bars indicate the between-participant standard error of the means.

simultaneous (i.e., as indicated by participants making each response equally often) when participants’ attention had been endogenously directed toward the tactile modality than when it had been directed toward the visual modality instead (see Fig. 1). Using a similar methodology, Zampini and his co-workers subsequently demonstrated a significant prior-entry between vision and audition (Zampini, Shore et al., 2005), and between vision and nociception (in the form of laser painful heat stimuli; Zampini et al., 2007). Taken together, the available evidence would therefore appear to support the conclusion that prior-entry effects can be observed no matter which modality participants endogenously direct their attention toward. One generalization that can be drawn from the various studies of endogenously attending to a particular sensory modality that have been published to date (see Table 1) is that the largest prior-entry effects tend to be reported in those studies that required their participants to make TOJs while much smaller prior-entry effects of all have been observed in those studies where a SJ task has been used (see also Schneider & Bavelier, 2003; Shore et al., 2001; Stelmach & Herdman, 1991; Van der Burg, Olivers, Bronkhorst, & Theeuwes, 2008; Wada, 2003; Zampini, Shore et al., 2005; Zhuang & Papathomas, 2009). The fact that the magnitude of the prior-entry effect appears to depend on the nature of the task that participants have to perform has led certain researchers (e.g., Schneider & Bavelier, 2003; Zampini, Shore et al., 2005) to argue that response bias may only be ruled out completely by the use of an SJ task. The underlying assumption here seems to be that the fact that larger prior-entry effects have been reported in orthogonal TOJ studies than in SJ studies must reflect some form of residual response bias affecting participants’ performance in the former (but not in the latter) case, since many researchers appear to believe that the two tasks are thought to measure the same thing (e.g., see Schneider & Bavelier, 2003; see also Zhuang & Papathomas, 2009). However, the influence of response bias can be effectively eliminated from one’s estimates of the magnitude of the prior-entry effect (no matter whether one is using an orthogonal or non-orthogonal TOJ design) by averaging participants’ performance on ‘‘Which came first?” and ‘‘Which came second?” versions of the TOJ task (see Shore et al., 2001). The logic here being that any response should have an equal and opposite effect on participants’ performance in the two cases. Hence, when performance in the ‘‘Which came first?” and

C. Spence, C. Parise / Consciousness and Cognition 19 (2010) 364–379

369

‘‘Which came second?” TOJ tasks is averaged, any response bias effects that may be present should be cancelled out, leaving just any residual attentional prior-entry effect (see below for a fuller discussion of the findings from Shore et al.’s study). Finally, it should be noted that some researchers are not so convinced that the TOJ and SJ tasks do, in fact, measure the same thing (see above).

4. Prior entry resulting from attention being directed to a spatial location In recent years, the focus of much of the prior-entry research has shifted (away from the study of the effects of attending to a particular sensory modality) toward assessing the effects of attending to a particular spatial location on the perception of temporal order and synchrony/asynchrony. Research now shows that both exogenous and endogenous spatial attentional orienting can give rise to significant prior-entry effects (e.g., Shore et al., 2001; Yates & Nicholls, 2009; though see also Schneider and Bavelier (2003)). While the majority of this research has focused on the effects/consequences of the spatial orienting of a person’s visual attention (e.g., Abrams & Law, 2000; Enns, Brehaut, & Shore, 1999; Hikosaka, Miyauchi, & Shimojo, 1993; Jas´kowski, 1993; Neumann, Esselmann, & Klotz, 1993; Scharlau, 2002; Scharlau & Ansorge, 2003; Scharlau, Ansorge, & Horstmann, 2006; Scharlau & Neumann, 2003a, 2003b; Schneider & Bavelier, 2003; Shore et al., 2001; Stelmach, Campsall, & Herdman, 1997; Stelmach & Herdman, 1991; Zackon, Casson, Stelmach, Faubert, & Racette, 1997; Zackon, Casson, Zafar, Stelmach, & Racette, 1999), similar results have also been reported in studies of unimodal auditory (Kanai, Ikeda, & Tayama, 2007) and tactile spatial attention (Yates & Nicholls, 2009), as well as in studies that have involved participants making crossmodal TOJs (Spence, Shore, & Klein, 2001). Unfortunately, however, the majority of these studies (especially the earlier studies) involved non-orthogonal TOJ designs. That said, orthogonal-responding TOJ paradigms (e.g., Shore et al., 2001; Yates & Nicholls, 2009), and SJ tasks (see Schneider & Bavelier, 2003) are now being used more frequently. As noted already, Shore et al. (2001) attempted to eliminate any residual response bias effects from their study of the effects of exogenous and endogenous visual spatial orienting on the prior-entry effect by averaging the performance of their participants on ‘Which came first?’ and ‘Which came second?’ orthogonal TOJ tasks. The participants in this study judged the order of presentation of two visual stimuli (a horizontal and a vertical line) presented from either the same or opposite sides of fixation (note that the two line segments formed a cross when they were presented from the same side of fixation). Participants’ attention was directed to one or other side by means of the transitory brightening of the outline of a square presented on either the left or right (the exogenous cue marked the location where the target might occur) or a central (endogenous) arrow precue at the start of each trial. The target stimuli in this experiment were just as likely to be presented on the same side as the exogenous peripheral cue as on the opposite side, and from the left or right. Importantly, however, the participants were given an incentive to direct their endogenous spatial attention in the direction indicated by the central arrow cue: in particular, the two visual TOJ stimuli (on cued trials) were more likely to appear at the cued location than they were to appear at the opposite location. Shore et al.’s (2001) results showed that exogenous visual orienting led to a significantly larger prior-entry effect than did endogenous orienting (mean prior-entry effects of 61 and 17 ms, respectively; see also Jas´kowski (1993), Stelmach and Herdman (1991); though see Prinzmetal, McCool et al. (2005)). Interestingly, the residual effect of response bias in Shore et al.’s (2001) orthogonal TOJ design (calculated as half the difference between the prior-entry effects reported in the ‘‘Which came first?” and ‘‘Which came second?” TOJ tasks) was estimated at 13 ms. Note that this value is much smaller than the 60 ms response bias effect observed in previous prior-entry research using a non-orthogonal TOJ design (see Frey, 1990). In other words, it would appear that the use of an orthogonal TOJ design successfully removed most of the response bias present in non-orthogonal TOJ tasks. Yates and Nicholls (2009) have recently reported a similar pattern of results in their study of the effects of exogenous and endogenous orienting of spatial attention on tactile TOJs. They reported a mean endogenous priorentry effect of 24 ms and a mean exogenous prior-entry effect of 16.5 ms in their TOJ studies. Shore et al. therefore concluded that attending to a spatial location can speed-up the relative arrival time of stimuli that are subsequently presented at that location. Schneider and Bavelier’s (2003) study of unimodal visual spatial prior entry, however, came to a rather different conclusion regarding the underlying causes of the prior-entry effect. They conducted four experiments designed to investigate the consequences of the peripheral exogenous spatial cuing (using single or multiple visual cues), central cuing (elicited by the presentation of an arrow at fixation), and gaze cuing (the gaze-deviated faces were again presented at fixation) of spatial attention on temporal perception. The participants in each experiment responded by making a TOJ regarding which visual stimulus (one red and the other green) had been presented first or, in another session (conducted on a different day), judging whether the two visual stimuli had been presented simultaneously or not. Schneider and Bavelier reported that the effect of exogenous spatial attentional orienting was very much reduced in the SJ task as compared to the orthogonal TOJ task. Nevertheless, significant prior-entry effects were reported in both tasks. The exogenous spatial cuing effect peaked at a cue-target SOA of 75 ms in the TOJ task and at an SOA of 40 ms in the SJ task, but was also pronounced at the 125 ms SOA in both tasks (note that they tested SOAs of 0, 40, 75, 125, 200, 500, and 1000 ms). By contrast, while the presentation of the central arrow cue resulted in a prior-entry effect in the TOJ task at the majority of non-zero SOAs (SOAs of 0, 100, 300, 600, 1000, and 1500 ms were tested), peaking at an SOA of 600 ms, analysis of the data from the SJ task only provided evidence of a significant prior-entry effect at the 600 ms SOA in the SJ task. Gaze-cuing led to a similar pattern of results although with a slightly different timecourse.

370

C. Spence, C. Parise / Consciousness and Cognition 19 (2010) 364–379

Fig. 2. Schneider and Bavelier (2003) outlined three different mechanisms that might result in behavioral performance in psychophysical temporal judgment tasks that might appear to provide evidence in support of the prior entry hypothesis. According to Titchener (1908), attention might speed-up the transmission of one stimulus relative to the other; either by reducing the transmission time of the attended stimulus or slowing the transmission of the unattended stimulus (see Spence, Shore, & Klein, 2001; 1 in figure). Alternatively, however, results consistent with the prior-entry effect might be cognitively mediated, that is, might alter the criteria within the decision process that compares the arrival times of the two stimuli and hence lead to a behavioral outcome indistinguishable from an attentional effect (2 in figure). Finally, in those studies in which attention is oriented by means of the presentation of a peripheral spatial cue, Schneider and Bavelier suggested that local sensory interactions between the cue and target stimuli (on the valid trials, where the cue and target are presented from the same location) might result in sensory facilitation which again could be confused with an attentional prior-entry effect (see 3 in diagram). Note that Schneider and Bavelier attempted to draw a distinction between sensory enhancement (or facilitation) and attention, arguing that they reflected separate processes. They also described the effects occurring at the level of decisional processes as attentional in nature (see their Fig. 1). However, it is important to note that many other researchers (ourselves included) describe effects on decisional processes in terms of response bias (rather than attention), while considering both attentional and sensory facilitation to represent different forms of attentional effect (i.e., different kinds of early sensory enhancement resulting from either the voluntary or automatic prioritization of certain of the information available at a given moment; e.g., see Naghavi & Nyberg, 2005, p. 392). For the case of unimodal spatial cuing this facilitation was suggested to result from the activation of the same pool of receptors by both the cue and first target stimulus, whereas in the case of crossmodal cuing studies sensory facilitation was attributed by Schneider and Bavelier to temporal ventriloquism effects instead (see Morein-Zamir et al., 2003). [Adapted, modified, and redrawn from Schneider & Bavelier, 2003, Fig. 1].

On the basis of these results, Schneider and Bavelier (2003) went onto conclude that, contrary to Titchener’s (1908) claim, endogenous attention (i.e., following the presentation of the central arrow cue) does not have much of an effect on arrival times. They also put forward a non-attentional account to explain at least part of the cuing effects reported following exogenous cuing. They suggested that the prior-entry effect reported there might actually partially reflect some form of spatiallyspecific sensory interaction (in their words, ‘‘sensory facilitation”) taking place between the cue and the first of the target stimuli on valid trials (i.e., on those trials where the cue and target stimulus are presented from the same side or spatial location; see Fig. 2). Note that in the case of the unimodal visual exogenous spatial cuing studies reported by Schneider and Bavelier, it is likely that the same group of neurons (in the ascending visual pathways) would have been used to process both the visual cue and the first of the visual target stimuli on the valid trials (that is, the cue and target fell within the same receptive fields (RFs) having been presented from more or less the same spatial location). Note though that the same argument cannot be applied to the results of Shore et al.’s (2001) study, since the RFs at the earliest cortical projection sites would likely be too small to accommodate both the cue and target stimulus used there (in valid trials). There are, however, several reasons to question Schneider and Bavelier’s (2003) claim that endogenous attention has little effect on arrival times. Note, for example, that while one of the two target stimuli would be presented from the location indicated by the central arrow cue (there were eight possible target locations arranged around a virtual circle centered on fixation in their Experiment 2), there was actually little incentive for participants to attend to the location indicated by the cue (since both the first and second target were equally likely to be presented at this location).3 Given that there was no strategic reason for participants to attend to the location indicated by the central arrow cue, it is unfortunate that Schneider and Bavelier failed to provide any independent confirmation that the central arrow cue had actually been effective in eliciting an endogenous shift of attention (see Spence, Shore, & Klein, 2001, on this problem, which is surprisingly common in studies of prior-entry; see also Schmidt, 2000). Note that a similar problem also affects the interpretation of the results of Schneider and Bavelier’s central gaze cuing study. It should also be pointed out that one assumption underlying much of Schneider and Bavelier’s reasoning is that TOJs and SJs actually measure the same thing. As yet, however, there seems no convincing reason to accept this claim (see above). In an elegant recent study, Weiss and Scharlau (2009) showed that the visual spatial prior-entry effect is reduced, but importantly not eliminated (it dropped from 54 ms to 44 ms), if participants were provided with feedback about the correctness of their TOJs on a trial-by-trial basis, and rewarded for making veridical (correct) responses. This result shows that while there may be a small strategic element to the prior-entry effect under certain conditions, the majority of the effect (at least in their study) reflects the obligatory consequences of attentional prioritization.

3

Our thanks to Ray Klein, for pointing out this possibility.

C. Spence, C. Parise / Consciousness and Cognition 19 (2010) 364–379

371

5. Prior-entry effects resulting from crossmodal exogenous spatial orienting Prior-entry effects have also been reported in a number of crossmodal exogenous spatial cuing studies (Eskes, Klein, Dove, Coolican, & Shore, 2007; Hongoh, Kita, & Soeta, 2008; Lupiánez, Baddeley, & Spence, 1999; Santangelo & Spence, 2009; Shimojo, Miyauchi, & Hikosaka, 1997; Spence & Lupiánez, 1998; Van Damme, Gallace, Spence, Crombez, & Moseley, 2009; Wada, 2003). For instance, the participants in an experiment by Spence and Lupiáñez were presented with a spatially-non-predictive visual cue on either the left or right at the start of each trial. The participants then had to make an unspeeded TOJ concerning which of two tones, one high-pitched and the other low pitched had been presented first (or second). One tone was presented on either side of fixation at SOAs of 15–250 ms (the SOA was varied using the method of constant stimuli). The onset of the spatially-non-predictive visual cue occurred 150 ms before the onset of the first tone. In spite of the spatially-non-predictive nature of the visual cue, a small but significant prior-entry effect was nevertheless still observed (M = 40 ms) no matter whether participants performed a ‘‘Which stimulus came first?” or a ‘‘Which stimulus came second?” version of the task, thus eliminating response bias as a potential contributing factor to this estimate of the prior-entry effect. In another study, Spence and his colleagues (Lupiánez et al., 1999; Spence & Lupiánez, 1998) presented a spatially-nonpredictive auditory cue (a brief white-noise burst) on either the left or right prior to a pair of visual and tactile stimuli, again one presented on either side of fixation (see Fig. 3). The participants now had to make an orthogonal TOJ regarding which modality had been presented first (or second), regardless of the side on which the first stimulus had been presented. Even

Fig. 3. Schematic illustration of a typical trial in one of the crossmodal TOJ studies conducted by Lupiánez et al. (1999). At the start of each trial, a spatiallynon-predictive auditory cue was briefly and unpredictably presented from a loudspeaker cone placed by the participant’s left or right hand. This was followed by the presentation of a pair of visual and tactile stimuli, one presented to either side of central fixation. These TOJ stimuli were separated by an SOA of 30–500 ms (varied using the method of constant stimuli). Participants made an unspeeded discrimination response regarding which modality (vision or touch) appeared to have been presented first (or second), while trying to ignore the auditory cue. (b) Mean proportion of vision-first (or visionsecond) responses as a function of the stimulus onset asynchrony (SOA) between the visual and tactile stimuli. Trials on which the visual stimulus was presented on the side of the auditory cue are shown by dotted line, while trials on which the tactile stimulus was presented on the auditorily-cue side are shown by solid line. The visual stimulus had to lead by a greater interval for the PSS to be achieved when the side of the tactile stimulus was cued than when the side of the visual stimulus was cued instead. The mean prior-entry effect (i.e., averaging over the ‘‘Which came 1st?” and ‘‘Which came 2nd?” resulting from the crossmodal exogenous orienting of spatial attention reported in this study was 141 ms.

372

C. Spence, C. Parise / Consciousness and Cognition 19 (2010) 364–379

though the participants were instructed to ignore the spatially-uninformative sound as much as possible, the results nevertheless once again showed that the stimulus subsequently presented on the side of the task-irrelevant cue sound was speeded-up relative to the stimulus that happened to be presented on the other side. A mean prior-entry effect of 121 ms was observed in the ‘‘Which came first?” study as compared to a mean prior-entry effect of 161 ms in the ‘‘Which came second?” version of the task (i.e., once again supporting the claim that the orthogonal nature of their TOJ task had effectively eliminated the response bias contribution to the prior-entry effect). Several researchers have also documented a crossmodal spatial cuing effect on unimodal visual TOJs resulting from the presentation of a spatially-non-predictive auditory cue on either the left or right (see Hongoh et al., 2008; Santangelo & Spence, 2009; Shimojo et al., 1997). Taken together, the results of the studies reported in this section therefore suggest that prior entry can be elicited by the exogenous orienting of spatial attention that follows the peripheral presentation of a spatially-non-predictive visual, auditory, or tactile cue. Once again, however, Schneider and Bavelier (2003) have suggested that sensory facilitation effects might help to explain the prior-entry effects reported in crossmodal exogenous spatial cuing studies. However, given that different sets of sensory receptors will, of necessity, initially be used to process the cue and target stimuli in such crossmodal studies (i.e., where the cue and target are presented to different sensory modalities), an alternative mechanism for sensory facilitation was needed to that proposed to account for intramodal exogenous spatial cuing effects (see above). Here, Schneider and Bavelier turned to the phenomenon of ‘temporal ventriloquism’ (see Morein-Zamir, Soto-Faraco, & Kingstone, 2003). Temporal ventriloquism is the term used to describe the temporal equivalent of the well-known spatial ventriloquism effect (e.g., Alais & Burr, 2004); the idea is that when auditory and visual stimuli occur slightly asynchronously, the brain may attempt to bind the two stimuli together by ‘pulling’ one stimulus into temporal alignment with the other. Schneider and Bavelier suggested that temporal ventriloquism might serve to pull the first target earlier in time (i.e., toward the time when the cue was presented) on the validly-cued trials (i.e., when the cue and first target stimulus were presented from the same spatial location). The important point to note here though is that subsequent research by Vroomen and Keetels (2006) and Keetels and Vroomen (2008) has shown that there is no spatial modulation of the temporal ventriloquism effect. That is, the stimuli in one sensory modality have just as much effect on the temporal perception of stimuli presented in the other modality regardless of whether they are presented from the same from versus from different locations. Hence, it turns out that the phenomenon of temporal ventriloquism simply cannot be used to explain the spatially-specific cuing effects seen in crossmodal exogenous cuing studies of the prior-entry effect. It remains an important question for future research to determine whether a plausible non-attentional (i.e., ‘sensory’ facilitation) alternative explanation for these crossmodal exogenous prior-entry effects can be developed (see Rowland, Quessy, Stanford, & Stein, 2007). Nevertheless, in the meantime, the possibility that sensory facilitation effects might influence performance should certainly be borne in mind whenever one is evaluating the prior-entry effects reported in exogenous spatial cuing studies (especially in unimodal cuing studies). Van Damme et al. (2009) recently conducted a study in which the participants had to make either unimodal TOJs concerning pairs of either tactile or auditory stimuli (consisting of brief vibrotactile stimuli or pure tone bursts), one presented on either side. Shortly before the onset of the first target (at a cue-target SOA of 250 ms), a picture was presented on one or other side for 150 ms. The picture consisted of one of three different image types: an image of physical threat (e.g., the picture of a pit-bull dog intending to bite), a general threat image (e.g., the picture of a jet exploding), or a neutral picture (e.g., the picture of a boat; see also West et al., in press). The visual cue was non-predictive with regard to the side on which the first auditory or tactile stimulus would be presented. The results showed a larger crossmodal exogenous spatial prior-entry effect following the presentation of body-specific threat pictures when the participants made tactile TOJs, while the general threat pictures resulted in the largest prior-entry effect for the auditory stimulus presented on the side of the picture instead. These results therefore suggest that the exogenous prior-entry effect elicited by the presentation of a visual cue can have differential effects depending upon the precise meaning of the cue. While the results of this study are intriguing, it is also important to note that the participants’ task was not orthogonal (i.e., the cue was presented on the left or right and participants had to report the side of the 1st auditory or tactile stimulus). Second, the blocked nature of the experimental design (with auditory and tactile targets being presented in separate blocks of experimental trials) means that any uncontrolled changes in participants’ attentional control settings may have had some influence on the pattern of results that were observed (cf. Folk, Remington, & Johnston, 1992). Thus, ideally this experiment should be repeated using an SJ task with the target modality varying on a trial-by-trial basis.

6. The cognitive neuroscience of prior entry Having reviewed the behavioral manifestation of prior entry, we turn now to the putative neural substrates underlying the effect. Neuroscience evidence demonstrating the modulation of early responses in sensory cortical areas would clearly provide strong support for the genuinely perceptual nature of prior entry. It would also help to rule out alternative accounts in terms of decisional-level effects (e.g., Pashler, 1998; Schneider & Bavelier, 2003). Several studies published over the last 15 years have suggested that covert spatial attention can indeed speed-up the processing of visual information (see Carrasco & McElree, 2001; Di Russo & Spinelli, 1999; Schuller & Rossion, 2001; Spinelli, Burr, & Morrone, 1994). However, the first study to explicitly look at the attentional modulation of TOJs using ERPs failed to demonstrate any effect of spatial attention on the timing of ERPs in visual cortex (McDonald et al., 2005). The participants in McDonald et al.’s study had to judge which of two visual stimuli, one red and the other green, had been presented first. One visual stimulus was presented from either

C. Spence, C. Parise / Consciousness and Cognition 19 (2010) 364–379

373

side of fixation. Participants’ spatial attention was exogenously directed to one side or the other by means of a brief non-predictive auditory tone presented 100–300 ms before the onset of the first visual stimulus on each and every trial. Psychophysical analysis of the behavioral data from McDonald et al.’s (2005) study revealed that the visual stimulus on the uncued side had to be presented 69 ms before the stimulus on the cued side in order for the two to be judged as simultaneous (see Hongoh et al., 2008; Santangelo & Spence, 2009, for similar results). Meanwhile, analysis of the ERP data highlighted a significant modulation of the strength of the posterior ERP signal in the 80–140 ms time-range. That is, the early neural activation in visual cortex associated with the processing of the visual target presented at the exogenously attended location was amplified, as has been observed in many previous ERP studies of spatial attention (e.g., Heinze, Luck, Mangun, & Hillyard, 1990; Hillyard, Vogel, & Luck, 1998). By contrast, there was no observable temporal shift in the latency of the early brain potentials associated with the presentation of the visual stimulus on the side of the auditory cue. That is, the contralateral N1 and P1 components were observed at the same latency as the ipsilateral components, no matter whether the visual stimulus was presented on the same or opposite side as the auditory cue.4 Although a small (c. 5 ms) P2 latency difference was observed over contralateral visual cortex, this effect could not be unambiguously interpreted as providing evidence of prior entry given the possibility of stimulus-related artifacts. McDonald et al. (2005) therefore concluded that the psychophysical shifts in the PSS that were observed in their study were not caused by a change in the latency of early perceptual processing, but rather that they arose from a crossmodal modulation of signal strength (i.e., gain-modulation) in early visual-cortical pathways. However, before generalizing from McDonald et al.’s conclusions, it is worth remembering that exogenous and endogenous attention can have qualitatively different effects on human perception and performance (see Corbetta & Shulman, 2002; Klein, 2004; Klein & Shore, 2000; Prinzmetal et al., 2009; Prinzmetal, McCool et al., 2005; Prinzmetal, Park et al., 2005; Spence & Santangelo, 2009; see also Zackon et al., 1999), and that McDonald and his colleagues only looked at the effects of crossmodal exogenous spatial orienting on ERPs. As noted earlier, it may be difficult to distinguish exogenous attentional effects from some kind of sensory (i.e., nonattentional) facilitation effects (cf. Schneider & Bavelier, 2003) in McDonald et al.’s study, given the use of an exogenous spatial cuing design. More recently, Vibell et al. (2007) reported a study in which their participants’ attention was directed to a sensory modality (rather than to a particular spatial location, as in McDonald et al.’s 2005, experiment), and which came to a qualitatively different conclusion (from that reached by McDonald and his colleagues) regarding the early neural underpinnings of the prior-entry effect. Using an orthogonal crossmodal TOJ paradigm (based on Spence, Shore, & Klein, 2001, earlier psychophysical studies), the attention of their participants was endogenously directed to either vision or touch. The participants had to determine on which side the first stimulus had been presented. A significant (38 ms) prior-entry effect was demonstrated behaviorally. Importantly, latency shifts of the early visual evoked potentials were also observed.5 That is, the latencies of the early visual P1, N1, N2 components, all peaked slightly (but significantly) earlier in time over contralateral (than over ipsilateral) cortex when the visual stimulus was attended as compared to when attention was directed to the tactile modality instead. (Note that the P1 and N1 components were temporally shifted by 3–4 ms, while the latency of the late P300 potential was shifted by 14 ms.) The mean amplitude of the P1 and N1 components for the attended visual stimuli were also increased over selected scalp sites (just as in McDonald et al.’s study), hence suggesting that the subjective perception of temporal order may result from some (as yet) unknown combination of neural latency and amplitude modulation effects. There are a number of reasons why the results of Vibell et al.’s (2007) ERP study may have been so different from those reported by McDonald et al. (2005) with regard to the neural correlates of the prior-entry effect. One possibility is that there may be differences between the mechanisms underlying the direction of attention to a sensory modality versus to a spatial location (cf. Macaluso, Frith, & Driver, 2002; Spence, Shore, & Klein, 2001). Alternatively, however, these differences may relate to the differences in the neural mechanisms known to underlie the exogenous versus endogenous orienting of spatial attention (see Corbetta & Shulman, 2002; Klein, 2004; Klein & Shore, 2000; Prinzmetal, McCool et al., 2005; Prinzmetal, Park et al., 2005; Schneider & Bavelier, 2003; Spence & Santangelo, 2009; Zackon et al., 1999; though see also Kerzel, Zarian and Souto (2009)). The difference might also be accounted for by the fact that participants’ attention was oriented on a trial-bytrial basis to one side or the other in McDonalds et al.’s study, while their attention was focused on one or other sensory modality for each block of trials in Vibell et al.’s study. Finally, however, it should be noted that McDonald et al. used a lower sampling rate (the ERP signals were digitized at 250 Hz as compared to a sampling rate of 500 Hz in Vibell et al.’s study) and this may also have led to McDonald et al. missing the small early effects (c. 3–4 ms) reported in Vibell et al.’s study due to the lower temporal resolution of the electrophysiological data that they collected. Vibell et al. (submitted for publication) recently conducted a follow-up ERP study in order to investigate whether the effects of endogenously attending to a sensory modality may be somehow qualitatively different from the effects of attending to a given spatial location on the prior-entry effect. In particular, they investigated whether directing attention to a specific location would result in similar latency shifts of the early visual evoked potentials as when participants’ attention had been endogenously directed to a particular sensory modality in Vibell et al.’s (2007) study. The experimental design was very similar to that reported by Vibell et al. (2007), although participants’ attention was now directed to either the left or right side 4 These early event-related potentials, which originate from extrastriate cortex, are assumed to represent the initial volley of neural input into these cortical areas. 5 Note that due to concerns relating to the possibility of neural interactions taking place between the two target stimuli, only those ERPs elicited by the visual stimuli, on those trials where they were presented first, were analyzed.

374

C. Spence, C. Parise / Consciousness and Cognition 19 (2010) 364–379

(i.e., rather than to the visual or tactile modality). In order to maintain the orthogonality of the experimental design, participants now reported which sensory modality (vision or touch) had been presented first. The psychophysical results once again showed a significant prior-entry effect (M = 28 ms), with participants reporting that the stimulus on the attended side had been perceived earlier than when attention had been directed to the other side. Importantly, however, while analysis of the ERP data revealed some modulations in the latency of various ERP components, they started somewhat later than in Vibell et al.’s (2007) study. While the N1, P2, and N2 showed significant effects of attention on both the amplitude and latency of the ERP components, amplitude (but not latency) effects were observed on the P1. When taken together, the results of Vibell et al.’s (2007, submitted for publication) two ERP studies therefore suggest that somewhat different mechanisms may underlie the prior-entry effects that result from endogenously attending to a sensory modality versus endogenously attending to a specific spatial location (cf. Macaluso et al., 2002; Spence, Shore, & Klein, 2001, on this point). In particular, it would appear that endogenously attending to a modality modulates earlier latency components than does exogenously attending to a particular location. The fact that Vibell et al. (submitted for publication) observed earlier ERP latency-shifting effects than reported in McDonald et al.’s (2005) study may reflect the fact that a blocked-cuing design was used in the former study, whereas participants’ attention was directed to one side or the other on a trial-by-trial basis in the latter study. In summary, the latest ERP research has now demonstrated just how early in human information processing the effects of prior entry can occur. That said, it is important to bear in mind that the ERP latency effects were (in all cases) much smaller than the prior-entry effects reported behaviorally. Nevertheless, the evidence reported by Vibell et al. (2007) shows that temporal information (at least at the millisecond timescales measured in studies of prior entry) can be (at least to a certain extent) represented temporally in the brain, thus speaking to a long-standing philosophical question about the neural representation of temporal information (see Dennett & Kinsbourne, 1992; Durgin & Sternberg, 2002; Köhler, 1947; Mellor, 1985; Roache, 1999; see also Eagleman et al., 2005). Vibell et al.’s results provide evidence relevant to Köhler’s (1947, p. 62) early claim that ‘‘Experienced order in time is always structurally identical with a functional order in the sequence of correlated brain processes”.

7. Prior entry resulting from clinical spatial attentional deficits While the majority of published studies of prior entry have investigated the effects of attention on the temporal processing of stimuli in normal participants, it is worth noting that spatial prior-entry effects have also been documented in neuropsychological patients suffering from clinical neglect and/or extinction as well. The deficits exhibited by these patients’ are typically characterized by a pathological failure to attend to stimuli presented on the side of space contralateral to their brain damage (which often involves damage to the superior temporal cortex and/or parietal lobe of the right hemisphere; see Driver & Vuilleumier, 2001; Karnath, Ferber, & Himmelbach, 2001). To date, researchers have reported evidence that is consistent with the prior entry of ipsilesional (when compared to contralesional) visual (Baylis, Simon, Baylis, & Rorden, 2002; Berberovic, Pisella, Morris, & Mattingley, 2004; Robertson, Mattingley, Rorden, & Driver, 1998; Rorden, Mattingley, Karnath, & Driver, 1997), auditory (Karnath, Zimmer, & Lewald, 2002), and tactile stimuli (Guerrini et al., 2003). Unfortunately, however, the interpretation of the results of many of these early studies is once again compromised by the response bias confound outlined earlier (given that a non-orthogonal ‘left versus right first’ TOJ task was used in many of these studies; e.g., Baylis et al., 2002; Berberovic et al., 2004; Karnath et al., 2002; Robertson et al., 1998; Rorden et al., 1997). That is, the patients may simply have preferred to respond to the stimulus on their good side when they are uncertain of which stimulus had been presented first. Recently, however, researchers have demonstrated that robust prior-entry effects can still be obtained in extinction patients under conditions where an orthogonal TOJ paradigm is used (see Dove, Eskes, Klein, & Shore, 2007; Sinnett, Juncadella, Rafal, Azañón, & Soto-Faraco, 2007): for example, the patients in a study by Sinnett et al. exhibited a mean visual prior-entry effect of 193 ms when judging whether a square or diamond had been presented first. They also exhibited an auditory priorentry effect (M = 69 ms) when judging whether the sound of a dog or a crow had been presented first. Meanwhile, Guerrini et al. (2003) demonstrated the prior entry of ipsilesional (as compared to contralesional) tactile stimuli using a ternary-response task in a group of right brain-damaged patients suffering from tactile extinction. Finally, Baylis et al. (2002) have shown the prior entry of the ipsilesional stimulus in extinction patients who had to judge whether two visual stimuli had been presented simultaneously or not. Given the ERP results reported in the preceding section, it would be interesting in future research to investigate whether the prior-entry effects reported in these neurophysiological patients have similar neural underpinnings to those seen following the spatial cuing of attention in normal participants. Finally, it should be noted that neglect/extinction patients are not the only clinical group to exhibit prior-entry effects. Moseley, Gallace, and Spence (2009) have now documented tactile prior entry in a group of patients suffering from chronic regional pain syndrome. The patients in this study perceived that tactile stimuli presented to their unaffected (left) hand as having been presented earlier in time than equivalent tactile stimuli presented to their painful right arm. These results were interpreted in terms of the patients finding it harder to direct their attention toward their affected (right) arm than to their other unaffected arm. Interestingly, a similar effect has also been reported in normal participants who have started to ‘disown’ their own right arm following the induction of the rubber hand illusion (see Botvinick & Cohen, 1998). That is, Moseley and his colleagues found that a tactile stimulus presented to the hand ‘experiencing’ the rubber hand illusion tended to be

C. Spence, C. Parise / Consciousness and Cognition 19 (2010) 364–379

375

perceived later in time than tactile stimuli presented to the other hand (Moseley et al., 2008). This result may reflect a reduction in the amount of attention being devoted to the ‘disowned’ limb (i.e., to the limb experiencing the illusion). Taken together, the results reported in this section therefore show that prior-entry effects may be common to a number of different clinical conditions. 8. Conclusions and directions for future research The last few years have seen a rapid growth of interest in studying the effects of attention on temporal perception in humans. The many methodological weaknesses that have hindered the correct interpretation of so much of the early research in this area have now been successfully eliminated in many of the studies that have been published recently. By eliminating such problems, psychologists have been able to provide more convincing empirical evidence in support of the existence of the prior-entry effect. The results of a number of such well-designed studies now show that attended stimuli typically reach awareness earlier than relatively less attended stimuli. Prior entry has been shown to occur regardless of the dimension or channel (e.g., sensory modality or spatial location) along which attention is oriented. Attentional facilitation has been reported no matter whether spatial attention is directed in an endogenous or exogenous manner (Shore et al., 2001; Yates & Nicholls, 2009; though see also Schneider & Bavelier, 2003). Although the extant research highlights the fact that decisional-level effects (e.g., response biases) have contributed (to a varying degree) to the results reported in many prior entry studies, it is also clear that even in the absence of a contribution from response bias, prior-entry effects are robust. Given the long history (over a century now) of the scientific study of the prior-entry effect, one would have expected that researchers would have moved beyond assessing people’s perception of the temporal order of simple beeps, flashes, and buzzes. However, that is simply not the case. To date, virtually all prior-entry research has been based on participants judging the temporal order (or simultaneity) of pairs of simple stimuli (though see Stolz, 1999, for an exception). This state of affairs contrasts with the recently changing focus of much of the research on multisensory perception that has started to move away from the study of pairs of simplistic and essentially (semantically) meaningless stimuli (e.g., beeps, flashes, and brief taps on the fingertip; de Gelder & Bertelson, 2003; see Spence, Shore, & Klein, 2001, for a review), stimuli that have no relation to one another (cf. Jackson, 1953; Spence, 2007). Indeed, the last few years have seen a shift toward the study of the temporal constraints on the perception of simultaneity/temporal order for more realistic (and typically complex) stimuli (e.g., Arrighi, Alais, & Burr, 2006; Kohlrausch & van de Par, 2005; Levitin, MacLean, Matthews, Chu, & Jensen, 2000; Vatakis, Ghazanfar, & Spence, 2008; Vatakis & Spence, 2007; Vatakis & Spence, 2008). Interestingly, this research has started to highlight the importance of anticipation (and perceived causality) in modulating crossmodal binding, and hence perhaps also prior entry itself (see Levitin et al., 2000; Petrini, Russell, & Pollick, 2009; Schutz & Kubovy, in press; Stekelenburg & Vroomen, 2007; Van Wassenhove, Grant, & Poeppel, 2005). It is further important to note here that in our everyday lives, sensory impressions are often complex and, what is more, the inputs perceived at around the same time by the different senses are often meaningfully-related to one another. Researchers studying temporal perception under conditions of divided attention have recently demonstrated that people’s ability to resolve the temporal order in which pairs of congruent (or matching) complex auditory and visual stimuli is sometimes worse than when judging pairs of incongruent stimuli (see Vatakis & Spence, 2007; Vatakis, Ghazanfar et al., 2008). Vatakis and Spence have recently argued that enhanced multisensory integration (leading to impaired temporal resolution) may occur for congruent audiovisual speech stimuli (i.e., when the speech sounds and lip-movements come from the same speaker and speech event) when compared to the reduced multisensory integration seen following the presentation of incongruent combinations of stimuli (i.e., where the voice belongs to one speech event while the lip-movements were associated with another). The more general point here is that while multisensory integration can give rise to enhanced

Fig. 4. (a) Auditory and visual stimuli used in Parise and Spence’s (2009) study of the effects of synesthetic congruency on audiovisual temporal perception. (b) Psychophysical results showing that performance was worse when the participants had to judge which of two synesthetically congruent stimuli was presented second than when having to judge pairs of synesthetically incongruent stimuli.

376

C. Spence, C. Parise / Consciousness and Cognition 19 (2010) 364–379

performance/perception under certain conditions, the flip side is that it can also lead to the loss of some of the information relating to the constituent unisensory signals. While early studies suggested that this ‘unity effect’ was restricted to the perception of audiovisual speech stimuli (see Vatakis, Ghazanfar et al., 2008; Vatakis, Navarra et al., 2008; Vatakis & Spence, 2008), recent research has now shown that neurocognitively normal individuals also find it harder to judge the temporal order of asynchronously-presented simple auditory and visual stimuli if they happen to be synesthetically matched than when judging synesthetically mismatched stimulus pairs (Parise & Spence, 2009). The synesthetically matched stimuli in one study consisted, for example, of large visual circles paired with low-frequency sounds and small circles paired with high-pitched sounds, while the crossmodal pairings were simply reversed in order to create the mismatched stimulus pairs (see Fig. 4). One fascinating question for future research in this area will therefore be to see whether prior entry has a more pronounced effect on the perception of pairs of auditory and visual stimuli that are mismatched than for pairs of stimuli that go together along some dimension (such as, for example, stimuli that are either semantically or synesthetically congruent). If audiovisual congruency results in increased binding for multisensory stimuli that have a higher probability of originating from the same external event (due to the unity effect) then the prediction is that prior entry might be expected to have a much smaller effect on the temporal perception of congruent (or matched), than of incongruent (or mismatched), stimuli (cf. Kanai, Ikeda, & Tayama, 2007; Nicol & Shore, 2007; Spence, 2007). To conclude, it may be that when pairs of sensory stimuli are presented at about the same time, attention can either prioritize the processing of one of the two stimuli if they are judged as being unrelated to one another (i.e., if they are ‘attributed’ by the brain to different objects or events – or if meaningfully-related but presented too far apart in space and/or time), or else help to bind them (across time and space) if they are perceived to be meaningfully-related to one another (see Neisser, 1976; Nicol et al., 2009; Spence, Baddeley, Zampini, James, & Shore, 2003; see Fig. 5).

Fig. 5. Schematic illustration demonstrating two of the ways in which attention might help to segregate/bind multisensory information. (a) When two stimuli refer to different events (or objects), in this case distinct visual and auditory events, attention (to the visual modality) serves to prioritize the perceptual time of arrival of one (attended) signal (visual) at the cost of the delayed arrival of the other (relatively) unattended signal (auditory; see 1). (b) By contrast, when the visual and auditory stimuli actually refer to the same multisensory event (or are assumed to do so), attention may help to bind the desynchronized unisensory signals that have become desynchronized (see 2) due to any differences in visual and auditory transduction latencies (see Spence, Shore, & Klein, 2001; Spence & Squire, 2003; Vibell et al., 2007; Vibell et al., submitted for publication). In this latter case, attention facilitates crossmodal binding and thus helps to maintain the perceptual experience of a unique multisensory event, having a unique temporal onset (see Neisser, 1976; Spence et al., 2003; see also Grossberg & Grunewald, 1997; Ley, Haggard, & Yarrow, 2009, for modeling approaches to the temporal binding problem). The factors modulating whether the human brain treats two or more unisensory stimuli as belonging to the same event or object or not is currently a subject of much debate (e.g., Ernst, 2007; Helbig & Ernst, 2007; Spence, 2007; Vatakis, Ghazanfar et al., 2008, Vatakis, Navarra et al., 2008; Vatakis & Spence, 2007, 2008), with recent progress coming from work that has started to model causal inference (Körding et al., 2007).

C. Spence, C. Parise / Consciousness and Cognition 19 (2010) 364–379

377

Acknowledgments We would like to thank Ray Klein for his helpful suggestions on an earlier version of this manuscript. References Abrams, R. A., & Law, M. B. (2000). Object-based visual attention with endogenous orienting. Perception & Psychophysics, 62, 818–833. Alais, D., & Burr, D. (2004). The ventriloquist effect results from near-optimal bimodal integration. Current Biology, 14, 257–262. Allan, L. G. (1975). The relationship between judgments of successiveness and judgments of order. Perception & Psychophysics, 18, 29–36. Arrighi, R., Alais, D., & Burr, D. (2006). Perceptual synchrony of audiovisual streams for natural and artificial motion sequences. Journal of Vision, 6, 260–268. Baylis, G. C., Simon, S. L., Baylis, L. L., & Rorden, C. (2002). Visual extinction with double simultaneous stimulation: What is simultaneous? Neuropsychologia, 40, 1027–1034. Berberovic, N., Pisella, L., Morris, A. P., & Mattingley, J. B. (2004). Prismatic adaptation reduces biased temporal order judgements in spatial neglect. Neuroreport, 15, 1199–1204. Blake, Z., Land, T., & Mollon, J. (2008). Relative latencies of cone signals measured by a moving vernier task. Journal of Vision, 8(16), 16. 1–11. Botvinick, M., & Cohen, J. (1998). Rubber hands ‘feel’ touch that eyes see. Nature, 391, 756. Buonomano, D. V., & Karmarkar, U. R. (2002). How do we tell time? The Neuroscientist, 8, 42–51. Cairney, P. T. (1975a). Bisensory order judgement and the prior entry hypothesis. Acta Psychologica, 39, 329–340. Cairney, P. T. (1975b). The complication experiment uncomplicated. Perception, 4, 255–265. Cardoso-Leite, P., Gorea, A., & Mamassian, P. (2007). Temporal order judgment and simple reaction times: Evidence for a common processing system. Journal of Vision, 7(6), 11. 1–14. Carrasco, M., & McElree, B. (2001). Covert attention accelerates the rate of visual information processing. Proceedings of the National Academy of Sciences USA, 98, 5363–5367. Carver, R. A., & Brown, V. (1997). Effects of amount of attention allocated to the location of visual stimulus pairs on perception of simultaneity. Perception & Psychophysics, 59, 534–542. Chica, A. B., & Christie, J. (2009). Spatial discrimination does improve temporal discrimination. Attention, Perception & Psychophysics, 71, 273–280. Corbetta, M., & Shulman, G. L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience, 3, 201–215. Correa, A., Sanabria, D., Spence, C., Tudela, P., & Lupiáñez, J. (2006). Selective temporal attention enhances the temporal resolution of visual perception: Evidence from a temporal order judgment task. Brain Research, 1070, 202–205. De Gelder, B., & Bertelson, P. (2003). Multisensory integration, perception and ecological validity. Trends in Cognitive Sciences, 7, 460–467. Dennett, D. C., & Kinsbourne, M. (1992). Time and the observer: The where and when of consciousness in the brain. Behavioral and Brain Sciences, 15, 183–247. Di Russo, F., & Spinelli, D. (1999). Electrophysiological evidence for an early attentional mechanism in visual processing in humans. Vision Research, 39, 2975–2985. Dove, M. E., Eskes, G., Klein, R. M., & Shore, D. (2007). A left attentional bias in chronic neglect: A case study using temporal order judgments. Neurocase, 13, 37–49. Drew, F. (1896). Attention: Experimental and critical. American Journal of Psychology, 7, 533–573. Driver, J., & Vuilleumier, P. (2001). Perceptual awareness and its loss in unilateral neglect and extinction. Cognition, 79, 39–88. Dunlap, K. (1910). The complication experiment and related phenomena. Psychological Review, 17, 157–191. Dunlap, K. (1917). A new complication apparatus. Journal of Experimental Psychology, 2, 89–91. Durgin, F. H., & Sternberg, S. (2002). The time of consciousness and vice versa. Consciousness and Cognition, 11, 284–290. Eagleman, D. M. (2008). Human time perception and its illusions. Current Opinion in Neurobiology, 18, 131–136. Eagleman, D. M., Tse, P. U., Buonomano, D., Janssen, P., Nobre, A. C., & Holcombe, A. O. (2005). Time and the brain: How subjective time relates to neural time. Journal of Neuroscience, 25, 10369–10371. Enns, J. T., Brehaut, J. C., & Shore, D. I. (1999). The duration of a brief event in the mind’s eye. Journal of General Psychology, 126, 355–372. Ernst, M. O. (2007). Learning to integrate arbitrary signals from vision and touch. Journal of Vision, 7(5/7), 1–14. Eskes, G. A., Klein, R. M., Dove, M. B., Coolican, J., & Shore, D. I. (2007). Comparing temporal order judgments and choice reaction time tasks as indices of exogenous spatial attention. Journal of Neuroscience Methods, 166, 259–265. Folk, C. L., Remington, R. W., & Johnston, J. C. (1992). Involuntary covert orienting is contingent on attentional control settings. Journal of Experimental Psychology: Human Perception and Performance, 18, 1030–1044. Frey, R. D. (1990). Selective attention, event perception and the criterion of acceptability principle: Evidence supporting and rejecting the doctrine of prior entry. Human Movement Science, 9, 481–530. Frey, R. D., & Wilberg, R. B. (1975). Selective attention and the judgment of temporal order. In J. Salmela (Ed.). Mouvement (Vol. 7, pp. 63–65). Quebec: Canadian Society for Psychomotor Learning and Sport Psychology. Frick, R. W. (1995). Accepting the null hypothesis. Memory & Cognition, 23, 132–138. Gibson, B. S., & Egeth, H. (1994). Inhibition and disinhibition of return: Evidence from temporal order judgments. Perception & Psychophysics, 56, 669–680. Grossberg, S., & Grunewald, A. (1997). Cortical synchronization and perceptual framing. Journal of Cognitive Neuroscience, 9, 117–132. Guerrini, C., Berlucchi, G., Bricolo, E., & Aglioti, S. M. (2003). Temporal modulation of spatial tactile extinction in right-brain-damaged patients. Journal of Cognitive Neuroscience, 15, 523–536. Hamlin, A. J. (1895). On the least observable interval between stimuli addressed to disparate senses and to different organs of the same sense. American Journal of Psychology, 6, 564–575. Heath, R. A. (1984). Response time and temporal order judgement in vision. Australian Journal of Psychology, 36, 21–34. Heinze, H. J., Luck, S. J., Mangun, G. R., & Hillyard, S. A. (1990). Visual event-related potentials index focused attention within bilateral stimulus arrays. I. Evidence for early selection. Electroencephalography & Clinical Neurophysiology, 75, 511–527. Helbig, H. B., & Ernst, M. O. (2007). Knowledge about a common source can promote visual-haptic integration. Perception, 36, 1523–1533. Hikosaka, O., Miyauchi, S., & Shimojo, S. (1993). Focal visual attention produces illusory temporal order and motion sensation. Vision Research, 33, 1219–1240. Hillyard, S. A., Vogel, E. K., & Luck, S. J. (1998). Sensory gain control (amplification) as a mechanism of selective attention: Electrophysiological and neuroimaging evidence. Philosophical Transactions of the Royal Society London B Biological Sciences, 353, 1257–1270. Hongoh, Y., Kita, S., & Soeta, Y. (2008). Separation between sound and light enhances audio–visual prior entry effect. IEICE Transactions on Information and Systems, E91-D, 1641–1648. Jackson, C. V. (1953). Visual factors in auditory localization. Quarterly Journal of Experimental Psychology, 5, 52–65. Jas´kowski, P. (1991). Two-stage model for order discrimination. Perception & Psychophysics, 50, 76–82. Jas´kowski, P. (1993). Selective attention and temporal-order judgement. Perception, 22, 681–689. Kanai, K., Ikeda, K., & Tayama, T. (2007). The effect of exogenous spatial attention on auditory information processing. Psychological Research, 71, 418–426. Karnath, H.-O., Ferber, S., & Himmelbach, M. (2001). Spatial awareness is a function of the temporal not the posterior parietal lobe. Nature, 411, 950–953. Karnath, H., Zimmer, U., & Lewald, J. (2002). Impaired perception of temporal order in auditory extinction. Neuropsychologia, 40, 1977–1982. Keetels, M., & Vroomen, J. (2008). Tactile-visual temporal ventriloquism and the effect of spatial disparity. Perception & Psychophysics, 70, 765–771.

378

C. Spence, C. Parise / Consciousness and Cognition 19 (2010) 364–379

Kelly, S. D. (2005). The puzzle of temporal experience. In A. Brook & K. Akins (Eds.), Cognition and the brain: The philosophy and neuroscience movement (pp. 208–240). Cambridge: Cambridge University Press. Kerzel, D., Zarian, L., & Souto, D. (2009). Involuntary cueing effects depend on accuracy measures: Stimulus and task dependence. Journal of Vision, 9(11), 16. 1–16. Klein, R. M. (2004). On the control of visual attention. In M. I. Posner (Ed.), Cognitive neuroscience of attention (pp. 29–44). New York, NY: Guilford Press. Klein, R. M., & Shore, D. I. (2000). Relationships among modes of visual orienting. In S. Monsell & J. Driver (Eds.), Control of cognitive processes: Attention and performance XVIII (pp. 195–208). Cambridge, MA: MIT Press. Köhler, W. (1947). Gestalt psychology: An introduction to new concepts in modern psychology. New York: Liveright Publication. Kohlrausch, A., & van de Par, S. (2005). Audio–visual interaction in the context of multi-media applications. In J. Blauert (Ed.), Communication acoustics (pp. 109–138). Berlin: Springer. Körding, K. P., Beierholm, U., Ma, W. J., Quartz, S., Tenenbaum, J. B., & Shams, L. (2007). Causal inference in multisensory perception. PLoS ONE, 2(9), e943. Lester, B. D., Hecht, L. N., & Vecera, S. P. (2009). Visual prior entry for foreground figures. Psychonomic Bulletin & Review, 16, 654–659. Levitin, D. J., MacLean, K., Matthews, M., Chu, L., & Jensen, E. (2000). The perception of cross-modal simultaneity (Or ‘‘The Greenwich observatory problem” revisited). In D. M. Dubois (Ed.), Computing Anticipatory Systems: CASYS’99. Third International Conference (CP517, pp. 323–329). . Ley, I., Haggard, P., & Yarrow, K. (2009). Optimal integration of auditory and vibrotactile information for judgments of temporal order. Journal of Experimental Psychology: Human Perception and Performance, 35, 1005–1019. Lupiánez, J., Baddeley, R., & Spence, C. (1999). Crossmodal links in exogenous spatial attention revealed by the orthogonal temporal order judgment task. Paper presented at the 1st international multisensory research conference: Crossmodal attention and multisensory integration. Oxford 1st–2nd October. . Macaluso, E., Frith, C. D., & Driver, J. (2002). Directing attention to locations and to sensory modalities: Multiple levels of selective processing revealed with PET. Cerebral Cortex, 12, 357–368. McDonald, J. J., Teder-Sälejärvi, W. A., & Hillyard, S. A. (2000). Involuntary orienting to sound improves visual perception. Nature, 407, 906–908. McDonald, J. J., Teder-Sälejärvi, W. A., Di Russo, F., & Hillyard, S. A. (2005). Neural basis of auditory-induced shifts in visual time-order perception. Nature Neuroscience, 8, 1197–1202. Mellor, D. H. (1985). Real time. Cambridge: Cambridge University Press. Mitrani, L., Shekerdjiiski, S., & Yakimoff, N. (1986). Mechanisms and asymmetries in visual perception of simultaneity and temporal order. Biological Cybernetics, 54, 159–165. Mollon, J. D., & Perkins, A. J. (1996). Errors of judgement at Greenwich in 1796. Nature, 380, 101–102. Morein-Zamir, S., Soto-Faraco, S., & Kingstone, A. (2003). Auditory capture of vision: Examining temporal ventriloquism. Cognitive Brain Research, 17, 154–163. Moseley, G. L., Gallace, A., & Spence, C. (2009). Space-based, but not arm-based, neglect in complex regional pain syndrome and its relationship to cooling of the affected limb. Brain, 132, 3142–3151. Moseley, G. L., Olthof, N., Venema, A., Don, S., Wijers, M., Gallace, A., et al (2008). Psychologically induced cooling of a specific body part caused by the illusory ownership of an artificial counterpart. Proceedings of the National Academy of Sciences USA, 105, 13168–13172. Naghavi, H. R., & Nyberg, L. (2005). Common fronto-parietal activity in attention, memory, and consciousness: Shared demands on integration? Consciousness and Cognition, 14, 390–425. Neisser, U. (1976). Cognition and reality: Principles and implications of cognitive psychology. San Francisco: Freeman. Neumann, O., Esselmann, U., & Klotz, W. (1993). Differential effects of visual-spatial attention on response latency and temporal-order judgment. Psychological Research, 56, 26–34. Nicol, J. R., & Shore, D. I. (2007). Perceptual grouping impairs temporal resolution. Experimental Brain Research, 183, 141–148. Nicol, J. R., Watter, S., Gray, K., & Shore, D. I. (2009). Object-based perception mediates the effect of exogenous attention on temporal resolution. Visual Cognition, 17, 555–573. Parise, C., & Spence, C. (2008). Synesthetic congruency modulates the temporal ventriloquism effect. Neuroscience Letters, 442, 257–261. Parise, C., & Spence, C. (2009). ‘When birds of a feather flock together’: Synesthetic associations modulate multisensory integration in non-synesthetes. PLoS ONE, 4(5), e5664. doi:10.1371/journal.pone.0005664. Pashler, H. E. (1998). The psychology of attention. Cambridge, MA: MIT Press. Pestilli, F., Viera, G., & Carrasco, M. (2007). How do attention and adaptation affect contrast sensitivity. Journal of Vision, 7(7), 9. 1–12. Petrini, K., Russell, M., & Pollick, F. (2009). When knowing can replace seeing in audiovisual integration of actions. Cognition, 110, 432–439. Prinzmetal, W., McCool, C., & Park, S. (2005). Attention: Reaction time and accuracy reveal different mechanisms. Journal of Experimental Psychology: General, 134, 73–92. Prinzmetal, W., Park, S., & Garrett, R. (2005). Involuntary attention and identification accuracy. Perception & Psychophysics, 67, 1344–1353. Prinzmetal, W., Zvinyatskovskiy, A., Gutierrez, P., & Dilem, L. (2009). Voluntary and involuntary attention have different consequences: The effect of perceptual difficulty. Quarterly Journal of Experimental Psychology, 62, 352–369. Roache, R. (1999). Mellor and Dennett on the perception of temporal order. Philosophical Quarterly, 49, 231–238. Robertson, I. H., Mattingley, J. B., Rorden, C., & Driver, J. (1998). Phasic alerting of neglect patients overcomes their spatial deficit in visual awareness. Nature, 395, 169–173. Rowland, B. A., Quessy, S., Stanford, T. R., & Stein, B. E. (2007). Multisensory integration shortens physiological response latencies. Journal of Neuroscience, 27, 5879–5884. Rolke, B., Dinkelbach, A., Hein, E., & Ulrich, R. (2008). Does attention impair temporal discrimination? Examining non-attentional accounts. Psychological Research, 72, 49–60. Rorden, C., Mattingley, J. B., Karnath, H.-O., & Driver, J. (1997). Visual extinction and prior entry: Impaired perception of temporal order with intact motion perception after unilateral parietal damage. Neuropsychologia, 35, 421–433. Santangelo, V., & Spence, C. (2009). Crossmodal exogenous orienting improves the accuracy of genuinely-unspeeded temporal order judgments. Experimental Brain Research, 194, 577–586. Scharlau, I. (2002). Leading, but not trailing, primes influence temporal order perception: Further evidence for an attentional account of perceptual latency priming. Perception & Psychophysics, 64, 1346–1360. Scharlau, I. (2007). Perceptual latency priming: A measure of attentional facilitation. Psychological Research, 71, 678–686. Scharlau, I., & Ansorge, U. (2003). Direct parameter specification of an attention shift: Evidence from perceptual latency priming. Vision Research, 43, 1351–1363. Scharlau, I., Ansorge, U., & Horstmann, G. (2006). Latency facilitation in temporal-order judgments: Time course of facilitation as a function of judgment type. Acta Psychologica, 122, 129–159. Scharlau, I., & Neumann, O. (2003a). Temporal parameters and time course of perceptual latency priming. Acta Psychologica, 113, 185–203. Scharlau, I., & Neumann, O. (2003b). Perceptual latency priming by masked and unmasked stimuli: Evidence for an attentional interpretation. Psychological Research, 67, 184–196. Schmidt, W. C. (2000). Endogenous attention and illusory line motion re-examined. Journal of Experimental Psychology: Human Perception and Performance, 26, 980–996. Schneider, K. A., & Bavelier, D. (2003). Components of visual prior entry. Cognitive Psychology, 47, 333–366. Schuller, A., & Rossion, B. (2001). Spatial attention triggered by eye gaze increases and speeds up early visual activity. Neuroreport, 12, 2381–2386. Schutz, M., & Kubovy, M. (in press). Causality in audio–visual sensory integration. Journal of Experimental Psychology: Human Perception & Performance.

C. Spence, C. Parise / Consciousness and Cognition 19 (2010) 364–379

379

Shimojo, S., Miyauchi, S., & Hikosaka, O. (1997). Visual motion sensation yielded by non-visually driven attention. Vision Research, 12, 1575–1580. Shore, D. I., & Spence, C. (2005). Prior entry. In L. Itti, G. Rees, & J. Tsotsos (Eds.), Neurobiology of attention (pp. 89–95). North Holland: Elsevier. Shore, D. I., Spence, C., & Klein, R. M. (2001). Visual prior entry. Psychological Science, 12, 205–212. Shore, D. I., Gray, K., Spry, E., & Spence, C. (2005). Spatial modulation of tactile temporal-order judgments. Perception, 34, 1251–1262. Sinnett, S., Juncadella, M., Rafal, R., Azañón, E., & Soto-Faraco, S. (2007). A dissociation between visual and auditory hemi-inattention: Evidence from temporal order judgements. Neuropsychologia, 45, 552–560. Spence, C. (2007). Audiovisual multisensory integration. Acoustical Science and Technology, 28, 61–70. Spence, C., Baddeley, R., Zampini, M., James, R., & Shore, D. I. (2003). Crossmodal temporal order judgments: When two locations are better than one. Perception & Psychophysics, 65, 318–328. Spence, C., & Lupiánez, J. (1998). Crossmodal links in attention revealed by the orthogonal temporal order judgment task. Paper presented at II Congreso de la Sociedad Espanyola de Psicologia Experimental (SEPEX 98). Spain: Granada [17th December]. Spence, C., Nicholls, M. E. R., & Driver, J. (2001). The cost of expecting events in the wrong sensory modality. Perception & Psychophysics, 63, 330–336. Spence, C., & Santangelo, V. (2009). Auditory attention. In C. Plack (Ed.), Auditory perception (pp. 249–270). Oxford: Oxford University Press. Spence, C., Shore, D. I., & Klein, R. M. (2001). Multisensory prior entry. Journal of Experimental Psychology: General, 130, 799–832. Spence, C., & Squire, S. B. (2003). Multisensory integration: Maintaining the perception of synchrony. Current Biology, 13, R519–R521. Spinelli, D., Burr, D. C., & Morrone, M. C. (1994). Spatial neglect is associated with increased latencies of visual evoked potentials. Visual Neuroscience, 11, 909–918. Stekelenburg, J., & Vroomen, J. (2007). Neural correlates of multisensory integration of ecologically valid audiovisual events. Journal of Cognitive Neuroscience, 19, 1964–1973. Stelmach, L. B., Campsall, J. M., & Herdman, C. M. (1997). Attentional and ocular movements. Journal of Experimental Psychology: Human Perception and Performance, 23, 823–844. Stelmach, L. B., & Herdman, C. M. (1991). Directed attention and perception of temporal order. Journal of Experimental Psychology: Human Perception and Performance, 17, 539–550. Sternberg, S., & Knoll, R. L. (1973). The perception of temporal order: Fundamental issues and a general model. In S. Kornblum (Ed.). Attention & performance (Vol. 4, pp. 629–685). London: Academic Press. Sternberg, S., Knoll, R. L., & Gates, B. A. (1971, November). Prior entry reexamined: Effect of attentional bias on order perception. Paper presented at the meeting of the Psychonomic Society, St. Louis, Missouri. Stolz, J. A. (1999). Word priming and temporal order judgments: Semantics turns back the clock. Canadian Journal of Experimental Psychology, 53, 316–322. Stone, S. A. (1926). Prior entry in the auditory–tactual complication. American Journal of Psychology, 37, 284–287. Titchener, E. B. (1908). Lectures on the elementary psychology of feeling and attention. New York: Macmillan. Ulrich, R. (1987). Threshold models of temporal-order judgments evaluated by a ternary response task. Perception & Psychophysics, 42, 224–239. Van Damme, S., Gallace, A., Spence, C., Crombez, G., & Moseley, G. L. (2009). Does the sight of physical threat induce a tactile processing bias? Modalityspecific facilitation of attention induced by viewing threatening pictures. Brain Research, 1253, 100–106. Van der Burg, E., Olivers, C. N. L., Bronkhorst, A. W., & Theeuwes, J. (2008). Audiovisual events capture attention: Evidence from temporal order judgments. Journal of Vision, 8(5), 2. 1–10. Vanderhaeghen, C., & Bertelson, P. (1974). The limits of prior entry: Nonsensitivity of temporal order judgments to selective preparation affecting choice reaction time. Bulletin of the Psychonomic Society, 4, 569–572. Van Eijk, R. L. J., Kohlrausch, A., Juola, J. F., & van de Par, S. (2008). Audio–visual synchrony and temporal order judgments: Effects of experimental method and stimulus type. Perception & Psychophysics, 70, 955–968. Van Wassenhove, V., Grant, K. W., & Poeppel, D. (2005). Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences USA, 102, 1181–1186. Vatakis, A., Ghazanfar, A., & Spence, C. (2008). Facilitation of multisensory integration by the ‘unity assumption’: Is speech special? Journal of Vision, 8(14), 1–11. Vatakis, A., Navarra, J., Soto-Faraco, S., & Spence, C. (2008). Audiovisual temporal adaptation of speech: Temporal order versus simultaneity judgments. Experimental Brain Research, 185, 521–529. Vatakis, A., & Spence, C. (2007). Crossmodal binding: Evaluating the unity assumption using audiovisual speech stimuli. Perception & Psychophysics, 69, 744–756. Vatakis, A., & Spence, C. (2008). Evaluating the influence of the ‘unity assumption’ on the temporal perception of realistic audiovisual stimuli. Acta Psychologica, 127, 12–23. Vibell, J., Klinge, C., Zampini, M., Spence, C., & Nobre, A. C. (2007). Temporal order is coded temporally in the brain: Early ERP latency shifts underlying prior entry in a crossmodal temporal order judgment task. Journal of Cognitive Neuroscience, 19, 109–120. Vibell, J., Klinge, C., Zampini, M., Spence, C., & Nobre, A. C. (submitted for publication). ERP study of the spatial prior-entry effect. Brain Research. Vroomen, J., & Keetels, M. (2006). The spatial constraint in intersensory pairing: No role in temporal ventriloquism. Journal of Experimental Psychology: Human Perception & Performance, 32, 1063–1071. Wada, Y. (2003). Crossmodal attention between vision and touch in temporal order judgment task. Shinrigaku Kenkyu, 74, 420–427. Watt, R. J. (1991). Understanding vision. Academic Press: London. Weiss, K., & Scharlau, I. (2009). Strategic influences on visual prior entry. Perception, 38(Suppl.), 17. West, G., Anderson, A., & Pratt, J. (in press). Motivationally significant stimuli show visual prior entry: Direct evidence for attentional capture. Journal of Experimental Psychology: Human Perception and Performance. Wilberg, R. B., & Frey, R. D. (1977). The prior entry phenomenon: In search of determinants of the effect. In B. Kerr (Ed.), Human performance and behavior (pp. 237–240). Banff: Canadian Society for Psychomotor Learning and Sport Psychology. Wilberg, R. B., & Frey, R. D. (1990). Prior entry: Information processing requirements and the judgment of temporal order. In H.-G. Geissler, M. H. Müller, & W. Prinz (Eds.), Psychophysical explorations of mental structures (pp. 253–267). Goettingen, Germany: Hogrefe and Huber Publishers. Yates, M. J., & Nicholls, M. E. R. (2009). Somatosensory prior entry. Perception & Psychophysics, 71, 847–859. Yeshurun, Y., & Carrasco, M. (1998). Attention improves or impairs visual performance by enhancing spatial resolution. Nature, 396, 72–75. Yeshurun, Y., & Levy, L. (2003). Transient spatial attention degrades temporal resolution. Psychological Science, 14, 225–231. Zackon, D. H., Casson, E. J., Stelmach, L., Faubert, J., & Racette, L. (1997). Distinguishing subcortical and cortical influences in visual attention. Investigative Opthalmology and Visual Science, 38, 364–371. Zackon, D. H., Casson, E. J., Zafar, A., Stelmach, L., & Racette, L. (1999). The temporal order judgment paradigm: Subcortical attentional contribution under exogenous and endogenous cuing conditions. Neuropsychologia, 37, 511–520. Zampini, M., Bird, K. S., Bentley, D. E., Watson, A., Barrett, G., Jones, A. K., et al (2007). ‘Prior entry’ for pain: Attention speeds the perceptual processing of painful stimuli. Neuroscience Letters, 414, 75–79. Zampini, M., Guest, S., Shore, D. I., & Spence, C. (2005a). Audio–visual simultaneity judgments. Perception & Psychophysics, 67, 531–544. Zampini, M., Shore, D. I., & Spence, C. (2005b). Audiovisual prior entry. Neuroscience Letters, 381, 217–222. Zhuang, X., & Papathomas, T. V. (2009). Prior entry for feature-based attention: Are objects of the attended color perceived earlier. Journal of Vision, 9(8), 144. 144a [Abstract]. .