Crossmodal attention

• Prof. Charles Spence, Crossmodal Research Laboratory, Department of Experimental Psychology, Oxford University

Crossmodal Attention

Charles Spence Crossmodal Research Laboratory, Department of Experimental Psychology, Oxford University

CORRESPONDENCE TO: Professor Charles Spence, Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford, OX1 3UD, United Kingdom. Tel: +44-1865-271364; Fax: +44-1865-310447

Email: charles.spence@psy.ox.ac.uk

Introduction

Attention refers to those processes that allow for the selective processing of incoming sensory stimuli. Mechanisms of attention help us to prioritize the processing of those stimuli that are most relevant to achieving our current goals and/or to performing the task at hand. However, our attention can also be captured by intrinsically salient or biological significant stimuli. Attended stimuli tend to be processed more thoroughly and more rapidly than other potentially distracting (‘unattended’) stimuli (Posner, 1978). Although attention research has traditionally considered selection among the competing inputs within just a single sensory modality at a time (most often vision; see Driver, 2001), the last couple of decades have seen a burgeoning of interest in the existence and nature of any crossmodal constraints on our ability to selectively attend to a particular sensory modality, spatial location, event, or object (Spence and Driver, 2004). In fact, crossmodal interactions in attention have now been demonstrated between most combinations of visual, auditory, tactile, olfactory, gustatory, and even painful stimuli (see Calvert, Spence and Stein, 2004).

It is important to note that attention can either be oriented endogenously or exogenously: People orient their attention endogenously when they voluntarily choose to attend to something, such as when listening to a particular individual at a noisy cocktail party, say, or when concentrating on the texture of the object that they happen to be holding in their hand. By contrast, exogenous orienting occurs in response to the reflexive (i.e., involuntary) capture of attention by the sudden onset of an unexpected event, such as when a person unexpectedly calls our name at a noisy cocktail party, or when a mosquito suddenly lands on our arm.

Attending to a sensory modality

One of the most fundamental questions in crossmodal attention research concerns the extent to which people can selectively direct their attention toward a particular sensory modality such as, for example, audition, at the expense of the processing of stimuli presented in the other modalities. Spence and his colleagues have conducted research showing that voluntarily (i.e., endogenously) attending to a particular sensory modality can result in the facilitation of people’s (speeded) spatial discrimination responses to stimuli presented in that modality when compared to situations in which their attention has been directed to another modality instead (Spence et al., 2000a, 2001a, 2002; see also Ashkenazi and Marks, 2004). The presentation of a non-predictive cue stimulus in a particular modality also results in the short-lasting exogenous orienting of attention toward the modality of the cue (see Spence et al., 2001 a; Turatto et al., 2002, 2004).

Interestingly, endogenously attending to a particular sensory modality does not always give rise to a particularly large effect on behavioural performance. For example, Alais, Morrone, and Burr (2006) reported little decrement in perceptual sensitivity when the participants in their study had to monitor the stimuli presented in two sensory modalities (audition and vision) rather than just one (audition or vision). The visual task in Alais et al.’s study consisted of ‘low-level’ contrast discrimination, while the auditory task involved participants having to discriminate the pitch of target sounds. The results showed that the auditory thresholds were slightly (but significantly) higher in the bimodal divided attention condition than in the unimodal focused attention condition. By contrast, visual thresholds were unaffected by whether performance was assessed in the focused or bimodal divided attention blocks.

The most parsimonious account of the data published to date on the effects of attending to a modality is that: 1) Generally-speaking, people find it easier to divide their attention between tasks presented in different modalities than between tasks presented within the same sensory modality (Hancock et al., 2007; Sarter, 2007; Wickens, 1992 see also Lavie, 2005); 2) The costs associated with attending to the wrong sensory modality appear to be larger than the benefits associated with focusing attention on a particular modality, when compared to performance in a neutral baseline divided attention condition (Alais et al., 2006; Spence et al., 2001a); 3) Crossmodal attentional effects are more likely to be observed when participants have to respond on the basis of higher-level stimulus attributes (such as relating to a target’s identity or spatial location (Soto-Faraco et al., 2002; Treisman and Davies, 1973). and 4) Tasks requiring speeded responding by participants are likely to show larger effects of attending to a modality than unspeeded responding tasks.

Over the years, the results of a number of studies utilizing a range of different behavioural tasks/paradigms have delivered results that are essentially consistent with these conclusions: For example, it appears that temporal attentional processing deficits associated with trying to process more than one near-simultaneously-presented stimulus, as indexed by phenomena such as the attentional blink (AB; e.g., Arnell, 2006; Arnell and Jenkins, 2004; Duncan et al., 1996), inattentional blindness (IB; Sinnett et al., 2006), or the repetition blindness/deafness (RB/RD; Soto-Faraco and Spence, 2002), are much more severe within a particular sensory modality than between different sensory modalities. Similarly, studies of perceptual load also support the existence of essentially independent pools auditory and visual of attentional resources (Rees et al., 2001; though see Otten et al., 2000; Tellinghuisen and Nowak, 2003). Interestingly, however, other dual-task processing deficits, such as, for example, the psychological refractory period (PRP), appear to be just as large no matter whether the sequentially-presented target stimuli are presented in the same modality versus in different modalities (e.g., Pashler, 1994; Spence, 2008).

What then of the neural correlates of focusing attention on a particular sensory modality? A growing-number of neuroimaging studies have now started to highlight the neural consequences of (or mechanisms underlying) the effects of focusing attention on a particular sensory modality (e.g., Johnson and Zatorre, 2005, 2006; Kawashima et al., 1995; Roland, 1982). These studies have shown increased neural activity in the cortical areas associated with the attended modality and suppressed neural activity in the cortical areas associated with the competing or ignored modalities, as one might have expected. For example, when attention is drawn away from an auditory event by the presence of a visual stimulus, and particularly by attending to (performing) a visual task (when compared with a non-competitive baseline condition), auditory cortex (especially secondary auditory cortical areas) shows decreased activity in response to auditory stimuli (Johnson and Zatorre, 2005; Laurienti et al., 2002; Shomstein and Yantis, 2004). By analyzing the functional connectivity between the auditory and visual cortical areas in participants performing auditory and visual tasks, Johnson and Zatorre (2005, 2006) were able to highlight a reciprocal inverse relationship with decreasing visual activation correlating with increased auditory activation and vice versa.

Crossmodal attention and visual dominance

Over the years, it has frequently been claimed that humans preferentially direct their attentional resources toward the visual modality (e.g., see Posner et al., 1976; Spence et al., 2001b). Evidence in support of this claim has come from a number of sources: One intriguing example that apparently supports the notion of visual dominance comes from research on the Colavita effect (see Colavita, 1974). In his now-classic study, Colavita reported that while people find it easy to make speeded modality discrimination/detection responses to auditory and visual stimuli when they are presented in isolation, they often fail to respond to auditory stimuli when they are presented at the same time as visual targets (see also Hecht and Reiner, 2009). There has been a recent resurgence of interest in studying the Colavita effect (see Spence, in press b, for a review). Researchers have attempted to account for the Colavita visual dominance effect in terms of participants having a tendency to direct their attention toward the visual modality, perhaps to make up for the inferior alerting properties of visual stimuli (see Posner et al., 1976). At present, though, it is unclear how many of the findings taken to support an attentional account of visual dominance can, in fact, be better accounted for in terms of visual estimates of stimulus attributes simply being more accurate (i.e., less variable) than those in the other modalities (e.g., Alais and Burr, 2004; Battaglia et al., 2003; Ernst and Banks, 2002; Morgan et al., 2008).

Crossmodal links in spatial attention

People can either orient their spatial attention overtly or covertly: Overt orienting occurs when we shift our eyes, head, hands, and/or tongue in order to more efficiently process a given environmental stimulus. By contrast, covert orienting occurs when we shift our attention without making any overt movements. Covert orienting is currently of most interest to cognitive neuroscientists studying crossmodal selective attention. It should, though, be noted that many of the same neural structures control both types of attentional orienting, and indeed, covert orienting has often been shown to occur as a pre-cursor to overt orienting (e.g., Rorden and Driver, 1999; Rorden et al., 2002). The majority of studies of crossmodal spatial attention published to date have adapted the spatial cuing paradigm first popularized by Posner back in the late 1970s (e.g., Posner, 1978; see Wright and Ward, 2008, for a recent review). In terms of the crossmodal links in spatial attention, the term crossmodal is typically used to refer to situations in which the orienting of a person’s spatial attention in one sensory modality (such as vision) results in a concomitant shift of attention in one or more of their other sensory modalities (such as audition or touch) to the same location (or object; Turatto et al., 2005) at the same time. The central question for researchers interested in crossmodal attention concerns how the brain’s attentional resources are coordinated, or linked, between the various spatial senses (i.e., vision, audition, and touch). How is it that people can select just that subset of information that is relevant to their current goals from amongst the abundance of multisensory information impinging on the various sensory receptors at any one time?

Most researchers fall into one of four camps regarding the nature (and even the very existence) of crossmodal links in spatial attention. According to the modality-specific attentional resources account (see Hancock et al., 2007; Sarter, 2007; Wickens, 1992), there are relatively independent visual, auditory, and tactile attentional systems in the human brain (see Figure 1A). According to this account (which posits that crossmodal links in spatial attention do not exist), people should be able to direct their visual attention to one location (as indicated schematically by the arrow in the figure) while at the same time directing their auditory or tactile attention in different directions (since the attentional systems are independent). Meanwhile, other cognitive neuroscientists have argued that there is only a single supramodal attentional system, such that people can only attend to a single location (or object) at any given time. The claim here is that people simply cannot ‘split’ their attention between different locations simultaneously (see Figure 1B). According to the supramodal account, all stimuli, no matter what their modality, that are presented from a location that is attended should receive preferential processing relative to stimuli that are presented elsewhere.

Figure 1: Schematic illustration of four of the ways in which researchers have conceptualized how crossmodal attentional resources might be configured across the ‘spatial’ senses: A) Independent modality-specific attentional resources (Wickens, 1992); B) Single supramodal attention system (Farah et al., 1989); C) Separable-but-linked attentional systems (Spence and Driver, 1996); and D) Hierarchical supramodal plus modality-specific attentional systems (Posner, 1990).

A third possibility is that there might be some intermediate form of organization of attentional resources instead. So, for example, according to Spence and Driver’s (1996) ‘separate-but-linked’ hypothesis (see Driver and Spence, 2004, for a review), there may be separate auditory, visual, and tactile attentional systems at the earliest levels of information processing. However, these attentional systems are subsequently linked, such that people’s attention is typically focused on the same region of space in the different modalities, but importantly does not always have to be (see Figure 1C). Posner (1990) has also proposed a somewhat different hybrid attentional system, one involving interconnected modality-specific, and supramodal, attentional systems (see Figure 1D).

Of course, any one of these models of crossmodal attention can be combined with the evidence concerning people’s ability to ‘attend to a modality’ (discussed above). Accordingly, the operation of any one of the four models highlighted in Figure 1 can presumably be modified by the biasing of a person’s attention toward, or away from, a particular sensory modality (or modalities; see Spence et al., 2001a). Given such a possibility, it therefore becomes all the more difficult to try and discriminate between these various models of crossmodal attention on the basis of behavioral data alone. This is one of the principal reasons why the various techniques of cognitive neuroscience, such as functional magnetic resonance imaging (fMRI) and transcranial magnetic stimulation (TMS), are increasingly being brought to bear on this topic (e.g., Chambers et al., 2004, 2007; Kida et al., 2007).

Crossmodal links in endogenous spatial attention

So what is the empirical evidence? Well, it has been shown that if people deliberately choose to direct their spatial attention to a particular location in one sensory modality, their endogenous attention in the other modalities will tend to follow to the same location, albeit at a somewhat reduced level (that is, the attentional benefits will be smaller; see Driver and Spence, 2004, for a review). So, for example, if participants are instructed to attend to their left hand because a tactile target is more likely to be presented there (than from the other hand), visual targets will also be responded to preferentially when they are presented by the left hand (than by the other hand). Evidence in support of the ‘separate-but-linked’ hypothesis of crossmodal links in endogenous spatial attention comes from the results of studies showing that although they find it difficult, people can nevertheless still direct their attention in different directions in different modalities at the same time (albeit with reduced efficiency). So, for example, under the appropriate experimental conditions, people can preferentially process visual stimuli presented by their left hand while simultaneously showing a small but significant attentional benefit when responding to tactile stimuli presented to their right hand. Such results are inconsistent with both the modality-specific and supramodal accounts of crossmodal spatial attention, and hence have been taken to provide support for one of the hybrid models, such as the separate-but-linked account (see also Chambers et al., 2004, 2007; Driver and Spence, 2004; Kida et al., 2007).

Crossmodal links in exogenous spatial attention

But what of the crossmodal links that constrain the deployment of exogenous spatial attention? In a typical exogenous orienting study, a spatially-nonpredictive cue is presented shortly before (typically within 300 ms) a target appears on either the same or opposite side. The target may either be presented in the same or different sensory modality as the cue. Importantly, however, the target is just as likely to be presented on the same, as on the opposite, side as the cue. Even though participants are often explicitly instructed to try and ignore the cue as much as possible (since it provides no useful information with regard to the likely target location), the results of innumerable studies have now shown that participants typically cannot ignore the cue (even after 1000’s of trials). Instead, they tend to respond more rapidly (and/or accurately) to targets presented at the cued, as opposed to the uncued location, at least when the target is presented within a few hundred milliseconds of the cue. These crossmodal attention effects have been shown to occur even under conditions where the modality of the cue is completely irrelevant to the participant’s task (i.e., when auditory cues are presented prior to participants performing an unimodal visual task; see Spence, 2001). Recent studies of exogenous crossmodal spatial attention demonstrate that perceptual sensitivity is also enhanced at the cued location (McDonald et al., 2000). What is more, people tend to become aware of stimuli presented at the cued location sooner than when the same stimuli are presented elsewhere, a phenomenon known as ‘prior entry’ (McDonald et al., 2005; Spence, in press a).

The presentation of auditory, tactile, or visual spatially-nonpredictive cues results in a rapid exogenous shift of spatial attention to the cued location. This attentional shift facilitates the subsequent processing of auditory, tactile, and visual targets at that location. While early studies of crossmodal exogenous orienting tended to present the stimuli from only one cue and target location on either side of a central fixation point, more recent studies have demonstrated that crossmodal exogenous spatial orienting effects can actually be quite spatially-specific (e.g., Gray et al., 2009; see Spence et al., 2004, for a review). In fact, studies have now it now appears that the spatial distribution of a person’s exogenous attention following the presentation of a peripheral cue depends on the modality of the cue, with visual cues typically giving rise to more spatially-focused attentional effects than tactile cues, which in turn seem to cue attention more narrowly than do auditory cues (see Gray et al., 2009; Spence et al., 2004). What is more, the narrow focus of spatial attention seen following visual cuing helps to explain why visual cues have not always been shown to influence auditory performance (especially when participants have had to make auditory elevation discrimination responses; see Prime et al., 2008; Spence, 2001).

Traditionally, it was thought that exogenous spatial cuing effects were automatic (e.g., see Spence, 2001). However, the latest research has shown that both intramodal and crossmodal cuing effects may be eliminated under conditions where the participant is engaged in another attentionally-demanding perceptual task at the same time. This is true regardless of whether that task is presented in the same or different sensory modality (see Spence and Santangelo, 2009, for a review). Interestingly, multisensory cues (e.g., audiovisual or audiotactile) seem to be capable of capturing a person’s spatial attention no matter what else they may be doing at the same time (see Spence and Santangelo, 2009).

Researchers are currently debating over the extent to which such facilitatory crossmodal effects should be considered in terms of crossmodal links in exogenous spatial attention versus in terms of the results of multisensory integration, as popularized by the work of Stein and his colleagues at the single cell level (see Bolognini et al., 2005; McDonald et al., 2001; Spence et al., 2004). One way to tease these two explanations apart in the future may be in terms of relative stimulus timing: For crossmodal exogenous attentional effects should peak when the onset of the cue precedes that of the target by up to 1-200 ms, whereas multisensory integration effects should be maximal when the cue and target are presented at around the same time (see Spence et al., 2004; Shore et al., 2006).

Crossmodal spatial links in inhibition of return

As the interval between the onset of the spatially-nonpredictive peripheral cue and target lengthens beyond around 3-400 ms, participants may start to respond more slowly to targets at the cued (as compared to the uncued) location. This phenomenon, known as ‘inhibition of return’ (IOR; Posner and Cohen, 1984), is typically reported in speeded simple detection studies, but has, on occasion, also been observed in speeded discrimination tasks as well (see Klein, 2000). Spence et al. (2000b) demonstrated that IOR occurs between all possible combinations of visual, auditory, and tactile stimuli. In their study, a random sequence of auditory, visual, and tactile stimuli were presented to either side of fixation with each target stimulus requiring a speeded simple detection response.

Maintaining crossmodal correspondence following posture change

Having demonstrated the existence of crossmodal links in both endogenous and exogenous spatial attention, and in IOR, between all possible combinations of auditory, visual, and tactile stimuli, one of the most important issues currently facing crossmodal attention researchers concerns how (and even whether) the brain updates the mapping (or correspondence) between the senses when people change their posture. Note that each of our senses initially codes information according to a different frame of reference: So, for example, at the earliest stages of information processing, visual stimuli are coded retinotopically, auditory stimuli tonotopically, and tactile stimuli somatotopically. The question therefore arises at to how the various cues processed by our different senses are coordinated into a common frame of reference for the control of attention, and subsequently, action (see Pöppel, 1979; Spence and Driver, 2004).

In order to investigate whether crossmodal links in spatial attention are updated following posture change, researchers typically conduct experiments in which the participants have to cross their hands over the midline (e.g., so that their left hand is in the right side of space and their right hand in the left), or else deviate their gaze (to either the left or right) while keeping their head fixed straight ahead (Spence and Driver, 2004; Spence et al., 2008). These postural manipulations are designed to vary the mapping between the senses. The results of several such studies have now shown that crossmodal links in spatial attention (both endogenous and exogenous) are updated following such changes in posture. So, for example, people find it easier to attend to tactile stimuli presented to their left hand and to visual stimuli on their left side when their hands are placed in an uncrossed posture (so that their left hand is on the left side of their body, and their right hand on the right). By contrast, they find it easier to concentrate on their left hand and right visual stimuli when their hands are crossed over the midline (such that their left hand now lies in the right hemispace; see Driver and Spence, 2004; Spence et al., 2008, for reviews). Similarly, Spence et al. (2004) have also demonstrated that auditory cues exogenously draw people’s visual attention to their location, regardless of the position of the eyes with respect to the head. Results such as these have led many researchers to conclude that the ‘space’ in which attention is directed is itself a multisensory construction (see Spence and Driver, 2004).

Neural underpinnings of crossmodal spatial attention

One of the most heavily investigated topics currently in crossmodal attention research concerns how (and where) such crossmodal links in spatial attention are mediated in the human brain (see Spence and Driver, 2004; Wright and Ward, 2008). One popular suggestion has been that multisensory maps, such as those found in the superior colliculus (SC; where spatially aligned maps of visual, auditory, and tactile space lie one superimposed on top of the other), might mediate at least some of the spatial attentional cuing effects that have been observed behaviorally in the laboratory. (The SC is a small sub-cortical brain structure that controls overt orienting of the oculomotor system; see Stein and Meredith, 1993.) However, there is currently much debate over the extent to which exogenous crossmodal spatial attention effects (typically observed by cognitive psychologists in awake human participants) and multisensory integration effects (typically observed at the single cell level in the superior colliculus, SC, in anaesthetized animals by neurophysiologists) actually represent the same underlying neural phenomenon (see above). That is, researchers are still trying to ‘bridge the gap’ between the different cognitive neuroscience methods and the different levels of analysis at which crossmodal spatial attention is currently being studied (see McDonald et al., 2001). One currently-popular idea here is that multisensory influences on ‘unimodal’ brain areas might arise as a result of feedback or back-projection influences upon them, from multisensory convergence-zones and/or attentional control structures (Driver and Noesselt, 2008; Driver and Spence, 1998; Stein and Stanford, 2008; Wright and Ward, 2008).

Applying crossmodal attention research

It seems likely that in the years to come, our growing understanding of the nature of the crossmodal links that constrain the deployment of spatial attention will increasingly help researchers to provide guidelines to facilitate the effective design of multimodal (or multisensory) user interfaces (Ferris and Sarter, 2008; Sarter, 2007; Spence and Ho, 2008). For example, research in the field of applied psychology has already shown that people find it particularly difficult to hold a conversation on a mobile phone, while simultaneously driving a vehicle (Spence and Read, 2003; see Ho and Spence, 2008, for a review). One of the major problems in this multisensory dual-task situation (a problem, note, which is not anticipated by the modality-specific resources account of crossmodal attention) may be that people find it difficult to attend visually out of the windscreen to watch the road ahead, while simultaneously trying to listen to the voice coming from the phone by their ear (due, presumably, to the existence of robust crossmodal links in endogenous spatial attention, see above). Spence and Read proposed that performance in this situation could be improved if the speaker’s voice were to be presented from directly in front of the driver, to take advantage of the underlying crossmodal links that constrain the deployment of endogenous spatial attention (see Driver and Spence, 2004; Ho and Spence, 2008).

Similarly, a better understanding of the nature of the crossmodal links underlying exogenous attentional orienting may also lead to the design of more effective non-visual (and multisensory) warning signals: Indeed, the latest research by Ho, Reed, and Spence (2007) has shown that bimodal cues (i.e., audiovisual or audiotactile) appear to capture the spatial attention of car drivers in the driving simulator far more effectively than unisensory warning signals (at least when the various unisensory cues are presented from the same spatial location, or direction; see Spence and Santangelo, 2009, for a review).

Summary

In conclusion, research on crossmodal attention has come a long way over the last 20 years or so. Scientists have highlighted the existence of extensive crossmodal interactions in attention between the senses (Calvert et al., 2004; Spence and Driver, 2004). While the majority of studies have tended to focus on audiovisual interactions, a growing body of research now shows similar constraints operating between many other pairs of sensory modalities as well. To date, most of the research has focused on crossmodal attention in “normal” healthy adult human participants. However, there is now growing interest in understanding any changes in crossmodal attention that may occur across the lifespan (e.g., Hugenschmidt et al., in press; Poliakoff et al., 2006), and following brain-damage (Brozzoli et al., 2006; Rapp and Hendel, 2003; Sinnett et al., 2007). There has even been some progress in extending these crossmodal research paradigms / theoretical approaches to the animal domain. To give but one example, chinchillas have recently been shown to exhibit decreased cochlear sensitivity when performing a visual (but not when performing an auditory task), with the magnitude of this decrease correlating with the demands of the animal’s visual task (see Delano et al., 2007).

References

• Alais, D. and Burr, D. (2004). The ventriloquist effect results from near-optimal bimodal integration. Current Biology, 14, 257-262.

• Alais, D., Morrone, C. and Burr, D. (2006). Separate attentional resources for vision and audition. Proceedings of the Royal Society B, 273, 1339-1345.

• Arnell, K. M. (2006). Visual, auditory, and cross-modality dual-task costs: Electrophysiological evidence for an amodal bottleneck on working memory consolidation. Perception & Psychophysics, 68, 447-457.

• Arnell, K. M. and Jenkins, R. (2004). Revisiting within modality and cross-modality attentional blinks: Effects of target-distractor similarity. Perception & Psychophysics, 66, 1147-1161.

• Ashkenazi, A. and Marks, L. E. (2004). Effect of endogenous attention on detection of weak gustatory and olfactory flavors. Perception & Psychophysics, 66, 596-608.

• Battaglia, P. W., Jacobs, R. A. and Aslin, R. N. (2003). Bayesian integration of visual and auditory signals for spatial localization. Journal of the Optical Society of America A, 20, 1391-1397.

• Bolognini, N., Frassinetti, F., Serino, A. and Làdavas, E. (2005). “Acoustical vision” of below threshold stimuli: Interaction among spatially converging audiovisual inputs. Experimental Brain Research, 160, 273-282.

• Brozzoli, C., Demattè, M. L., Pavani, F., Frassinetti, F. and Farnè, A. (2006). Neglect and extinction: Within and between sensory modalities. Restorative Neurology and Neruoscience, 24, 217-232.

• Calvert, G. A., Spence, C. and Stein, B. E. (Eds.). (2004). The handbook of multisensory processes. Cambridge, MA: MIT Press.

• Chambers, C. D., Payne, J. M. and Mattingley, J. B. (2007). Parietal disruption impairs reflexive spatial attention within and between sensory modalities. Neuropsychologia, 45, 1715-1724.

• Chambers, C. D., Stokes, M. G. and Mattingley, J. B. (2004). Modality-specific control of strategic spatial attention in parietal cortex. Neuron, 44, 925-930.

• Colavita, F. B. (1974). Human sensory dominance. Perception & Psychophysics, 16, 409-412.

• Delano, P. H., Elgueda, D., Hamame, C. M. and Robles, L. (2007). Selective attention to visual stimuli reduces cochlear sensitivity in chinchillas. Journal of Neuroscience, 27, 4146-4153.

• Driver, J. (2001). A selective review of selective attention research from the past century. British Journal of Psychology, 92, 53-78.

• Driver, J. and Noesselt, T. (2008). Multisensory interplay reveals crossmodal influences on ‘sensory-specific’ brain regions, neural responses, and judgments. Neuron, 57, 11-23.

• Driver, J. and Spence, C. (1998). Crossmodal attention. Current Opinion in Neurobiology, 8, 245-253.

• Driver, J. and Spence, C. (2004). Crossmodal spatial attention: Evidence from human performance. In C. Spence and J. Driver (Eds.), Crossmodal space and crossmodal attention (pp. 179-220). Oxford, UK: Oxford University Press.

• Duncan, J., Martens, S. and Ward, R. (1997). Restricted attentional capacity within but not between sensory modalities. Nature, 387, 808-810.

• Ernst, M. O. and Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415, 429-433.

• Farah, M. J., Wong, A. B., Monheit, M. A. and Morrow, L. A. (1989). Parietal lobe mechanisms of spatial attention: Modality-specific or supramodal? Neuropsychologia, 27, 461-470.

• Ferris, T. K. and Sarter, N. B. (2008). Cross-modal links among vision, audition, and touch in complex environments. Human Factors, 50, 17-26.

• Gray, R., Mohebbi, R. and Tan, H. Z. (2009). The spatial resolution of crossmodal attention: Implications for the design of multimodal interfaces. ACM Transactions on Applied Perception, 6, 1-14.

• Hancock, P. A., Oron-Gilad, T. and Szalma, J. L. (2007). Elaborations of the multiple-resource theory of attention. In A. F. Kramer, D. A. Wiegmann, and A. Kirlik (Eds.), Attention: From theory to practice (pp. 45-56). Oxford: Oxford University Press.

• Hecht, D. and Reiner, M. (2009). Sensory dominance in combinations of audio, visual and haptic stimuli. Experimental Brain Research, 193, 307-314.

• Ho, C., Reed, N. and Spence, C. (2007). Multisensory in-car warning signals for collision avoidance. Human Factors, 49, 1107-1114.

• Ho, C. and Spence, C. (2008). The multisensory driver: Implications for ergonomic car interface design. Aldershot: Ashgate Publishing.

• Hugenschmidt, C. E., Peiffer, A. M., McCoy, T. P., Hayasaka, S. and Laurienti, P. J. (in press). Preservation of crossmodal selective attention in healthy aging. Experimental Brain Research.

• Johnson, J. A. and Zatorre, R. J. (2005). Attention to simultaneous unrelated auditory and visual events: Behavioral and neural correlates. Cerebral Cortex, 15, 1609-1620.

• Johnson, J. A. and Zatorre, R. J. (2006). Neural substrates for dividing and focusing attention between simultaneous auditory and visual events. Neuroimage, 31, 1673-1681.

• Kawashima, R., O'Sullivan, B. T. and Roland, P. E. (1995). Positron-emission tomography studies of cross-modality inhibition in selective attentional tasks: Closing the "mind's eye". Proceedings of the National Academy of Science, USA, 92, 5969-5972.

• Kida, T., Inui, K., Wasaka, T., Akatsuka, K., Tanaka, E.and Kakigi, R. (2007). Time-varying cortical activations related to visual-tactile cross-modal links in spatial selective attention. Journal of Neurophysiology, 97, 3585-3596.

• Klein, R. (2000). Inhibition of return. Trends in Cognitive Sciences, 4, 138-147.

• Laurienti, P. J., Burdette, J. H., Wallace, M. T., Yen, Y.-F., Field, A. S.and Stein, B. E. (2002). Deactivation of sensory-specific cortex by cross-modal stimuli. Journal of Cognitive Neuroscience, 14, 1-10.

• Lavie, N. (2005). Distracted and confused?: Selective attention under load. Trends in Cognitive Sciences, 9, 75-82.

• McDonald, J. J., Teder-Sälejärvi, W. A. and Hillyard, S. A. (2000). Involuntary orienting to sound improves visual perception. Nature, 407, 906-908.

• McDonald, J. J., Teder-Sälejärvi, W. A., Di Russo, F. and Hillyard, S. A. (2005). Neural basis of auditory-induced shifts in visual time-order perception. Nature Neuroscience, 8, 1197-1202.

• McDonald, J. J., Teder-Sälejärvi, W. A. and Ward, L. M. (2001). Multisensory integration and crossmodal attention effects in the human brain. Science, 292, 1791.

• Morgan, M. L., DeAngelis, G. C. and Angelaki, D. E. (2008). Multisensory integration in macaque visual cortex depends on cue reliability. Neuron, 59, 662-673.

• Otten, L. J., Alain, C. and Picton, T. W. (2000). Effects of visual attentional load on auditory processing. NeuroReport, 11, 875-880.

• Pashler, H. (1994). Dual-task interference in simple tasks: Data and theory. Psychological Bulletin, 116, 220-244.

• Poliakoff, E., Ashworth, S., Lowe, C. and Spence, C. (2006). Vision and touch in ageing: Crossmodal selective attention and visuotactile spatial interactions. Neuropsychologia, 44, 507-517.

• Pöppel, E. (1973). Comments on "Visual system's view of acoustic space". Nature, 243, 231.

• Posner, M. I. (1978). Chronometric explorations of mind. Hillsdale, NJ: Erlbaum.

• Posner, M. I. (1990). Hierarchical distributed networks in the neuropsychology of selective attention. In A. Caramazza (Ed.), Cognitive neuropsychology and neurolinguistics: Advances in models of cognitive function and impairment (pp. 187-210). Hillsdale, NJ: Erlbaum.

• Posner, M. I. and Cohen, Y. (1984). Components of visual orienting. In H. Bouma and D. G. Bouwhuis (Eds.), Attention and performance: Control of language processes (Vol. 10, pp. 531-556). Hillsdale, NJ: Erlbaum.

• Posner, M. I., Nissen, M. J. and Klein, R. M. (1976). Visual dominance: An information-processing account of its origins and significance. Psychological Review, 83, 157-171.

• Prime, D. J., McDonald, J. J., Green, J. and Ward, L. M. (2008). When crossmodal attention fails: A controversy resolved? Canadian Journal of Experimental Psychology, 62, 192-197.

• Rapp, B. and Hendel, S. K. (2003). Principles of cross-modal competition: Evidence from deficits of attention. Psychonomic Bulletin & Review, 10, 210-219.

• Rees, G., Frith, C. and Lavie, N. (2001). Processing of irrelevant visual motion during performance of an auditory attention task. Neuropsychologia, 39, 937-949.

• Roland, P. E. (1982). Cortical regulation of selective attention in man. A regional cerebral blood flow study. Journal of Neurophysiology, 48, 1059-1078.

• Rorden, C. and Driver, J. (1999). Does auditory attention shift in the direction of an upcoming saccade? Neuropsychologia, 37, 357-377.

• Rorden, C., Greene, K., Sasine, G. M. and Baylis, G. C. (2002). Enhanced tactile performance at the destination of an upcoming saccade. Current Biology, 12, 1-6.

• Sarter, N. B. (2007). Multiple-resource theory as a basis for multimodal interface design: Success stories, qualifications, and research needs. In A. F. Kramer, D. A. Wiegmann and A. Kirlik (Eds.), Attention: From theory to practice (pp. 187-195). Oxford: Oxford University Press.

• Shomstein, S. and Yantis, S. (2004). Control of attention shifts between vision and audition in human cortex. Journal of Neuroscience, 24, 10702-10706.

• Shore, D. I., Barnes, M. E. and Spence, C. (2006). The temporal evolution of the crossmodal congruency effect. Neuroscience Letters, 392, 96-100.

• Sinnett, S., Costa, A. and Soto-Faraco, S. (2006). Manipulating inattentional blindness within and across sensory modalities. Quarterly Journal of Experimental Psychology, 59, 1425-1442.

• Sinnett, S., Juncadella, M., Rafal, R., Azañón, E and Soto-Faraco, S. (2007). A dissociation between visual and auditory hemi-inattention: Evidence from temporal order judgements. Neuropsychologia, 45, 552-560.

• Soto-Faraco, S. and Spence, C. (2002). Modality-specific auditory and visual temporal processing deficits. Quarterly Journal of Experimental Psychology (A), 55, 23-40.

• Soto-Faraco, S. Spence, C., Fairbank, K., Kingstone, A., Hillstrom, A. P. and Shapiro, K. (2002). A crossmodal attentional blink between vision and touch. Psychonomic Bulletin & Review, 9, 731-738.

• Spence, C. (2001). Crossmodal attentional capture: A controversy resolved? In C. Folk and B. Gibson (Eds.), Attention, distraction and action: Multiple perspectives on attentional capture. Advances in psychology, 133 (pp. 231-262). Amsterdam: Elsevier Science BV.

• Spence, C. (2008). Cognitive neuroscience: Searching for the bottleneck in the brain. Current Biology, 18, R965-R968.

• Spence, C. (in press a). Prior entry: Attention and temporal perception. To appear in A. C. Nobre and J. T. Coull (Eds.), Attention and time. Oxford: Oxford University Press.

• Spence, C. (in press b). Explaining the Colavita visual dominance effect. Progress in Brain Research.

• Spence, C., Bentley, D. E., Phillips, N., McGlone, F. P. and Jones, A. K. P. (2002). Selective attention to pain: A psychophysical investigation. Experimental Brain Research, 145, 395-402.

• Spence, C. and Driver, J. (1996). Audiovisual links in endogenous covert spatial attention. Journal of Experimental Psychology: Human Perception and Performance, 22, 1005-1030.

• Spence, C. and Driver, J. (Eds.) (2004). Crossmodal space and crossmodal attention. Oxford, UK: Oxford University Press.

• Spence, C. and Ho, C. (2008). Multisensory warning signals for event perception and safe driving. Theoretical Issues in Ergonomics Science, 9, 523-554.

• Spence, C., Kettenmann, B., Kobal, G. and McGlone, F. P. (2000a). Selective attention to the chemosensory modality. Perception & Psychophysics, 62, 1265-1271.

• Spence, C., Lloyd, D., McGlone, F., Nicholls, M. E. R. and Driver, J. (2000b). Inhibition of return is supramodal: A demonstration between all possible pairings of vision, touch and audition. Experimental Brain Research, 134, 42-48.

• Spence, C., Nicholls, M. E. R. and Driver, J. (2001a). The cost of expecting events in the wrong sensory modality. Perception & Psychophysics, 63, 330-336.

• Spence, C., Pavani, F., Maravita, A. and Holmes, N. P. (2008). Multi-sensory interactions. In M. C. Lin and M. A. Otaduy (Eds.), Haptic rendering: Foundations, algorithms, and applications (pp. 21-52). Wellesley, MA: AK Peters.

• Spence, C. and Read, L. (2003). Speech shadowing while driving: On the difficulty of splitting attention between eye and ear. Psychological Science, 14, 251-256.

• Spence, C. and Santangelo, V. (2009). Capturing spatial attention with multisensory cues. Hearing Research. http://dx.doi.org/10.1016/j.heares.2009.04.015

• Spence, C., Shore, D. I. and Klein, R. M. (2001b). Multisensory prior entry. Journal of Experimental Psychology: General, 130, 799-832.

• Stein, B. E. and Meredith, M. A. (1993). The merging of the senses. Cambridge, MA: MIT Press.

• Stein, B. E. and Stanford, T. R. (2008). Multisensory integration: Current issues from the perspective of the single neuron. Nature Reviews Neuroscience, 9, 255-267.

• Tellinghuisen, D. J. and Nowak, E. J. (2003). The inability to ignore auditory distractors as a function of visual task perceptual load. Perception & Psychophysics, 65, 817-828.

• Treisman, A. M. and Davies, A. (1973). Divided attention to ear and eye. In S. Kornblum (Ed.), Attention and performance (Vol. 4, pp. 101-117). New York: Academic Press.

• Turatto, M., Benso, F., Galfano, G., Gamberini, L. and Umilta, C. (2002). Non-spatial attentional shifts between audition and vision. Journal of Experimental Psychology: Human Perception & Performance, 28, 628-639.

• Turatto, M., Galfano, G., Bridgeman, B. and Umiltà, C. (2004). Space-independent modality-driven attentional capture in auditory, tactile and visual systems. Experimental Brain Research, 155, 301-310.

• Turatto, M., Mazza, V. and Umiltà, C. (2005). Crossmodal object-based attention: Auditory objects affect visual processing. Cognition, 96, B55-B64.

• Wickens, C. D. (1992). Engineering psychology and human performance (2nd. Edition). NY: HarperCollins.

• Wright, R. D. and Ward, L. M. (2008). Orienting of attention. Oxford: Oxford University Press.

Recommended reading

• Spence, C. and Driver, J. (Eds.) (2004). Crossmodal space and crossmodal attention. Oxford, UK: Oxford University Press.

• Wright, R. D. and Ward, L. M. (2008). Chapter 8: Crossmodal attention shifts. In R. D. Wright and L. M. Ward, Orienting of attention (pp. 199-227). Oxford: Oxford University Press.

External links

University of Oxford, Department of Experimental Psychology, Crossmodal Research Laboratory: http://psyweb.psy.ox.ac.uk/xmodal/default.htm

Sponsored by: Naotsugu Tsuchiya, California Institute of Technology, Pasadena, CA, USA

Reviewed by: Anonymous

Accepted on: 2010-03-17 17:41:08 GMT