- Email: [email protected]
Effects of auditory and tactile warning on response to visual hazards under a noisy environment
Applied Ergonomics 60 (2017) 58e67
Contents lists available at ScienceDirect
Applied Ergonomics journal homepage: www.elsevier.com/locate/apergo
Effects of auditory and tactile warning on response to visual hazards under a noisy environment Atsuo Murata a, *, Takashi Kuroda a, Waldemar Karwowski b a b
Dept. of Intelligent Mechanical Systems, Graduate School of Natural Science and Technology, Okayama University, Okayama, Japan Dept. of Industrial Engineering & Management Systems, University of Central Florida, Orlando, USA
a r t i c l e i n f o
a b s t r a c t
Article history: Received 22 April 2016 Received in revised form 8 August 2016 Accepted 4 November 2016
A warning signal presented via a visual or an auditory cue might interfere with auditory or visual information inside and outside a vehicle. On the other hand, such interference would be certainly reduced if a tactile cue is used. Therefore, it is expected that tactile cues would be promising as warning signals, especially in a noisy environment. In order to determine the most suitable modality of cue (warning) to a visual hazard in noisy environments, auditory and tactile cues were examined in this study. The condition of stimulus onset asynchrony (SOA) was set to 0ms, 500ms, and 1000ms. Two types of noises were used: white noise and noise outside a vehicle recorded in a real-world driving environment. The noise level LAeq (equivalent continuous A-weighted sound pressure level) inside the experimental chamber of each type of noise was adjusted to approximately 60 dB (A), 70 dB (A), and 80 dB (A). As a result, it was verified that tactile warning was more effective than auditory warning. When the noise outside a vehicle from a real-driving environment was used as the noise inside the experimental chamber, the reaction time to the auditory warning was not affected by the noise level. © 2016 Elsevier Ltd. All rights reserved.
Keywords: Auditory cue Tactile cue Automotive warning system Noise outside a vehicle White noise SOA
1. Introduction As the information on an automotive display contains a lot of text and images, drivers tend to be distracted by the displays and become inattentive to a variety of hazards. Driving a car imposes a characteristically heavy workload on visual perception, cognitive information processing, and manual responses. Drivers often simultaneously perform two or more tasks; for example, they adjust the volume of a radio or CD player while driving. Such sharing of attention may lead to slow reactions to a warned situation or larger deviations of own vehicle and induce fatal or nonfatal crashes (Young and Stanton, 2004; Stanton and Young, 2005). With the progress in by-wire and information technology, visual and cognitive workload while driving increases, and the driver-vehicle interaction is getting increasingly complicated (Lee et al., 2001; Dukic et al., 2006; Gkikas, 2013; Castro, 2009). Consequently, drivers tend to be distracted by a variety of
* Corresponding author. Dept. of Intelligent Mechanical Systems, Graduate School of Natural Science and Technology, Okayama University, Okayama 3-1-1, Tsushimanaka, Kita-ku, Okayama, 700-8530, Japan. E-mail addresses: [email protected] (A. Murata), [email protected] (W. Karwowski). http://dx.doi.org/10.1016/j.apergo.2016.11.002 0003-6870/© 2016 Elsevier Ltd. All rights reserved.
secondary tasks such as the operation of switches for a CD player or air conditioner while driving (Regan et al., 2009), which increases the risk of inattentive driving. Lees and Lee (2007) showed that an automotive collision warning system (CWS) can enhance hazard identification and management if false alarms are tolerated. Lee, McGehee, Brown, and Marshall (2006) examined the effectiveness of various warning modalities in making distracted drivers attentive during severe braking situations that exceeded adaptive cruse control (ACC) capability. The power of touch via tactile sense was suggested by Denworth (2015) and Linden (2015). The potential applicability of the tactile sense to automotive warning systems has gained more attention towards enhancing driver safety (Jones and Sarter, 2008). Jones and Sarter (2008) reviewed the utilization of the sense of touch as a medium for information representation. They concluded that sense of touch represents a promising means for communications in human-vehicle systems. Recently, the tendencies of cross-modal information processing and design (Driver and Spence, 1998; Driver, 2001; Spence and Driver, 1994, 1997a, 1997b, 2004; Jones et al., 2008) have emerged as a major research topic in the design of automotive warning systems. Presenting information via multiple modalities such as vision, audition, and touch is thought to be a promising way to
A. Murata et al. / Applied Ergonomics 60 (2017) 58e67
reduce transmission errors and enhance safety. A better understanding of spatial and temporal cross-modal links is essential to enable better application of these properties to automotive warning design. Ferris et al. (2006) and Ferris and Sarter (2008) found significant asymmetric performance effects of cross-modal spatial links on the basis of the following results. Auditory cues shortened response latencies for collocated visual targets, while a similar result was not observed for visual cues. Responses to contra-lateral targets were faster for tactually cued auditory targets and each combination of visual and tactile cues. Fitch et al. (2007) showed that tactile warning reduced correct localization response times by 257 ms and increased the percentage of correct localization by 52% (from 32% to 84%) relative to auditory warning, which indicates that tactile warning by seat vibration might be effective for warning drivers of the direction of a crash threat. The visual and auditory perceptual systems are more activated in driving than the tactile perceptual system. Therefore, tactile warning is expected to lead to a faster response to a hazard. It has been shown that tactile warning presentation is effective in a driving environment where hazards are ubiquitous Ho et al., 2006c; Van Erp, 2005; Van Erp and Veen, 2001; Van Erp and Veen, 2003). Ho and Spence (2005), Ho et al. (2006a, 2006c), and Ho and Spence (2008) explored the effectiveness of tactile warning for front and rear locations. Murata and Tanaka (2011), Murata et al. (2012a) and Murata and Nakagawa (2012)examined how tactile warning is effective for promoting responses to hazards in left and right locations. Moreover, Murata et al. (2012b) and Murata et al. (2013) investigated the effectiveness of tactile warning by apparent movement. They found that tactile warning by apparent movement can more quickly transmit directional cues than the simultaneous stimulation of two tactors or a single-point stimulation. Scott and Gray (2008) engaged in the development of assistance systems, in particular, non-visual and multisensory collisionwarning systems for drivers, and examined how the presentation of auditory, tactile, and multisensory warning signals is effective in alerting a driver and rapidly orienting his or her spatial attention to the direction of the potential danger. Like many other studies, they demonstrated that the use of tactile warning improved the response of drivers to time-critical events that could potentially result in a crash. Meng, Gray, Ho, Ahtamad, and Spence (2015) assessed the effectiveness of various tactile warnings using a simulated car-following task, and suggested that dynamic tactile cues that look as though they are approaching the torso can be effectively used to communicate information concerning external events. If a warning signal is presented via visual or auditory stimulus, auditory or visual interference with other information, such as the noise outside a vehicle, might arise. On the other hand, if a tactile cue is used, such interference between the same (auditory or visual) modalities does not occur, because much information on the driving environment is visual or auditory and there are few cases where drivers are exposed to high vibration in recent road environment (Refer to Fig. 10 as for the effectiveness (less susceptibility to vibration) of tactile cue in a real-world running). Therefore, it is expected that a tactile signal would be very promising as a warning signal especially in a noisy environment. However, the studies cited above on the effectiveness of tactile warning systems have only paid attention to the advantage of tactile cues, and did not compare the response speed between tactile and auditory cues and verify the effectiveness of tactile cues under a high level of background noise. Although Gray (2011) showed that looming warnings had significantly faster brake reaction times as compared with the other non-looming warnings, the effect of the ambient noise outside of the vehicle on the response speed to an auditory warning was not
59
discussed. Murata et al. (2014) verified that tactile warning was more and more effective with an increase in the noise level inside the experimental chamber. The reaction time to auditory warning was significantly affected by the noise level, while the reaction time to tactile warning was not affected by the noise level at all. Moreover, the stimulus onset asynchrony (SOA) condition did not remarkably affect the reaction time to either auditory or tactile warnings. However, they used white noise as the background noise surrounding the experimental chamber, though it is desirable to use noise recorded in a real-world driving environment. As the auditory cue was also a pure tone of 1 kHz, it is possible that the perceptibility of the auditory cue might be significantly affected by the spectral power density of the surrounding noise of a similar type (Reference to Fig. 6). Therefore, it was hypothesized that if the interference between the surrounding (background) noise and the auditory cue was mitigated, it might be possible that the auditory cue is effective even under a high noise level. It was explored whether similar results to Murata et al. (2014) could be obtained even when the noise outside a vehicle from a real-driving environment was used. In order to determine the most suitable modality of cue (warning) to a visual hazard under noisy environment, and to examine the effect of the type of ambient noise on the processing speeds of auditory and tactile cues, the following four experimental factors were used, and the effects of the experimental factors on the reaction to a target (hazard) were examined: (1) cue modality (auditory or tactile cue), (2) SOA (0 ms, 500 ms, and 1000 ms), (3) type of surrounding noise inside an experimental chamber (white noise and noise outside a vehicle in a real-driving environment) and (4) noise level LAeq (equivalent continuous A-weighted sound pressure level) inside the experimental chamber (about 60 dB(A), 70 dB(A), and 80 dB(A)). 2. Method 2.1. Participants Ten healthy males aged from 21 to 24 years took part in the experiment. All participants had held a driver's license for 3e4 years. All participants provided informed consent after receiving a brief explanation of the aim and the contents of the experiment. 2.2. Apparatus A PC (Mouse Computer, MDV-ADS7210S) and a projector (NEC, LT245, 2200 lumens) were used to control a simulated driving task and a reaction task to a visual hazard. A steering controller (Logicool, LPRC-14000) was connected to the PC so that the participant could operate the vehicle on the simulated driving display. The participant was required to operate both accelerator and brake, and maintain a speed between 70 km/h and 80 km/h in the simulated driving task. The display (I.O DATA, LCD-A154VH-V) for maintaining the velocity within the predetermined range was installed 326 mm from the steering controller. The steering controller (Logicool, LPRC-14000) used for this task was attached with an accelerator and brake. For the presentation of the auditory cue, two speakers (Onkyo, Powered Speaker System OP1) were placed in front of the participant. Speakers were connected to the PC to control the auditory cue. The auditory cue was a pure tone of 1 kHz, and presented to the ears of a participant with LAeq of about 60 dB(A). LAeq (equivalent continuous A-weighted sound pressure level during 10 s) was measured using a sound level meter (RION, NL-42). The value (about 60 dB(A)) of LAeq was empirically determined so that the auditory cue is audible enough and not too noisy. As it was impossible to adjust LAeq to just 60 dB(A) due to the property of LAeq
60
A. Murata et al. / Applied Ergonomics 60 (2017) 58e67
(this measure is always variable around a fixed value), this was set to about 60 dB(A). Another speaker (Elecom, MS-131SV) was used to present the noise. This speaker was installed behind the participant and connected to the PC (Acer, Aspire V5). The installation of the three speakers is depicted in Fig. 1. Four tactors with a diameter of 45 mm (Acouve Laboratory Inc., Vp216) were installed on the surface of the driver's seat as shown in Fig. 2. The tactors were connected to the PC for controlling the simulated driving task to control their outputs. In the studies of tactile warning system (Ho et al., 2005; 2006a; Meng et al., 2015), the characteristics of tactors are usually represented using not velocity or acceleration but frequency. Therefore, the characteristics of tactile cue were described as follows so that reproducibility is assured. The frequency and the intensity of outputs were empirically set to 64 Hz and 10 V respectively using a function generator (Gw Instek, SFG-2004) so that the participant could detect the vibration most sensitively. Many previous studies attached tactors around the waist or torso of participants (Meng et al., 2015; Ho et al., 2005; 2006a, 2006b, 2006c). From a practical viewpoint, drivers tend to avoid attaching tactors around their waist or torso. Therefore, it is more plausible and practical to place tactors on the surface of the driver's seat. Fig. 2. Location of tactors on the surface of driver's seat.
2.3. Task Using the driving simulator system, the participants were required to simultaneously carry out a simulated driving task (main task) and a reaction task to a hazard randomly presented on one of two locations (on the right or left of the display of driving simulator) (secondary task). In the primary driving simulator task, the participant was required to minimize the deviation from the predetermined line, and maintain lane location using a steering wheel. The participant was also required to operate both the accelerator and brake, and maintain a speed of between 70 km/h and 80 km/h (see Fig. 3). This was programmed using an interpreter type programming language HSP (Hot Soup Processor). In the secondary task (reaction task to a hazard), the participant was required to react to a visual hazard (see Fig. 4) presented on the left or the right of the display in front of the participant using the brake. A warning was presented to the participant 0 ms, 500 ms,
and 1000 ms before a visual hazard appeared. In other words, the SOA (Stimulus Onset Asynchrony) was set to 0ms, 500ms, and 1000ms (see Fig. 5). The warning was presented to the left or the right display using the following warning presentation modes (cues): (1) auditory cue (pure tone of 1 kHz) and (2) tactile cue. Catch trials were added to the secondary task so that premature responses were avoided. Catch trials randomly occupied 2 trials out of the total number of trials (12 trials). In the catch trial, the visual hazard was not presented to the participant, and only the warning was presented to the participant. The participant had to confirm that the visual hazard existed whenever the warning was presented. When the participant detected the visual hazard, he was required to brake quickly and decelerate the vehicle to 30 km/h. After this procedure was completed, the participant was required to increase the vehicle velocity to between 70 km/h and 80 km/h
Fig. 1. Outline of experimental system.
A. Murata et al. / Applied Ergonomics 60 (2017) 58e67
Fig. 3. Display of velocity maintenance task.
again, and maintain the velocity within this range. The auditory cue was presented to the participant via two speakers placed (1200 mm away from the participant) in front of the participant. The tactile cue was transmitted to the participant via four tactors (vibrotransducers) (see Fig. 2) placed on the sitting surface such that these tactors were in contact with the left and the right thighs. The cue to the left or right target was presented via two tactors each (two tactors were placed on the left thigh, and two tactors were on the right thigh). For both (a) white noise and (b) noise outside the vehicle, the background noise level LAeq (equivalent continuous A-weighted sound pressure level) inside the experimental chamber was adjusted to approximately 60 dB(A), 70 dB(A), and 80 dB(A). The power spectral densities of the white noise and the noise from outside a vehicle recorded in a real-world driving environment are demonstrated in Fig. 6(a) and (b), respectively. 2.4. Design and procedure The temperature within the experimental chamber was controlled to 25 C. The humidity (although not controllable) ranged from 20 to 25%. Experimental factors were noise type (white noise and noise outside a vehicle), warning modality (auditory and tactile), background noise level LAeq (equivalent continuous Aweighted sound pressure level) inside the experimental chamber (approximately 60 dB(A), 70 dB(A), and 80 dB(A)), and SOA (0 ms, 500 ms, and 1000 ms). All of the above were within-subject variables. It is certain that the warning should be presented to a driver as soon as a hazard was detected. However, it is expected that the reaction to a cue (whether is tactile or auditory) for SOA of 0 ms is delayed as compared to that for SOA of 500ms and 1000ms. Therefore, three SOAs (0, 500, and 1000 ms) were used to examine whether a cue presented before a hazard appears leads to the enhanced preparedness of a participant and thus to a faster and accurate reaction to a visual hazard. First, the participant was required to adjust the seat so that they could conduct the experimental task comfortably. After explaining the primary and secondary tasks described above to the participant, the participant was permitted to practice each task separately, and practice both tasks simultaneously. After the participant reported that he had practiced enough and completely understood the tasks, and the experimenter confirmed this, the experiment was started. According to Ferris and Sarter (2008), the duration of both auditory and tactile cues were determined as 1000 ms. For the tactile cues, the left and right visual hazards were cued by two tactors installed on the left and the right, respectively. In the case of the auditory
61
cues, left and right visual hazards were cued by two speakers installed on the front left and the front right, respectively. Participants were required to carry out both the primary tracking task and the secondary task simultaneously as quickly and accurately as possible for 15 min for each condition. Participants were also required to firstly confirm a visual hazard and then react to it. Experiments under white noise and noise from outside a vehicle in real driving conditions were conducted one day apart for all participants. The order of performance of the two noise types was counterbalanced across the participants. The six conditions of warning modality (two levels) and noise level (three levels) were randomly carried out by each participant for each noise type. For each condition, twelve trials (two catch trials were included) were conducted for each SOA. The presentation of SOA was also randomized in a total of 36 trials. A break of about 3 min was inserted between two conditions. It took about 105 min to complete the experiment for each noise type. The evaluation measures were: the reaction time to a visual hazard, the percentage incorrect reactions to a visual hazard, and the tracking error in the simulated driving task from the onset of a cue to the end of the response.
2.5. Comparison of sensitivity to tactile cue between real-world running and laboratory experiments It is not certain whether tactile warning is effective under the real-world running environment. Therefore, the sensitivities to a tactile warning (cue) were also investigated as a function of four placement sites of tactors on the driver's seat in Fig. 2 (front left, front right, rear left, rear right) and compared between real-world running and laboratory experiments. Ten participants above also took part in this experiment. The experiment was conducted on a separate day. In the laboratory experiment, using the same main task (simulated driving task), the sensitivity to a tactile cue was investigated as a function of location of tactors. The sensitivity to a tactile cue was measures by adjusting the frequency of tactor via a function generator. The sensitivity was measured using both ascending and descending sequences. The frequency ranged from 20 Hz to 200 Hz. In the ascending sequence, the initial value of sensitivity (20 Hz) was increased by 10 Hz, and the threshold value of each participant was determined. Each participant was required to tell the experimenter the frequency at which he began to feel the vibration. In the descending sequence, the initial value of sensitivity (200 Hz) was decreased by 10 Hz, and the threshold value was determined. Each participant was required to tell the experimenter the frequency at which the participant could not percept the vibration any more. The mean frequency of ascending and descending sequence was also calculated. The order of four locations was randomized across the participants. The order of ascending and descending measurements was counterbalanced across the participants. For each location of tactor, a total of ten measurements were made for both ascending and descending conditions. Therefore, a total of 80 measurements were conducted. In the real-world running experiment, the sensitivity was measured when running about 20 km along a highway with a mean speed of 55 km/h (this was within a legally permitted speed). The participant was required to obey the same procedure as the laboratory experiment. According to a similar procedure to the laboratory experiment, the ascending, descending, and the mean frequencies were obtained. Two experimenters rode the vehicle (Toyota, WISH 1.8X) together, and engaged in the measurement. A total of 80 measurements were conducted only during running.
62
A. Murata et al. / Applied Ergonomics 60 (2017) 58e67
Fig. 4. Explanation of visual hazard.
2.6. Statistical rationale for analyzing effects of experimental factors on reaction time of judgment to visual hazard Although there were four experimental factors (SOA, cue modality, background noise type, and background noise level) that affected the performance measures, in particular, reaction time for the judgment of a visual hazard, it is generally not recommended to carry out a four-way ANOVA, because a four-way interaction is not so easy to interpret. Therefore, two-way and three-way ANOVAs were conducted to examine how the experimental factors affected performance measures such as reaction time according to the
following strategy. (i) As it was judged that the more important factors were the background (ambient) noise type and the cue modality, a two-way (background noise type by cue modality) ANOVA was conducted in the first place to investigate how these two factors affected performance measures. (ii) Second, the combination of background noise and cue modality, that is, the cue presentation condition (which cue modality is presented under which background noise type) was regarded as one experimental factor, and a three-way
A. Murata et al. / Applied Ergonomics 60 (2017) 58e67
63
(cue presentation condition by background noise level by SOA) was carried out. (iii) If there existed an experimental factor that is not involved in a significant interaction, two- or three-way ANOVA was further conducted for each level of this factor after this factor was excluded from ANOVA. 3. Results 3.1. Percentage incorrect reactions
Fig. 5. Explanation of SOA (Stimulus Onset Asynchrony).
The incorrect reactions included the following: a no-brake reaction, and a reaction to a catch trial. For each combination of cue modality and background (ambient) noise type (four condition), each participant conducted a total of 108 judgment trials. The percentage incorrect reactions for each participant was defined as
Fig. 6. Power spectrum of (a)white noise and (b)noise outside a vehicle that includes a variety of noise during driving.
64
A. Murata et al. / Applied Ergonomics 60 (2017) 58e67
the percentage of incorrect reactions to a total number of reactions to a visual hazard (108 trials). The mean percentage incorrect reactions was very low and as follows: auditory cue under white noise: 0.28%, tactile cue under white noise: 0.28%, auditory cue under noise outside a vehicle: 0%, and tactile cue under noise outside a vehicle: 0.28%. The cue and the noise did not affect the error. 3.2. Tracking error in a simulated driving task A two-way (background noise type by noise level) ANOVA (Analysis of Variance) conducted on the tracking error revealed no significant main effects, and an interaction. As a result of carrying out a three-way (cue presentation condition (combination of noise type and cue) by noise level by SOA) ANOVA on the percentage error trials, no significant main effects and interactions were detected. The summary of the tracking errors is as follows: auditory cue under white noise: 0.37 m, tactile cue under white noise: 0.4 m, auditory cue under noise outside a vehicle: 0.39 m, and tactile cue under noise outside a vehicle: 0.38 m. A similar two-way (noise type by noise level) ANOVA conducted on the tracking error revealed no significant main effects and an interaction. A similar three-way (cue presentation condition (combination of noise type and cue) by noise level by SOA) ANOVA on the tracking error detected no significant main effects or interactions. Thus, the tracking error in the simulated driving task was not affected by noise type, noise level, SOA, or cue modality.
Fig. 8. Reaction time as a function of background noise level and cue modality (SOA:500ms). (a) white noise, (b) noise outside of a vehicle. *: p < 0.05, **: p < 0.01.
3.3. Reaction time Figs.7e9 show reaction time as a function of noise level and cue modality when SOA is 0 ms, 500 ms, and 1000 ms respectively. According to the strategy stated in Section 2.6, ANOVA was conducted. First, according to strategy (i), a two-way (background (ambient) noise type by cue modality) ANOVA was performed on the reaction time, detecting significant main effects of noise type (F (1,9 ¼ 11.943, p < 0.01) and cue (F (1,9) ¼ 47.173, p < 0.01). The background noise type by cue modality interaction was not significant. Second, according to strategy (ii), a three-way (cue presentation condition (background noise type, cue modality) by background noise level of by SOA) ANOVA was then conducted on the reaction time. Significant main effects of cue presentation condition (F
Fig. 7. Reaction time as a function of background noise level and cue modality (SOA:0ms). (a) white noise, (b) noise outside of a vehicle. *: p < 0.05, **: p < 0.01.
Fig. 9. Reaction time as a function of background noise level and cue modality (SOA:1000ms). (a) white noise, (b) noise outside of a vehicle. *: p < 0.05, **: p < 0.01.
(3,27) ¼ 10.865, p < 0.01), background noise level (F (2,18) ¼ 6.385, p < 0.01), and SOA (F (2,18) ¼ 236.151, p < 0.01) were detected. A significant interaction between cue presentation condition (background noise type, cue modality) and background noise level was also found (F (6,54) ¼ 14.176, p < 0.01). As the factor SOA was not involved in any significant interaction in three-way ANOVA above, according to strategy (iii), a three-way (background noise level by cue modality by noise type) ANOVA performed on the reaction time for SOA of 0ms revealed significant main effects of background noise type (F (1,9) ¼ 33.588, p < 0.01), cue modality (F (1,9) ¼ 6.747, p < 0.05), and background noise level (F (2,18) ¼ 12.463, p < 0.01). Significant interactions between background noise level and cue modality (F (2,18) ¼ 11.838, p < 0.01) and between noise type and background noise level interaction (F (2,18) ¼ 15.082, p < 0.01) were detected. A similar three-way ANOVA was carried out on the reaction time for SOA of 500ms. Significant main effects of noise type (F (1,9) ¼ 18.136, p < 0.01), cue (F (1,9) ¼ 9.572, p < 0.05), and background noise level (F (2,18) ¼ 7.810, p < 0.01) were detected. A significant interaction between noise type and background noise level (F (2,18) ¼ 11.299, p < 0.01) was detected. Interactions between cue and background noise level (F (2,18) ¼ 24.508, p < 0.01) and among background noise type, cue modality, and background noise level (F (2,18) ¼ 6.267, p < 0.01) were also significant. A three-way ANOVA was also carried out on the reaction time for SOA of 1000ms. Significant main effects of noise type (F
A. Murata et al. / Applied Ergonomics 60 (2017) 58e67
65
hand, the reaction time to a hazard was affected by the noise type, the warning modality, SOA, and the noise level LAeq inside the experimental chamber. Therefore, the effects of the four experimental factors on the reaction time to a visual hazard are discussed below. 4.2. Effects of experimental factors on reaction time to visual hazard
Fig. 10. Sensitivity of tactor at four sites compared between (A)real-world running and (B)laboratory.
(1,9) ¼ 15.822, p < 0.01), cue (F (1,9) ¼ 12.139, p < 0.01), and background noise level (F (2,18) ¼ 7.810, p < 0.01) were detected. A significant interaction between background noise type and background noise level (F (2,18) ¼ 4.315, p < 0.05) was detected. 3.4. Comparison of sensitivity to tactile cue between real-world running and laboratory experiments The comparisons of sensitivity to a tactile cue between realworld running and laboratory experiments are shown in Fig. 10(A) and (B). The sensitivity to a tactile cue did not differ between real-world running and laboratory experiments, which means that the tactile warning is usable even in real-world running environment. A two-way (measurement environment (real-world running and laboratory experiments) by location of tactor) ANOVA conducted on the sensitivities to the tactile cue revealed no significant main effects or interactions for the ascending measurement, the descending measurement, and the mean of both ascending and descending measurements. 4. Discussion 4.1. Effects of experimental factors on percentage incorrect reactions and tracking error The percentage of incorrect reactions to a hazard and the tracking error in a simulated driving task were not affected by the background noise type, the warning (cue) modality, SOA, and the noise level LAeq inside the experimental chamber. On the other
The type of auditory cue was the same as that of Murata et al. (2014). Murata et al.(2014) used white noise, the noise level of which changed from 60 dB(A) to 90 dB (A) in 10 dB(A) steps, as the ambient (background) noise. In this study, the noise from a realworld driving environment was also used as ambient (background) noise to which the participants were exposed during the experiment. Under the noise (a) (white noise) equal to or more than 70 dB(A), Murata et al. (2014) showed that the reaction time of the auditory cue increased with the increase of the noise level LAeq inside the experimental chamber. As shown in Figs. 7e9, similar results were obtained here for noise (a) (white noise). For noise (b) (noise from outside a vehicle), the reaction time in response to the auditory warning was not as remarkably affected by the noise level (see Figs. 7e9). This indicates that the auditory cue is effective irrespective of the noise level when the noise from outside a vehicle was used as the ambient noise. The interference between ambient noise and the auditory cue must be subtle when the noise from outside a vehicle (noise (b)) is used, leading to the reduced effect of the noise level of the ambient noise on the perception and the response to the auditory cue. This suggests that a combination of auditory cue and noise affects the effectiveness of the auditory cue. In other words, the combination of an auditory cue (pure tone of 1 kHz) and noise recorded outside a vehicle is more compatible than that of an auditory cue (pure tone of 1 kHz) and white noise. When noise recorded outside a vehicle was used as the ambient noise inside the experiment chamber, a pure tone is effective as the auditory cue, because the reaction time to the auditory cue is not affected by the level of the noise level LAeq. These results also indicate that the compatibility of the auditory cue and external noise should be taken into account when designing an auditory warning system. Overall, the tactile cue was more effective than the auditory cue (see Figs. 7e9). Even if viewed from individual data, the tactile cue also tended to be more effective than the auditory cue (This was consistently observed for all participants). In both this study and Murata et al. (2014), the advantage of tactile cues over the auditory cues was demonstrated under the ambient noise. It must be noted that while this study investigated both types of ambient noises, Murata et al. (2014) investigated only one type of ambient noise (white noise (a)). The response to the auditory cue was not affected by the noise level of both types of noises. It must be noted that reaction time to a tactile cue under noise recorded outside a vehicle was shorter than that under white noise. Moreover, the reaction time to a tactile cue under white noise (noise (a)) was nearly equal to the reaction time to an auditory cue under noise from outside a vehicle (noise (b)) irrespective of the noise level LAeq. The response speed under noise from outside a vehicle was faster than that under white noise for each cue modality, SOA, and noise level LAeq (see Figs. 7e9). All participants were licensed drivers and accustomed to driving under real-world driving environment. While noise from outside a vehicle must be more familiar than white noise, white noise was likely to be very annoying for all participants. Such familiarity or annoyance might lead to the difference in response speed between white noise and noise from outside of the vehicle for both auditory and tactile warnings. The SOA condition affected the reaction time to auditory or tactile warnings under both white noise and noise from outside a
66
A. Murata et al. / Applied Ergonomics 60 (2017) 58e67
vehicle (see Figs. 7e9). When SOA was equal to 0 ms, the reaction time was prolonged for both auditory and tactile cues, and for both types of noises (a) and (b). For SOA of 500 ms or 1000 ms, the reaction time was lower than that for SOA of 0 ms. Participants were required to confirm that the visual hazard existed whenever a warning was presented, and the catch trials were inserted to avoid premature responses to an auditory or a visual cue. The SOA of 0 ms did not necessarily lead to a fast response to a cue. it is certain that the warning should be presented to a driver as soon as a hazard was detected. Ideally, the warning should be presented to a driver as soon as a hazard was detected by a warning system. As shown in Figs. 7e9, the reaction to both tactile and auditory cues for SOA of 0 ms tended to be slower as compared with that for SOA of 500 ms and 1000 ms. In this manner, it was verified that a cue before a hazard appears leads to the preparedness of a participant and thus to a faster reaction of a participant. In summary, the SOA condition of 0 ms is not suitable for warning presentation. SOA should be set to 500ms or 1000ms, which is enough to prepare for the forthcoming hazard. Therefore, it should be desirable that a warning system in advance predict the hazard before it actually occurs so that drivers can prepare for the hazard. On the basis of these results, tactile warning appears very promising as a warning cue under noisy environments. Moreover, the auditory cue was not affected by the ambient noise level when the noise recorded from a real-world driving environment was used. The nature of cross-modal links in spatial attention demonstrates that responses to a target presented in one sensory modality can be facilitated by the prior presentation of a cue (warning) by another sensory modality (Spence and Driver, 2004). On the basis of these results, tactile warning is very promising as a warning signal to a visual hazard especially in a noisy environment. Further, the results also suggest that an auditory cue of pure tone of 1 kHz is less affected by the ambient noise when noise recorded outside a vehicle is used than when white noise is used, which indicates that the compatibility of the auditory cue and external noise is an important design factor for an auditory warning system. 4.3. Effectiveness of tactile warning under real-world driving This study demonstrated the effectiveness of a tactile cue in a laboratory experiment. It is important to investigate whether the results holds even under real-world running. As it was judged it necessary to demonstrate the effectiveness of tactor under realworld environment, the comparative study on the sensitivity of tactor under both laboratory and real-world driving environments was conducted. As shown in Fig. 10(A) and (B), the sensitivity of the tactor did not differ between the real-world running environment and the laboratory experiment for ascending and descending measurements and the mean of both measurements. This indicates that the tactile warning is practically usable even in real-world running environments. Based on this result, it can be speculated that the results are applicable even under real-world running. 4.4. Implication for design of automotive warning system The design implications can be summarized as follows. Based on the results that the tactile cues are robust to the noise level, it is suggested that they should be utilized as a means of automotive warning systems. This study indicates that an auditory warning system should be designed by taking into account the compatibility between the auditory cue and an ambient (background) noise. The results are also indicative of carefully selecting the type of noise to investigate the effectiveness of warning systems with accuracy. As
the SOA condition of 0 ms led to slower a response to both auditory and tactile cues, SOA of 500ms or 1000ms should be used for drivers to be prepared enough for the forthcoming time-critical events or hazards. The effectiveness of simultaneous presentations of both auditory and tactile cues is suggested, for example, in Ho et al. (2006b; 2008), Murata et al. (2012a), and Murata and Kanbayashi (2013). Future research should carry out a comparative study of effectiveness between tactile warning and auditory-tactile warning. Although warnings regarding hazards on the left or right were studied here, future research should explore the effectiveness of non-visual cues when drivers are provided with not less than four directions. The pure tone of 1 kHz was used in this study. The effectiveness of looming auditory cues was suggested by Gray (2011). Ho et al. (2005) investigated the effectiveness of various auditory cues in capturing a driver's visual attention. Future work should explore the most suitable type of auditory cue. The limitations of this study are stated. Each condition was conducted for a relatively short period of time (9e10 min) in regards to real world driving. Drivers must usually drive for more than 9e10 min in real world driving, and must conduct an avoidance task as unexpectedly required by a warning system during driving. Therefore, it is speculated that the effectiveness of a warning system is affected by time spent in the vehicle. In future research, it must be explored how an auditory or tactile warning system may be affected by time spent in the vehicle. As the participants in this study were from the population of young male (licensed) drivers, it is not certain whether the results can be generalized to other populations such as females or older adults. How the results may be affected when tested with a wider demographic such as sex difference or age should be addressed in future research. Finally, we discuss the risk compensation phenomenon that appears with the development of such a preventive safety system. Preventive safety by means of an automotive warning system is desirable under conditions where the driver's workload is high, thus having limited resources left to analyze the situation before reacting to it. However, it must be noted that such a system is not necessarily beneficial and may have negative effects if the corresponding risk is undermined. We must give plenty of thoughts to whether drivers really drive safely when vehicles are equipped with safety warning systems. Analysis of the behavior of drivers of a vehicle with safety features such as seat belts and air bags indicated that these drivers tended to drive more aggressively (Wilde, 1982; Evans, et al., 1982; Evans and Wasielewski, 1982; Evans and Graham, 1991). Such behavior can be explained by the theory of risk homeostasis (Wilde, 1982). Such an adverse effect to new preventive safety technologies must be considered.
5. Conclusions In order to determine the most suitable modality of cue (warning) to a visual hazard under noisy environments, the effects of the following four conditions on the response speed to a hazard were examined: (1) cue modality (auditory and tactile cue), (2) SOA (0ms, 500ms, and 1000ms), (3) noise type (white noise and noise outside a vehicle), and (4) noise level LAeq inside the experimental chamber (approximately 60 dB(A), 70 dB(A), and 80 dB(A)). We investigated whether a tactile cue under a noisy environment is more effective for quickening the response to a hazard than an auditory cue. It was also explored whether the response speed to an auditory cue is different according to the type of the ambient noise. The conclusions and implications can be stated as follows:
A. Murata et al. / Applied Ergonomics 60 (2017) 58e67
1. The reaction time to the tactile warning was not affected by both the white noise level and the noise level from outside a vehicle. As the response to the tactile cue was faster than that of the auditory cue irrespective of the noise type, it can be concluded that the tactile cue is more robust to the noise level. This supports the utilization of tactile cues as a means of automotive warning system. 2. The reaction time to the auditory warning was significantly affected by the white noise level, while the reaction time to the auditory warning was not affected by the noise level from outside a vehicle. This indicates that the compatibility between an auditory cue and the ambient noise should be taken into account when designing an auditory warning system. 3. The reaction time to both auditory and tactile cues tended to be shortened under the noise recorded outside a vehicle than under the white noise, which suggests that the type of noise should be carefully selected to investigate the effectiveness of the warning system. 4. The SOA of 0 ms did not necessarily lead to a fast response to both auditory and tactile cues for noise conditions. The SOA condition of 0 ms is not suitable when designing an auditory warning system. SOA of 500ms or 1000ms should be selected so that drivers are adequately prepared for the forthcoming hazards. Acknowledgments This work was partly supported by Grant-in Aids for Scientific Research (B) (grant numbers 22310101 & 26282095), Japan Society for the Promotion of Science (JSPS). References Castro, C., 2009. Human Factors of Visual and Cognitive Performance in Driving. CRC Press. Denworth, L., 2015. The special power of touch. Sci. Am. Mind 26 (4), 30e39. Driver, J., Spence, C., 1998. Attention and the cross-modal construction of space. Trends Cognitive Sci. 2 (7), 254e262. Driver, J., 2001. A selective review of selective attention research from the past century. Br. J. Psychol. 92, 53e78. Dukic, T., Hanson, L., Falkmer, T., 2006. Effects of drivers' age and push button locations on visual time off road, steering wheel deviation and safety perception. Ergonomics 49 (1), 78e92. Evans, L., Graham, J.D., 1991. Risk reduction or risk compensation? The case of mandatory safety-belt use law. J. Risk Uncertain. 4 (1), 61e73. Evans, L., Wasielewski, P., 1982. Do accident-involved drivers exhibit riskier everyday driving behavior? Accid. Analysis Prev. 14 (1), 57e64. Evans, L., Wasielewski, P., von Buseck, C.R., 1982. Compulsory seat belt usage and driver risk-taking behavior. Hum. Factors 24 (1), 41e48. Ferris, T., Sarter, N.B., 2008. Cross-modal links among vision, audition, and touch in complex environments. Hum. Factors 50 (1), 17e26. Ferris, T., Penfold, R., Hameed, S., Sarter, N., 2006. The implications of crossmodal links in attention for the design of multimodal interfaces: a driving simulator study. Proc. Hum. Factors Ergonomics Soc. 50th Annu. Meet. 406e409. Fitch, G.M., Kiefer, R.J., Hankey, J.M., Kleiner, B.M., 2007. Toward developing an approach for alerting drivers to the direction of a crash threat. Hum. Factors 49 (4), 710e720. Gkikas, N., 2013. Automotive Ergonomics-driver-vehicle Interaction. CRC Press. Gray, R., 2011. Looming auditory collision warning for driving. Hum. Factors 53 (1), 63e74. Ho, C., Spence, C., 2005. Assessing the effectiveness of various auditory cues in capturing a driver's visual attention. J. Exp. Psychol. Appl. 11 (3), 157e174.
67
Ho, C., Spence, C., 2008. The Multisensory Driver-implications for Ergonomic Car Interface Design-. Ashgate. Ho, C., Tan, H.Z., Spence, C., 2005. Using spatial vibrotactile cues to direct visual attention in driving scenes. Transp. Res. Part F 8 (6), 397e412. Ho, C., Reed, N., Spence, C., 2006a. Assessing the effectiveness of “intuitive” vibrotactile warning signals in preventing front-to-rear collisions in a driving simulator. Accid. Analysis Prev. 38 (5), 989e996. Ho, C., Reed, N., Spence, C., 2006b. Multisensory in-car warning signals for collision avoidance. Hum. Factors 49 (6), 1107e1114. Ho, C., Tan, H.Z., Spence, C., 2006c. The differential effect of vibrotactile and auditory cues on visual spatial attention. Ergonomics 49 (7), 724e738. Jones, L.A., Sarter, N.B., 2008. Tactile displays:Guidance for their design and application. Hum. Factors 50 (1), 90e111. Jones, C.M., Gray, R., Spence, C., Tan, H.Z., 2008. Directing visual attention with spatially informative and noninformative tactile cues. Exp. Brain Res. 186, 659e669. Lee, J.D., Caven, B., Haake, S., Brown, T.L., 2001. Speech-based interaction with invehicle computers: the effect of speech-based e-mail on drivers' attention to the roadway. Hum. Factors 43 (5), 631e640. Lee, J.D., McGehee, D.V., Brown, T.L., Marshall, D., 2006. Effects of adaptive cruise control and alert modality on driver performance. Transp. Res. Rec. 1980, 49e56. Lees, M.N., Lee, J.D., 2007. The influence of distraction and driving context on driver response to imperfect collision warning systems. Ergonomics 50 (8), 1264e1286. Linden, D.J., 2015. Touch, Viking Penguin, UK. Meng, F., Gray, R., Ho, C., Ahtamad, M., Spence, C., 2015. Dynamic vibrotactile signals for forward collision avoidance warning systems. Hum. Factors 57 (2), 329e346. Murata, A., Kanbayashi, M., 2013. Effectiveness of automotive warning system presented with multiple sensory modalities. Proc. HCI2013 22, I88eI97. Murata, A., Nakagawa, Y., 2012. Effectiveness of warning presentation with directional cue that makes use of cross-modal link. Proc. SICE2012 1379e1384. Murata, A., Tanaka, K., 2011. Basic study on effectiveness of tactile interface for warning presentation in driving environment. Int. J. Knowl. Eng. Softw. Data Paradigms 3 (1), 95e105. Murata, A., Kanbayashi, M., Hayami, T., 2012a. Effectiveness of automotive warning system presented with multiple sensory modalities. Proc. SICE2012 920e925. Murata, A., Kemori, Moriwaka, M., 2012b. Basic study on automotive warning presentation to front/rear hazard by vibrotactile stimulation. Proc. AHFE2012 421e430. Murata, A., Kemori, S., Moriwaka, M., 2013. Proposal of automotive 8-directional warning system that makes use of tactile apparent movement. Proc. HCI2013 22, I98eI107. Murata, A., Kuroda, T., Kanbayashi, M., 2014. Effectiveness auditory and vibrotactile cuing of driver attention under noisy environment. Proc. AHFE2014 3247e3256. Regan, M.A., Lee, J.D., Young, K.L., 2009. Driver Distraction-theory, Effects, and Mitigation-. CRC Press. Scott, J.J., Gray, R., 2008. A comparison of tactile, visual, and auditory warnings for rear-end collision prevention in simulated driving. Hum. Factors 50 (2), 264e275. Spence, C., Driver, J., 1994. Covert spatial orienting in audition: exogenous and endogenous mechanisms. J. Exp. Psychol. Hum. Percept. Perform. 20 (3), 555e574. Spence, C., Driver, J., 1997a. Cross-modal links in attention between audition, vision, and touch: implications for interface design. Int. J. Cognitive Ergonomics 1 (4), 351e373. Spence, C., Driver, J., 1997b. On measuring selective attention to a specific sensory modality. Percept. Psychophys. 59 (3), 389e403. Spence, C., Driver, J., 2004. Crossmodal Space and Crossmodal Attention. Oxford University Press. Stanton, N.A., Young, M.S., 2005. Driver behavior with adaptive control. Ergonomics 48 (10), 1294e1313. Van Erp, J.B.F., van Veen, H.A.H.C., 2001. Vibro-tactile information processing in automobiles. Proc. Of EuroHaptics 2001, 99e104. Van Erp, J.B.F., van Veen, H.A.H.C., 2003. A multi-purpose tactile test for astronauts in the international space station. Proc. EuroHaptics 2003, 405e408. Van Erp, J.B.F., 2005. Presenting directions with a vibrotactile torso display. Ergonomics 48 (3), 302e313. Wilde, G.J.S., 1982. The theory of risk homeostasis: implications for safety and health. Risk Anal. 2 (4), 209e225. Young, M.S., Stanton, N.A., 2004. Taking the load off: investigations of how adaptive cruise control affects mental workload. Ergonomics 47 (9), 1014e1035.
Contents lists available at ScienceDirect
Applied Ergonomics journal homepage: www.elsevier.com/locate/apergo
Effects of auditory and tactile warning on response to visual hazards under a noisy environment Atsuo Murata a, *, Takashi Kuroda a, Waldemar Karwowski b a b
Dept. of Intelligent Mechanical Systems, Graduate School of Natural Science and Technology, Okayama University, Okayama, Japan Dept. of Industrial Engineering & Management Systems, University of Central Florida, Orlando, USA
a r t i c l e i n f o
a b s t r a c t
Article history: Received 22 April 2016 Received in revised form 8 August 2016 Accepted 4 November 2016
A warning signal presented via a visual or an auditory cue might interfere with auditory or visual information inside and outside a vehicle. On the other hand, such interference would be certainly reduced if a tactile cue is used. Therefore, it is expected that tactile cues would be promising as warning signals, especially in a noisy environment. In order to determine the most suitable modality of cue (warning) to a visual hazard in noisy environments, auditory and tactile cues were examined in this study. The condition of stimulus onset asynchrony (SOA) was set to 0ms, 500ms, and 1000ms. Two types of noises were used: white noise and noise outside a vehicle recorded in a real-world driving environment. The noise level LAeq (equivalent continuous A-weighted sound pressure level) inside the experimental chamber of each type of noise was adjusted to approximately 60 dB (A), 70 dB (A), and 80 dB (A). As a result, it was verified that tactile warning was more effective than auditory warning. When the noise outside a vehicle from a real-driving environment was used as the noise inside the experimental chamber, the reaction time to the auditory warning was not affected by the noise level. © 2016 Elsevier Ltd. All rights reserved.
Keywords: Auditory cue Tactile cue Automotive warning system Noise outside a vehicle White noise SOA
1. Introduction As the information on an automotive display contains a lot of text and images, drivers tend to be distracted by the displays and become inattentive to a variety of hazards. Driving a car imposes a characteristically heavy workload on visual perception, cognitive information processing, and manual responses. Drivers often simultaneously perform two or more tasks; for example, they adjust the volume of a radio or CD player while driving. Such sharing of attention may lead to slow reactions to a warned situation or larger deviations of own vehicle and induce fatal or nonfatal crashes (Young and Stanton, 2004; Stanton and Young, 2005). With the progress in by-wire and information technology, visual and cognitive workload while driving increases, and the driver-vehicle interaction is getting increasingly complicated (Lee et al., 2001; Dukic et al., 2006; Gkikas, 2013; Castro, 2009). Consequently, drivers tend to be distracted by a variety of
* Corresponding author. Dept. of Intelligent Mechanical Systems, Graduate School of Natural Science and Technology, Okayama University, Okayama 3-1-1, Tsushimanaka, Kita-ku, Okayama, 700-8530, Japan. E-mail addresses: [email protected] (A. Murata), [email protected] (W. Karwowski). http://dx.doi.org/10.1016/j.apergo.2016.11.002 0003-6870/© 2016 Elsevier Ltd. All rights reserved.
secondary tasks such as the operation of switches for a CD player or air conditioner while driving (Regan et al., 2009), which increases the risk of inattentive driving. Lees and Lee (2007) showed that an automotive collision warning system (CWS) can enhance hazard identification and management if false alarms are tolerated. Lee, McGehee, Brown, and Marshall (2006) examined the effectiveness of various warning modalities in making distracted drivers attentive during severe braking situations that exceeded adaptive cruse control (ACC) capability. The power of touch via tactile sense was suggested by Denworth (2015) and Linden (2015). The potential applicability of the tactile sense to automotive warning systems has gained more attention towards enhancing driver safety (Jones and Sarter, 2008). Jones and Sarter (2008) reviewed the utilization of the sense of touch as a medium for information representation. They concluded that sense of touch represents a promising means for communications in human-vehicle systems. Recently, the tendencies of cross-modal information processing and design (Driver and Spence, 1998; Driver, 2001; Spence and Driver, 1994, 1997a, 1997b, 2004; Jones et al., 2008) have emerged as a major research topic in the design of automotive warning systems. Presenting information via multiple modalities such as vision, audition, and touch is thought to be a promising way to
A. Murata et al. / Applied Ergonomics 60 (2017) 58e67
reduce transmission errors and enhance safety. A better understanding of spatial and temporal cross-modal links is essential to enable better application of these properties to automotive warning design. Ferris et al. (2006) and Ferris and Sarter (2008) found significant asymmetric performance effects of cross-modal spatial links on the basis of the following results. Auditory cues shortened response latencies for collocated visual targets, while a similar result was not observed for visual cues. Responses to contra-lateral targets were faster for tactually cued auditory targets and each combination of visual and tactile cues. Fitch et al. (2007) showed that tactile warning reduced correct localization response times by 257 ms and increased the percentage of correct localization by 52% (from 32% to 84%) relative to auditory warning, which indicates that tactile warning by seat vibration might be effective for warning drivers of the direction of a crash threat. The visual and auditory perceptual systems are more activated in driving than the tactile perceptual system. Therefore, tactile warning is expected to lead to a faster response to a hazard. It has been shown that tactile warning presentation is effective in a driving environment where hazards are ubiquitous Ho et al., 2006c; Van Erp, 2005; Van Erp and Veen, 2001; Van Erp and Veen, 2003). Ho and Spence (2005), Ho et al. (2006a, 2006c), and Ho and Spence (2008) explored the effectiveness of tactile warning for front and rear locations. Murata and Tanaka (2011), Murata et al. (2012a) and Murata and Nakagawa (2012)examined how tactile warning is effective for promoting responses to hazards in left and right locations. Moreover, Murata et al. (2012b) and Murata et al. (2013) investigated the effectiveness of tactile warning by apparent movement. They found that tactile warning by apparent movement can more quickly transmit directional cues than the simultaneous stimulation of two tactors or a single-point stimulation. Scott and Gray (2008) engaged in the development of assistance systems, in particular, non-visual and multisensory collisionwarning systems for drivers, and examined how the presentation of auditory, tactile, and multisensory warning signals is effective in alerting a driver and rapidly orienting his or her spatial attention to the direction of the potential danger. Like many other studies, they demonstrated that the use of tactile warning improved the response of drivers to time-critical events that could potentially result in a crash. Meng, Gray, Ho, Ahtamad, and Spence (2015) assessed the effectiveness of various tactile warnings using a simulated car-following task, and suggested that dynamic tactile cues that look as though they are approaching the torso can be effectively used to communicate information concerning external events. If a warning signal is presented via visual or auditory stimulus, auditory or visual interference with other information, such as the noise outside a vehicle, might arise. On the other hand, if a tactile cue is used, such interference between the same (auditory or visual) modalities does not occur, because much information on the driving environment is visual or auditory and there are few cases where drivers are exposed to high vibration in recent road environment (Refer to Fig. 10 as for the effectiveness (less susceptibility to vibration) of tactile cue in a real-world running). Therefore, it is expected that a tactile signal would be very promising as a warning signal especially in a noisy environment. However, the studies cited above on the effectiveness of tactile warning systems have only paid attention to the advantage of tactile cues, and did not compare the response speed between tactile and auditory cues and verify the effectiveness of tactile cues under a high level of background noise. Although Gray (2011) showed that looming warnings had significantly faster brake reaction times as compared with the other non-looming warnings, the effect of the ambient noise outside of the vehicle on the response speed to an auditory warning was not
59
discussed. Murata et al. (2014) verified that tactile warning was more and more effective with an increase in the noise level inside the experimental chamber. The reaction time to auditory warning was significantly affected by the noise level, while the reaction time to tactile warning was not affected by the noise level at all. Moreover, the stimulus onset asynchrony (SOA) condition did not remarkably affect the reaction time to either auditory or tactile warnings. However, they used white noise as the background noise surrounding the experimental chamber, though it is desirable to use noise recorded in a real-world driving environment. As the auditory cue was also a pure tone of 1 kHz, it is possible that the perceptibility of the auditory cue might be significantly affected by the spectral power density of the surrounding noise of a similar type (Reference to Fig. 6). Therefore, it was hypothesized that if the interference between the surrounding (background) noise and the auditory cue was mitigated, it might be possible that the auditory cue is effective even under a high noise level. It was explored whether similar results to Murata et al. (2014) could be obtained even when the noise outside a vehicle from a real-driving environment was used. In order to determine the most suitable modality of cue (warning) to a visual hazard under noisy environment, and to examine the effect of the type of ambient noise on the processing speeds of auditory and tactile cues, the following four experimental factors were used, and the effects of the experimental factors on the reaction to a target (hazard) were examined: (1) cue modality (auditory or tactile cue), (2) SOA (0 ms, 500 ms, and 1000 ms), (3) type of surrounding noise inside an experimental chamber (white noise and noise outside a vehicle in a real-driving environment) and (4) noise level LAeq (equivalent continuous A-weighted sound pressure level) inside the experimental chamber (about 60 dB(A), 70 dB(A), and 80 dB(A)). 2. Method 2.1. Participants Ten healthy males aged from 21 to 24 years took part in the experiment. All participants had held a driver's license for 3e4 years. All participants provided informed consent after receiving a brief explanation of the aim and the contents of the experiment. 2.2. Apparatus A PC (Mouse Computer, MDV-ADS7210S) and a projector (NEC, LT245, 2200 lumens) were used to control a simulated driving task and a reaction task to a visual hazard. A steering controller (Logicool, LPRC-14000) was connected to the PC so that the participant could operate the vehicle on the simulated driving display. The participant was required to operate both accelerator and brake, and maintain a speed between 70 km/h and 80 km/h in the simulated driving task. The display (I.O DATA, LCD-A154VH-V) for maintaining the velocity within the predetermined range was installed 326 mm from the steering controller. The steering controller (Logicool, LPRC-14000) used for this task was attached with an accelerator and brake. For the presentation of the auditory cue, two speakers (Onkyo, Powered Speaker System OP1) were placed in front of the participant. Speakers were connected to the PC to control the auditory cue. The auditory cue was a pure tone of 1 kHz, and presented to the ears of a participant with LAeq of about 60 dB(A). LAeq (equivalent continuous A-weighted sound pressure level during 10 s) was measured using a sound level meter (RION, NL-42). The value (about 60 dB(A)) of LAeq was empirically determined so that the auditory cue is audible enough and not too noisy. As it was impossible to adjust LAeq to just 60 dB(A) due to the property of LAeq
60
A. Murata et al. / Applied Ergonomics 60 (2017) 58e67
(this measure is always variable around a fixed value), this was set to about 60 dB(A). Another speaker (Elecom, MS-131SV) was used to present the noise. This speaker was installed behind the participant and connected to the PC (Acer, Aspire V5). The installation of the three speakers is depicted in Fig. 1. Four tactors with a diameter of 45 mm (Acouve Laboratory Inc., Vp216) were installed on the surface of the driver's seat as shown in Fig. 2. The tactors were connected to the PC for controlling the simulated driving task to control their outputs. In the studies of tactile warning system (Ho et al., 2005; 2006a; Meng et al., 2015), the characteristics of tactors are usually represented using not velocity or acceleration but frequency. Therefore, the characteristics of tactile cue were described as follows so that reproducibility is assured. The frequency and the intensity of outputs were empirically set to 64 Hz and 10 V respectively using a function generator (Gw Instek, SFG-2004) so that the participant could detect the vibration most sensitively. Many previous studies attached tactors around the waist or torso of participants (Meng et al., 2015; Ho et al., 2005; 2006a, 2006b, 2006c). From a practical viewpoint, drivers tend to avoid attaching tactors around their waist or torso. Therefore, it is more plausible and practical to place tactors on the surface of the driver's seat. Fig. 2. Location of tactors on the surface of driver's seat.
2.3. Task Using the driving simulator system, the participants were required to simultaneously carry out a simulated driving task (main task) and a reaction task to a hazard randomly presented on one of two locations (on the right or left of the display of driving simulator) (secondary task). In the primary driving simulator task, the participant was required to minimize the deviation from the predetermined line, and maintain lane location using a steering wheel. The participant was also required to operate both the accelerator and brake, and maintain a speed of between 70 km/h and 80 km/h (see Fig. 3). This was programmed using an interpreter type programming language HSP (Hot Soup Processor). In the secondary task (reaction task to a hazard), the participant was required to react to a visual hazard (see Fig. 4) presented on the left or the right of the display in front of the participant using the brake. A warning was presented to the participant 0 ms, 500 ms,
and 1000 ms before a visual hazard appeared. In other words, the SOA (Stimulus Onset Asynchrony) was set to 0ms, 500ms, and 1000ms (see Fig. 5). The warning was presented to the left or the right display using the following warning presentation modes (cues): (1) auditory cue (pure tone of 1 kHz) and (2) tactile cue. Catch trials were added to the secondary task so that premature responses were avoided. Catch trials randomly occupied 2 trials out of the total number of trials (12 trials). In the catch trial, the visual hazard was not presented to the participant, and only the warning was presented to the participant. The participant had to confirm that the visual hazard existed whenever the warning was presented. When the participant detected the visual hazard, he was required to brake quickly and decelerate the vehicle to 30 km/h. After this procedure was completed, the participant was required to increase the vehicle velocity to between 70 km/h and 80 km/h
Fig. 1. Outline of experimental system.
A. Murata et al. / Applied Ergonomics 60 (2017) 58e67
Fig. 3. Display of velocity maintenance task.
again, and maintain the velocity within this range. The auditory cue was presented to the participant via two speakers placed (1200 mm away from the participant) in front of the participant. The tactile cue was transmitted to the participant via four tactors (vibrotransducers) (see Fig. 2) placed on the sitting surface such that these tactors were in contact with the left and the right thighs. The cue to the left or right target was presented via two tactors each (two tactors were placed on the left thigh, and two tactors were on the right thigh). For both (a) white noise and (b) noise outside the vehicle, the background noise level LAeq (equivalent continuous A-weighted sound pressure level) inside the experimental chamber was adjusted to approximately 60 dB(A), 70 dB(A), and 80 dB(A). The power spectral densities of the white noise and the noise from outside a vehicle recorded in a real-world driving environment are demonstrated in Fig. 6(a) and (b), respectively. 2.4. Design and procedure The temperature within the experimental chamber was controlled to 25 C. The humidity (although not controllable) ranged from 20 to 25%. Experimental factors were noise type (white noise and noise outside a vehicle), warning modality (auditory and tactile), background noise level LAeq (equivalent continuous Aweighted sound pressure level) inside the experimental chamber (approximately 60 dB(A), 70 dB(A), and 80 dB(A)), and SOA (0 ms, 500 ms, and 1000 ms). All of the above were within-subject variables. It is certain that the warning should be presented to a driver as soon as a hazard was detected. However, it is expected that the reaction to a cue (whether is tactile or auditory) for SOA of 0 ms is delayed as compared to that for SOA of 500ms and 1000ms. Therefore, three SOAs (0, 500, and 1000 ms) were used to examine whether a cue presented before a hazard appears leads to the enhanced preparedness of a participant and thus to a faster and accurate reaction to a visual hazard. First, the participant was required to adjust the seat so that they could conduct the experimental task comfortably. After explaining the primary and secondary tasks described above to the participant, the participant was permitted to practice each task separately, and practice both tasks simultaneously. After the participant reported that he had practiced enough and completely understood the tasks, and the experimenter confirmed this, the experiment was started. According to Ferris and Sarter (2008), the duration of both auditory and tactile cues were determined as 1000 ms. For the tactile cues, the left and right visual hazards were cued by two tactors installed on the left and the right, respectively. In the case of the auditory
61
cues, left and right visual hazards were cued by two speakers installed on the front left and the front right, respectively. Participants were required to carry out both the primary tracking task and the secondary task simultaneously as quickly and accurately as possible for 15 min for each condition. Participants were also required to firstly confirm a visual hazard and then react to it. Experiments under white noise and noise from outside a vehicle in real driving conditions were conducted one day apart for all participants. The order of performance of the two noise types was counterbalanced across the participants. The six conditions of warning modality (two levels) and noise level (three levels) were randomly carried out by each participant for each noise type. For each condition, twelve trials (two catch trials were included) were conducted for each SOA. The presentation of SOA was also randomized in a total of 36 trials. A break of about 3 min was inserted between two conditions. It took about 105 min to complete the experiment for each noise type. The evaluation measures were: the reaction time to a visual hazard, the percentage incorrect reactions to a visual hazard, and the tracking error in the simulated driving task from the onset of a cue to the end of the response.
2.5. Comparison of sensitivity to tactile cue between real-world running and laboratory experiments It is not certain whether tactile warning is effective under the real-world running environment. Therefore, the sensitivities to a tactile warning (cue) were also investigated as a function of four placement sites of tactors on the driver's seat in Fig. 2 (front left, front right, rear left, rear right) and compared between real-world running and laboratory experiments. Ten participants above also took part in this experiment. The experiment was conducted on a separate day. In the laboratory experiment, using the same main task (simulated driving task), the sensitivity to a tactile cue was investigated as a function of location of tactors. The sensitivity to a tactile cue was measures by adjusting the frequency of tactor via a function generator. The sensitivity was measured using both ascending and descending sequences. The frequency ranged from 20 Hz to 200 Hz. In the ascending sequence, the initial value of sensitivity (20 Hz) was increased by 10 Hz, and the threshold value of each participant was determined. Each participant was required to tell the experimenter the frequency at which he began to feel the vibration. In the descending sequence, the initial value of sensitivity (200 Hz) was decreased by 10 Hz, and the threshold value was determined. Each participant was required to tell the experimenter the frequency at which the participant could not percept the vibration any more. The mean frequency of ascending and descending sequence was also calculated. The order of four locations was randomized across the participants. The order of ascending and descending measurements was counterbalanced across the participants. For each location of tactor, a total of ten measurements were made for both ascending and descending conditions. Therefore, a total of 80 measurements were conducted. In the real-world running experiment, the sensitivity was measured when running about 20 km along a highway with a mean speed of 55 km/h (this was within a legally permitted speed). The participant was required to obey the same procedure as the laboratory experiment. According to a similar procedure to the laboratory experiment, the ascending, descending, and the mean frequencies were obtained. Two experimenters rode the vehicle (Toyota, WISH 1.8X) together, and engaged in the measurement. A total of 80 measurements were conducted only during running.
62
A. Murata et al. / Applied Ergonomics 60 (2017) 58e67
Fig. 4. Explanation of visual hazard.
2.6. Statistical rationale for analyzing effects of experimental factors on reaction time of judgment to visual hazard Although there were four experimental factors (SOA, cue modality, background noise type, and background noise level) that affected the performance measures, in particular, reaction time for the judgment of a visual hazard, it is generally not recommended to carry out a four-way ANOVA, because a four-way interaction is not so easy to interpret. Therefore, two-way and three-way ANOVAs were conducted to examine how the experimental factors affected performance measures such as reaction time according to the
following strategy. (i) As it was judged that the more important factors were the background (ambient) noise type and the cue modality, a two-way (background noise type by cue modality) ANOVA was conducted in the first place to investigate how these two factors affected performance measures. (ii) Second, the combination of background noise and cue modality, that is, the cue presentation condition (which cue modality is presented under which background noise type) was regarded as one experimental factor, and a three-way
A. Murata et al. / Applied Ergonomics 60 (2017) 58e67
63
(cue presentation condition by background noise level by SOA) was carried out. (iii) If there existed an experimental factor that is not involved in a significant interaction, two- or three-way ANOVA was further conducted for each level of this factor after this factor was excluded from ANOVA. 3. Results 3.1. Percentage incorrect reactions
Fig. 5. Explanation of SOA (Stimulus Onset Asynchrony).
The incorrect reactions included the following: a no-brake reaction, and a reaction to a catch trial. For each combination of cue modality and background (ambient) noise type (four condition), each participant conducted a total of 108 judgment trials. The percentage incorrect reactions for each participant was defined as
Fig. 6. Power spectrum of (a)white noise and (b)noise outside a vehicle that includes a variety of noise during driving.
64
A. Murata et al. / Applied Ergonomics 60 (2017) 58e67
the percentage of incorrect reactions to a total number of reactions to a visual hazard (108 trials). The mean percentage incorrect reactions was very low and as follows: auditory cue under white noise: 0.28%, tactile cue under white noise: 0.28%, auditory cue under noise outside a vehicle: 0%, and tactile cue under noise outside a vehicle: 0.28%. The cue and the noise did not affect the error. 3.2. Tracking error in a simulated driving task A two-way (background noise type by noise level) ANOVA (Analysis of Variance) conducted on the tracking error revealed no significant main effects, and an interaction. As a result of carrying out a three-way (cue presentation condition (combination of noise type and cue) by noise level by SOA) ANOVA on the percentage error trials, no significant main effects and interactions were detected. The summary of the tracking errors is as follows: auditory cue under white noise: 0.37 m, tactile cue under white noise: 0.4 m, auditory cue under noise outside a vehicle: 0.39 m, and tactile cue under noise outside a vehicle: 0.38 m. A similar two-way (noise type by noise level) ANOVA conducted on the tracking error revealed no significant main effects and an interaction. A similar three-way (cue presentation condition (combination of noise type and cue) by noise level by SOA) ANOVA on the tracking error detected no significant main effects or interactions. Thus, the tracking error in the simulated driving task was not affected by noise type, noise level, SOA, or cue modality.
Fig. 8. Reaction time as a function of background noise level and cue modality (SOA:500ms). (a) white noise, (b) noise outside of a vehicle. *: p < 0.05, **: p < 0.01.
3.3. Reaction time Figs.7e9 show reaction time as a function of noise level and cue modality when SOA is 0 ms, 500 ms, and 1000 ms respectively. According to the strategy stated in Section 2.6, ANOVA was conducted. First, according to strategy (i), a two-way (background (ambient) noise type by cue modality) ANOVA was performed on the reaction time, detecting significant main effects of noise type (F (1,9 ¼ 11.943, p < 0.01) and cue (F (1,9) ¼ 47.173, p < 0.01). The background noise type by cue modality interaction was not significant. Second, according to strategy (ii), a three-way (cue presentation condition (background noise type, cue modality) by background noise level of by SOA) ANOVA was then conducted on the reaction time. Significant main effects of cue presentation condition (F
Fig. 7. Reaction time as a function of background noise level and cue modality (SOA:0ms). (a) white noise, (b) noise outside of a vehicle. *: p < 0.05, **: p < 0.01.
Fig. 9. Reaction time as a function of background noise level and cue modality (SOA:1000ms). (a) white noise, (b) noise outside of a vehicle. *: p < 0.05, **: p < 0.01.
(3,27) ¼ 10.865, p < 0.01), background noise level (F (2,18) ¼ 6.385, p < 0.01), and SOA (F (2,18) ¼ 236.151, p < 0.01) were detected. A significant interaction between cue presentation condition (background noise type, cue modality) and background noise level was also found (F (6,54) ¼ 14.176, p < 0.01). As the factor SOA was not involved in any significant interaction in three-way ANOVA above, according to strategy (iii), a three-way (background noise level by cue modality by noise type) ANOVA performed on the reaction time for SOA of 0ms revealed significant main effects of background noise type (F (1,9) ¼ 33.588, p < 0.01), cue modality (F (1,9) ¼ 6.747, p < 0.05), and background noise level (F (2,18) ¼ 12.463, p < 0.01). Significant interactions between background noise level and cue modality (F (2,18) ¼ 11.838, p < 0.01) and between noise type and background noise level interaction (F (2,18) ¼ 15.082, p < 0.01) were detected. A similar three-way ANOVA was carried out on the reaction time for SOA of 500ms. Significant main effects of noise type (F (1,9) ¼ 18.136, p < 0.01), cue (F (1,9) ¼ 9.572, p < 0.05), and background noise level (F (2,18) ¼ 7.810, p < 0.01) were detected. A significant interaction between noise type and background noise level (F (2,18) ¼ 11.299, p < 0.01) was detected. Interactions between cue and background noise level (F (2,18) ¼ 24.508, p < 0.01) and among background noise type, cue modality, and background noise level (F (2,18) ¼ 6.267, p < 0.01) were also significant. A three-way ANOVA was also carried out on the reaction time for SOA of 1000ms. Significant main effects of noise type (F
A. Murata et al. / Applied Ergonomics 60 (2017) 58e67
65
hand, the reaction time to a hazard was affected by the noise type, the warning modality, SOA, and the noise level LAeq inside the experimental chamber. Therefore, the effects of the four experimental factors on the reaction time to a visual hazard are discussed below. 4.2. Effects of experimental factors on reaction time to visual hazard
Fig. 10. Sensitivity of tactor at four sites compared between (A)real-world running and (B)laboratory.
(1,9) ¼ 15.822, p < 0.01), cue (F (1,9) ¼ 12.139, p < 0.01), and background noise level (F (2,18) ¼ 7.810, p < 0.01) were detected. A significant interaction between background noise type and background noise level (F (2,18) ¼ 4.315, p < 0.05) was detected. 3.4. Comparison of sensitivity to tactile cue between real-world running and laboratory experiments The comparisons of sensitivity to a tactile cue between realworld running and laboratory experiments are shown in Fig. 10(A) and (B). The sensitivity to a tactile cue did not differ between real-world running and laboratory experiments, which means that the tactile warning is usable even in real-world running environment. A two-way (measurement environment (real-world running and laboratory experiments) by location of tactor) ANOVA conducted on the sensitivities to the tactile cue revealed no significant main effects or interactions for the ascending measurement, the descending measurement, and the mean of both ascending and descending measurements. 4. Discussion 4.1. Effects of experimental factors on percentage incorrect reactions and tracking error The percentage of incorrect reactions to a hazard and the tracking error in a simulated driving task were not affected by the background noise type, the warning (cue) modality, SOA, and the noise level LAeq inside the experimental chamber. On the other
The type of auditory cue was the same as that of Murata et al. (2014). Murata et al.(2014) used white noise, the noise level of which changed from 60 dB(A) to 90 dB (A) in 10 dB(A) steps, as the ambient (background) noise. In this study, the noise from a realworld driving environment was also used as ambient (background) noise to which the participants were exposed during the experiment. Under the noise (a) (white noise) equal to or more than 70 dB(A), Murata et al. (2014) showed that the reaction time of the auditory cue increased with the increase of the noise level LAeq inside the experimental chamber. As shown in Figs. 7e9, similar results were obtained here for noise (a) (white noise). For noise (b) (noise from outside a vehicle), the reaction time in response to the auditory warning was not as remarkably affected by the noise level (see Figs. 7e9). This indicates that the auditory cue is effective irrespective of the noise level when the noise from outside a vehicle was used as the ambient noise. The interference between ambient noise and the auditory cue must be subtle when the noise from outside a vehicle (noise (b)) is used, leading to the reduced effect of the noise level of the ambient noise on the perception and the response to the auditory cue. This suggests that a combination of auditory cue and noise affects the effectiveness of the auditory cue. In other words, the combination of an auditory cue (pure tone of 1 kHz) and noise recorded outside a vehicle is more compatible than that of an auditory cue (pure tone of 1 kHz) and white noise. When noise recorded outside a vehicle was used as the ambient noise inside the experiment chamber, a pure tone is effective as the auditory cue, because the reaction time to the auditory cue is not affected by the level of the noise level LAeq. These results also indicate that the compatibility of the auditory cue and external noise should be taken into account when designing an auditory warning system. Overall, the tactile cue was more effective than the auditory cue (see Figs. 7e9). Even if viewed from individual data, the tactile cue also tended to be more effective than the auditory cue (This was consistently observed for all participants). In both this study and Murata et al. (2014), the advantage of tactile cues over the auditory cues was demonstrated under the ambient noise. It must be noted that while this study investigated both types of ambient noises, Murata et al. (2014) investigated only one type of ambient noise (white noise (a)). The response to the auditory cue was not affected by the noise level of both types of noises. It must be noted that reaction time to a tactile cue under noise recorded outside a vehicle was shorter than that under white noise. Moreover, the reaction time to a tactile cue under white noise (noise (a)) was nearly equal to the reaction time to an auditory cue under noise from outside a vehicle (noise (b)) irrespective of the noise level LAeq. The response speed under noise from outside a vehicle was faster than that under white noise for each cue modality, SOA, and noise level LAeq (see Figs. 7e9). All participants were licensed drivers and accustomed to driving under real-world driving environment. While noise from outside a vehicle must be more familiar than white noise, white noise was likely to be very annoying for all participants. Such familiarity or annoyance might lead to the difference in response speed between white noise and noise from outside of the vehicle for both auditory and tactile warnings. The SOA condition affected the reaction time to auditory or tactile warnings under both white noise and noise from outside a
66
A. Murata et al. / Applied Ergonomics 60 (2017) 58e67
vehicle (see Figs. 7e9). When SOA was equal to 0 ms, the reaction time was prolonged for both auditory and tactile cues, and for both types of noises (a) and (b). For SOA of 500 ms or 1000 ms, the reaction time was lower than that for SOA of 0 ms. Participants were required to confirm that the visual hazard existed whenever a warning was presented, and the catch trials were inserted to avoid premature responses to an auditory or a visual cue. The SOA of 0 ms did not necessarily lead to a fast response to a cue. it is certain that the warning should be presented to a driver as soon as a hazard was detected. Ideally, the warning should be presented to a driver as soon as a hazard was detected by a warning system. As shown in Figs. 7e9, the reaction to both tactile and auditory cues for SOA of 0 ms tended to be slower as compared with that for SOA of 500 ms and 1000 ms. In this manner, it was verified that a cue before a hazard appears leads to the preparedness of a participant and thus to a faster reaction of a participant. In summary, the SOA condition of 0 ms is not suitable for warning presentation. SOA should be set to 500ms or 1000ms, which is enough to prepare for the forthcoming hazard. Therefore, it should be desirable that a warning system in advance predict the hazard before it actually occurs so that drivers can prepare for the hazard. On the basis of these results, tactile warning appears very promising as a warning cue under noisy environments. Moreover, the auditory cue was not affected by the ambient noise level when the noise recorded from a real-world driving environment was used. The nature of cross-modal links in spatial attention demonstrates that responses to a target presented in one sensory modality can be facilitated by the prior presentation of a cue (warning) by another sensory modality (Spence and Driver, 2004). On the basis of these results, tactile warning is very promising as a warning signal to a visual hazard especially in a noisy environment. Further, the results also suggest that an auditory cue of pure tone of 1 kHz is less affected by the ambient noise when noise recorded outside a vehicle is used than when white noise is used, which indicates that the compatibility of the auditory cue and external noise is an important design factor for an auditory warning system. 4.3. Effectiveness of tactile warning under real-world driving This study demonstrated the effectiveness of a tactile cue in a laboratory experiment. It is important to investigate whether the results holds even under real-world running. As it was judged it necessary to demonstrate the effectiveness of tactor under realworld environment, the comparative study on the sensitivity of tactor under both laboratory and real-world driving environments was conducted. As shown in Fig. 10(A) and (B), the sensitivity of the tactor did not differ between the real-world running environment and the laboratory experiment for ascending and descending measurements and the mean of both measurements. This indicates that the tactile warning is practically usable even in real-world running environments. Based on this result, it can be speculated that the results are applicable even under real-world running. 4.4. Implication for design of automotive warning system The design implications can be summarized as follows. Based on the results that the tactile cues are robust to the noise level, it is suggested that they should be utilized as a means of automotive warning systems. This study indicates that an auditory warning system should be designed by taking into account the compatibility between the auditory cue and an ambient (background) noise. The results are also indicative of carefully selecting the type of noise to investigate the effectiveness of warning systems with accuracy. As
the SOA condition of 0 ms led to slower a response to both auditory and tactile cues, SOA of 500ms or 1000ms should be used for drivers to be prepared enough for the forthcoming time-critical events or hazards. The effectiveness of simultaneous presentations of both auditory and tactile cues is suggested, for example, in Ho et al. (2006b; 2008), Murata et al. (2012a), and Murata and Kanbayashi (2013). Future research should carry out a comparative study of effectiveness between tactile warning and auditory-tactile warning. Although warnings regarding hazards on the left or right were studied here, future research should explore the effectiveness of non-visual cues when drivers are provided with not less than four directions. The pure tone of 1 kHz was used in this study. The effectiveness of looming auditory cues was suggested by Gray (2011). Ho et al. (2005) investigated the effectiveness of various auditory cues in capturing a driver's visual attention. Future work should explore the most suitable type of auditory cue. The limitations of this study are stated. Each condition was conducted for a relatively short period of time (9e10 min) in regards to real world driving. Drivers must usually drive for more than 9e10 min in real world driving, and must conduct an avoidance task as unexpectedly required by a warning system during driving. Therefore, it is speculated that the effectiveness of a warning system is affected by time spent in the vehicle. In future research, it must be explored how an auditory or tactile warning system may be affected by time spent in the vehicle. As the participants in this study were from the population of young male (licensed) drivers, it is not certain whether the results can be generalized to other populations such as females or older adults. How the results may be affected when tested with a wider demographic such as sex difference or age should be addressed in future research. Finally, we discuss the risk compensation phenomenon that appears with the development of such a preventive safety system. Preventive safety by means of an automotive warning system is desirable under conditions where the driver's workload is high, thus having limited resources left to analyze the situation before reacting to it. However, it must be noted that such a system is not necessarily beneficial and may have negative effects if the corresponding risk is undermined. We must give plenty of thoughts to whether drivers really drive safely when vehicles are equipped with safety warning systems. Analysis of the behavior of drivers of a vehicle with safety features such as seat belts and air bags indicated that these drivers tended to drive more aggressively (Wilde, 1982; Evans, et al., 1982; Evans and Wasielewski, 1982; Evans and Graham, 1991). Such behavior can be explained by the theory of risk homeostasis (Wilde, 1982). Such an adverse effect to new preventive safety technologies must be considered.
5. Conclusions In order to determine the most suitable modality of cue (warning) to a visual hazard under noisy environments, the effects of the following four conditions on the response speed to a hazard were examined: (1) cue modality (auditory and tactile cue), (2) SOA (0ms, 500ms, and 1000ms), (3) noise type (white noise and noise outside a vehicle), and (4) noise level LAeq inside the experimental chamber (approximately 60 dB(A), 70 dB(A), and 80 dB(A)). We investigated whether a tactile cue under a noisy environment is more effective for quickening the response to a hazard than an auditory cue. It was also explored whether the response speed to an auditory cue is different according to the type of the ambient noise. The conclusions and implications can be stated as follows:
A. Murata et al. / Applied Ergonomics 60 (2017) 58e67
1. The reaction time to the tactile warning was not affected by both the white noise level and the noise level from outside a vehicle. As the response to the tactile cue was faster than that of the auditory cue irrespective of the noise type, it can be concluded that the tactile cue is more robust to the noise level. This supports the utilization of tactile cues as a means of automotive warning system. 2. The reaction time to the auditory warning was significantly affected by the white noise level, while the reaction time to the auditory warning was not affected by the noise level from outside a vehicle. This indicates that the compatibility between an auditory cue and the ambient noise should be taken into account when designing an auditory warning system. 3. The reaction time to both auditory and tactile cues tended to be shortened under the noise recorded outside a vehicle than under the white noise, which suggests that the type of noise should be carefully selected to investigate the effectiveness of the warning system. 4. The SOA of 0 ms did not necessarily lead to a fast response to both auditory and tactile cues for noise conditions. The SOA condition of 0 ms is not suitable when designing an auditory warning system. SOA of 500ms or 1000ms should be selected so that drivers are adequately prepared for the forthcoming hazards. Acknowledgments This work was partly supported by Grant-in Aids for Scientific Research (B) (grant numbers 22310101 & 26282095), Japan Society for the Promotion of Science (JSPS). References Castro, C., 2009. Human Factors of Visual and Cognitive Performance in Driving. CRC Press. Denworth, L., 2015. The special power of touch. Sci. Am. Mind 26 (4), 30e39. Driver, J., Spence, C., 1998. Attention and the cross-modal construction of space. Trends Cognitive Sci. 2 (7), 254e262. Driver, J., 2001. A selective review of selective attention research from the past century. Br. J. Psychol. 92, 53e78. Dukic, T., Hanson, L., Falkmer, T., 2006. Effects of drivers' age and push button locations on visual time off road, steering wheel deviation and safety perception. Ergonomics 49 (1), 78e92. Evans, L., Graham, J.D., 1991. Risk reduction or risk compensation? The case of mandatory safety-belt use law. J. Risk Uncertain. 4 (1), 61e73. Evans, L., Wasielewski, P., 1982. Do accident-involved drivers exhibit riskier everyday driving behavior? Accid. Analysis Prev. 14 (1), 57e64. Evans, L., Wasielewski, P., von Buseck, C.R., 1982. Compulsory seat belt usage and driver risk-taking behavior. Hum. Factors 24 (1), 41e48. Ferris, T., Sarter, N.B., 2008. Cross-modal links among vision, audition, and touch in complex environments. Hum. Factors 50 (1), 17e26. Ferris, T., Penfold, R., Hameed, S., Sarter, N., 2006. The implications of crossmodal links in attention for the design of multimodal interfaces: a driving simulator study. Proc. Hum. Factors Ergonomics Soc. 50th Annu. Meet. 406e409. Fitch, G.M., Kiefer, R.J., Hankey, J.M., Kleiner, B.M., 2007. Toward developing an approach for alerting drivers to the direction of a crash threat. Hum. Factors 49 (4), 710e720. Gkikas, N., 2013. Automotive Ergonomics-driver-vehicle Interaction. CRC Press. Gray, R., 2011. Looming auditory collision warning for driving. Hum. Factors 53 (1), 63e74. Ho, C., Spence, C., 2005. Assessing the effectiveness of various auditory cues in capturing a driver's visual attention. J. Exp. Psychol. Appl. 11 (3), 157e174.
67
Ho, C., Spence, C., 2008. The Multisensory Driver-implications for Ergonomic Car Interface Design-. Ashgate. Ho, C., Tan, H.Z., Spence, C., 2005. Using spatial vibrotactile cues to direct visual attention in driving scenes. Transp. Res. Part F 8 (6), 397e412. Ho, C., Reed, N., Spence, C., 2006a. Assessing the effectiveness of “intuitive” vibrotactile warning signals in preventing front-to-rear collisions in a driving simulator. Accid. Analysis Prev. 38 (5), 989e996. Ho, C., Reed, N., Spence, C., 2006b. Multisensory in-car warning signals for collision avoidance. Hum. Factors 49 (6), 1107e1114. Ho, C., Tan, H.Z., Spence, C., 2006c. The differential effect of vibrotactile and auditory cues on visual spatial attention. Ergonomics 49 (7), 724e738. Jones, L.A., Sarter, N.B., 2008. Tactile displays:Guidance for their design and application. Hum. Factors 50 (1), 90e111. Jones, C.M., Gray, R., Spence, C., Tan, H.Z., 2008. Directing visual attention with spatially informative and noninformative tactile cues. Exp. Brain Res. 186, 659e669. Lee, J.D., Caven, B., Haake, S., Brown, T.L., 2001. Speech-based interaction with invehicle computers: the effect of speech-based e-mail on drivers' attention to the roadway. Hum. Factors 43 (5), 631e640. Lee, J.D., McGehee, D.V., Brown, T.L., Marshall, D., 2006. Effects of adaptive cruise control and alert modality on driver performance. Transp. Res. Rec. 1980, 49e56. Lees, M.N., Lee, J.D., 2007. The influence of distraction and driving context on driver response to imperfect collision warning systems. Ergonomics 50 (8), 1264e1286. Linden, D.J., 2015. Touch, Viking Penguin, UK. Meng, F., Gray, R., Ho, C., Ahtamad, M., Spence, C., 2015. Dynamic vibrotactile signals for forward collision avoidance warning systems. Hum. Factors 57 (2), 329e346. Murata, A., Kanbayashi, M., 2013. Effectiveness of automotive warning system presented with multiple sensory modalities. Proc. HCI2013 22, I88eI97. Murata, A., Nakagawa, Y., 2012. Effectiveness of warning presentation with directional cue that makes use of cross-modal link. Proc. SICE2012 1379e1384. Murata, A., Tanaka, K., 2011. Basic study on effectiveness of tactile interface for warning presentation in driving environment. Int. J. Knowl. Eng. Softw. Data Paradigms 3 (1), 95e105. Murata, A., Kanbayashi, M., Hayami, T., 2012a. Effectiveness of automotive warning system presented with multiple sensory modalities. Proc. SICE2012 920e925. Murata, A., Kemori, Moriwaka, M., 2012b. Basic study on automotive warning presentation to front/rear hazard by vibrotactile stimulation. Proc. AHFE2012 421e430. Murata, A., Kemori, S., Moriwaka, M., 2013. Proposal of automotive 8-directional warning system that makes use of tactile apparent movement. Proc. HCI2013 22, I98eI107. Murata, A., Kuroda, T., Kanbayashi, M., 2014. Effectiveness auditory and vibrotactile cuing of driver attention under noisy environment. Proc. AHFE2014 3247e3256. Regan, M.A., Lee, J.D., Young, K.L., 2009. Driver Distraction-theory, Effects, and Mitigation-. CRC Press. Scott, J.J., Gray, R., 2008. A comparison of tactile, visual, and auditory warnings for rear-end collision prevention in simulated driving. Hum. Factors 50 (2), 264e275. Spence, C., Driver, J., 1994. Covert spatial orienting in audition: exogenous and endogenous mechanisms. J. Exp. Psychol. Hum. Percept. Perform. 20 (3), 555e574. Spence, C., Driver, J., 1997a. Cross-modal links in attention between audition, vision, and touch: implications for interface design. Int. J. Cognitive Ergonomics 1 (4), 351e373. Spence, C., Driver, J., 1997b. On measuring selective attention to a specific sensory modality. Percept. Psychophys. 59 (3), 389e403. Spence, C., Driver, J., 2004. Crossmodal Space and Crossmodal Attention. Oxford University Press. Stanton, N.A., Young, M.S., 2005. Driver behavior with adaptive control. Ergonomics 48 (10), 1294e1313. Van Erp, J.B.F., van Veen, H.A.H.C., 2001. Vibro-tactile information processing in automobiles. Proc. Of EuroHaptics 2001, 99e104. Van Erp, J.B.F., van Veen, H.A.H.C., 2003. A multi-purpose tactile test for astronauts in the international space station. Proc. EuroHaptics 2003, 405e408. Van Erp, J.B.F., 2005. Presenting directions with a vibrotactile torso display. Ergonomics 48 (3), 302e313. Wilde, G.J.S., 1982. The theory of risk homeostasis: implications for safety and health. Risk Anal. 2 (4), 209e225. Young, M.S., Stanton, N.A., 2004. Taking the load off: investigations of how adaptive cruise control affects mental workload. Ergonomics 47 (9), 1014e1035.