Valid representation of a highly dynamic collision avoidance scenario in a driving simulator

Transportation Research Part F 31 (2015) 54–66

Contents lists available at ScienceDirect

Transportation Research Part F journal homepage: www.elsevier.com/locate/trf

Valid representation of a highly dynamic collision avoidance scenario in a driving simulator Ilka Zöller a,⇑, Alexander Betz b, Nicole Mautes a, Lucas Scholz b, Bettina Abendroth a, Ralph Bruder a, Hermann Winner b a b

Technische Universität Darmstadt, Institut für Arbeitswissenschaft, Otto-Berndt-Straße 2, 64287 Darmstadt, Germany Technische Universität Darmstadt, Fachgebiet Fahrzeugtechnik, Otto-Berndt-Straße 2, 64287 Darmstadt, Germany

a r t i c l e

i n f o

Article history: Received 12 December 2013 Received in revised form 10 March 2014 Accepted 9 March 2015

Keywords: Driving simulator Validity Collision avoidance Driver behavior Wheeled mobile

a b s t r a c t In Germany the second-most frequent accidents in road traffic are rear-end collisions. For this reason rear-end collisions are quite important for accident research and the development of driving safety systems. To examine the functionality and to design the human– machine-interface of new driving safety systems, especially in the early development phase, subject tests are necessary. Because of the great hazard potential of such safety critical scenarios for the test persons, they are often conducted in a driving simulator (DS). Accordingly, validity is an important qualification to ensure that the findings collected in a simulated test environment can be directly transferred to the real world. This paper regards the question of driving behavior validity of DS in critical situations. There are hardly any validation studies which analyze the driving behavior in a specific collision avoidance situation. The validation study described in this paper aims to evaluate the behavioral validity of a fixed-base simulator in a collision avoidance situation. For this reason a field study from 2007 was replicated in a fixed-base simulator environment. The main questions of this validation study were if the driver can notice an active hazard braking system and if the driving behavior in a static simulator can be valid in such a critical situation. The key finding of the study states that there is no driving behavior validity in a static driving simulator for the tested dynamic scenario. The missing vestibular feedback causes a different behavior of the participants in field and simulator. The resulting absence of comparability leads to non-valid performance indicators. But these indicators are key parameters for analyzing the function and acceptance of active braking systems. So the question arises, which motion performance does a motion base have to provide in order to achieve valid acceleration simulation of such a highly dynamic collision avoidance scenario. The DS’s performance is measured in workspace, velocity and acceleration. Ó 2015 Elsevier Ltd. All rights reserved.

⇑ Corresponding author. Tel.: +49 6151 166289. E-mail addresses: [email protected] (I. Zöller), [email protected] (A. Betz), [email protected] (N. Mautes), Lucas-scholz@ mgc-mainz.de (L. Scholz), [email protected] (B. Abendroth), [email protected] (R. Bruder), [email protected] (H. Winner). http://dx.doi.org/10.1016/j.trf.2015.03.004 1369-8478/Ó 2015 Elsevier Ltd. All rights reserved.

I. Zöller et al. / Transportation Research Part F 31 (2015) 54–66

55

1. Introduction According to figures from the German Federal Statistics Office (Destatis, 2013), the number of rear-end collisions (collision with a leading vehicle and/or one that is in the process of stopping, is already stopped, or is just beginning to accelerate after a stop) with 2703 accidents in February 2013 appeared relatively high. This corresponds to an equity ratio of 17% of the total number of accidents with personal injuries in this month (15,835). In February 2012 the number of rear-end collisions was 18% higher than in 2013 (3201). According to this, the accident rate is decreasing gradually, but the number of such accidents is still high. The German Automobile Club analyzed the causes of accidents and published that rear-end collisions are the second-most frequent reason for car accidents (ADAC, 2011). To avoid such accidents the automotive industry develops safety systems (such as ABS, braking assistant). To assess and design new safety systems, driving tests in a simulation environment are often performed. The main reason for this fact is that simulators provide a secure environment for the test person (Blana, 1996) and the test vehicle. For this reason, driving simulators (DS) admit to analyze the human–machine interaction according to active safety systems already in the early phases of the engineering process of such technological systems. Particularly for critical situations, using the DS is a cost-efficient way to reach an adequate evaluation. In addition to the mentioned benefits, it has been taken into consideration that DS are merely an attempt at reproducing reality – therefore second-rate, regardless of how exact the model is. Against this background it becomes particularly urgent to enquire whether the transferability of the data gathered in the simulator to a real driver behavior can be guaranteed. A lot of research can be found that examines this question of validity of driving behavior (e.g. Bella, 2005; Blaauw, 1982; Engström, Johansson, & Ostlund, 2005; Godley, Fildes, & Triggs, 2002; Jamson & Mouta, 2004). However, there are almost no scientific validity studies on rear-end situations with/without system intervention. The study treated in this paper analyzes the validity of a fixed-based driving simulator with 180° FOV in a rear-end collision avoidance scenario. The analyzed question is, if a fixed-based simulator with a large FOV is realistic enough to provide valid results, or if a moving-base is needed for such a dynamic situation. Another question investigated in this paper, is how much space an adequate moving-base simulator would need. 2. Basic methodology of validation Existing validation methods have their origin in a differentiation from Mudd (1968) and McCormick (1970), who divided validity into a physical and a behavioral aspect (Blana, 1996). The physical aspect refers to the physical correspondence between the simulator and the real vehicle. The behavioral validity refers to the aspect whether the behavior of participants driving in a DS is comparable to the behavior shown in real environment. Generally the behavioral aspect is supposed to be more important for the examination of a specific task of driving (Blana, 1996). Therefore this paper focuses on driver behavior validity and answers the question of whether drivers’ behavior in a simulator can be transferred to driving behavior in the field. The validation methodology for behavioral validity was proposed by Brown for flight simulation in 1975. According to Brown (1975), the main question is how well the task in the simulator duplicates the conditions of the real task. For analyzing this he distinguishes between quantitative indices of performance and subjective criteria. Quantitative indices are, on the one hand, the characteristics of the flight which were analyzed by comparing control performances (e.g. rate of lateral motion) in the simulator with them expected in an actual flight. On the other hand, they are physiological measurements (e.g. heart rate, skin conductance level) which provide objective and quantitative values. The subjective validation is based on the opinions of experienced pilots. Furthermore, Brown (1975) mentioned that the simulator always represented a specific aircraft. This methodology is transferable to driving simulators, as both are human–machine-systems. The validation study described in this paper refers to the methodology from Brown (1975); two studies (field and simulator) are conducted under similar conditions (e.g. identically chosen subject groups, identical road geometry, identical programmed behavior of the leading vehicle and the same safety system). In order to validate the behavioral validity, the recorded data of both studies are then compared. The selection of the analyzed data depends on the goal of the field study. Thus the question can be answered, if data gathered in a simulated environment matches the results of the field study. In the analysis of driving behavior validity a distinction is made between absolute and relative validity (Reed & Green, 1995). When there is a numerical correspondence between the behavior data in the simulator and in the real world (both studies under same conditions) it is called absolute validity (Reed & Green, 1995). When there is a correspondence between effects of different variations in different situations it is called relative validity (Törnros, 1998). In the present study the absolute validity is investigated, because it is more general. To analyze if the measurements show absolute validity, statistical comparisons of the means are carried out, for example, with a t-test. If there is no statistically significant difference between the means of a parameter in field and simulation (p > 0.05), absolute validity can be confirmed.

56

I. Zöller et al. / Transportation Research Part F 31 (2015) 54–66

3. Validation study design 3.1. Evaluation goal The field study was conducted at the Technische Universität Darmstadt in 2007 (i.a. Fecher et al., 2008). The Institute of Automotive Engineering (FZD) and the Institute of Ergonomics (IAD) joined forces to conduct this study. The aim was to analyze the drivers’ acceptance and behavior while an intervention of an active hazard braking system in an imminent rear-end collision scenario. The specific question was how the test person would act in a collision avoidance situation with and without an active hazard braking system and during a system intervention when a collision avoidance situation does not exist. According to this, specific performance indicators (e.g. effectiveness, reaction time) are chosen for the question of validity. The measurements and selected performance indicators are further described in Section 3.5. The formed hypotheses are described in Section 4. 3.2. Participants As the field study was conducted in 2007, most of the design aspects of the present validation study are predefined. This includes the collective of participants. Fig. 1 shows the composition of subjects in field and simulator study. The field study involved N = 60 subjects. Ages ranged from 23 to 69 years (mean = 43.6 years, SD = 15.7 years). Gender was almost balanced (32 men and 28 women). The composition of the subjects for the simulator study was selected according to the same group criteria (‘‘younger drivers’’ aged up to 40 and ‘‘elderly drivers’’ aged 50 years old or over) to the field study, shown in Fig. 1. The participants were not selected on the basis of their driving experience or reaction times, but comparability concerning this criteria is assumed because of the large number of participants (N > 30) and the assumption of a normal distribution. N = 45 subjects, 27 men and 18 women, had participated in the simulator study. There were hardly any experiences with DS among the participants. Their ages ranged from 19 to 65 years (mean = 41.2 years, SD = 14.9 years). 3.3. Testing environment The simulator study was conducted in the fixed-base IAD driving simulator (Fig. 2), which is equipped with a 180° FOV. Three high definition projectors with a resolution of 1920  1200 pixels and a luminance of 6000 lumen are used to realize this field of view. The mockup consists of a full size Chevrolet Aveo. The steering wheel (SensoWheel) is equipped with an electronic force-feedback system, the pedals also emit a mechanical feedback force. The front car speakers provide the driver navigation prompts and driving noises. The simulation runs with the driving simulation program SILAB. In the field study a Honda Legend was used as experimental vehicle. The vehicle was equipped with an active braking system whose deceleration could be adjusted through the variables brake pressure and intervention duration. CAN-Data was recorded with the software DEWESoft 6.4. To create a rear-end collision situation that is highly reproducible and safe for the test person the Experimental Vehicle for Unexpected Target Approach (EVITA) (Hoffmann, 2008) was used in the field study. EVITA is an evaluation method for rear-end collision situations. The tests (unexpected target approach) are conducted in real world test drives on a proving ground without safety risk for the test person. EVITA consist of a setup using a towing vehicle, an actively braking trailer and a following car (Fig. 3). The test starts with a constant follow-up run (vrel between all three systems is zero) and a close-coupled car trailer combination. Suddenly, the trailer detaches from the towing car (cable winch opens) and brakes actively to cause a critical situation for the test person in the following car behind the trailer. Meanwhile, the towing car keeps up the initial velocity. Regardless of the test person’s reaction the towing car avoids the

Total number of subjects

18 16 14 12 10 8

17 15

6

15

Field study

15 13

12

Simulator study

10

4

8

2 0 male (<40)

male (50-69)

female (<40) female (50-69)

Different groups of subjects Fig. 1. Composition of the subjects in the field study (N = 60) and the simulator study (N = 45).

I. Zöller et al. / Transportation Research Part F 31 (2015) 54–66

57

Fig. 2. Fixed-base driving simulator at IAD, TU Darmstadt.

Fig. 3. EVITA setup (Hoffmann, 2008).

rear-end collision by reaccelerating the trailer by closing the cable winch. The trailer reaccelerates to the velocity of the towing car and gets pulled out of the collision area. In the driving simulator study a follow-up car like EVITA was simulated. 3.4. Experimental design The field study took place at the Technische Universität Darmstadt testing ground, at the August-Euler-airfield in Griesheim (Darmstadt). The road geometry can be inferred from Fig. 4. For the driving simulator study the test track was modeled in the simulation software SILAB. In both studies the experimental design was identical. After a practice drive to get used of the testing environment each participant had to drive four laps on the real or simulated test track, each beginning at the starting point shown in Fig. 4 and corresponding to one of the four variants shown in Table 1. A cone-lined lane on the runway forced the driver to follow the confederate vehicle EVITA without lateral displacement. The confederate vehicle EVITA drove with a constant speed of 60 km/h. The driving task for the test person included following the confederate car at a distance of 20–25 m. To provide a reference, a display informed the driver about his current distance in both studies. In order to produce a typical collision avoidance scenario, the subjects had to perform a secondary task while following EVITA (reading navigation instructions of a piece of paper in the lower area of the center console). The aim was to visually distract the subjects from the traffic situation. The reason for this experimental setup was that distraction is a common cause of accidents (e.g. NHTSA, 2009), and different distraction conditions have specific influences of the driving behavior (Muhrer & Vollrath, 2011). In two variants an unexpected risk of collision was created in the ‘‘Baseline/AHBS’’-zone (Fig. 4) during the distracted follow-up run. Therefore the confederate vehicle (EVITA) suddenly braked (deceleration value: 8 m/s2), the moment that the TTC (time to collision) had fallen below seven seconds (Fig. 5). In each of these variants different system characteristics of the active braking guard were tested. In the ‘‘Baseline’’ variant the test person had no support from the Active Hazard Braking – system, while during the ‘‘AHBS’’ variant there was a full active deceleration (deceleration value: 10 m/s2) of the braking-system for 1.3 seconds, which supported the driver. In order to analyze the driver’s response and behavior in the event of authorized proper system intervention as well as an unwarranted system intervention, a non-critical situation with a ‘‘False Tripping’’ of the system was realized. In the variant ‘‘Empty Run’’ nothing happened, in order to reduce the expectations of the participants. A randomized order of the variants ‘‘Baseline’’, ‘‘AHBS’’ and ‘‘False Tripping’’ should ensure

58

I. Zöller et al. / Transportation Research Part F 31 (2015) 54–66

Starting Point

„False Tripping“ -zone

„Baseline/AHBS“ -zone

Fig. 4. Course sketch of the testing ground from TU Darmstadt in Griesheim.

Table 1 Experiment variants with the different system characteristics of the active hazard braking. Variants

Risk of collision

Deceleration experimental vehicle (m/s2)

Deceleration EVITA (m/s2)

‘‘Baseline’’ ‘‘AHBS’’ ‘‘False Tripping’’ ‘‘Empty Run’’

Yes Yes No No

0 10 10 0

8 8 0 0

Fig. 5. Braking of the confederate vehicle (EVITA).

equal treatment. In each randomization the second variant was the ‘‘Empty Run’’ to ensure that the subjects would not anticipate the next critical situation. Critically speaking, only the first critical situation can be used for further analysis because of the given surprise effect. Table 1 shows the different system characteristics of each variant. Accordingly, in both studies the participants got no information about the active hazard braking system to achieve the desired surprise effect during the critical situation. The general process of the experiment was equivalent in field and simulator study. After being welcomed by the investigators, the subjects had to complete a questionnaire about their driving and simulator experience as well as some other personal information. As an exception of the DS study, the participants drove then an initial drive for 10 min. Then they got their final introduction on the first and secondary task, such as in the field study, before the experiment consisting of four laps started on the previously shown test track. After the variants ‘‘AHBS’’ and ‘‘False Tripping’’ the subjects had to respond on some questions about the system intervention. After the test ride the subjects had to answer two further questionnaires, one about the system interventions and one about the realism of the driving simulator. 3.5. Measurements and performance indicators Vehicle dynamic variables such as the steering angle, brake pressure, accelerator pedal position and speed were measured. Variables such as distance, relative speed and TTC (time to collision) were calculated during the study. To monitor

I. Zöller et al. / Transportation Research Part F 31 (2015) 54–66

59

the drivers’ actions and the motoring environment video recordings were used. Furthermore the results from eye tracking were analyzed. The data presented here was used to determine specific performance indicators that are important for analyzing the functional quality of an active braking system (Hoffmann, 2008). How these indicators in the field and simulator study are consistent will be analyzed in the context of hypotheses tests in Section 4. 3.5.1. Effectiveness To analyze the behavior and reaction of drivers with and without a collision avoidance system, such as Forward Collision Warning, a common method is to expose the subject to repeated collision avoidance situations and then calculate the average response over those hazardous situations (Aust, 2012). As such, for the authorized system intervention (‘‘AHBS’’) the ‘‘effectiveness’’, which complies with the speed difference Dv during the evaluation period (Hoffmann, 2008), is a major criterion. The reason, therefore, is that the ‘‘effectiveness’’ enables conclusions to the system performance and so to the accident avoidance potential. The assessment period lasts 2 s, beginning with the system intervention, and ending with an imaginary rear-end collision (Hoffmann, 2008). Fig. 6 shows the idealized velocity curve over time for the ‘‘AHBS’’ variant. 3.5.2. Reaction time Another important criterion to characterize different driver reactions in the case of hazardous situations is the reaction time. According to Hoffmann (2008) it consists of three variables, visual distraction time, foot movement time and actuation time (Fig. 7). The visual distraction time describes the time duration between the system intervention (TTC < 7 s) and the first glance towards the street. The foot movement time begins with the first movement of the foot from the gas pedal and ends with the first contact with the brake pedal. If the drivers’ foot is already placed over the braking pedal the foot movement time is set to zero. The actuation time describes the time between the first contact with the brake pedal and the achievement of a braking pressure of 60 bar. Finally it must be mentioned, that the performance data of the braking systems in the Honda Legend and the simulator cannot be compared directly. This is caused since the physical behavior of the brake system’s hydraulics and mechanics is not simulated in the DS. Because of this, the actuation time cannot be analyzed. 3.5.3. Drivers’ reactions and forgiveness The indicators described above are important measurements to assess the functional quality of the active braking system. Another important question relates to the case of ‘‘False Tripping’’ and, therefore, a non-critical situation, in which an intervention of the braking system is unintentional. For this variant two indicators are analyzed. First of all the objective driver reaction (e.g. accelerating, stepping on the brake) is an important aspect. Also interesting is the subjective feeling of disturbance. This ‘‘forgiveness’’ (Hoffmann, 2008) is measured through a questionnaire. 4. Validation results First of all it has to be mentioned that some data sets of the simulator study are not usable. This is a result of various issues experienced during data collection. The first problem was the occurrence of simulator sickness, which caused the abortion of the simulation in 22% of the test drives. Another issue was the high degree of difficulty of the task of driving (following the vehicle ahead with constant distance of 20–25 m), which in turn activates the active hazard braking system. For this issue two possible causes can be named: a fixed-base simulator does not provide the sensation of acceleration forces and a two dimensional view makes it difficult to estimate the distance to EVITA. Other participants were not able to perform the secondary task as they were fully occupied with the task of driving. As a result these participants were not distracted during the simulation.

1 Driver distraction 2 Braking EVITA 3 System action 4 Gaze attention 5 Driver brakes

Evaluation period

Fig. 6. Idealized velocity curve over time for the ‘‘AHBS’’ variant (Hoffmann, 2008).

60

I. Zöller et al. / Transportation Research Part F 31 (2015) 54–66

visual distraction time

foot movement time

actuation time

reaction time

Fig. 7. Components of the reaction time according to Hoffmann (2008).

For the contemplated measurement effectiveness to assess the functional quality of an active braking system the variant ‘‘Baseline’’ provides a basis for comparison, because there is no system intervention and so only the driver reaction is measured. For the variant ‘‘Baseline’’ only N = 10 datasets of the simulator study are usable, because the other subjects were not distracted. N = 18 data sets of the field study are usable. The first hypothesis to be tested is H1. H1. The effectiveness from baseline, as first lap or not, in field and simulator study is equal. In order to take into account a potential surprise effect of the first experienced critical situation, the hypothesis is divided into two parts. First the effectiveness from ‘‘Baseline’’ as first lap is examined through a statistical analysis. The mean values were built of all usable datasets for the effectiveness in field and simulator study. These mean values were than compared with a two-sided t-test. The results (t = 0.231, p = 0.838) show no significant difference (at a significance level of 5%) between the effectiveness in field and simulator for the variant ‘‘Baseline’’ as first lap. The effectiveness from ‘‘Baseline’’ not as first lap was proved similarly. The results (t = 0.775, p = 0.468) show no significant difference. In summary it can be said that hypothesis 1 should not be rejected. There are no significant differences between the effectiveness of baseline as the first or as another lap or between field and simulator test. So, the driver behavior in the variant ‘‘Baseline’’ in both environments is equal and the validity cannot be denied. 4.1. Correct system intervention 4.1.1. Comparison of the indicator ‘‘effectiveness’’ in field and simulator The next chapter compares the indicator effectiveness for the ‘‘AHBS’’ variant measured in field or in simulator against each other in order to prove the transferability and, thus, the validity of these indicators between the two environments. In the case of validity the effectiveness is then compared with the results of the baseline variant. Regarding to the variant ‘‘AHBS’’ hypothesis H2 can be proposed. H2. The effectiveness from AHBS in field and simulator study is equal. In the simulator study only 7 of the 45 data sets are usable for the variation ‘‘AHBS’’. This is because in 17 cases the AHBS system was not activated, activated too late or activated before EVITA had braked. Additionally one subject did not drive in convoy, two subjects had braked before the confederate vehicle (EVITA) braked and 8 subjects were not distracted. Owing to the small quantity of usable data sets (N < 10), the statistical evaluation of the effectiveness of the variant AHBS for the simulator study will be waived. 4.1.2. Comparison of the indicator ‘‘reaction time’’ in field and simulator The reaction time is an interesting parameter because it makes it possible to compare different settings of system parameters (providing a full system intervention with 10 m/s2 or a partial intervention) with regard to the functionality and to the subjects’ reaction. To analyze the reaction time for the variant ‘‘AHBS’’ hypothesis H3 can be proposed. H3. The reaction time during the lap ‘‘AHBS’’ in field and simulator study is equal. Only three data sets of the simulator study can be used to analyze the hypothesis. The reason for this, is that only three of the seven subjects used the brake pedal during the ‘‘AHBS’’. Without the use of the brake pedal the actuation time, which is one component of the reaction time, cannot be established. Due to the vast amount of data and the missing system comparability a statistical analysis can be waived. 4.2. Unwarranted system intervention 4.2.1. Comparison of the indicator ‘‘driver reaction’’ in field and simulator For the unwarranted system intervention more data sets were usable because it was not necessary for the intervention that the subjects achieve the following distance of 25 m. As a result there are N = 20 usable data sets in the field environment and N = 32 usable data sets in the simulator study. The following hypothesis has been developed: H4. There are no significant differences in the drivers’ reactions during the variant ‘‘False Tripping’’ in the field and in the simulator study.

61

I. Zöller et al. / Transportation Research Part F 31 (2015) 54–66

The results show that in the field study (N = 20) drivers reacted in four ways, whereas four various reactions can be found in the simulator study (N = 32). The following Fig. 8 shows the distribution between the six different drivers’ reactions in both studies. In the simulator study (N = 32) no driver reacted with braking with releasing or braking to standstill, while in the field study 35% of the subjects reacted in this way. A possible explanation for this is that the subjects could not feel a vestibular difference during the ‘‘False Tripping’’ in the simulator and, moreover, they know that the test ride in the simulator is riskless. In the simulator study 69% of the drivers accelerated like before, or they accelerated more. Only one person (3%) moved the foot from the accelerator to the brake pedal. Because of the differences in the driver reactions, the hypothesis has to be rejected. The driver’s reaction during the variant ‘‘False Tripping’’ in the field and the simulator study are not similar and therefore the simulation results are not a valid representation of reality. This result was expected in a static simulator without vestibular feedback. 4.2.2. Comparison of the indicator ‘‘Forgiveness’’ in field and simulator Additional, the subjects were asked how disturbing the false tripping was. The answers of N = 37 subjects in the field and N = 29 subjects in the simulator study were analyzed and the following hypothesis was made: H5. The subjectively experienced annoyance in field and simulator study is similar. Fig. 9 demonstrates that the subjectively experienced annoyance in the simulator study is lower than in the field study. In the field study 76% of the subjects had experienced the false tripping as very disturbing and in the simulator study only 35% felt the same way. One possible reason for this difference is that the subjects in the simulator cannot feel the acceleration forces. Because of the differences in the subjectively experienced annoyance this hypothesis also has to be rejected. The subjectively experienced annoyance during the variant ‘‘False Tripping’’ in the field and the simulator study are not similar and therefore not valid. 5. Dynamic performance 5.1. Method to investigate the performance demand of DS Static driving simulators represent the most basic simulator setup. There is a wide variety of static simulators reaching from simple systems such as a personal computer setup similar to applications from the entertainment industry to more advanced systems like the used DS of IAD with enhanced periphery and field of view. Although, there is one inherent system limitation – lack of a dynamic driving experience. Negative side effects such as simulator sickness increase due to the lack adequate vestibular feedback (Straus, 2005) and the question arises of whether the driver behavior data that is collected is valid or not. The present study investigates the dynamic requirements of DS that are necessary to perform a dynamic DS study of the introduced rear-end collision situation. The method is presented schematically in Fig. 10. The identified requirements are compared with the state of the art DS systems used by the automobile industry in order to see if the available technology is sufficient for the introduced test case. This analysis is based on the previously introduced field study of the rear-end collision analysis. The longitudinal acceleration of the test drives is fed through the ‘‘ideal motion cueing algorithm’’ (MCA) that transforms the captured vehicle dynamics into DS dynamics in terms of horizontal motion of the DS’s center of gravity (CG) (Betz, Winner, Ancochea, & Graupner, 2012). Since this study represents a longitudinal maneuver, the lateral information is neglected. The motion of the CG is evaluated with respect to longitudinal stroke, velocity and acceleration demand of the DS.

Field study

Simulator study

Percent of subjects [%]

80 69

70

60 60 50 40 25

30 20

12

16 10 5

10 0

0

3

0

0

0 Continue Reduce Remove foot Move foot Breaking with Breaking to a stepping on stepping on from the over the brake releasing standstill the accelerator the accelerator accelerator pedal pedal pedal pedal

Different driver reactions Fig. 8. Drivers’ reactions by ‘‘False Tripping’’ in the simulator study (N = 32) and the field study (N = 20).

62

I. Zöller et al. / Transportation Research Part F 31 (2015) 54–66

Forgiveness 76

Percent of subjects [%]

80 70 60 50

35

40 30 20 10

Field study

24

Simulator study

17

14

8 10

5

8 3

0 undisturbing

little disturbing

rather disturbing

disturbing

very disturbing

Different experienced annoyance Fig. 9. Forgiveness of the ‘‘False Tripping’’ in the simulator study (N = 29) and the field study (N = 37).

The MCA is shown in Fig. 10. The parameterization requires the limitation of tilt rate and acceleration as well as the washout parameters sv and sd. To not disturb the driver’s acceleration perception due to cabin tilt, the tilt rate and acceleration have to be limited. Therefore, the low-pass filter (LP) is parameterized in such manner that the worst case acceleration step input does not violate human perception thresholds for tilt motion. The amplitude of the worst case step input is assumed to be according to following equation:

amplitude ¼ g  scaling factor; scaling factor 2 ½0:5; 0:7; 1 The washout parameters (sv and sd) are determined accordingly to the loop gain stability margin. Further details concerning the MCA are found in (Betz, Winner et al., 2012). The used parameters are presented in Table 2. 5.2. Results The results shown in Section 4 indicate that a fixed-base simulator cannot provide valid driver behavior results in the exemplarily chosen highly dynamic scenario. The question arises if validity of fixed-base simulators is obtained in any other highly dynamic scenarios. While there was equal effectiveness during the variant ‘‘Baseline’’ in field and simulator study, the active braking system (‘‘AHBS’’) could not generate same driver reaction as most drivers did not even perceive this braking activity. The question arises, which workspace would be necessary to create a valid simulation setup for such a highly dynamic collision avoidance scenario. In the course of the redesign of the Daimler simulator, the Daimler AG considered similar thoughts and came to the following result (Zeeb, 2010): ‘‘To induce a much better longitudinal motion sensation with a scaling factor close to 1:1 for all possible acceleration and deceleration scenarios even a several ten meter long sledge would not be sufficient, but would increase the technical and financial effort tremendously, especially when the [above mentioned] mandatory requirements for drive dynamic experiments have to be fulfilled.’’ For further information concerning the demands of DS the introduced methodology from Section 5.1 is applied and leads to following results for maximum stroke, velocity and acceleration. All three demands are determined for different scaling factors – 1 (solid graph), 0.7 (dashed graph) and 0.5 (dotted graph). The scaling factors are meant to be exemplary and are chosen accordingly to common values from literature (Greenberg, Artz, & Cathey, 2003). Those values usually result from the

Fig. 10. Schematically procedure of DS’s performance analysis.

63

I. Zöller et al. / Transportation Research Part F 31 (2015) 54–66 Table 2 Parameterization of the motion cueing algorithm. Scale factor

LP cut-off frequency (xLPF , s1)

LP damping ratio (1LPF , s1)

Feedback gain (sv, s)

Feedback gain (sd, s)

1 0.7 0.5

0.56 0.71 0.83

1.59 1.20 0.75

4.49

19.8

Fig. 11. DS’s performance requirements resulting from N = 292 test drives (Field study).

Table 3 Parameterization of the Motion cueing algorithm. Institution

Stroke in m  m

Velocity in m/s

Acceleration in m/s2

Forschungsinstitut für Kraftfahrwesen und Fahrzeugmotoren Stuttgart (FKFS, 2010) Daimler (Zeeb, 2010) Peugeot Société Anonyme (PSA) (Chapron & Colinot, 2007) University of Iowa (Clark, Sparks, & Carmein, 2001) Toyota (Murano, Yonekawa, Aga, & Nagiri, 2009)

10  7 12  0 10  5.5 20  20 35  20

±10 ±3 ±6.1 ±6.1

±10 ±5 ±6.1 ±4.9

combination of available DS and the concerned driving maneuver with the goal of a scaling factor as close to 1 as possible. Thus, the chosen scaling factors are only meant to show its influence onto the DS’s performance demands without any opinion of the authors of this study. The demanded scaling factor has to be determined by the responsible supervisor of the respective experiment. The solid graph shows the demands for a scaling factor of 1. In this case the acceleration simulation is performed as measured in the real world test drive. The maximum stroke demand is shown as a cumulative distribution function (Fig. 11, upper diagram). The results show that the maximum stroke demand of about 105 m is not an outlier. When scaling the measured vehicle dynamics the demand is reduced as shown by the dashed and dotted graph. It has to be stressed that even the 0.5 scaled maneuvers require maximum strokes between 10 m and 23 m. Besides the stroke demand the system

64

I. Zöller et al. / Transportation Research Part F 31 (2015) 54–66

Fig. 12. CAD model of WMDS (left side) and resulting hardware prototype (right side).

performance depends on maximum velocity and acceleration capability, which are also related to the scaling factor, are shown in Fig. 11 (center and lower diagram). When comparing the results of this analysis to state of the art DS (shown in Table 3) it becomes clear that today’s systems are not capable performing the introduced test maneuver by scaling with a factor of 1. Even scaling with 0.5 is a hard task (stroke = 23 m; velocity = 8.5 m/s; acceleration = 6 m/s2) for those systems and reaches their performance limitations. While peak acceleration and velocity of high end DS are roughly sufficient, the stroke demand (scaling 0.5: 23 m; 0.7: 48 m; 1.0: 105 m) goes beyond all known system limits. The question arises why there are no systems providing a longer stroke since the peak acceleration and velocity capabilities are available. The reason for this is a result of the kind of motion system being used. To support the motion capability of the strongly limited hexapods, rail system and turntables are used for the unbound degrees of freedom of vehicles (x, y, yaw). Those rail systems grow with enhanced workspace and cause increasing system mass. This linkage is very cost intensive and represents the physical limit of the state of the art driving simulators. 6. Discussion and conclusion First of all it can be mentioned that valid data can be gained in a static driving simulator according to some driver behavior indicators in a rear-end collision situation. This is discussed in more detail below. The ‘‘effectiveness’’ shows valid results in the ‘‘Baseline’’ variant without system intervention. However it must be made clear that it is not possible to offer all targeted performance indicators in a valid way with the fixed-base driving simulator used in this study. In the driving simulator study the system intervened only in few cases because the conditions needed to make it viable have not been met (Section 4.1). For the subjects it is more difficult to follow a confederate vehicle within a defined distance with a defined speed in a simulated than in a real environment. The distance estimation seems to be difficult because of the lack of dynamic simulation and 3D-view. The results also indicate that correct speed estimation requires vestibular feedback. The number of generated critical situations is correspondingly lower because they were subordinated to the speed- and distance conditions. Despite a good visual simulation, with Over-HD-Quality and a sufficient FOV horizontal for an adequate speed perception, (according to Jamson, 2000, 120° are enough) it is hard to achieve the driving task. In addition, only three subjects used the brake in the ‘‘ABHS’’ variant, so that the reaction time could not be analyzed. The subjects did not recognize the hazard or recognized it too late. The reason for this is likely the lack of vestibular feedback for such a dynamic scenario. Ways to improve feedback and degree of reality of a driving simulator are shown below. Due to the lack of comparable conditions it was not possible to assess validity for the effectiveness of a static DS in a critical rear-end collision situation with active system intervention. The question could not be answered, if the ‘‘effectiveness’’ of the variant ‘‘AHBS’’ is greater than the effectiveness of ‘‘Baseline’’. For this reason simulators similar to the IAD driving simulator are not suitable to perform such dynamic tests with system interventions. Besides the lack of vestibular feedback the small number of usable data sets was another problem of the simulator study which leads to a lack of comparability with the field study. One possible approach to answer this question would be to substantially increase the number of participants. This would help to counteract the number of failures caused by simulator sickness. However it must be noted that increasing the number of participants substantially leads to an enormous increase in cost and effort. Against this background the advantages of a simulator study are lost (lower costs) and a field study would be preferred. So there is an economical limit of DS studies because the increasing numbers of participants reduces the advantage of low costs in simulated environments. With the previously mentioned approach in Section 5 the vestibular feedback in a driving simulator is consistent with real movements in such a situation. Thus, the vestibular sensory channel is satisfied. It can be expected that with these improvements, the reaction times become valid.

I. Zöller et al. / Transportation Research Part F 31 (2015) 54–66

65

The results from Section 5.2 support the statement of Zeeb (2010), the state of the art technology in DS are not capable of providing the investigated performance demands for the investigation through reasonable effort. It has to be stressed, that the investigated rear-end collision scenario is limited to longitudinal acceleration simulation. It becomes even more demanding when performing test with horizontal accelerations (x- and y-direction), such as turning. Thus, a new technology is required that allows for enhanced workspace while keeping the complexity and costs low. There is one alternative DS concept known from literature that seems promising for the demanded workspace requirement. The idea of a wheeled mobile DS (WMDS) was introduced in a patent of Donges (2002). The authors of this paper have researched this new WMDS concept since 2010 (Betz, Butry, Junietz, Wagner, & Winner, 2013; Betz, Hämisch, Müller, & Winner, 2012; Betz, Winner et al., 2012). The concept is introduced in Fig. 12. Wheeled Mobile Driving Simulators (WMDS) solve the core problem of very high moving mass and costs. The main idea is based on the assumption that a wheeled system, whose propulsion is limited by friction forces of the tire, is suitable to simulate the dynamics of vehicles that are also limited by tire friction forces. The wheeled mobile platform, with three conventional, powered and active steerable wheels, allows translational motion and yaw. On top of the platform a tilt system is to be mounted to provide pitch, roll and heave. The system provides built in stroke fitting real world vehicle motion. The workspace is only limited by the available size of the hall. Avoiding the conventional rail systems, which mainly cause the moving mass increase, results in a light weight concept as shown schematically in Fig. 12. The presented racing cockpit only illustrates the position where the dome is supposed to be mounted. In summary, there are two main points of criticism regarding the use of a fixed-base driving simulator for a highly dynamic scenario like the one concerned here. First, there is a lack of speed and distance perception despite a large field of view with high-resolution. Perhaps a 3D-simulation could provide a better feedback. The second point of criticism concerns the lack of vestibular feedback in a static simulator. A superior solution could be reached with a self-propelled motion platform as described above. In addition, an acoustic feedback and eventually seat-belt pre-tensioners could help to intensify the hazard situation to the subject (because the subject has problems to realize the change of the distance and the braking of EVITA). Thanks to the vestibular feedback provided by the motion platform the subjects could also perceive their own braking, not only the EVITA reaction. Acknowledgements The validation study described in this paper was supported by the German Research Foundation (Deutsche Forschungsgemeinschaft – DFG) as part of the research project ‘‘Bestimmung und Quantifizierung von Gestaltungsmerkmalen einer realitätsnahen Fahrsimulation’’. The field study was supported by the German AKTIV Projekt. References Aust, M. L. (2012). Effects of forward collision warning and repeated event exposure on emergency braking. Transportation Research Part F, 18, 34–46. Bella, F. (2005). Validation of a driving simulator for work zone design. Transportation Research Record, 1937, 136–144. Betz, A., Butry, A., Junietz, P., Wagner, P., & Winner, H. (2013). Driving dynamics control of a wheeled mobile driving simulator utilizing an omnidirectional motion base for urban traffic simulation. In Future active safety technology toward zero-traffic-accident (FASTzero), 22–26. September 2013, Nagoya, Japan. Betz, A., Hämisch, R., Müller, M., & Winner, H. (2012). Concept analysis of a wheeled mobile driving simulator showing an omnidirectional motion base for urban traffic simulation. In SIMVEC – Berechnung, Simulation und Erprobung im Fahrzeugbau 2012, 20. und 21. November 2012, Baden-Baden. Betz, A., Winner, H., Ancochea, M., & Graupner, M. (2012). Motion analysis of a wheeled mobile driving simulator for urban traffic situations. In Proceedings of driving simulation conference 2012 (S. 123–136). Frankreich: Actes INRETS. Blaauw, G. (1982). Driving experience and task demands in simulator and instrumented car: A validation study. Human Factors, 24, 473–486. Blana, E. (1996). Driving simulator validation studies: A literature review. University of Leeds: Institute of Transport Studies, Working Paper 480. Brown, J. L. (1975). Visual elements in flight simulator. Report of Working Group 34. Washinton DC: Committee on Vision, Assembly of Behavioural and Social Sciences. Chapron, T., & Colinot, J.-P. (2007). The new PSA Peugeot-Citroën Advanced Driving Simulator Overall design and motion cue algorithm. In Proceedings of driving simulator conference. Iowa City. Clark, A., Sparks, H., & Carmein, J. (2001). Unique features and capabilities of the NADS motion system. In Proceedings of the 17th international technical conference on the enhanced safety of vehicles (ESV). Amsterdam. Destatis (2013). Verkehr – Verkehrsunfälle. Fachserie 8 Reihe 7, Wiesbaden: Destasis. Donges, E. (2002). Fahrsimulator. Patent application DE 101 06 150 A 1, BMW AG. Engström, J., Johansson, E., & Ostlund, J. (2005). Effects of visual and cognitive load in real and simulated motorway driving. Transportation Research Part F, 8, 97–120. Fecher, N., Hoffmann, J., Winner, H., Fuchs, K., Abendroth, B., & Bruder, R. (2008). Analysis of driver behaviour in autonomous emergency hazard braking situations. In FISITA world automotive congress, 14–19 September 2008, München. Fisita. FKFS, Forschungsinstitut für Kraftfahrwesen und Fahrzeugmotoren Stuttgart (2010). Der Stuttgarter Fahrsimulator – Automobilforschung auf europäischem Spitzenniveau. Accessed 12.08.2013. Godley, S., Fildes, B., & Triggs, T. (2002). Driving simulator validation for speed research. Accident Analysis and Prevention, 34, 589–600. Greenberg, J., Artz, B., & Cathey, L. (2003). The effect of lateral motion cues during simulated driving. In Proceedings of driving simulation conference North America. Dearborn, Michigan. Hoffmann, J. (2008). Das Darmstädter Verfahren (EVITA) zum Testen und Bewerten von Frontalkollisionsgegenmaßnahmen. Fortschritt-Berichte VDI Reihe 12 Nr. 693, Düsseldorf, ISBN 978-3-18-369312-2. Jamson, H. (2000). Driving Simulation validity: issues of field of view and resolution. In Proceedings driving simulation conference (S. 57–64). Paris. Jamson, H., & Mouta, S. (2004). More bang for your buck? A cross-cost simulator evaluation study. In Proceedings of the driving simulator conference (S.321– 332). Paris. McCormick, E. J. (1970). Human factors engineering (3 Auflage). New York: McGraw-Hill.

66

I. Zöller et al. / Transportation Research Part F 31 (2015) 54–66

Mudd, S. (1968). Assessment of the fidelity of dynamic flight simulators. Human Factors, 10, 351–358. Muhrer, E., & Vollrath, M. (2011). The effect of visual and cognitive distraction on driver’s anticipation in a simulated car following scenario. Transportation Research Part F, 14, 555–566. Murano, T., Yonekawa, T., Aga, M., & Nagiri, S. (2009). Development of high-performance driving simulator. SAE International of Passenger Cars – Mechanical Systems, 2(1), 661–669. Reed, M., & Green, P., (1995). Validation of a low-cost driving simulator using a telephone dialling task. UMTRI-95-19, Final Report, Transportation Research Institute, University Of Michigan. Straus, S. (2005). New, improved, comprehensive and automated driver’s license test and vision screening system. Department of Transportation and Federal Highway Administration. May 2005. Törnros, J. (1998). Driving behavior in a real and a simulated road tunnel – A validation study. Accident Analysis and Prevention, 30(4), 497–503. Zeeb, E. (2010). Daimler’s new full-scale, high-dynamic driving simulator – A technical overview. In Proceedings of the driving simulator conference Europe 2010 (pp. 157–165), 9–10 September 2010 in Paris, France.