- Email: [email protected]
Vehicle Interface for the Prolab2 Demonstrator
Copyright © IFAC Human-Oriented Design of Advanced RobotIcs Systems, Vienna, Austria, 1995
ASSISTANCE DRIVING AND DRIVER/VEHICLE INTERFACE FOR THE PROLAB2 DEMONSTRATOR N.LE FORT-PIAT·, and P.PLECZON and S.CHALARD •• ·Univer6ity of Technology de Compiegne, HEUDIASYC Laboratory, URA-CNRS 817, compiegne, FRANCE **INGENIA-DIALEXIS, Village d'wtrepriu,31324 Labege lnnopole ceder, FRANCE
Abstract. In the framework of the Prometheus program and the sequel of this program, we are particularly interested by the development of an intelligent co-pilot in order to assist the driver. The development of such driver assistant is based on studies on car driving especially in cognitive psychology in order to take into account the characteristics of the different drivers . This kind of support systems is designed to help the driver in situations where he has some difficulties and is generally embedded in a larger architecture which includes real-time machine vision capability. It use sensors information about driver (behavior and intentions) and about vehicle and environment states. In this paper, we particularly focus on the design of the driver assistant system, its decision capabilities and its Human-Machine Interface. In order to validate our work, a demonstrator has been embedded on a Peugeot 605 and tested on real driving contexts. Key Words. Decision support systems, Man/machine Interface, Distributed Artificial Intelligence
at the right time. So, for this driver assistance system, the goal is to realize a global and complete system allowing to schedule all the driving contexts (highway, road, crossroads) . Only a few projects take into consideration urban situations yet a lot of accidents arise in this context implying pedestrians.
1. INTRODUCTION Numerous projects are concerned by safety of road traffic and driving comfort.ln Europe, Projects like Prometheus and DRIVE had conduct to the development of driver support systems (Onken 1994, AbouKhaled and AI. 1995), recognition sJ'stems, specific electronic equipments like AICC (Autonomous Intelligent Cruise Control) (Schwertberger 1994), Route guidance (Carminat (Chassang 1994) , Melissa, ...), automatic braking , ... Most of the warning assistants developped have no competence for autonomous interventions in the vehicle. It 's the case for DAISY (Onken 1994), DWA (Driver Warning Assistant, (Enkelmann 1993)) and the Prolab2 demonstrator. These support systems are designed to help the driver in all traffic situations (lane-keeping, overtaking, crossing , ... ) except for DAISY dedicated to german autobahn .
2. THE FUNCTIONAL ARCHITECTURE
The decision capabilities cover aspects like supervision and diagnosis of situation , planning and monitoring of manoeuvres . The objective of the supervision is to analyze the current situation and to alarm or to warn the driver in order to anticipate critical situations. The prediction of the future situation allows to compute the feasibility of a manoeuvre or the risk associated to this manoeuvre. The Human-Machine Interface has in charge the interaction between the co-pilot and the driver (Pleczon and Chalard 1994). The driver can choose between three information modes : alarm mode is an emergency warning level ; advice mode is a decision support interface intended to help the driver to undertake somef'manoeuvres such as overtaking (qualitative aid) ; assistance mode is the last step between co-piloting and automated piloting. The system provides the driver with the best action to undertake regarding security parameters (quantitative aid : speed, timing ,.. ). According to the context , the specific situation and
In the prolab2 project , the objective is to increase the perception and the decision capabilities of the driver. The driver is an important part of the man-machine system due to his ability to adapt to a variety of situations, but in certain circumstances, when the human limitations in information perception are overridden or when the driver vigilance decreases, the driver assistance system shall support the driver in order to enhance driving safety and comfort . The more important is that the driver must get the relevant information 107
ception module. The driver information (Pleczon and Chalard 1994) has in charge to select and to present the adequate messages to the driver. According to the context, the specific situation and the level of assistance choosen, sound or/and visual messages are proposed to the driver.
the level of assistance chosen, sound or/and visual messages are proposed to the driver. The functional architecture of the driver assistance system is decomposed in five modules realizing the following functions: perception, data fusion, supervision, sensors planning and driver information. The perception module has in charge of sensor's data processing. The on-board perception system is composed of several cameras, used to provide information about the static and dynamic aspects of the local environment (structure of the roadway, current locat.ion of the vehicle , other moving obstacles, ... ). To each physical sensors corresponds vision processing functions. The perception module processes data at two levels: t.he lower level concerns the basic vision algorithms like 3D data acquisit.ion , edges detection, road segmentation: the higher level concerns sophisticated algorithms allowing from data given by the lower level, to identify and to recognize the kind of obstacles (vehicle. pedestrians, bicycles, .. )
3. SITUATION ASSESSMENT From data provided by the different sensors installed on the vehicle and merged by the dynamic data manager, a diagnosis of the situation is made. This diagnosis is based on knowledge concerning: • the static environment kind of contexts (highway, crossroad or road), number of lanes, road markings • the dynamic environment : presence of obstacles in zones of interest, relative position and speed of the obstacles, kind of obstacles (car, truck, bicycle and pedestrian)
Sensors
commands
• the intentions of the driver: provided by sensors installed on the vehicle like foot brake, left or right indicator, steering angle Sensors Planning
This diagnosis integrates a lateral controller in order to monitor the lateral behaviour of the vehicle by checking its position in the lane; and a longitudinal controller to monitor the longitudinal velocity of the vehicle with respect to the highway code (speed Iimi t) the current characteristics of the road (curvature) and the presence of obstacles. The situation assessment is based on the analysis of the situation and its probable evolution. Thus, it's possible to detect potential risks and in these cases to prevent the driver. In case of emergency, the supervisor identifies the nature of the danger, its location in space, the danger level and sends to the driver information module the danger level as well as possible manoeuvres (overtaking, ... ) or actions (brake, slow down).
requests
Figure 1: Architecture of the assistance syst.em The data fusion module (Rombaut and Meizel 1994) merges the information providing by the perception module and builds up a real-time map of the environment. The supervisor module (AbouKhaled and AI. 199.5) analyzes the current situation by using the information contained in the real-time map of the environment. Depending on the current situation , this module can ask for more information concerning obstacles or road marks, by sending requests to the sensors planner. The supervisor generates messages to the driver information module when the situation becomes dangerous or when the driver wants to engage a manoeuvre. From the requests sent by the supervisor, the sensors planner decides which physical sensors must be activated and specifies for this sensor characteristics like t.he functioning mode (continuous or punctual) and transmits t.he commands to the per-
4. HMI DESIGN METHODOLOGY
The design of this HMI is based on a cogmtlve analysis of drivers in the different scenarios that ProLab 2 copilot can deal with. The aim of this analysis is to know and understand the drivers' potential deficiencies in order to define the most
J08
be alternatively displayed corresponding to the three modes: (1) a white red-bordered circle for the alarm mode; (2) a white redbordered triangle for the advisory mode; (3) a blue white-bordered rectangle for the assistance mode.
adequate information. The first step is to identify the nature of the information on a conceptual point of view. CU~J1H1VC
:ul:llysis "f Jnvcr's acuviucs
AnalysIs Cif driver's pOlcntial dcficcnCIC!l
TechnICal con"tr.llllts
• Alarm mode : Only full screen icons are displayed, for the information must be easily and quickly perceived . In addition, at the very moment where the alarm is displayed, the background of the display flashes (black to red - 10 Hz) during about 500 ms. This coloured flash enhances the detectabilitv of the signal. .
H;lnlw;ltc
ConccPlual
conslr.lInL"
!\.1ISS Ul~.\
rrCUIC\lCln
HM I
• Advisory mode: The information displayed at that help level is more complex. In the advisory mode, the driver chooses to look at the display at a certain moment in order to get information before executing the intended manoeuvre. This is why the HMI displays a synthetic representation of the current environment as well as the status (possible or dangerous) of the intended manoeuvre. The goal of the synthetic representation of the environment is to help the driver focus his attention on the most immediate danger (according to the current situation or to the driver's current intention). Because of the small size of the display, we chose a representation based on distortion rather than an ordinary spatial representation of the obstacles. The advantages of distortion based representation have been highlighted by many different studies such as : the star representation used in process supervision (Coekin 1968), the Chernoff faces (Chernoff 19(3) and the Coutaz " status man" (Coutaz 1990). The representation we chose consists in a kiT\d of " vehicle safe zone". The safe zone is pictured as an ellipse , the shape of which can be distorted at the imaginary impact with an obstacle (see figure 4. The distortion varies with the level of danger. Furthermore, a colour coding reinforces the distortion information (green = no danger. yellow = potential danger. red = danger).
Figure 2: context hierarchy
The second step is, on the one hand , to define the way the information are provided (visual or auditive channel , graphics and sounds characteristics, choice of the display and sound devices) and on the other hand to predict possible misuse by the copilot (misinterpretations of the information, learning difficulties, using the copilot to drive faster, etc.). The HMI specification phase is constrained by technical constraints (perception limitation for instance) and hardware constraints (characteristics of available displays). The third step is the assessment of the HMI with drivers. This step may lead to software redesign or to new conceptual propositions. Prolab2 copilot human-machine interface (HMI) includes two major features. The first one is the opportunity for the driver to choose one of the three heip levels the system offers according to his driving habits. The second one is that the copilot HMI is a multimodal interface using both graphic information (LCD display) and sound information (q uadraphonic sound icons).
Graphic information
• Assistance mode : In the assistance mode. the HMI displays two types of information on the LCD : (1) an icon indicating the best action to undertake (straight ahead. overtaking, etc.) on the left ;art of the screen ; (2) an icon reinforcing the current context if necessary (dangerous bend for instance). Furthermore, an advised speed is shown on a specific device which is integrated into the tachymeter.
Graphics are displayed on an LCD display mounted into the dashboard. Information 15 suited for the different help levels. • Mode indication : A mode indication icon is continuously displayed on the upper left corner of the screen to indicate (1) that the copilot is operational and (2) the currently selected mode. Therefore. three icons can
109
5. DIAGNOSIS OF DRIVING SITUATIONS AND HMI
5.2 Lane-changing For a lane-changing , a pre-diagnosis is made before triggering the monitoring of the driver's actions. This pre-diagnosis consists in checking if:
5.1 Vehicle following An important part of the analysis concerns the verification of respect of safety distances between the vehicles. \Vhen vehicles are detected in the vicinity of the reference vehicle, safety distances are computed and compared with the current dist.ances between the vehicles. Considering a particular situation given in fig-
• the left line is not a continuous line • if the manoeuvre is possible : for that the time and the distance needed to execute the manoeuvre are computed.
/"
-/
//
2
//
I
4
./
../
/
,/
./.
...:./
./
6
5
a: correct safety distance (green chevrons)
Figure 3: Vehicle-following ure :3. In this situation. there is a vehicle in front of the reference vehicle R. To diagnose this situation. we compute safety distances between the two vehicles. The safety distance is function of t he relative speed between the two vehicles.
b: Moderately critical safety distance ( yello w chevrons)
where
• Ds is the safety distance , • Dr (v~) is the distance during covered during the reaction time at speed Vr , c . Cnucal safety distance (red chevron)
•
D("['~, ~/~ ,
F) is the distance covered while the reference vehicle reaches the speed of the previous vehicle , with F the braking force
Figure 4: Advisory mode The time and the distance are determined from initial lateral position and orientation of the vehicle and taking into account the lane width. In order to estimate this data, the behaviour of the vehicule is simulated using the dynamic model of th e vehicl e. Csing this data. we determine if the
• Dc is a constant distance imposed between
the t\\lO vehicles (if they are at the same speed), • D( ,/~) is the distance covered by the front vehicle during the reaction and the braking time. Four distances dl. d2 , d3 and d4 are comput.ed corresponding to four differents quantities of breaking_ The comparison with the actual distance measured by the perceptive system gives the corresponding message sent to the driver information module. The figure 4 presents the display when the safety distance is correct (4.a), when the safety distance is too short (see 4.b , and when a corrective action is required (see 4.c).
,
,
- - - ---------- - --------------------~----------------
..: - ( 1,
c + (t)
Figure ·5: Lane-changing manoeuvre manoeuvre is possible , by checking if the safety
110
distances , e- (t) and e+( t), with the vehicles placed on the future lane are respected at the beginning and at the end of the manoeuvre. If this test. is
1f fnmt.vchidc :he n
SLOWDQWr-,:
Figure 7: context hierarchy
Figure 6: Overtaking is possible tru e. t he message presented in the figure 6 is displayed
attention on a pack until conditions become false. The Situation Analysis task includes 210 production rules to realize the supervision of the highway and crossroads contexts. The Danger controller task includes 80 production rules. Communications between the differents tasks are organised around a share memory zone and mailbox facilities. The different tasks depending of the realization, use different kinds of semaphors to manage their conflicts concerning the access to indivisible resources.
6. SOFTWARE AND HARDWARE ARCHITECTURE
The purpose of the supervisor is to analyze the current situation. Its implementation involves the management. of the various knowledge sources (vehicl e. infrast.ruct.ure. obst.acles , et.c.) and the handling of a huge quantity of perceptual information. These data are often uncertain and incomplete because of sensors limitations. So we have to cope \\'ith the following problems:
The hardware architecture (fig 8) is based on several MVME167 cards (supervisor. data fusion), a transvision system (based on transputers T800) , and a radstone card (morphologic processor). and Three PC486. Communications are realized by VME bus, by serial links between the MVME cards and the PC, by a VSB bus to control the acquisition Process. The results of the situation assessment software are sent to the HMI software that we have designed. This software stands on a 486 PC and receives the data via a Vehicle Area l\'etwork (VAN). The HMI software uses the situation assessment data to select the best information to be sent to the driver according to the current context.
• Representation of the vanous knowledge sources. • Real time diagnosis of the situation according to th e current information. • Defini t ion of the warning strategies. To implement the software architecture, a realt.ime multitasking operating system is used. In (his environment , a task is associated to each function of the driver assistance system. For the situation analysis and the danger controller. the expert system shell Super developped in our laboratory. is used. This shell is integrated in the multitasking environment. like t.hat expert. systems associated to th e previous funct.ions are considered like the others tasks.
7. CONCLUSION The ProLab2 demonstrator has been realized by nin e French laboratories (composing the french PRO-ART group) . in collaboration \'iith PSA and Renault corporations. Real experiments of the demonstrator were realized and had allowed to optimize the global behavior of the copilot. The behavior of the driver assistance system tested in real contexts has been in idequation with t he functional specifications and the results were very encouraging to the satifactions of the automobile companies. The HMI software integrated in ProLab2 is fully operational and can provide the driver with three different help levels in many re-
.-\ multi-expert architecture has been developed (0 take into account. on the one hand , knowledge about the environment of the vehicle and its behavlOr and .on the other hand , knowledge about the dri ver and its needs. The know ledge base are organized in order to decrease the inference time by dividing the base into several packs of rules. each pack corresponding to a given context (see figure I). Thus t.he expert module may focus its
III
- - - ' - - ------- - ----- ----- ----- -' - - ---- - - --- ~ - .
Figure 8: The Pro-Lab II Hardware Architecture al istic conditions. Both graphics and sounds are used to inform the driver. Moreover , sound information given by the way of quadraphonic sound icons seem to be quite adequate to inform the driver quickly about the danger type and its location. The demonstrator ProLab2 (embedded in a Peugeot 605) had been presented to the Board Members Meeting (BMM'94) in Paris , 18-21 october 1994, to the PREDIT meeting in Paris, 7-9 February 1995 , and to the Final Presentation of PRO- CHIP and PRO-ART in Toulouse, 9 June 1995 . The near future works concern improvments of algorithms to suppress false alarms and to reduce system response-time . For long-term research Man-Machine Cooperation, data fusion aspects for intelligent vehicles and situation analysis taking account incertainties and data validity must. be performed in order to give back more efficient the help prov iding by the assistant sytem (Rombaut. and C herfaoui 1995).
Coekin , J.A (1968). A versatile presentation of parameters for rapid recognition of total state. In : IEEE International Symposium on ManMachine Systems. Coutaz, J. (1990). In : Interfaces Homm esOrdinateur: Conception et Realisation. Enkelmann , w . (1993). Reali zation of a driver 's warning assistant for intersections. In : Proceedi ngs of the IEEE Intelligent Vehicles Symposium. Tokyo Japan . pp . 84-89. Onken , R . (1994). Daisy, an adaptative, knowledge based driver monitoring and warning system. In: Proceedings of the IEEE Intelligent Vehicles Symposium. Paris, France. pp. 544-549. Pleczon, P. and S. Chalard (1994). The human m achine interface of prolab2 co-pilot. In : Proceedings of the IEEE Intelligent Vehicles Symposzum. Paris , France. Rombaut , M . and D . Meizel (1994). Dynamic data temporal multisensor fusion in the prometheus prolab2 demonstrator. In : IEEE International Conference on Robotics and A utomation. San Diego , California.
REFERENCES AbouKhaled, O. and AI. (1995). A real time supervision of a vehicle in a dynamic environment . In: Proceedi ngs of th e IFAC Intelligent A utonomous Vehicles. Espoo, Finlande.
Rombaut , M. and V . Cherfaoui (1995). Human machine cooperation for vehicle driving. In: Human Orien ted Design of Advanced Robotics Systems DARS'95 . .Vienna Austria.
Chassang. R . (1994). Carminat : Going towards an operational service in 1996. In : Proceedings of th e First \lForld Congress on applications of Transport Telematics and Int elligent llehicle-Highway systems. Paris, France. pp. 2456-246l.
Schwertberger , w. (1994). Autonomous intelligent cruise control in commercial vehicles. In: Proceedings of th e First World Congress on applications of Transport Telematics and Intelligent Vehicle-Highway systems. Paris , France.
C hernoff, H. (1973). Using faces to represent points in k-dimension space graphically. In : Journal Amer.Stat.Association. Vol. 68. pp . 361-:368.
112
ASSISTANCE DRIVING AND DRIVER/VEHICLE INTERFACE FOR THE PROLAB2 DEMONSTRATOR N.LE FORT-PIAT·, and P.PLECZON and S.CHALARD •• ·Univer6ity of Technology de Compiegne, HEUDIASYC Laboratory, URA-CNRS 817, compiegne, FRANCE **INGENIA-DIALEXIS, Village d'wtrepriu,31324 Labege lnnopole ceder, FRANCE
Abstract. In the framework of the Prometheus program and the sequel of this program, we are particularly interested by the development of an intelligent co-pilot in order to assist the driver. The development of such driver assistant is based on studies on car driving especially in cognitive psychology in order to take into account the characteristics of the different drivers . This kind of support systems is designed to help the driver in situations where he has some difficulties and is generally embedded in a larger architecture which includes real-time machine vision capability. It use sensors information about driver (behavior and intentions) and about vehicle and environment states. In this paper, we particularly focus on the design of the driver assistant system, its decision capabilities and its Human-Machine Interface. In order to validate our work, a demonstrator has been embedded on a Peugeot 605 and tested on real driving contexts. Key Words. Decision support systems, Man/machine Interface, Distributed Artificial Intelligence
at the right time. So, for this driver assistance system, the goal is to realize a global and complete system allowing to schedule all the driving contexts (highway, road, crossroads) . Only a few projects take into consideration urban situations yet a lot of accidents arise in this context implying pedestrians.
1. INTRODUCTION Numerous projects are concerned by safety of road traffic and driving comfort.ln Europe, Projects like Prometheus and DRIVE had conduct to the development of driver support systems (Onken 1994, AbouKhaled and AI. 1995), recognition sJ'stems, specific electronic equipments like AICC (Autonomous Intelligent Cruise Control) (Schwertberger 1994), Route guidance (Carminat (Chassang 1994) , Melissa, ...), automatic braking , ... Most of the warning assistants developped have no competence for autonomous interventions in the vehicle. It 's the case for DAISY (Onken 1994), DWA (Driver Warning Assistant, (Enkelmann 1993)) and the Prolab2 demonstrator. These support systems are designed to help the driver in all traffic situations (lane-keeping, overtaking, crossing , ... ) except for DAISY dedicated to german autobahn .
2. THE FUNCTIONAL ARCHITECTURE
The decision capabilities cover aspects like supervision and diagnosis of situation , planning and monitoring of manoeuvres . The objective of the supervision is to analyze the current situation and to alarm or to warn the driver in order to anticipate critical situations. The prediction of the future situation allows to compute the feasibility of a manoeuvre or the risk associated to this manoeuvre. The Human-Machine Interface has in charge the interaction between the co-pilot and the driver (Pleczon and Chalard 1994). The driver can choose between three information modes : alarm mode is an emergency warning level ; advice mode is a decision support interface intended to help the driver to undertake somef'manoeuvres such as overtaking (qualitative aid) ; assistance mode is the last step between co-piloting and automated piloting. The system provides the driver with the best action to undertake regarding security parameters (quantitative aid : speed, timing ,.. ). According to the context , the specific situation and
In the prolab2 project , the objective is to increase the perception and the decision capabilities of the driver. The driver is an important part of the man-machine system due to his ability to adapt to a variety of situations, but in certain circumstances, when the human limitations in information perception are overridden or when the driver vigilance decreases, the driver assistance system shall support the driver in order to enhance driving safety and comfort . The more important is that the driver must get the relevant information 107
ception module. The driver information (Pleczon and Chalard 1994) has in charge to select and to present the adequate messages to the driver. According to the context, the specific situation and the level of assistance choosen, sound or/and visual messages are proposed to the driver.
the level of assistance chosen, sound or/and visual messages are proposed to the driver. The functional architecture of the driver assistance system is decomposed in five modules realizing the following functions: perception, data fusion, supervision, sensors planning and driver information. The perception module has in charge of sensor's data processing. The on-board perception system is composed of several cameras, used to provide information about the static and dynamic aspects of the local environment (structure of the roadway, current locat.ion of the vehicle , other moving obstacles, ... ). To each physical sensors corresponds vision processing functions. The perception module processes data at two levels: t.he lower level concerns the basic vision algorithms like 3D data acquisit.ion , edges detection, road segmentation: the higher level concerns sophisticated algorithms allowing from data given by the lower level, to identify and to recognize the kind of obstacles (vehicle. pedestrians, bicycles, .. )
3. SITUATION ASSESSMENT From data provided by the different sensors installed on the vehicle and merged by the dynamic data manager, a diagnosis of the situation is made. This diagnosis is based on knowledge concerning: • the static environment kind of contexts (highway, crossroad or road), number of lanes, road markings • the dynamic environment : presence of obstacles in zones of interest, relative position and speed of the obstacles, kind of obstacles (car, truck, bicycle and pedestrian)
Sensors
commands
• the intentions of the driver: provided by sensors installed on the vehicle like foot brake, left or right indicator, steering angle Sensors Planning
This diagnosis integrates a lateral controller in order to monitor the lateral behaviour of the vehicle by checking its position in the lane; and a longitudinal controller to monitor the longitudinal velocity of the vehicle with respect to the highway code (speed Iimi t) the current characteristics of the road (curvature) and the presence of obstacles. The situation assessment is based on the analysis of the situation and its probable evolution. Thus, it's possible to detect potential risks and in these cases to prevent the driver. In case of emergency, the supervisor identifies the nature of the danger, its location in space, the danger level and sends to the driver information module the danger level as well as possible manoeuvres (overtaking, ... ) or actions (brake, slow down).
requests
Figure 1: Architecture of the assistance syst.em The data fusion module (Rombaut and Meizel 1994) merges the information providing by the perception module and builds up a real-time map of the environment. The supervisor module (AbouKhaled and AI. 199.5) analyzes the current situation by using the information contained in the real-time map of the environment. Depending on the current situation , this module can ask for more information concerning obstacles or road marks, by sending requests to the sensors planner. The supervisor generates messages to the driver information module when the situation becomes dangerous or when the driver wants to engage a manoeuvre. From the requests sent by the supervisor, the sensors planner decides which physical sensors must be activated and specifies for this sensor characteristics like t.he functioning mode (continuous or punctual) and transmits t.he commands to the per-
4. HMI DESIGN METHODOLOGY
The design of this HMI is based on a cogmtlve analysis of drivers in the different scenarios that ProLab 2 copilot can deal with. The aim of this analysis is to know and understand the drivers' potential deficiencies in order to define the most
J08
be alternatively displayed corresponding to the three modes: (1) a white red-bordered circle for the alarm mode; (2) a white redbordered triangle for the advisory mode; (3) a blue white-bordered rectangle for the assistance mode.
adequate information. The first step is to identify the nature of the information on a conceptual point of view. CU~J1H1VC
:ul:llysis "f Jnvcr's acuviucs
AnalysIs Cif driver's pOlcntial dcficcnCIC!l
TechnICal con"tr.llllts
• Alarm mode : Only full screen icons are displayed, for the information must be easily and quickly perceived . In addition, at the very moment where the alarm is displayed, the background of the display flashes (black to red - 10 Hz) during about 500 ms. This coloured flash enhances the detectabilitv of the signal. .
H;lnlw;ltc
ConccPlual
conslr.lInL"
!\.1ISS Ul~.\
rrCUIC\lCln
HM I
• Advisory mode: The information displayed at that help level is more complex. In the advisory mode, the driver chooses to look at the display at a certain moment in order to get information before executing the intended manoeuvre. This is why the HMI displays a synthetic representation of the current environment as well as the status (possible or dangerous) of the intended manoeuvre. The goal of the synthetic representation of the environment is to help the driver focus his attention on the most immediate danger (according to the current situation or to the driver's current intention). Because of the small size of the display, we chose a representation based on distortion rather than an ordinary spatial representation of the obstacles. The advantages of distortion based representation have been highlighted by many different studies such as : the star representation used in process supervision (Coekin 1968), the Chernoff faces (Chernoff 19(3) and the Coutaz " status man" (Coutaz 1990). The representation we chose consists in a kiT\d of " vehicle safe zone". The safe zone is pictured as an ellipse , the shape of which can be distorted at the imaginary impact with an obstacle (see figure 4. The distortion varies with the level of danger. Furthermore, a colour coding reinforces the distortion information (green = no danger. yellow = potential danger. red = danger).
Figure 2: context hierarchy
The second step is, on the one hand , to define the way the information are provided (visual or auditive channel , graphics and sounds characteristics, choice of the display and sound devices) and on the other hand to predict possible misuse by the copilot (misinterpretations of the information, learning difficulties, using the copilot to drive faster, etc.). The HMI specification phase is constrained by technical constraints (perception limitation for instance) and hardware constraints (characteristics of available displays). The third step is the assessment of the HMI with drivers. This step may lead to software redesign or to new conceptual propositions. Prolab2 copilot human-machine interface (HMI) includes two major features. The first one is the opportunity for the driver to choose one of the three heip levels the system offers according to his driving habits. The second one is that the copilot HMI is a multimodal interface using both graphic information (LCD display) and sound information (q uadraphonic sound icons).
Graphic information
• Assistance mode : In the assistance mode. the HMI displays two types of information on the LCD : (1) an icon indicating the best action to undertake (straight ahead. overtaking, etc.) on the left ;art of the screen ; (2) an icon reinforcing the current context if necessary (dangerous bend for instance). Furthermore, an advised speed is shown on a specific device which is integrated into the tachymeter.
Graphics are displayed on an LCD display mounted into the dashboard. Information 15 suited for the different help levels. • Mode indication : A mode indication icon is continuously displayed on the upper left corner of the screen to indicate (1) that the copilot is operational and (2) the currently selected mode. Therefore. three icons can
109
5. DIAGNOSIS OF DRIVING SITUATIONS AND HMI
5.2 Lane-changing For a lane-changing , a pre-diagnosis is made before triggering the monitoring of the driver's actions. This pre-diagnosis consists in checking if:
5.1 Vehicle following An important part of the analysis concerns the verification of respect of safety distances between the vehicles. \Vhen vehicles are detected in the vicinity of the reference vehicle, safety distances are computed and compared with the current dist.ances between the vehicles. Considering a particular situation given in fig-
• the left line is not a continuous line • if the manoeuvre is possible : for that the time and the distance needed to execute the manoeuvre are computed.
/"
-/
//
2
//
I
4
./
../
/
,/
./.
...:./
./
6
5
a: correct safety distance (green chevrons)
Figure 3: Vehicle-following ure :3. In this situation. there is a vehicle in front of the reference vehicle R. To diagnose this situation. we compute safety distances between the two vehicles. The safety distance is function of t he relative speed between the two vehicles.
b: Moderately critical safety distance ( yello w chevrons)
where
• Ds is the safety distance , • Dr (v~) is the distance during covered during the reaction time at speed Vr , c . Cnucal safety distance (red chevron)
•
D("['~, ~/~ ,
F) is the distance covered while the reference vehicle reaches the speed of the previous vehicle , with F the braking force
Figure 4: Advisory mode The time and the distance are determined from initial lateral position and orientation of the vehicle and taking into account the lane width. In order to estimate this data, the behaviour of the vehicule is simulated using the dynamic model of th e vehicl e. Csing this data. we determine if the
• Dc is a constant distance imposed between
the t\\lO vehicles (if they are at the same speed), • D( ,/~) is the distance covered by the front vehicle during the reaction and the braking time. Four distances dl. d2 , d3 and d4 are comput.ed corresponding to four differents quantities of breaking_ The comparison with the actual distance measured by the perceptive system gives the corresponding message sent to the driver information module. The figure 4 presents the display when the safety distance is correct (4.a), when the safety distance is too short (see 4.b , and when a corrective action is required (see 4.c).
,
,
- - - ---------- - --------------------~----------------
..: - ( 1,
c + (t)
Figure ·5: Lane-changing manoeuvre manoeuvre is possible , by checking if the safety
110
distances , e- (t) and e+( t), with the vehicles placed on the future lane are respected at the beginning and at the end of the manoeuvre. If this test. is
1f fnmt.vchidc :he n
SLOWDQWr-,:
Figure 7: context hierarchy
Figure 6: Overtaking is possible tru e. t he message presented in the figure 6 is displayed
attention on a pack until conditions become false. The Situation Analysis task includes 210 production rules to realize the supervision of the highway and crossroads contexts. The Danger controller task includes 80 production rules. Communications between the differents tasks are organised around a share memory zone and mailbox facilities. The different tasks depending of the realization, use different kinds of semaphors to manage their conflicts concerning the access to indivisible resources.
6. SOFTWARE AND HARDWARE ARCHITECTURE
The purpose of the supervisor is to analyze the current situation. Its implementation involves the management. of the various knowledge sources (vehicl e. infrast.ruct.ure. obst.acles , et.c.) and the handling of a huge quantity of perceptual information. These data are often uncertain and incomplete because of sensors limitations. So we have to cope \\'ith the following problems:
The hardware architecture (fig 8) is based on several MVME167 cards (supervisor. data fusion), a transvision system (based on transputers T800) , and a radstone card (morphologic processor). and Three PC486. Communications are realized by VME bus, by serial links between the MVME cards and the PC, by a VSB bus to control the acquisition Process. The results of the situation assessment software are sent to the HMI software that we have designed. This software stands on a 486 PC and receives the data via a Vehicle Area l\'etwork (VAN). The HMI software uses the situation assessment data to select the best information to be sent to the driver according to the current context.
• Representation of the vanous knowledge sources. • Real time diagnosis of the situation according to th e current information. • Defini t ion of the warning strategies. To implement the software architecture, a realt.ime multitasking operating system is used. In (his environment , a task is associated to each function of the driver assistance system. For the situation analysis and the danger controller. the expert system shell Super developped in our laboratory. is used. This shell is integrated in the multitasking environment. like t.hat expert. systems associated to th e previous funct.ions are considered like the others tasks.
7. CONCLUSION The ProLab2 demonstrator has been realized by nin e French laboratories (composing the french PRO-ART group) . in collaboration \'iith PSA and Renault corporations. Real experiments of the demonstrator were realized and had allowed to optimize the global behavior of the copilot. The behavior of the driver assistance system tested in real contexts has been in idequation with t he functional specifications and the results were very encouraging to the satifactions of the automobile companies. The HMI software integrated in ProLab2 is fully operational and can provide the driver with three different help levels in many re-
.-\ multi-expert architecture has been developed (0 take into account. on the one hand , knowledge about the environment of the vehicle and its behavlOr and .on the other hand , knowledge about the dri ver and its needs. The know ledge base are organized in order to decrease the inference time by dividing the base into several packs of rules. each pack corresponding to a given context (see figure I). Thus t.he expert module may focus its
III
- - - ' - - ------- - ----- ----- ----- -' - - ---- - - --- ~ - .
Figure 8: The Pro-Lab II Hardware Architecture al istic conditions. Both graphics and sounds are used to inform the driver. Moreover , sound information given by the way of quadraphonic sound icons seem to be quite adequate to inform the driver quickly about the danger type and its location. The demonstrator ProLab2 (embedded in a Peugeot 605) had been presented to the Board Members Meeting (BMM'94) in Paris , 18-21 october 1994, to the PREDIT meeting in Paris, 7-9 February 1995 , and to the Final Presentation of PRO- CHIP and PRO-ART in Toulouse, 9 June 1995 . The near future works concern improvments of algorithms to suppress false alarms and to reduce system response-time . For long-term research Man-Machine Cooperation, data fusion aspects for intelligent vehicles and situation analysis taking account incertainties and data validity must. be performed in order to give back more efficient the help prov iding by the assistant sytem (Rombaut. and C herfaoui 1995).
Coekin , J.A (1968). A versatile presentation of parameters for rapid recognition of total state. In : IEEE International Symposium on ManMachine Systems. Coutaz, J. (1990). In : Interfaces Homm esOrdinateur: Conception et Realisation. Enkelmann , w . (1993). Reali zation of a driver 's warning assistant for intersections. In : Proceedi ngs of the IEEE Intelligent Vehicles Symposium. Tokyo Japan . pp . 84-89. Onken , R . (1994). Daisy, an adaptative, knowledge based driver monitoring and warning system. In: Proceedings of the IEEE Intelligent Vehicles Symposium. Paris, France. pp. 544-549. Pleczon, P. and S. Chalard (1994). The human m achine interface of prolab2 co-pilot. In : Proceedings of the IEEE Intelligent Vehicles Symposzum. Paris , France. Rombaut , M . and D . Meizel (1994). Dynamic data temporal multisensor fusion in the prometheus prolab2 demonstrator. In : IEEE International Conference on Robotics and A utomation. San Diego , California.
REFERENCES AbouKhaled, O. and AI. (1995). A real time supervision of a vehicle in a dynamic environment . In: Proceedi ngs of th e IFAC Intelligent A utonomous Vehicles. Espoo, Finlande.
Rombaut , M. and V . Cherfaoui (1995). Human machine cooperation for vehicle driving. In: Human Orien ted Design of Advanced Robotics Systems DARS'95 . .Vienna Austria.
Chassang. R . (1994). Carminat : Going towards an operational service in 1996. In : Proceedings of th e First \lForld Congress on applications of Transport Telematics and Int elligent llehicle-Highway systems. Paris, France. pp. 2456-246l.
Schwertberger , w. (1994). Autonomous intelligent cruise control in commercial vehicles. In: Proceedings of th e First World Congress on applications of Transport Telematics and Intelligent Vehicle-Highway systems. Paris , France.
C hernoff, H. (1973). Using faces to represent points in k-dimension space graphically. In : Journal Amer.Stat.Association. Vol. 68. pp . 361-:368.
112