- Email: [email protected]
Validation of a new bybrid system for diagnosing driver state
riv The most recent findings from accident studies indicate that we only know about lowering of vigilance when it causes accidents and through its adverse effects on road safety statistics. In France, more than 30% of fatal road traffic accidents are blamed on driver failures caused by inattention, drowsiness or a health problem. The percentage is approximately the same for the rest of Europe, North America and dapan. This paper describes the vali-
dation of a new system for detecting, in real time, driver behavior pro blems during actual driving. Dtlriny the idst decade, alrr~~t one hundred and seventy-five such systems have been developed or granted international patents, but none have been fitted to vehicles. The systems developed so far by manufacturers have not proved to be reliabie or acceptable to drivers and have more or less been aban-
doned. This, then, is the context in which we have made this contribution. Its main aim is to validate a sys tern using hybrid technology consist ing of fifteen q so coi~plemer~tary sensors combined with on-line learning, diagnosis and decisionmaking. This new system is able to detect driver deficiencies in real time. It is able to identify the cause and evaluate the severity of the deterioration in the driver’s state, and if necessary warn the driver or a traf-
li DRIVER
fit control center. In extreme cases, it acts automatically to slow the vehicle down and bring it to a halt at the side of the road. Ten subjects drove an experimental vehicle for two hours. This vehicle was equipped with sensors that were able to record the following parameters. e Parameters relating to the physiology and behavior of the driver: EEG, EOG, eye movements and driver posture. 0 Mechanical parameters relating to the vehicle: steering angle, brake pedal pressure, movement of the accelerator pedal, engine speed, force exerted on the steering wheel, lateral distance, lateral acceleration, position of the vehicle on the carriageway, standard deviation of the steering angle, standard deviation of speed, white line detection using infrared telemetry, standard deviation of lateral distance, minimum lateral distance, time taken to cross lane lines, position of the nearside and offside of the vehicle, static and dynamic obstacle detection. l Footage was taken of the road and traffic upstream and downstream from the vehicle.
Generalized Gaussian Neural Networks (GGNN),independent component analysis and harmonic analysis were used to identify driver states.
The table Results OJ system ualidation presents the results obtained by the driver behavior and action diagnosis system on the vehicle. This table gives the following durations, for all the subjects together. * False negative detections (no detection): the system failed to detect hypovigilance when the physiological indicators showed that the
VIGILANCE
subject was hypovigilant, or even likely to cause an accident. This is the most dangerous index to be reduced. * False positive detections (false detection or false alarm): the system identii fied hypovigilance or a state likely to cause an accident in the driver, but this detection was not confirmed by on-line physiological monitoring. * When the system and the physiological indicators both confirm the absence of hypovigilance we talk of true positive detections. When they simultaneously confirm hypovigilance we refer-to true negatives. Whenever hypovigilance is detected an alarm or warning is automatically given to the driver and road users. The performance of the system was assessed in terms of its sensitivity and selectivity using the following expressions: Sensitivity =
true detections ~-.----z 93.6% true detections + false negative detections
true negative detections ___ _ 99,47 Selectivity = 0 true negative detections + false alarms Our main conclusion relates to estimated performance, which attains a level never before reached by any system. On an international level, this is the first time that a dedicated on-line system for observing driver vigilance has achieved levels of sensitivity and selectivity of 94% and 99% respectively. The average rate of false negative detections was 6.4%: of the 10.2 minute total duration of lowered vigilance that was observed among the subjects in two hours of driving, only 0,65 minute were undetected by the system. The 39 seconds of hypovigilance which the system failed to detect are of course important, and reducing this amount further is one of our aims for the future. The modifications that are necessary in order to improve the system’s performance were identified in the first stages of development , and just involve the implementation of new sensors and processors and the modification of algorithms. In comparison, the systems developed in Japan and the United States have still failed to achieve their desired levels of performance. Our success is the culmination of national and community research activities in this sphere since the third European framework.
In addition to this technological approach, other means of tackling the adverse road safety consequences of impaired driving behavior should be considered. For example: - increasing driver awareness of the importance of their state while driving; - development of a battery of selfdiagnosis tests; analysis of the effects of short interruptions in driving; - development of techniques to test driving aptitude; -.. investigation of the correlation between the organization of work and performance of the driving task; - regulation and the development of preventive measures; - widening the scope of tirese measures to take in the drivers of light vehicles, vans and lorries.
Y
De Waard has conducted surveys in European Union countries to assess the acceptability of a system to diagnose driver state. I-Ie has shown
that, on average, 72% of drivers think that such a system would be beneficial for road safety and that it would reduce the number of road traffic violations by 83%. In general, drivers felt that such a system would be acceptable.
Conclusion
accident in 69% and 31% of cases respectively. The total averaqe duration of hypovigilance for each subject was 10.2 minutes, representing 8.5% of the time driven in the tests. This result agrees with that obtained in a study of professional lorry drivers on a motorway.
The results given in this paper have validated the new hybrid system for diagnosing driver state. The following general conclusions can be drawn from the study.
Suitably modified harmonic and factor analysis techniques have proved very effective in processing the input data on-line. The learning of the system with a GGNN reached optimum performance and met the objectives we had set ourselves.
All subjects exhibited reductions in vigilance with levels defined as hypovigilance and likely to cause an
The system was validated by comparing physiological criteria,
learning and on-line diagnosis. Its high level of performance was achieved as a result of the quality of learning selected. The false alarm rate was only about 0.5%. The global absolute error varied between 3 and 12.7%. The average rate of false negative detections was 6.4%. The performance of the system in terms of sensitivity and selectivity were 93.6% and 99.4% respectively. Moreover, surveys conducted in the European Union have shown that this type of system is acceptable to road users. Further development is required to improve its performance still further and enable it to be+ personalized for individual drivers.
dation of a new system for detecting, in real time, driver behavior pro blems during actual driving. Dtlriny the idst decade, alrr~~t one hundred and seventy-five such systems have been developed or granted international patents, but none have been fitted to vehicles. The systems developed so far by manufacturers have not proved to be reliabie or acceptable to drivers and have more or less been aban-
doned. This, then, is the context in which we have made this contribution. Its main aim is to validate a sys tern using hybrid technology consist ing of fifteen q so coi~plemer~tary sensors combined with on-line learning, diagnosis and decisionmaking. This new system is able to detect driver deficiencies in real time. It is able to identify the cause and evaluate the severity of the deterioration in the driver’s state, and if necessary warn the driver or a traf-
li DRIVER
fit control center. In extreme cases, it acts automatically to slow the vehicle down and bring it to a halt at the side of the road. Ten subjects drove an experimental vehicle for two hours. This vehicle was equipped with sensors that were able to record the following parameters. e Parameters relating to the physiology and behavior of the driver: EEG, EOG, eye movements and driver posture. 0 Mechanical parameters relating to the vehicle: steering angle, brake pedal pressure, movement of the accelerator pedal, engine speed, force exerted on the steering wheel, lateral distance, lateral acceleration, position of the vehicle on the carriageway, standard deviation of the steering angle, standard deviation of speed, white line detection using infrared telemetry, standard deviation of lateral distance, minimum lateral distance, time taken to cross lane lines, position of the nearside and offside of the vehicle, static and dynamic obstacle detection. l Footage was taken of the road and traffic upstream and downstream from the vehicle.
Generalized Gaussian Neural Networks (GGNN),independent component analysis and harmonic analysis were used to identify driver states.
The table Results OJ system ualidation presents the results obtained by the driver behavior and action diagnosis system on the vehicle. This table gives the following durations, for all the subjects together. * False negative detections (no detection): the system failed to detect hypovigilance when the physiological indicators showed that the
VIGILANCE
subject was hypovigilant, or even likely to cause an accident. This is the most dangerous index to be reduced. * False positive detections (false detection or false alarm): the system identii fied hypovigilance or a state likely to cause an accident in the driver, but this detection was not confirmed by on-line physiological monitoring. * When the system and the physiological indicators both confirm the absence of hypovigilance we talk of true positive detections. When they simultaneously confirm hypovigilance we refer-to true negatives. Whenever hypovigilance is detected an alarm or warning is automatically given to the driver and road users. The performance of the system was assessed in terms of its sensitivity and selectivity using the following expressions: Sensitivity =
true detections ~-.----z 93.6% true detections + false negative detections
true negative detections ___ _ 99,47 Selectivity = 0 true negative detections + false alarms Our main conclusion relates to estimated performance, which attains a level never before reached by any system. On an international level, this is the first time that a dedicated on-line system for observing driver vigilance has achieved levels of sensitivity and selectivity of 94% and 99% respectively. The average rate of false negative detections was 6.4%: of the 10.2 minute total duration of lowered vigilance that was observed among the subjects in two hours of driving, only 0,65 minute were undetected by the system. The 39 seconds of hypovigilance which the system failed to detect are of course important, and reducing this amount further is one of our aims for the future. The modifications that are necessary in order to improve the system’s performance were identified in the first stages of development , and just involve the implementation of new sensors and processors and the modification of algorithms. In comparison, the systems developed in Japan and the United States have still failed to achieve their desired levels of performance. Our success is the culmination of national and community research activities in this sphere since the third European framework.
In addition to this technological approach, other means of tackling the adverse road safety consequences of impaired driving behavior should be considered. For example: - increasing driver awareness of the importance of their state while driving; - development of a battery of selfdiagnosis tests; analysis of the effects of short interruptions in driving; - development of techniques to test driving aptitude; -.. investigation of the correlation between the organization of work and performance of the driving task; - regulation and the development of preventive measures; - widening the scope of tirese measures to take in the drivers of light vehicles, vans and lorries.
Y
De Waard has conducted surveys in European Union countries to assess the acceptability of a system to diagnose driver state. I-Ie has shown
that, on average, 72% of drivers think that such a system would be beneficial for road safety and that it would reduce the number of road traffic violations by 83%. In general, drivers felt that such a system would be acceptable.
Conclusion
accident in 69% and 31% of cases respectively. The total averaqe duration of hypovigilance for each subject was 10.2 minutes, representing 8.5% of the time driven in the tests. This result agrees with that obtained in a study of professional lorry drivers on a motorway.
The results given in this paper have validated the new hybrid system for diagnosing driver state. The following general conclusions can be drawn from the study.
Suitably modified harmonic and factor analysis techniques have proved very effective in processing the input data on-line. The learning of the system with a GGNN reached optimum performance and met the objectives we had set ourselves.
All subjects exhibited reductions in vigilance with levels defined as hypovigilance and likely to cause an
The system was validated by comparing physiological criteria,
learning and on-line diagnosis. Its high level of performance was achieved as a result of the quality of learning selected. The false alarm rate was only about 0.5%. The global absolute error varied between 3 and 12.7%. The average rate of false negative detections was 6.4%. The performance of the system in terms of sensitivity and selectivity were 93.6% and 99.4% respectively. Moreover, surveys conducted in the European Union have shown that this type of system is acceptable to road users. Further development is required to improve its performance still further and enable it to be+ personalized for individual drivers.