The ANN-based computing of drowsy level

Available online at www.sciencedirect.com

Expert Systems with Applications Expert Systems with Applications 36 (2009) 2534–2542 www.elsevier.com/locate/eswa

The ANN-based computing of drowsy level Muhammed B. Kurt a, Necmettin Sezgin a,*, Mehmet Akin a, Gokhan Kirbas b, Muhittin Bayram a a

Department of Electrical and Electronics Engineering, University of Dicle, 21280 Diyarbakir, Turkey b Faculty of Medicine, University of Dicle, Diyarbakir, Turkey

Abstract We have developed a new method for automatic estimation of vigilance level by using electroencephalogram (EEG), electromyogram (EMG) and eye movement (EOG) signals recorded during transition from wakefulness to sleep. In the previous studies, EEG signals and EEG signals with EMG signals were used for estimating vigilance levels. In the present study, it was aimed to estimate vigilance levels by using EEG, EMG and EOG signals. The changes in EEG, EMG and EOG were diagnosed while transiting from wakefulness to sleep by using wavelet transform and developed artificial neural network (ANN). EEG signals were separated to its subbands using wavelet transform, LEOG (Left EOG), REOG (Right EOG) and chin EMG was used in ANN process for increasing the accuracy of the estimation rate by evaluating their tonic levels and also used in data preparation stage to verify and eliminate the movement artifacts. Then, training and testing data sets consist of the EEG subbands (delta, theta, alpha and beta); LEOG, REOG and EMG signals were applied to the ANN for training and testing the system which gives three situations for the vigilance level of the subject: Awake, drowsy, and sleep. The accuracy of estimation is about 97–98% while the accuracy of the previous study which used only EEG was 95–96% and the study which used EEG with EMG was 98–99%. The reason of decreasing the percentage of present study according to the last study is because of the increase of the input data. Ó 2008 Elsevier Ltd. All rights reserved. Keywords: EEG; EMG; EOG; Wavelet; Neural network; Drowsy; Sleep

1. Introduction One of the important applications of EEG processing is the study of the time course of alertness and vigilance of operators who perform monotonous but attentiondemanding tasks (air traffic controllers, lorry drivers, etc.). The objective is to avoid potential accidents generated by decreased vigilance using a real-time system which can continuously monitor vigilance, thereby, preventing accidents caused by attention deficit (Subasi, Kiymik, Akin, & Erogul, 2005). In this study, our aim was to develop a new method that can estimate the vigilance state of an arbitrary subject with *

Corresponding author. Fax: +90 4882135113. E-mail addresses: [email protected] (M.B. Kurt), necmettinsezgin@ gmail.com (N. Sezgin), [email protected] (M. Akin), gkirbas@dicle. edu.tr (G. Kirbas), [email protected] (M. Bayram). 0957-4174/$ - see front matter Ó 2008 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2008.01.085

a more accurate rate. In order to solve the same problem given in this paper, several neural networks had been suggested in different studies (Akin, Kurt, Sezgin, & Bayram, 2007; Ben Khalifa, Bedoui, Dogui, & Alexandre, 2004; Kiymik, Akin, & Subasi, 2004; Vuckovic, Radivojevic, Chen, & Popovic, 2002, etc.). However, in those studies EEG signals and EEG with EMG signals were used in different experimental conditions, and simultaneous usage of EEG, EMG and EOG signals was not given. This is the novelty that our paper brings. In practice, the EEG, EOG and EMG are simultaneously recorded on continuously moving chart paper, so that relationship among the three can be seen immediately. Nowadays, these measures are recorded in a computer media by means of a polysomnograph which is a biomedical instrument designed for the measurement of physiologic variables of sleep. A computerized analysis of the recordings aims to facilitate the time consuming and

M.B. Kurt et al. / Expert Systems with Applications 36 (2009) 2534–2542

difficult visual inspection and automatically extract characteristic features of the sleep. Among these three measures, the EEG signals do a pretty good job of state discrimination, and thus, it is of primary importance in polysomnography. EEG distinguishes between the states of vigilance, that is, wakefulness and sleep, and to some extent between the ‘levels’ of vigilance within a state. In previous studies, sleep stage classification was mostly based on the spectral analysis of EEG recordings. Principe, Gala, and Chang (1989) designed a finite automation that was capable of categorizing the sleep into seven different stages. It was shown that a limited number of electrodes and spectral analysis of characteristic bands could be used as a classifier (Doghramji, Merrill, & Sangal, 1997; Jung, Makeig, Stensmo, & Sejnowski, 1997). More recently, some studies (Halici, 1999) have concentrated on detecting the information on drowsiness available from a full EEG spectrum. Vuckovic et al. (2002) present a method for classifying alert versus drowsy states from 1-s long sequences of fullspectrum EEG recordings in an arbitrary subject. This method uses time series of interhemispheric and intrahemispheric cross-spectral densities of full-spectrum EEG recordings as the input to an artificial neural network (ANN) with two discrete outputs: drowsy and alert. Subasi et al. (2005) used 5-s long sequences of full-spectrum EEG recordings for classifying alert versus drowsy states in an arbitrary subject. This study used the wavelet-based neural network model trained with the Levenberg–Marquardt algorithm to discriminate the alertness level of the subject, and the classification results of this study were 93.3% alert, 96.6% drowsy, and 90% sleep. During the transition from wakefulness (or stage 0) to sleep stage 1, the EEG changes from a low amplitude high frequency pattern to a high amplitude low frequency pattern. This high amplitude low frequency pattern shows distinct dynamics across the stage 0–stage 1 sleep cycle (Gelir & Ardi, 2000). EMGs, muscles activities, getting lower tonus of EMG while transition from 0 to stage 1 (Gelir & Ardi, 2000). EOG waves are getting slower depending on the absence of REMs (Gelir & Ardi, 2000).When awake, humans and several other vertebrates show low voltage (10–30 lV), fast (16–25 Hz) EEG activity and high tonus in EMG (Gelir & Ardi, 2000; Richard, 2001). When relaxed, humans also show sinusoidal alpha activity of about 20–40 lV and 8–12 Hz. Passing from wakefulness to sleep stage 1 is characterized by progressively slower frequencies and higher voltage activities in the EEG, lower tonus in EMG and slower eye movements in EOG (Gelir & Ardi, 2000). 1.1. Stage 0 (wakefulness) This stage is just before the start of the sleep. EEG: In this passing stage from wakefulness to sleep stage 1, in the EEG signals, low voltage (about 10– 34 lV) pattern with fast (16–25 Hz) activity called ‘‘beta” activity decreases. With the eyes closed and the subjects

2535

relaxed, a sinusoidal 8–12 Hz pattern observed and called ‘‘alpha” activity is abundant. When a person becomes sleepy, the alpha and beta waves increasingly give place to other waves of low frequency known as the ‘‘theta” waves (from four to eight per second) (Hazarika, 1997). EMG: The EMG may be high or moderate, depending on the degree of muscle tension (Subasi et al., 2005). EOG: Rapid-eye-movements (REMs) may be abundant or scarce, depending on the amount of visual scanning (Horner, 2001). 1.2. Sleep stage 1 This stage tends to be short. EEG: Alpha activity decreases (<50% of the record), activation is scarce, and the EEG consists mostly of low voltage, mixed frequency activity, much of it at 3–7 Hz (Richard, 2001). EMG: The EMG is moderate to low (Subasi et al., 2005). EOG: REMs are absent, but slow rolling eye movements appear (Richard, 2001). 2. Materials and methods 2.1. Subjects In the present study, EEG, EOG and EMG signals were obtained from 10 subjects. The group consisted of four females and six males with ages ranging from 18 to 65 years and a mean age of 33.5 years. Subjects with normal intelligence and without mental disorders were included in the study after passing the neurological screening. All recordings were performed in accordance with the medical ethical standards. The subjects were not sleep deprived, they had no deviations from their usual circadian cycle, and they took no medicine or alcohol. Two experts with extended experience of interpreting the sleep data evaluated and rated the recordings used for this study. Each of them inspected the recordings, and then agreed as to which record sequences clearly indicated awake, drowsy or sleep states of the subject. 2.2. Obtaining the EEG, EOG and EMG data sets The EEG, EMG and EOG data used in this study were obtained from Dicle University, Sleep Laboratory, Faculty of Medicine in Diyarbakir. During the records silver surface electrodes were used. C3-A2 standard settlement is applied to the subject of experiment to record EEG data. Standard EOG derivations use E1 superolateral to left eye and E2 on right cheekbone. Surface submental EMG is recorded using two electrodes below the chin. Data were recorded on the papers and at the same time to a PC. Signals of interest are generated from the brain (i.e., cortex and deeper structures), the eye movements and the chin muscles for EEG, EOG and EMG, respectively. In Fig. 1,

2536

M.B. Kurt et al. / Expert Systems with Applications 36 (2009) 2534–2542

2.3. Wavelet transform

Fig. 1. EEG, EMG and EOG record points.

the electrode displacements for EEG, EOG and EMG signals are illustrated. The detection of specific patterns in the EEG, EOG and EMG signals is difficult due to the presence of noise and artifacts. Artifacts occur due to body movements, electrode displacements and other interfering signals. In this study, chin EMG end eye movements were not only used in ANN process for increasing the accuracy of the estimation rate by evaluating its tonic levels but also used in data preparation stage to verify and eliminate the movement artifacts. In this study, artifacts and noises were removed from the signal sets by filtering and visual inspection (see Figs. 2–4). The signals during the 7 h episodes were recorded. Digital signals are taken per 20 min for each block. Below it are shown the EEG, LEOG, REOG and EMG signals of awake, drowsy and sleep levels for training ANN.

The wavelet transform specifically permits to discriminate between non-stationary signals with different frequency features (Misiti, Misiti, Oppenheim, & Poggi, 1996). A signal is stationary if it does not change much over time. Fourier transform can be applied to the stationary signals. However, like EEG, a plenty of signals contain non-stationary or transitory characteristics, so that it is not a good idea to apply Fourier transform to such signals (Misiti et al., 1996). Among the methods used to analyse non-stationary signal like EEG, it was shown that wavelet analysis is the most efficient method in the frequency domain (Akin, 2002), and discrete wavelet transform (DWT) analysis of EEG recordings was proven to be a powerful tool for detecting transitions from an alert to a drowsy state (Conradt et al., 1999; Jung et al., 1997). The wavelet transform decomposes a signal into a set of basic functions called wavelets. These basic functions are obtained by dilations, contractions and shifts of a unique function called wavelet prototype (Misiti et al., 1996). For the input signal x(t), wavelet transform is defined as continuous wavelet transform (CWT) and discrete wavelet transform (DWT). CWT is defined as Z CWTða; bÞ ¼ X ðtÞWa;b ðtÞdt where  denotes complex conjugate, a 2 Rþ represents the scale parameter, b 2 Rþ represents the translation and Wa;b ðtÞ is obtained by scaling the prototype wavelet WðtÞ at time b and scaling by a   1 tb Wa;b ðtÞ ¼ pffiffiffi W a a Generally in wavelet applications, orthogonal dyadic functions are chosen as the mother wavelet. This transform is often discritisized in a and b on a dyadic grid with the time remaining continuous. The mother wavelet is defined as

Fig. 2. Training signal of awake level.

Fig. 3. Training signal of drowsy level.

Fig. 4. Training signal of sleep level.

Wj;k ðtÞ ¼ 2j=2 Wð2j t  kÞ where fWj;k ðtÞ; j; k; 2 Zg for L2(R). DWT analyses the signal at different frequency bands with different resolutions by decomposing the signal into a coarse approximation and detailed information. DWT employs two sets of functions, called scaling functions and wavelet functions, which are associated with lowpass and highpass filters, respectively. The decomposition of the signal into the different frequency bands is simply obtained by successive highpass and lowpass filtering of the time domain signal (Polikar, 2001). The original signal x(n) is first passed through a half band highpass filter g(n) and lowpass filter h(n). After the filtering, half of the samples can be eliminated according to the Nyquist’s rule, since the signal now has the highest frequency of p/2 radians instead of p. The signal can

M.B. Kurt et al. / Expert Systems with Applications 36 (2009) 2534–2542

therefore be subsampled by 2, simply by discarding every other sample (Misiti et al., 1996). This procedure constitutes one level of decomposition and can mathematically be expressed as follows: X x½n  g½2k  n Y high ½k ¼ X Y low ½k ¼ x½n  h½2k  n where Yhigh[k] and Ylow[k] are the outputs of the highpass and lowpass filters, respectively, after subsampling by 2 Fig. 5. The above procedure, which is also known as subband coding, can be repeated for further decomposition. At every level, the filtering and subsampling will result in half the number of samples and half the frequency band spends. 2.4. Artificial neural network Neural networks are used as a powerful means in engineering area after the development, especially, in computer technology. The fundamental characteristic of the neural networks is an adaptive, non-algorithmic and parallel-distributed memory (Halici, 1999). Neural networks are modeled by the inspiration from biological neural system and have more simple structure. Many neural networks were developed for resembling several characteristics of biological neural networks such as learning and reacting. Some characteristics, however, are realized with an engineering approach instead of neuropsychological one (Khahill & Duchene, 1999). The ANN raised high hopes concerning promising results with respect to analysis, classification, pattern recognition, and functional monitoring (Jung et al., 1997). This has been especially true for areas such as signal processing, including the analysis and interpretation of EEG recordings during cognitive load, rest, or sleep (Gevins & Smith, 1999; Peters, Pfurtscheller, & Flyvbjerg, 2001). 2.4.1. Neural network classifier An ANN consists of one input layer, one or more hidden layers, and one output layer. An input vector is applied to the input layer, where all of the inputs are distributed to each unit in the hidden layer. All of the units have weight

S 0-32Hz

cA 1 0-16Hz cA 2 0-8Hz cA 3 0-4Hz

cD 1 16-32Hz cD 2 8-16Hz

cD3 4-8Hz

Fig. 5. The subband coding algorithm.

2537

vectors, which are multiplied by these input vectors. Each unit sums these inputs and produces a value that is transformed by a non-linear activation function, for which we used the common asymmetric sigmoid function. The output of the final layer is then computed by multiplying the output vector from the hidden layer by the weights into the final layer. More summations and activations of these units then give the actual output of the network. The determination of appropriate number of hidden layers is one of the most critical tasks in neural network design. Unlike the input and output layers, one starts with no prior knowledge as to the number of hidden layers. A network with too few hidden nodes would be incapable of differentiating between complex patterns leading to only a linear estimate of the actual trend. In contrast, if the network has too many hidden nodes, it will follow the noise in the data due to over-parameterization, leading to poor generalization for untrained data. With increasing number of hidden layers, training becomes excessively time consuming. The most popular approach to finding the optimal number of hidden layers is by trial and error (Basheer & Hajmeer, 2000; Haykin, 1994). In this study, the neural network developed consists of one input layer, two hidden layers, and one output layer. Training algorithms are an integral part of ANN model development. An appropriate topology may still fail to give a better model, unless trained by a suitable training algorithm. A good training algorithm will shorten the training time while also achieving a better accuracy. Therefore, the training process is an important characteristic of ANNs, whereby representative examples of the knowledge are iteratively presented to the network, so that it can integrate this knowledge within its structure. There are a number of training algorithms used to train a MLPNN, and a frequently used one is called the backpropagation training algorithm (Basheer & Hajmeer, 2000; Guler & Ubeyli, 2004; Haykin, 1994). 2.4.2. Performance indicators of the neural network 2.4.2.1. Measuring error. Given a random set of initial weights, the outputs of the network will be very different from the desired classifications. As the network is trained, the weights of the system are continually adjusted to reduce the difference between the output of the system and the desired response. The difference is referred to as the error and can be measured in several ways. The most common measurements are sum square error (SSE) and mean square error (MSE). SSE is the average of the squares of the difference between each output and the desired output (Basheer & Hajmeer, 2000; Fausett, 1994; Haykin, 1994). 2.4.2.2. Cross-validation. Cross-validation is a highly recommended criterion for stopping the training of a network. During performance analysis of network, cross-validation can be used for determining the final training. In general, it is known that a network with enough weights will always learn the training set better as the number of iterations is increased. However, neural network researchers have

2.4.2.3. Classification and regression. Neural networks are used for both classification and regression. In classification, the aim is to assign the input patterns to one of several classes, usually represented by outputs restricted to lie in the range from 0 to 1, so that they represent the probability of class membership. While the classification is carried out, a specific pattern is assigned to a specific class according to the characteristic features selected for it. In regression, desired output and actual network output results can be shown on the same graph and performance of network can be evaluated in this way (Basheer & Hajmeer, 2000; Haykin, 1994). 3. Experimental study In this study 17,0667 EEG, EOG and EMG data are divided into 750 data for each epoch. In words, firstly we removed the artifacts by visual inspection from the EEG signals that were simultaneously recorded with the EMG and EOG signals in which artifacts are easier to identify than EEG signals. That is, when we encountered artifacts in the EMG and EOG then the corresponding part of EEG signals with artifacts was removed from the EEG signal sets. EEG signals were divided into four subfrequencies by using discrete wavelet transformation. The data in these four bands, filtered EMG spectrum, left and right EOG signals were applied to inputs of the developed neural network to produce one of awake, drowsy, and sleep outputs. Using wavelet transform each EEG was separated to its subbands. A wavelet of depth 3 (i.e., 3 levels) was designed. In this study first a lowpass prefilter having cutoff frequency of 32 Hz was used to seperate artifacts and any other noisy signals from the original EEG signals. Then EEG data were divided into low and high wavelet coefficients. And again these low and high wavelet coefficients are divided into their subhigh and sublow wavelet coefficients. From the original EEG signal we got delta (range of 0–4 Hz), theta (range of 4–8 Hz), alpha (range of 8– 12 Hz) and beta (range of 12–32 Hz). The result of wavelet transform of 750 EEG data and their frequency components are shown in Figs. 6 and 7, respectively. EEG signals for an person awakened contain beta and alpha waves as shown in Figs. 6 and 7, while for a person passing from awake to sleep, the beta wave give place to

1 0 -1 0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

delta

1 0 -1

theta

1 0 -1

alpha

1 0 -1 1

beta

found that this decrease in the training set error was not always coupled to better performance in the test. When the network is trained too much, the network memorizes the training patterns and does not generalize well. The training holds the key to an accurate solution, so the criterion to stop training must be very well described. The aim of the stop criterion is to maximize the network’s generalization (Basheer & Hajmeer, 2000). We performed the following cross-validation procedure for training the network as a way to control the overfitting of training data. We selected 30 of 120 data sets for training the ANN.

EEG signal

M.B. Kurt et al. / Expert Systems with Applications 36 (2009) 2534–2542

0 -1

time (sec)

Fig. 6. Simulated EEG waveform and its subbands due to wavelet transform for an awake person.

beta(db) alpha(db) theta(db) delta(db) EEG signal(db)

2538

30 20 10 0 30 20 10 0 30 20 10 0 30 20 10 0 30 20 10 0

0

5

10

15

20

25

30

0

5

10

15

20

25

30

0

5

10

15

20

25

30

0

5

10

15

20

25

30

0

5

10

15

20

25

30

Frequency (Hz)

Fig. 7. Frequency components of EEG signal and its subbands for awake person.

other waves which have low frequencies. It can be easily seen that in Figs. 8 and 9, the EEG signal contains less beta and alpha waves while the person is passing to sleep. When a person starts to sleep (it is the starting of stage 1) the EEG wave gets slower since beta and alpha waves give place to delta and theta waves. This occurence is shown in Figs. 10 and 11. 3.1. Performance analysis of ANN Neural networks employing back-propagation were trained with a training set and checked with a test set. The neural network found the input–output maps by analyzing the training set repeatedly. This is called the network-training phase. Most of the neural network design

1 0 -1

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

1 0 -1

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

theta

1 0 -1

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

alpha

1 0 -1

beta

1 0 -1

time (sec) Fig. 8. Simulated EEG waveform and its subbands due to wavelet transform for a drowsy person.

beta(db) alpha(db) theta(db) delta(db) EEG signal(db)

delta EEG signal

M.B. Kurt et al. / Expert Systems with Applications 36 (2009) 2534–2542

30 20 10 0 30 20 10 0 30 20 10 0 30 20 10 0 30 20 10 0

2539

0

5

10

15

20

25

30

0

5

10

15

20

25

30

0

5

10

15

20

25

30

0

5

10

15

20

25

30

0

5

10

15

20

25

30

Frequency(Hz)

beta(db) alpha(db) theta(db) delta(db) EEG signal(db)

Fig. 11. Frequency components of EEG signal and its subbands for sleeping person. 30 20 10 0 30 20 10 0 30 20 10 0 30 20 10 0 30 20 10 0

0

5

10

15

20

25

30

0

5

10

15

20

25

30

0

5

10

15

20

25

30

0

5

10

15

20

25

30

0

5

10

15

20

25

30

Frequency(Hz)

delta EEG signal

Fig. 9. Frequency components of EEG signal and its subbands for drowsy person.

1 0 -1

theta

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

1 0 -1 1 0 -1

alpha

effort was spent in the training phase. Training was normally slow because the network weights were updated based on the error information. It was necessary to monitor how well the network was learning. One of the simplest methods is to observe how the square differs between the network’s output and the desired response changes over training iterations. The curve of the SSE versus iteration is called the training curve. Training SSE curve of neural network is shown in Fig. 13. According to Fig. 13, as the network learns, the error converges to zero. A neural network is subject to what is known as the memorization of training data. Also it is known as the statistical phenomenon of over-fitting when it is over-trained. If a network over-fits or memorizes the training data, its generalized performance on other sample populations, such as, the test file or on records for which prospective predictions are to be made, is likely to be severely compromised. Therefore, the most important criterion is choosing the number of

1 0 -1

beta

1 0 -1

time(sec)

Fig. 10. Simulated EEG waveform and its subbands due to wavelet transform for a sleeping person.

Fig. 12. Multi-layer neural networks model.

2540

M.B. Kurt et al. / Expert Systems with Applications 36 (2009) 2534–2542

Fig. 13. SSE and learning rate versus iteration number.

iterations for training. Cross-validation is one of the most powerful methods to stop the training. In principle, the training curve decreases exponentially to zero or a small constant. Just how small in magnitude this constant depends on the situation and judgement must be used to find what error value is appropriate for the problem. When the error in the cross-validation has increased, the training should be stopped because the point of best generalization has been reached. Since SSE was converging to a small constant, approximately zero, training of the neural network was determined to be successful. Classification is based on the partition of every section of the space formed by EEG wavelet signals and determination of a partitioning function related with those sections; in the case of the ignorance of the mathematical forms of the partitioning functions, first a learning activity should be realized. Learning activity provides the determination of the real values of these functions with the aid of the examples from each class (training set). Since the classifiers are based on deciding by learning, they lead to more successful results with respect to the traditional methods (non-learning). The multi-layer neural networks model which was used in this study is given in Fig. 12. The training characteristics of neural network used in this study are as follows:Structure: Layer number: 4 (one input, two hidden and one output)Training parameters:

EMG recordings have been tested to develop network. The responses of the network to these test signals are shown in Table 1. Below figures are test signals of awake, drowsy and sleep level, respectively Figs. 14–16. As shown in Table 1; 1–10 epochs are awake, 11–20 epochs are drowsy, and 21–30 are sleep epochs. According to these results, accuracy of the network classification which was calculated as a mean value ± standard deviation (SD) is presented in Table 2. As seen in Table 1, the classification percentages of ANN on the test data are about 97–98%. This final table clearly shows that when EEG signals are used with EOG and EMG signals as the input parameters of ANN, accuracy of classification is evidently increased with respect to the same classification system using only EEG signals for sleep stage classification computed in Akin, Bayram, Erog˘lu, and Sezgin (2003) and Kiymik et al. (2004), however, decreased with respect to the system using EEG with EMG signals for classification computed in Akin et al. (2007). 3.2. Effect of EOG and EMG signals in the experiment The structure of the neural network, usage of EOG and EMG with EEG, experimental conditions (such as physiological state of subjects, recording points of EEG signals, and interfering signals) are all the important factors that affect the result. Obtaining such a good result depends on all of these, not only one. Therefore we did not claim that the EOG and EMG signals have the key role in the vigilance estimation. All we said was that in all vigilance estimation studies from the EEG, the main role belongs to EEG signals, which potentially carry all information

Fig. 14. Awake test signal.

Adaptive learning coefficient: 0.0004 Momentum coefficient: 0.95 Sum-squared error-SSE: 0.0004 Activation function: tangent sigmoid For the EEG, EOG and EMG signals also, a digital filter was used in order to remove the ECG artifacts and noise from the signals. There is not any instability or roughness in training process of the network. This shows the convenience of the parameters chosen to train the network. The trained network was tested with EEG, EOG and EMG signals. As a result it was seen that by observing the output vector produced by the network, several types of EEG, EOG and

Fig. 15. Drowsy test signal.

Fig. 16. Sleep test signal.

M.B. Kurt et al. / Expert Systems with Applications 36 (2009) 2534–2542 Table 1 30 test signals obtained by trained artifical neural network, used EEG, EMG and EOG Test signal

Awake (%)

Drowsy (%)

Sleep (%)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

98 99 98 95 99 97 99 96 97 97 05 07 04 10 11 06 04 07 13 17 11 07 05 13 16 05 07 05 09 04

17 15 08 21 06 12 03 13 08 10 98 97 99 97 96 98 99 97 95 96 13 08 07 25 20 08 12 10 13 08

08 12 10 14 04 08 02 10 08 09 05 09 02 10 12 04 05 10 07 09 99 98 99 95 97 98 99 99 97 98

Table 2 Classification performance of the ANN State of vigilance

Accuracy (%) (mean ± SD)

Awake Drowsy Sleep

97.5 ± 1.79 97.2 ± 1.23 97.9 ± 0.92

related with the vigilance level. However, as explained in the introduction, there may be some situations when the state of vigilance is not clear in the EEG because of artifacts and/or of uncertainty of transition pattern from one state to another. In these cases, experts use EOG and EMG signals for a more accurate decision. Therefore, we decided simultaneous usage of EEG, EMG and EOG signals so that a high accuracy vigilance level can be achieved with respect to the usage of EEG only. 4. Conclusion In this study, we have tried to estimate the level of vigilance of an arbitrary subject. For this purpose, the wavelet transform of the EEG signals was taken, and delta, theta, alpha, and beta subfrequencies were extracted. Moreover, chin EMG and EOG signals were used in ANN process for increasing the accuracy of the estimation rate by evalu-

2541

ating EMG tonic levels and eye movements. In this study, artifacts and noises were removed from the signal sets by filtering and visual inspection. A relationship between simultaneous EEG, EMG and EOG related to vigilance states could have been established. This relationship was used during decision (neural network) stage. If a similar relationship between EEG, EMG and EOG signals can be established for a given problem, then the method can be extended to that problem as well. But the usage of EMG and EOG in eliminating artifacts from simultaneous EEG signals can always be used in all EEG signal applications. During the transition from wakefulness to sleep, EEG changes from a low amplitude-high frequency pattern to a high amlitude-low frequency pattern. This high amplitude-low frequency pattern shows distinct dynamics across the wakefulness to stage 1 sleep cycle. When the frequency components of the subfrequencies were controlled, the beta and alpha activities decreased during the transition from wakefulness to sleep stage 1. It was also noticed that while the EMG signals have high EMG tonus in awake, as the person was passing from awake to sleep, the high EMG tonus gets lower, and in the sleep there occur low EMG tonus in EMG signals. Eye movements also get slower while transition from awake to sleep. The developed neural network has been trained and tested with specially prepared data sets. According to the response of the ANN to the test data sets, we calculated mean and SD of the results of the ANN output. It was observed that the estimation accuracy was about 97–98% while the accuracy of the previous study at which only EEG was used 95–96% (Akin et al., 2003; Kiymik et al., 2004) and the accuracy of the study in which EEG was used with EMG was 98–99% (Akin et al., 2007). According to these results, the level of drowsyness can be estimated better by using together EEG, EOG and EMG signals rather than using only EEG signals. In this study the accuracy percentage is decreased with respect to study (Akin et al., 2007). This happened because of increasing the input data. However in the present study, in reality, the accuracy is more robust and acceptable. As a result, in this study it can be said that the estimation of vigilance level of an arbitrary healthy subject is powered by using EOG with EMG and EEG. The application of this study will be helpful for neurologist to analyse the awake-sleep correlation. Moreover, according to these obtained results, this study is very important in the way of making a device that helps traffic controllers and vehicle derivers to determine his/her drowsiness level to prevent the traffic accidents resulting from drowsiness. References Akin, M. (2002). Comparison of wavelet transform and FFT methods in the analysis of EEG signals. Journal of Medical Systems, 26, 241–247. Akin, M., Bayram, M., Erog˘lu, O., & Sezgin, N. (2003). Determining of doze level analysing EEG signals by using wavelet transform and

2542

M.B. Kurt et al. / Expert Systems with Applications 36 (2009) 2534–2542

neural networks. International Journal of Computational Intelligence, 302–305. Akin, M., Kurt, M. B., Sezgin, N., & Bayram, M. (2007). Estimating vigilance level by using EEG and EMG signals. Neural computing & applications. Berlin: Springer-Verlag London Limited. Basheer, I. A., & Hajmeer, M. (2000). Artificial neural networks: Fundamentals, computing, design, and application. Journal of Microbiological Methods, 43, 3–31. Ben Khalifa, K., Bedoui, M.H., Dogui, M., & Alexandre, F. (2004). Alertness states classification by SOM and LVQ neural networks. ICIT’04, Istanbul. Conradt, R., Brandenburg, U., Penzel, T., Hasan, J., Varri, A., & Peter, J. H. (1999). Vigilance transitions in reaction time test: A method of describing the state of alertness more objectively. Clinical Neurophysiology, 110, 1499–1509. Doghramji, K., Merrill, M. M., & Sangal, S. B. (1997). A normative study of the maintenance of wakefulness test (MWT). Electroencephalography and Clinical Neurophysiology, 103, 554–562. Fausett, L. (1994). Fundamentals of neural networks architectures, algorithms, and applications. Englewood Cliffs: Prentice-Hall. Gelir, E., & Ardi, S. (2000). Insan Uyku Evrelerinin Standart Terminoloji. Yontem ve Skorlama El Kitabi. Gevins, A., & Smith, M. E. (1999). Detecting transient cognitive impairment with EEG pattern recognition methods. Aviation Space and Environmental Medicine, 70(10), 1018–1024. Guler, N. F., & Ubeyli, E. D. (2004). Wavelet-based neural network analysis of ophthalmic artery Doppler signals. Computers in Biology and Medicine, 34(7), 601–613. Halici, U. (1999). Lecture notes on neural networks, the dynamic of brain multidisipliner. Diyarbakir. Haykin, S. (1994). Neural networks: A comprehensive foundation. New York: Macmillan.

Hazarika, N. (1997). Classification of EEG signals using the wavelet transform. Signal Processing, 59(1), 61. Horner, R. (2001). PSL 472/1472 Sleep Physiology, Last updated 20/12/ 2001. Jung, T. P., Makeig, S., Stensmo, M., & Sejnowski, T. J. (1997). Estimating alertness from the EEG power spectrum. IEEE Transactions on Biomedical Engineering, 44, 60–69. Khahill, M., & Duchene, J. (1999). Detection and classification of multiple events in piecewise stationary signals. Journals of Signal Processing. pii: S0165-684(98)00236-9. Kiymik, M. K., Akin, M., & Subasi, A. (2004). Automatic recognition of alertness level by using wavelet transform and artificial neural network. Journal of Neuroscience Methods, 139(2), 231–240. Misiti, M., Misiti, Y., Oppenheim, G., & Poggi, J. M. (1996). Wavelet toolbox users guide copyright. Math Works Inc. Peters, B. O., Pfurtscheller, G., & Flyvbjerg, H. (2001). Automatic differentiation of multichannel EEG signals. IEEE Transactions on Biomedical Engineering, 48, 111–116. Polikar, R. (2001). Multiresolution analyses: The discrete wavelet transform. http://www.public.iastate.edu/rpolikar/ WAVELETS/ WTPART4.HTML. Principe, J. C., Gala, S. K., & Chang, T. G. (1989). Sleep staging automation based on the theory of evidence. IEEE Transactions on Biomedical Engineering, 36, 503–509. Subasi, A., Kiymik, M. K., Akin, M., & Erogul, O. (2005). Automatic recognition of vigilance state by using a wavelet-based artificial neural network. Neural Computing Applications, 14, 45–55. Vuckovic, A., Radivojevic, V., Chen, A. C. N., & Popovic, D. (2002). Automatic recognition of alertness and drowsiness from EEG by an artificial neural network. Medical Engineering & Physics, 24, 349–360.