- Email: [email protected]
Time-locked and phase-locked features of P300 event-related potentials (ERPs) for brain–computer interface speller
Biomedical Signal Processing and Control 5 (2010) 243–251
Contents lists available at ScienceDirect
Biomedical Signal Processing and Control journal homepage: www.elsevier.com/locate/bspc
Time-locked and phase-locked features of P300 event-related potentials (ERPs) for brain–computer interface speller Dong Ming a,b , Xingwei An a , Youyuan Xi a , Yong Hu b,∗ , Baikun Wan a , Hongzhi Qi a , Longlong Cheng a , Zhaojun Xue a a b
Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, PR China Department of Orthopaedics and Traumatology, Li Ka Shing Faculty of Medicine, University of Hong Kong, Hong Kong, China
a r t i c l e
i n f o
Article history: Received 30 August 2009 Received in revised form 26 July 2010 Accepted 2 August 2010 Available online 26 August 2010 Keywords: Brain–computer interface Evoked response potentials Phase reset Event-related spectral perturbation Inter-trial coherence
a b s t r a c t The brain–computer interface P300 speller is aimed to help those patients unable to activate muscles to spell words by utilizing their brain activity. However, a problem associated with the use of this brain–computer interface paradigm is the generation mechanics of P300 related to responses to visual stimuli. Herein, we investigated the event-related potential (ERP) response for the P300-based brain–computer interface speller. A signal preprocessing method integrated coherent average, principal component analysis (PCA) and independent component analysis (ICA) to reduce the dimensions and noise in the raw data. The time–frequency analysis was based on wavelet and two characteristic parameters of event-related spectral perturbation (ERSP) and inter-trial coherence (ITC) were computed to indicate the evoked response (time-locked) and phase reset (phase-locked) activity, respectively. Results demonstrated that the proposed method was valid for the time-locked and phase-locked feature extraction and both the evoked response and phase reset contributed to the genesis of the P300 signal. These electrophysiological responses characteristics of ERPs would be used for BCI P300 speller design and its signal processing strategies. © 2010 Elsevier Ltd. All rights reserved.
1. Introduction A brain–computer interface (BCI) provides alternative communication and control channels to convey messages and commands from the brain to the external world [1], especially for those patients with severe neurological or muscular diseases. At present, electroencephalogram (EEG) is the major brainwave signal used by non-invasive BCIs. One strategy of EEG-based BCI involves the use of event related potential (ERP) that exploits the electrophysiological responses to a certain event. The most robust feature of the ERP is a positive displacement occurring around 300 ms after stimulus, termed the P300 or P3 [2]. The P300 was first utilized in BCI as a speller [3]. A major technical problem in the P300-based BCI speller is the robustness of the classification of the response from background noise to improve the BCI system performance. Furthermore, it remains controversial whether ERPs are generated by evoked response or by phase reset with the outward stimulus [4]. ERP is traditionally considered to reflect transient, fixed latency, and fixed polarity evoked responses to a stimulus [5–8]. In other words, the ERP has a time-
∗ Corresponding author at: Duchess of Kent Children’s Hospital, The University of Hong Kong, Hong Kong, China. Tel.: +852 2817 7111; fax: +852 2855 0684. E-mail address: [email protected] (Y. Hu). 1746-8094/$ – see front matter © 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.bspc.2010.08.001
locked relation with stimulation. Another competing view suggests that at least part of the ERP is generated by a reorganization of ongoing oscillations in the EEG; i.e., a portion of ongoing EEG to a phase-locked relationship with stimulation. Non-additive processes typical for a phase reset were recently shown to be involved in the generation of the ERP [9], with the conclusion that phase resetting existed in the human EEG, while phase concentration or phase locking was observed in the alpha range EEG [10]. Importantly, many of arguments used to test the prediction of the evoked and phase reset model have been argued for predictive validation [11]. For example, a predictor of the phase reset model is empirical evidence for phase concentration in the absence of a power increase. While a reset of phase will not lead to a power change, the superposition of an evoked response on background EEG activity must lead to a power change. It was also suggested that both phase and amplitude dynamics should be considered, as both the evoked activity and phase reset of ongoing EEG activity contribute substantially to the different auditory Go and NoGo ERP components [12]. In this previous study [12], phase locking mainly contributed to the exogenous ERP components, while evoked activity related to the cognitive processing mainly contributed to the endogenous ERP components. The aim of the present study was to investigate both the evoked response (termed time-locked) and phase reset (termed phaselocked) activities that contribute to the genesis of the P300 signal
244
D. Ming et al. / Biomedical Signal Processing and Control 5 (2010) 243–251
2.1. Stimuli and data acquisition
Fig. 1. Main components of event-related potentials, including exogenous components and endogenous components. The P1, N1, P2, N2, and P3 (also termed P300) are the main components.
in brain–computer interface speller. Two characteristic parameters of event-related spectral perturbation (ERSP) and inter-trial coherence (ITC) around 300 ms of the signal were applied to indicate the evoked response and phase reset activity, respectively.
2. Materials and methods ERP waveform consists of a sequence of positive and negative voltage deflections [13], labeled as P1, N1, P2, N2, and P3 (also termed P300) as shown in Fig. 1. The initial peak (P1) is an obligatory sensory response that is elicited by visual stimuli without cognitive processes. The P1 wave is strongly influenced by stimulus parameters such as luminance. The early sensory responses are called exogenous components to indicate their dependence on external rather than internal factors. By contrast, the P300 wave depends entirely on the task performed by the subject, and is not directly influenced by the physical properties of the eliciting stimulus. The P300 wave is therefore termed an endogenous component to indicate its dependence on internal rather than external factors. P300 as a constituent of the ERP is considered a potential BCI control signal [14]. P300 is a positive EEG defection that occurs during 200–700 ms (typically 300 ms) after stimulus onset, and is typically recorded over the central-parietal scalp [15]. The response is evoked by attention to rare stimuli in a random series of stimulus events (i.e., the oddball paradigm). P300 was used in the BCI P300 speller system because it appears to be closely associated with the cognitive processes. The system consists of stimulus, data acquisition, feature extraction, pattern recognition, and result display (Fig. 2). This study focuses on the feature extraction part.
We used the EEG dataset from Dataset IIb (P300 speller paradigm) obtained from the BCI Competition 2003 data bank [16]. The signals (band-pass filtered from 0.1 to 60 Hz and digitized at 240 Hz) of 64 channels according to the standard electrode position nomenclature of American electroencephalographic society were collected from the subject in three sessions [16]. The first two sessions are used to train the classifier. And the third session is use as the test session. In this study, we only use the first two sessions to extract the P300 feature. Each session consisted of a number of runs. In each run, the subject focused attention on a series of characters. And, totally there are 42 characters to be focused on. For each character epoch, user display was as follows: the matrix was displayed for a 2.5-s period, and during this time each character had the same intensity (i.e., the matrix was blank). Subsequently, each row and column in the matrix was randomly intensified for 100 ms (i.e., resulting in 12 different stimuli of six rows and six columns (Fig. 3). After intensification of a row/column, the matrix was blank for 75 ms. Row/column intensifications were block randomized in blocks of 12. The sets of 12 intensifications were repeated 15 times for each character epoch (i.e., any specific row/column was intensified 15 times, resulting in 180 total intensifications for each character epoch). Each character epoch was followed by a 2.5-s period during which time the matrix was blank. This period informed the user that this character was completed and to focus on the next character in the word that was displayed on the top of the screen (the current character was shown in parentheses). We analyze the signals acquired from the stimulation to 1 s after. For each character, it contains 15 blocks. And each block contains 12 trials (i.e. the stimulation of 6 rows and 6 columns). The sample rate is 240 Hz with 64 channels. So for each character, a 64 × 180 × 240 matrix (64 channels × 15 blocks × 12 trials × 240 Hz) will be generated. 2.2. Data preprocessing and feature extraction This process can be separated into two parts: preprocessing and feature extraction. For preprocessing, the coherence average, principal component analysis (PCA), and independent component analysis (ICA) were used to reduce dimensions and improve signal to noise ratio (SNR). The time–frequency features were then extracted and analyzed. 2.2.1. Data preprocessing It would be difficult to identify the P300 in a single trial without pre-processing. In this study, a Butterworth filter was used as the low-pass filter with a cut-off frequency of 30 Hz. The signals were then processed using coherence average, PCA and ICA by the analysis tool EEGLAB 5.02 (http://sccn.ucsd.edu/eeglab/).
Fig. 2. Major components of the system. These components include (a) the stimulus, where the subject responds to different stimulus in their EEG, (b) data acquisition, (c) data processing for extraction of the features of the signals, and (d) pattern recognition. The results are then displayed on a monitor.
D. Ming et al. / Biomedical Signal Processing and Control 5 (2010) 243–251
245
Fig. 3. User display and the assignment of Stimulus Code for this paradigm. In this example, the users’ task is to spell the word ‘SEND’ (one character at a time). For each character, all rows and columns in the matrix were intensified a number of times (e.g., the third row in this example) as described in the text. The right part illustrates the assignment of the variable Stimulus Code to different row/column intensifications.
2.2.1.1. Coherence average. The coherence average is commonly used to process weak signals, such as EEG-P300, with a strong noise and to improve SNR of signals. SNR was defined as follows: P SNR = 2 , ı
P ı2 /N
∧
= X(t)X T (t),
(3)
(1)
where P is the power of ideal P300 signal and ı2 is the power of the noise. If the noise is assumed as a stationary random signal with a mean value of 0, then the variance of the noise is ı2 . After a coherence average of N samples with the same Stimulus-Code, the variance of the noise will be reduced to ı2 /N, so the new P300 SNR will becomes N times larger. SNR =
Step 1: estimate the sample covariance matrix of the highdimensional EEG signal X(t) after processed by coherence average.
=
NP . ı2
(2)
The data of each electrode was separated after stimulation for 1 s. At sampling rate of 240 Hz, a 180 × 240 matrix (15 blocks × 12 trials × 240 Hz) was generated. Data from the row and column trial including the focused character were respectively averaged across all 15 blocks. The averaged row and column data were connected end to end. Then for each electrode there will be a 1 × 480 vector (1 × 240 for row:1 × 240 for column), and for each character there will be a 64 × 480 matrix (Fig. 4).
2.2.1.2. Principle component analysis. Principle component analysis (PCA) involves a mathematical procedure that transforms a number of possibly correlated variables into a smaller number of uncorrelated variables termed principal components. The first principal component accounts for as much of the variability in the data as possible, and each succeeding component accounts for as much of the remaining variability as possible. PCA is theoretically an optimal linear scheme (in terms of least mean square error) for compressing a set of high dimensional vectors into a set of lower dimensional vectors and then reconstructing the original set. PCA is a non-parametric analysis and the answer is unique and independent of any hypothesis regarding data probability distribution. Importantly, PCA presents a method of compressing the high resolution data into a format for ICA to extract the required information–increasing computational efficiency. The steps to process the EEG data by PCA are as follows:
where X(t) = [x1 (t), x2 (t), ..., xp (t)] and xi (t) is a normalized time series from ith sampling channel of EEG with zero mean value and p is the total number of sampling channels of EEG. Step 2: calculate the eigenvalues 1 , 2 , . . . p and eigenvectors e1 , e2 , . . . ep of the covariance matrix . Here 1 , 2 , . . . p ≥ 0. We can select p characteristic signals y1 , y1 , . . . yp . Here, Y = [y1 , y2 , . . . , yp ]T = E T X,
(4)
E = [e1 , e2 , . . . , ep ],
(5)
ET
(6)
E = ,
= diag{1 , 1 , · · ·p }.
(7)
p Step 3: choose principle component. i / i=1 i is the weight of yi n p
in Y. Find a suitable n, where / i=1 i ≥ 80%. i=1 i Step 4: form the new low-dimensional signal Y*. Y* = [y1 , y2 , . . . yn ]T . 2.2.1.3. Independent component analysis. ICA is a statistical and computational technique for revealing an observed multidimensional random vector into components that are statistically as independent from each other as possible. In practical situations, we cannot generally find a representation where the components are really independent, although we can at least find components that are as independent as possible. This leads us to the following definition of ICA. Given a set of observations of variables (y1 (t), y2 (t). . .yn (t)), such as above-mentioned low-dimensional signal Y*, where t is the time or sample index, assume that the observations are generated as a linear mixture of independent components:
⎛
⎞
⎛
⎞
y1 (t) s1 (t) ⎜ y2 (t) ⎟ ⎜ s2 (t) ⎟ ⎜ . ⎟ = A⎜ . ⎟, ⎝ .. ⎠ ⎝ .. ⎠ yn (t) sn (t)
(8)
where A is a matrix determined by the Infomax ICA based on stochastic gradient learning rules. Infomax explicitly tries to maximize the joint entropy of a nonlinear function of the separated
246
D. Ming et al. / Biomedical Signal Processing and Control 5 (2010) 243–251
Fig. 4. Signals processed by averaging the rows and columns, respectively, which include the P300 when a word is spelt. The vertical axis stands for 64 electrodes, and the horizontal axis stands for time (unit: s). The first 1 s stands for the averaging of the epochs when the correlative rows are intensified. And the second 1 s stand for the averaging of the epochs when the correlative columns are intensified.
outputs; however, it implicitly minimizes the mutual information between the separated outputs so as to make them mutually independent [17]. Independent component analysis now consists of estimating both the matrix A and the si (t), when we only observe the yi (t). Note that we assumed here that the number of independent components was equal to the number of observed variables. Alternatively, we could define ICA as follows: find a linear transformation given by a matrix W, so that the random variables yi , i = 1, 2. . .n are as independent as possible.
⎛
⎞
⎛
⎞
s1 (t) y1 (t) ⎜ s2 (t) ⎟ ⎜ y2 (t) ⎟ ⎜ . ⎟ = W ⎜ . ⎟. ⎝ .. ⎠ ⎝ .. ⎠ sn (t) yn (t)
(9)
This formulation is not that different from that described above, since after estimating A, its inverse A−1 gives W. It can be shown that the problem is well defined, that is, the model can be estimated if all the components si (t) are non-gaussian or only one component is gaussian. This is a fundamental requirement that also explains the main difference between ICA and factor analysis, in which the non-gaussian nature of the data is not taken into account. In fact, ICA could be considered as non-gaussian factor analysis, since in factor analysis we are also modeling the data as linear mixtures of some underlying factors [17].
exponential wavelet is defined as follows: (t) = w(t)exp(j2t),
(10)
where w(t) is the window function. In the present study, we used the Hanning Window, w(t) = 0.5(1 − cos(2t/(N − 1))). The definition of the time–frequency analysis for the signal x is:
F(f, t) =
x(u)|f |p W (f (u − t))exp[−j2f (u − t)]du,
(11)
where f and t stand for the frequency and the time, respectively. While p is a constant, p ≥ 0, we usually choose p = 0, 0.5, 1. 2.2.2.2. Definition of ERSP and ITC. To test the hypothesis of the phase reset, the wavelet transform was performed for each individual trial, and absolute values of the resulting transforms were averaged. ERSP is delimited as follows: ERSP(f, t) =
1 n
|Fk (f, t)|,
(12)
where n stands for the total number of the trials. Fk (f, t) is the time–frequency distribution of the kth trial. ERSP reflects the influence to the power spectrum by the stimulation.
2.2.2. Feature extraction The wavelet-based time–frequency analysis is used to clarify the time course of the evoked and phase-resetting EEG contributions to the ERPs. The event-related spectral perturbation (ERSP) indicates changes in power (in dB) as a function of frequency over the time course of the ERP. The inter-trial coherence (ITC) provides a measure of phase locking (with a range of 0–1 covering from no coupling to complete phase locking), again as a function of frequency over the time course of the ERP. These two measures thus allow insight into the interplay of the evoked and phase locking mechanisms as a function of time. 2.2.2.1. Time–frequency analysis. To determine the phase and time course of oscillatory activity, we used the complex exponential form of the sinusoidal wavelet to analyze the power spectral and the phase spectral properties. The analytical expression of the complex
Fig. 5. Results processed by PCA and ICA, which include the P300 when a word is spelt. The vertical axis stands for the serial number of the independent component, and the horizontal axis stands for time (unit: s).
D. Ming et al. / Biomedical Signal Processing and Control 5 (2010) 243–251
247
Fig. 6. Time–frequency analysis of 10 averaged epochs in the Cz electrode. (a) The ERSP (in dB) and ITC value of 10 averaged epochs in the Cz electrode without P300 signals. There were no differences towards different time and frequency. The ERSP is random and very low, while the ITC values of the signals from 0 to 15 Hz were large than the other frequency. However, there was no difference among the different times. (b) The ERSP (in dB) and ITC values of 10 averaged epochs in the Cz electrode with P300 signals. The ERSP towards 300 ms are larger than in the other period, and the ITC values from 200 to 400 ms are also large.
ITC is used to identify the degree of phase-locking [18], with definition as:
gle trials. ITC values between 0 and 1 are strictly phase-locked to stimulus onset across trials [19,20].
1 ITC(f, t) = n
3. Results and discussion
n Fk (f, t) . |Fk (f, t)|
(13)
k=1
The degree of phase-locking was calculated by ITC, which reflects the homogeneity of the instantaneous phase across sin-
Fig. 4 showed a 64 channel P300 ERP signal. Processed by PCA and ICA, the result can be seen in Fig. 5, while the signal dimension is reduced from 64 to 2 (two independent components). Accord-
248
D. Ming et al. / Biomedical Signal Processing and Control 5 (2010) 243–251
Fig. 7. Time–frequency analysis of 20 averaged epochs in the Cz electrode. (a) The ERSP (in dB) and ITC value of 20 averaged epochs in the Cz electrode with P300 signals. The ERSPs towards 300 ms are larger than in the other period, and the ITC values from 200 to 400 ms are also large. The differences are larger than those of 10 averaged epochs. (b) The ERSP (in dB) and ITC value of 20 averaged epochs in the Cz electrode without P300 signals. There were no differences towards different time and frequency, while the ERSP and ITC values towards 200 ms are a little larger than those in the other period, which may caused by the visual evoked potentials. In the other period, the ERSP and the ITC values of the signals from 0 to 15 Hz were larger than for the other frequency.
ing to the prior knowledge of P300 latency, the second component is considered to reflect the P300. Furthermore, the time domain properties of the P300 can be determined from this component. However, we can only observe the time delay of P300 signals from the time analysis, without any information on the genesis of P300 signals. ERSP reflects the influence of the stimulation on the power spectrum, and can prove the evoked response theory. ITC provides a measure of phase locking (with a range of 0–1 reflecting no coupling
to complete phase locking). If P300 is generated by evoked activity, ERSP of the signals with a P300 component towards 300 ms should be larger than the other period. If P300 is generated by phase resetting, the phases of the signals without P300 should be random and the phases of the signals with P300 should be locked. The coherent average technique was used to eliminate the influence of the random phase and to enhance the phase resetting and evoked response feature of P300. Firstly, signals of each electrode were processed by coherent average technique to reduce the ran-
D. Ming et al. / Biomedical Signal Processing and Control 5 (2010) 243–251
249
Fig. 8. Time–frequency analysis of 20 averaged epochs processed by PCA and ICA. (a) The ERSP (in dB) and ITC value of 20 averaged epochs with the P300 component, which have already been processed by PCA and ICA. The ERSP towards 300 ms are larger than for the other period, and the ITC values from 200 to 400 ms are also large. (b) The ERSP (in dB) and ITC value of 20 averaged epochs without the P300 component, which have already been processed by PCA and ICA. There were also no differences towards different time and frequency. The ERSP value and ITC value were random and low.
dom influence and to enhance ERSP and ITC values. ERSP and ITC are shown as functions of frequency and time in Figs. 6–8. In these figures, the upper left marginal panel along y-axis presents the mean spectrum during the baseline period. The marginal panel along x-axis under the ERSP image shows the maximum (green) and minimum (blue) ERSP values relative to baseline power at each frequency. The lower left marginal panel along y-axis shows the mean ITC across the imaged time range. The marginal panel along x-axis under the ITC image shows ERP, which is produced by ITC across the data spectral pass band.
Results from the Cz electrode can be seen in Fig. 6a–b. Cz was the major electrode related to the event-related potentials. ERSP and ITC values of the signals with the P300 component towards 300 ms are obviously larger than those without P300. ERP can thus be determined. The classical P300 deflection is clearly shown in ERP (Fig. 6a), while there is no such information in Fig. 6b. ERSP (in dB) and ITC values of 20 epochs can be seen in Fig. 7, in which the difference between signals with P300 and signals without P300 is obvious. The periodicity of ERP values can be seen in Fig. 7a, while there are obvious positive peaks in Fig. 7b, suggest-
250
D. Ming et al. / Biomedical Signal Processing and Control 5 (2010) 243–251
4. Conclusion By analyzing the genesis of ERP in the time–frequency domain via computing ITC and ERSP of P300, both time-locked and phaselocked features exist during the use of brain–computer interface P300 speller. The joint analysis of ITC and ERSP demonstrated the validity of our proposed method for feature extraction integrating coherent average, principal component analysis (PCA), independent component analysis (ICA) and wavelet-based time–frequency analysis. It would provide useful information to develop new BCI P300 speller, as well as extracting and analyzing of other evoked potentials. The above discussed literature shows that the question about the mechanisms generating the ERP in the human EEG is still far from being answered. It is noteworthy that several studies yielding evidence for phase resetting argue that phase reset may be only one mechanism which is involved in ERP generation, but they also provide evidence for an evoked response. So, a mixture of these two mechanisms is proposed by some recent publications in this field. The crucial point, nevertheless, is to quantify the contribution of each mechanism. It will be necessary to get an idea to which proportion either phase resetting or evoked response accounts for the generation of a distinct ERP component at a specific brain site. Acknowledgments Fig. 9. Feature comparison of P300 windows from 250 to 450 ms below 15 Hz for 10 averaged epochs (marked as “10”), 20 averaged epochs (marked as “20”), and 20 averaged epochs with the P300 component (marked as “20 PCA-ICA”). (a) The ERSP (in dB) and (b) ITC.
ing that the P300 component does exist. Furthermore, as the P300 component appeared in the ERP, corresponding values of ERSP and ITC appeared the positive peak; this phenomenon was observed in most electrodes. ERP, ERSP, and ITC towards 300 ms are obviously larger than in the other period, proving the presence of coherence of the phase between trials and the influence cause by evoked response. All of above data were determined from the raw signal without processing by PCA and ICA, and results are not obvious if there are insufficient signals to be averaged. Thus, many experiments are required. Signals, which have already been processed by PCA and ICA, are then analyzed. As we know there is only one component in each EEG signal, then there will be only one figure for the epochs without the P300 and one figure for the epochs with the P300. ERSP and ITC values of 20 respective epochs without and with P300 can be seen in Fig. 8. Comparisons of P300 features were conducted in Fig. 9 to test the performance of the above methods for the time-locked and phaselocked feature extraction. We define a P300 window from 250 to 450 ms and the differences of ERSP and ITC below 15 Hz between the P300 window and the other time domain was used as P300 features. From Fig. 9, PCA and ICA plays a very important role for ERSP and ITC value enhancement of P300 signal. Actually, after P300 enhancement, little effort may be needed in the future to classify the target and nontarget responses. In a sense, this algorithm is a kind of gray-box to effectively extract P300 features, which is different from the pattern recognition methods. Thus, it can be easily combined with the classification methods. ERSP reflects the influence on the power spectrum by the stimulation, while ITC reflects the homogeneity of the instantaneous phase across single trial. Both ERSP and ITC obviously show the P300 components in these trials. Thus, the above analysis suggests that both phase resetting and evoked activity contribute to the genesis of the P300 component. However, the quality of the method should be evaluated with more classification methods and accuracies in further investigation in real P300 data.
This research was supported by National Natural Science Foundation of China (No. 30970875, No. 90920015, and No. 60501005), Joint Project of National Natural Science Foundation of China–Royal Society of Edinburgh of UK (No. 30910494/C1009), The National High Technology Research & Development Program of China (No. 2007AA04Z236), Biomedical Engineering Key Program (No. 07ZCKFSF01300) and Intentional Cooperation Key Program (No. 08ZCGHHZ00300) of Tianjin Science Technology Support Plan. This work was partially supported by S.K. Yee Medical Foundation (202206). References [1] H. Li, Y. Li, C. Guan, An effective BCI speller based on semi-supervised learning, Conference Proceedings of IEEE Engineering in Medicine and Biology Society 1 (2006) 1161–1164. [2] N. Xu, X.R. Gao, B. Hong, X.B. Miao, S.K. Gao, F.S. Yang, BCI competition 2003 – data set IIb: enhancing P300 wave detection using ICA-based subspace projections for BCI applications, IEEE Transactions on Biomedical Engineering 51 (6) (2004) 1067–1072. [3] L.A. Farwell, E. Donchin, Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials, Electroencephalography and Clinical Neurophysiology 70 (6) (1988) 510–523. [4] W. Klimesch, P. Sauseng, S. Hanslmayr, W. Gruber, R. Freunberger, Event-related phase reorganization may explain evoked neural dynamics, Neuroscience and Biobehavioral Reviews 31 (7) (2007) 1003–1016. [5] A.C. Merzagora, M. Butti, R. Polikar, M. Izzetoglu, S. Bunce, S. Cerutti, A.M. Bianchi, B. Onaral, Model comparison for automatic characterization and classification of average ERPs using visual oddball paradigm, Clinical Neurophysiology 120 (2) (2009) 264–274. [6] C. Kranczioch, S. Debener, C.S. Herrmann, A.K. Engel, EEG gamma-band activity in rapid serial visual presentation, Experimental Brain Research 169 (2) (2006) 246–254. [7] O. Fehér, R. Schnell, Visual event related potentials the origin of wave P300 a computer model, Acta Biologica Hungarica 53 (4) (2002) 445–464. [8] M. Saito, T. Ishida, Cognitive resource model for the information-processing of task-irrelevant visual stimuli, Psychiatry & Clinical Neurosciences 56 (2) (2002) 145–151. [9] S. Hanslmayr, W. Klimesch, P. Sauseng, W. Gruber, M. Dopplemayr, R. Freunberger, T. Pecherstorfer, N. Birbaumer, Alpha phase reset contributes to the generation of ERPs, Cerebral Cortex 17 (1) (2007) 1–8. [10] S. Makeig, M. Westerfield, T.P. Jung, S. Enghoff, J. Townsend, E. Courchesne, T.J. Sejnowski, Dynamic brain sources of visual evoked responses, Science 295 (5555) (2002) 690–694. [11] P. Sauseng, W. Klimesch, W. Gruber, S. Hanslmayr, R. Freunberger, M. Doppelmayr, Are event-related potential components generated by phase resetting of brain oscillations? A critical discussion, Neuroscience 146 (4) (2007) 1435–1444.
D. Ming et al. / Biomedical Signal Processing and Control 5 (2010) 243–251 [12] R.J. Barry, Evoked activity and EEG phase resetting in the genesis of auditory Go/NoGo ERPs, Biological Psychology 80 (3) (2009) 292–299. [13] S.J. Luck (Ed.), An Introduction to the Event-Related Potential Technique, The MIT Press, 2005, pp. 28–36. [14] F. Nijboer, E.W. Sellers, J. Mellinger, M.A. Jordan, T. Matuz, A. Furdea, S. Halder, U. Mochty, D.J. Krusienski, T.M. Vaughan, J.R. Wolpaw, N. Birbaumer, A. Kübler, A P300-based brain–computer interface for people with amyotrophic lateral sclerosis, Clinical Neurophysiology 119 (8) (2008) 1909–1916. [15] P.K. Ackles, J.R. Jennings, M.G.H. Coles (Eds.), Advances in Psychophysiology, JAI Press, 1987, pp. 1–79. [16] F. Sharbrough, G.E. Chatrian, R. Lesser, H. Lüders, M. Nuwer, T.W. Picton, American electroencephalographic society guidelines for standard electrode position nomenclature, Journal of Clinical Neurophysiology 8 (2) (1991) 200–202.
251
[17] H. Aapo, E. Oja, Independent component analysis: algorithms and applications, Neural Networks 13 (4–5) (2000) 411–430. [18] L.I. Fuentemilla, J. Marco-Pallarés, C. Grau, Modulation of spectral power and of phase resetting of EEG contributes differentially to the generation of auditory event-related potentials, NeuroImage 30 (3) (2006) 909–916. [19] M. Valencia, M. Alegre, J. Artieda, Phase measures in the study of brain responses, Conference Proceedings of IEEE Engineering in Medicine and Biology Society 3 (2003) 2467–2470. [20] A. Delorme, S. Makeig, EEGLAB: an open source toolbox for analysis of singletrial EEG dynamics including independent component analysis, Journal of Neuroscience Methods 134 (1) (2004) 9–21.
Contents lists available at ScienceDirect
Biomedical Signal Processing and Control journal homepage: www.elsevier.com/locate/bspc
Time-locked and phase-locked features of P300 event-related potentials (ERPs) for brain–computer interface speller Dong Ming a,b , Xingwei An a , Youyuan Xi a , Yong Hu b,∗ , Baikun Wan a , Hongzhi Qi a , Longlong Cheng a , Zhaojun Xue a a b
Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, PR China Department of Orthopaedics and Traumatology, Li Ka Shing Faculty of Medicine, University of Hong Kong, Hong Kong, China
a r t i c l e
i n f o
Article history: Received 30 August 2009 Received in revised form 26 July 2010 Accepted 2 August 2010 Available online 26 August 2010 Keywords: Brain–computer interface Evoked response potentials Phase reset Event-related spectral perturbation Inter-trial coherence
a b s t r a c t The brain–computer interface P300 speller is aimed to help those patients unable to activate muscles to spell words by utilizing their brain activity. However, a problem associated with the use of this brain–computer interface paradigm is the generation mechanics of P300 related to responses to visual stimuli. Herein, we investigated the event-related potential (ERP) response for the P300-based brain–computer interface speller. A signal preprocessing method integrated coherent average, principal component analysis (PCA) and independent component analysis (ICA) to reduce the dimensions and noise in the raw data. The time–frequency analysis was based on wavelet and two characteristic parameters of event-related spectral perturbation (ERSP) and inter-trial coherence (ITC) were computed to indicate the evoked response (time-locked) and phase reset (phase-locked) activity, respectively. Results demonstrated that the proposed method was valid for the time-locked and phase-locked feature extraction and both the evoked response and phase reset contributed to the genesis of the P300 signal. These electrophysiological responses characteristics of ERPs would be used for BCI P300 speller design and its signal processing strategies. © 2010 Elsevier Ltd. All rights reserved.
1. Introduction A brain–computer interface (BCI) provides alternative communication and control channels to convey messages and commands from the brain to the external world [1], especially for those patients with severe neurological or muscular diseases. At present, electroencephalogram (EEG) is the major brainwave signal used by non-invasive BCIs. One strategy of EEG-based BCI involves the use of event related potential (ERP) that exploits the electrophysiological responses to a certain event. The most robust feature of the ERP is a positive displacement occurring around 300 ms after stimulus, termed the P300 or P3 [2]. The P300 was first utilized in BCI as a speller [3]. A major technical problem in the P300-based BCI speller is the robustness of the classification of the response from background noise to improve the BCI system performance. Furthermore, it remains controversial whether ERPs are generated by evoked response or by phase reset with the outward stimulus [4]. ERP is traditionally considered to reflect transient, fixed latency, and fixed polarity evoked responses to a stimulus [5–8]. In other words, the ERP has a time-
∗ Corresponding author at: Duchess of Kent Children’s Hospital, The University of Hong Kong, Hong Kong, China. Tel.: +852 2817 7111; fax: +852 2855 0684. E-mail address: [email protected] (Y. Hu). 1746-8094/$ – see front matter © 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.bspc.2010.08.001
locked relation with stimulation. Another competing view suggests that at least part of the ERP is generated by a reorganization of ongoing oscillations in the EEG; i.e., a portion of ongoing EEG to a phase-locked relationship with stimulation. Non-additive processes typical for a phase reset were recently shown to be involved in the generation of the ERP [9], with the conclusion that phase resetting existed in the human EEG, while phase concentration or phase locking was observed in the alpha range EEG [10]. Importantly, many of arguments used to test the prediction of the evoked and phase reset model have been argued for predictive validation [11]. For example, a predictor of the phase reset model is empirical evidence for phase concentration in the absence of a power increase. While a reset of phase will not lead to a power change, the superposition of an evoked response on background EEG activity must lead to a power change. It was also suggested that both phase and amplitude dynamics should be considered, as both the evoked activity and phase reset of ongoing EEG activity contribute substantially to the different auditory Go and NoGo ERP components [12]. In this previous study [12], phase locking mainly contributed to the exogenous ERP components, while evoked activity related to the cognitive processing mainly contributed to the endogenous ERP components. The aim of the present study was to investigate both the evoked response (termed time-locked) and phase reset (termed phaselocked) activities that contribute to the genesis of the P300 signal
244
D. Ming et al. / Biomedical Signal Processing and Control 5 (2010) 243–251
2.1. Stimuli and data acquisition
Fig. 1. Main components of event-related potentials, including exogenous components and endogenous components. The P1, N1, P2, N2, and P3 (also termed P300) are the main components.
in brain–computer interface speller. Two characteristic parameters of event-related spectral perturbation (ERSP) and inter-trial coherence (ITC) around 300 ms of the signal were applied to indicate the evoked response and phase reset activity, respectively.
2. Materials and methods ERP waveform consists of a sequence of positive and negative voltage deflections [13], labeled as P1, N1, P2, N2, and P3 (also termed P300) as shown in Fig. 1. The initial peak (P1) is an obligatory sensory response that is elicited by visual stimuli without cognitive processes. The P1 wave is strongly influenced by stimulus parameters such as luminance. The early sensory responses are called exogenous components to indicate their dependence on external rather than internal factors. By contrast, the P300 wave depends entirely on the task performed by the subject, and is not directly influenced by the physical properties of the eliciting stimulus. The P300 wave is therefore termed an endogenous component to indicate its dependence on internal rather than external factors. P300 as a constituent of the ERP is considered a potential BCI control signal [14]. P300 is a positive EEG defection that occurs during 200–700 ms (typically 300 ms) after stimulus onset, and is typically recorded over the central-parietal scalp [15]. The response is evoked by attention to rare stimuli in a random series of stimulus events (i.e., the oddball paradigm). P300 was used in the BCI P300 speller system because it appears to be closely associated with the cognitive processes. The system consists of stimulus, data acquisition, feature extraction, pattern recognition, and result display (Fig. 2). This study focuses on the feature extraction part.
We used the EEG dataset from Dataset IIb (P300 speller paradigm) obtained from the BCI Competition 2003 data bank [16]. The signals (band-pass filtered from 0.1 to 60 Hz and digitized at 240 Hz) of 64 channels according to the standard electrode position nomenclature of American electroencephalographic society were collected from the subject in three sessions [16]. The first two sessions are used to train the classifier. And the third session is use as the test session. In this study, we only use the first two sessions to extract the P300 feature. Each session consisted of a number of runs. In each run, the subject focused attention on a series of characters. And, totally there are 42 characters to be focused on. For each character epoch, user display was as follows: the matrix was displayed for a 2.5-s period, and during this time each character had the same intensity (i.e., the matrix was blank). Subsequently, each row and column in the matrix was randomly intensified for 100 ms (i.e., resulting in 12 different stimuli of six rows and six columns (Fig. 3). After intensification of a row/column, the matrix was blank for 75 ms. Row/column intensifications were block randomized in blocks of 12. The sets of 12 intensifications were repeated 15 times for each character epoch (i.e., any specific row/column was intensified 15 times, resulting in 180 total intensifications for each character epoch). Each character epoch was followed by a 2.5-s period during which time the matrix was blank. This period informed the user that this character was completed and to focus on the next character in the word that was displayed on the top of the screen (the current character was shown in parentheses). We analyze the signals acquired from the stimulation to 1 s after. For each character, it contains 15 blocks. And each block contains 12 trials (i.e. the stimulation of 6 rows and 6 columns). The sample rate is 240 Hz with 64 channels. So for each character, a 64 × 180 × 240 matrix (64 channels × 15 blocks × 12 trials × 240 Hz) will be generated. 2.2. Data preprocessing and feature extraction This process can be separated into two parts: preprocessing and feature extraction. For preprocessing, the coherence average, principal component analysis (PCA), and independent component analysis (ICA) were used to reduce dimensions and improve signal to noise ratio (SNR). The time–frequency features were then extracted and analyzed. 2.2.1. Data preprocessing It would be difficult to identify the P300 in a single trial without pre-processing. In this study, a Butterworth filter was used as the low-pass filter with a cut-off frequency of 30 Hz. The signals were then processed using coherence average, PCA and ICA by the analysis tool EEGLAB 5.02 (http://sccn.ucsd.edu/eeglab/).
Fig. 2. Major components of the system. These components include (a) the stimulus, where the subject responds to different stimulus in their EEG, (b) data acquisition, (c) data processing for extraction of the features of the signals, and (d) pattern recognition. The results are then displayed on a monitor.
D. Ming et al. / Biomedical Signal Processing and Control 5 (2010) 243–251
245
Fig. 3. User display and the assignment of Stimulus Code for this paradigm. In this example, the users’ task is to spell the word ‘SEND’ (one character at a time). For each character, all rows and columns in the matrix were intensified a number of times (e.g., the third row in this example) as described in the text. The right part illustrates the assignment of the variable Stimulus Code to different row/column intensifications.
2.2.1.1. Coherence average. The coherence average is commonly used to process weak signals, such as EEG-P300, with a strong noise and to improve SNR of signals. SNR was defined as follows: P SNR = 2 , ı
P ı2 /N
∧
= X(t)X T (t),
(3)
(1)
where P is the power of ideal P300 signal and ı2 is the power of the noise. If the noise is assumed as a stationary random signal with a mean value of 0, then the variance of the noise is ı2 . After a coherence average of N samples with the same Stimulus-Code, the variance of the noise will be reduced to ı2 /N, so the new P300 SNR will becomes N times larger. SNR =
Step 1: estimate the sample covariance matrix of the highdimensional EEG signal X(t) after processed by coherence average.
=
NP . ı2
(2)
The data of each electrode was separated after stimulation for 1 s. At sampling rate of 240 Hz, a 180 × 240 matrix (15 blocks × 12 trials × 240 Hz) was generated. Data from the row and column trial including the focused character were respectively averaged across all 15 blocks. The averaged row and column data were connected end to end. Then for each electrode there will be a 1 × 480 vector (1 × 240 for row:1 × 240 for column), and for each character there will be a 64 × 480 matrix (Fig. 4).
2.2.1.2. Principle component analysis. Principle component analysis (PCA) involves a mathematical procedure that transforms a number of possibly correlated variables into a smaller number of uncorrelated variables termed principal components. The first principal component accounts for as much of the variability in the data as possible, and each succeeding component accounts for as much of the remaining variability as possible. PCA is theoretically an optimal linear scheme (in terms of least mean square error) for compressing a set of high dimensional vectors into a set of lower dimensional vectors and then reconstructing the original set. PCA is a non-parametric analysis and the answer is unique and independent of any hypothesis regarding data probability distribution. Importantly, PCA presents a method of compressing the high resolution data into a format for ICA to extract the required information–increasing computational efficiency. The steps to process the EEG data by PCA are as follows:
where X(t) = [x1 (t), x2 (t), ..., xp (t)] and xi (t) is a normalized time series from ith sampling channel of EEG with zero mean value and p is the total number of sampling channels of EEG. Step 2: calculate the eigenvalues 1 , 2 , . . . p and eigenvectors e1 , e2 , . . . ep of the covariance matrix . Here 1 , 2 , . . . p ≥ 0. We can select p characteristic signals y1 , y1 , . . . yp . Here, Y = [y1 , y2 , . . . , yp ]T = E T X,
(4)
E = [e1 , e2 , . . . , ep ],
(5)
ET
(6)
E = ,
= diag{1 , 1 , · · ·p }.
(7)
p Step 3: choose principle component. i / i=1 i is the weight of yi n p
in Y. Find a suitable n, where / i=1 i ≥ 80%. i=1 i Step 4: form the new low-dimensional signal Y*. Y* = [y1 , y2 , . . . yn ]T . 2.2.1.3. Independent component analysis. ICA is a statistical and computational technique for revealing an observed multidimensional random vector into components that are statistically as independent from each other as possible. In practical situations, we cannot generally find a representation where the components are really independent, although we can at least find components that are as independent as possible. This leads us to the following definition of ICA. Given a set of observations of variables (y1 (t), y2 (t). . .yn (t)), such as above-mentioned low-dimensional signal Y*, where t is the time or sample index, assume that the observations are generated as a linear mixture of independent components:
⎛
⎞
⎛
⎞
y1 (t) s1 (t) ⎜ y2 (t) ⎟ ⎜ s2 (t) ⎟ ⎜ . ⎟ = A⎜ . ⎟, ⎝ .. ⎠ ⎝ .. ⎠ yn (t) sn (t)
(8)
where A is a matrix determined by the Infomax ICA based on stochastic gradient learning rules. Infomax explicitly tries to maximize the joint entropy of a nonlinear function of the separated
246
D. Ming et al. / Biomedical Signal Processing and Control 5 (2010) 243–251
Fig. 4. Signals processed by averaging the rows and columns, respectively, which include the P300 when a word is spelt. The vertical axis stands for 64 electrodes, and the horizontal axis stands for time (unit: s). The first 1 s stands for the averaging of the epochs when the correlative rows are intensified. And the second 1 s stand for the averaging of the epochs when the correlative columns are intensified.
outputs; however, it implicitly minimizes the mutual information between the separated outputs so as to make them mutually independent [17]. Independent component analysis now consists of estimating both the matrix A and the si (t), when we only observe the yi (t). Note that we assumed here that the number of independent components was equal to the number of observed variables. Alternatively, we could define ICA as follows: find a linear transformation given by a matrix W, so that the random variables yi , i = 1, 2. . .n are as independent as possible.
⎛
⎞
⎛
⎞
s1 (t) y1 (t) ⎜ s2 (t) ⎟ ⎜ y2 (t) ⎟ ⎜ . ⎟ = W ⎜ . ⎟. ⎝ .. ⎠ ⎝ .. ⎠ sn (t) yn (t)
(9)
This formulation is not that different from that described above, since after estimating A, its inverse A−1 gives W. It can be shown that the problem is well defined, that is, the model can be estimated if all the components si (t) are non-gaussian or only one component is gaussian. This is a fundamental requirement that also explains the main difference between ICA and factor analysis, in which the non-gaussian nature of the data is not taken into account. In fact, ICA could be considered as non-gaussian factor analysis, since in factor analysis we are also modeling the data as linear mixtures of some underlying factors [17].
exponential wavelet is defined as follows: (t) = w(t)exp(j2t),
(10)
where w(t) is the window function. In the present study, we used the Hanning Window, w(t) = 0.5(1 − cos(2t/(N − 1))). The definition of the time–frequency analysis for the signal x is:
F(f, t) =
x(u)|f |p W (f (u − t))exp[−j2f (u − t)]du,
(11)
where f and t stand for the frequency and the time, respectively. While p is a constant, p ≥ 0, we usually choose p = 0, 0.5, 1. 2.2.2.2. Definition of ERSP and ITC. To test the hypothesis of the phase reset, the wavelet transform was performed for each individual trial, and absolute values of the resulting transforms were averaged. ERSP is delimited as follows: ERSP(f, t) =
1 n
|Fk (f, t)|,
(12)
where n stands for the total number of the trials. Fk (f, t) is the time–frequency distribution of the kth trial. ERSP reflects the influence to the power spectrum by the stimulation.
2.2.2. Feature extraction The wavelet-based time–frequency analysis is used to clarify the time course of the evoked and phase-resetting EEG contributions to the ERPs. The event-related spectral perturbation (ERSP) indicates changes in power (in dB) as a function of frequency over the time course of the ERP. The inter-trial coherence (ITC) provides a measure of phase locking (with a range of 0–1 covering from no coupling to complete phase locking), again as a function of frequency over the time course of the ERP. These two measures thus allow insight into the interplay of the evoked and phase locking mechanisms as a function of time. 2.2.2.1. Time–frequency analysis. To determine the phase and time course of oscillatory activity, we used the complex exponential form of the sinusoidal wavelet to analyze the power spectral and the phase spectral properties. The analytical expression of the complex
Fig. 5. Results processed by PCA and ICA, which include the P300 when a word is spelt. The vertical axis stands for the serial number of the independent component, and the horizontal axis stands for time (unit: s).
D. Ming et al. / Biomedical Signal Processing and Control 5 (2010) 243–251
247
Fig. 6. Time–frequency analysis of 10 averaged epochs in the Cz electrode. (a) The ERSP (in dB) and ITC value of 10 averaged epochs in the Cz electrode without P300 signals. There were no differences towards different time and frequency. The ERSP is random and very low, while the ITC values of the signals from 0 to 15 Hz were large than the other frequency. However, there was no difference among the different times. (b) The ERSP (in dB) and ITC values of 10 averaged epochs in the Cz electrode with P300 signals. The ERSP towards 300 ms are larger than in the other period, and the ITC values from 200 to 400 ms are also large.
ITC is used to identify the degree of phase-locking [18], with definition as:
gle trials. ITC values between 0 and 1 are strictly phase-locked to stimulus onset across trials [19,20].
1 ITC(f, t) = n
3. Results and discussion
n Fk (f, t) . |Fk (f, t)|
(13)
k=1
The degree of phase-locking was calculated by ITC, which reflects the homogeneity of the instantaneous phase across sin-
Fig. 4 showed a 64 channel P300 ERP signal. Processed by PCA and ICA, the result can be seen in Fig. 5, while the signal dimension is reduced from 64 to 2 (two independent components). Accord-
248
D. Ming et al. / Biomedical Signal Processing and Control 5 (2010) 243–251
Fig. 7. Time–frequency analysis of 20 averaged epochs in the Cz electrode. (a) The ERSP (in dB) and ITC value of 20 averaged epochs in the Cz electrode with P300 signals. The ERSPs towards 300 ms are larger than in the other period, and the ITC values from 200 to 400 ms are also large. The differences are larger than those of 10 averaged epochs. (b) The ERSP (in dB) and ITC value of 20 averaged epochs in the Cz electrode without P300 signals. There were no differences towards different time and frequency, while the ERSP and ITC values towards 200 ms are a little larger than those in the other period, which may caused by the visual evoked potentials. In the other period, the ERSP and the ITC values of the signals from 0 to 15 Hz were larger than for the other frequency.
ing to the prior knowledge of P300 latency, the second component is considered to reflect the P300. Furthermore, the time domain properties of the P300 can be determined from this component. However, we can only observe the time delay of P300 signals from the time analysis, without any information on the genesis of P300 signals. ERSP reflects the influence of the stimulation on the power spectrum, and can prove the evoked response theory. ITC provides a measure of phase locking (with a range of 0–1 reflecting no coupling
to complete phase locking). If P300 is generated by evoked activity, ERSP of the signals with a P300 component towards 300 ms should be larger than the other period. If P300 is generated by phase resetting, the phases of the signals without P300 should be random and the phases of the signals with P300 should be locked. The coherent average technique was used to eliminate the influence of the random phase and to enhance the phase resetting and evoked response feature of P300. Firstly, signals of each electrode were processed by coherent average technique to reduce the ran-
D. Ming et al. / Biomedical Signal Processing and Control 5 (2010) 243–251
249
Fig. 8. Time–frequency analysis of 20 averaged epochs processed by PCA and ICA. (a) The ERSP (in dB) and ITC value of 20 averaged epochs with the P300 component, which have already been processed by PCA and ICA. The ERSP towards 300 ms are larger than for the other period, and the ITC values from 200 to 400 ms are also large. (b) The ERSP (in dB) and ITC value of 20 averaged epochs without the P300 component, which have already been processed by PCA and ICA. There were also no differences towards different time and frequency. The ERSP value and ITC value were random and low.
dom influence and to enhance ERSP and ITC values. ERSP and ITC are shown as functions of frequency and time in Figs. 6–8. In these figures, the upper left marginal panel along y-axis presents the mean spectrum during the baseline period. The marginal panel along x-axis under the ERSP image shows the maximum (green) and minimum (blue) ERSP values relative to baseline power at each frequency. The lower left marginal panel along y-axis shows the mean ITC across the imaged time range. The marginal panel along x-axis under the ITC image shows ERP, which is produced by ITC across the data spectral pass band.
Results from the Cz electrode can be seen in Fig. 6a–b. Cz was the major electrode related to the event-related potentials. ERSP and ITC values of the signals with the P300 component towards 300 ms are obviously larger than those without P300. ERP can thus be determined. The classical P300 deflection is clearly shown in ERP (Fig. 6a), while there is no such information in Fig. 6b. ERSP (in dB) and ITC values of 20 epochs can be seen in Fig. 7, in which the difference between signals with P300 and signals without P300 is obvious. The periodicity of ERP values can be seen in Fig. 7a, while there are obvious positive peaks in Fig. 7b, suggest-
250
D. Ming et al. / Biomedical Signal Processing and Control 5 (2010) 243–251
4. Conclusion By analyzing the genesis of ERP in the time–frequency domain via computing ITC and ERSP of P300, both time-locked and phaselocked features exist during the use of brain–computer interface P300 speller. The joint analysis of ITC and ERSP demonstrated the validity of our proposed method for feature extraction integrating coherent average, principal component analysis (PCA), independent component analysis (ICA) and wavelet-based time–frequency analysis. It would provide useful information to develop new BCI P300 speller, as well as extracting and analyzing of other evoked potentials. The above discussed literature shows that the question about the mechanisms generating the ERP in the human EEG is still far from being answered. It is noteworthy that several studies yielding evidence for phase resetting argue that phase reset may be only one mechanism which is involved in ERP generation, but they also provide evidence for an evoked response. So, a mixture of these two mechanisms is proposed by some recent publications in this field. The crucial point, nevertheless, is to quantify the contribution of each mechanism. It will be necessary to get an idea to which proportion either phase resetting or evoked response accounts for the generation of a distinct ERP component at a specific brain site. Acknowledgments Fig. 9. Feature comparison of P300 windows from 250 to 450 ms below 15 Hz for 10 averaged epochs (marked as “10”), 20 averaged epochs (marked as “20”), and 20 averaged epochs with the P300 component (marked as “20 PCA-ICA”). (a) The ERSP (in dB) and (b) ITC.
ing that the P300 component does exist. Furthermore, as the P300 component appeared in the ERP, corresponding values of ERSP and ITC appeared the positive peak; this phenomenon was observed in most electrodes. ERP, ERSP, and ITC towards 300 ms are obviously larger than in the other period, proving the presence of coherence of the phase between trials and the influence cause by evoked response. All of above data were determined from the raw signal without processing by PCA and ICA, and results are not obvious if there are insufficient signals to be averaged. Thus, many experiments are required. Signals, which have already been processed by PCA and ICA, are then analyzed. As we know there is only one component in each EEG signal, then there will be only one figure for the epochs without the P300 and one figure for the epochs with the P300. ERSP and ITC values of 20 respective epochs without and with P300 can be seen in Fig. 8. Comparisons of P300 features were conducted in Fig. 9 to test the performance of the above methods for the time-locked and phaselocked feature extraction. We define a P300 window from 250 to 450 ms and the differences of ERSP and ITC below 15 Hz between the P300 window and the other time domain was used as P300 features. From Fig. 9, PCA and ICA plays a very important role for ERSP and ITC value enhancement of P300 signal. Actually, after P300 enhancement, little effort may be needed in the future to classify the target and nontarget responses. In a sense, this algorithm is a kind of gray-box to effectively extract P300 features, which is different from the pattern recognition methods. Thus, it can be easily combined with the classification methods. ERSP reflects the influence on the power spectrum by the stimulation, while ITC reflects the homogeneity of the instantaneous phase across single trial. Both ERSP and ITC obviously show the P300 components in these trials. Thus, the above analysis suggests that both phase resetting and evoked activity contribute to the genesis of the P300 component. However, the quality of the method should be evaluated with more classification methods and accuracies in further investigation in real P300 data.
This research was supported by National Natural Science Foundation of China (No. 30970875, No. 90920015, and No. 60501005), Joint Project of National Natural Science Foundation of China–Royal Society of Edinburgh of UK (No. 30910494/C1009), The National High Technology Research & Development Program of China (No. 2007AA04Z236), Biomedical Engineering Key Program (No. 07ZCKFSF01300) and Intentional Cooperation Key Program (No. 08ZCGHHZ00300) of Tianjin Science Technology Support Plan. This work was partially supported by S.K. Yee Medical Foundation (202206). References [1] H. Li, Y. Li, C. Guan, An effective BCI speller based on semi-supervised learning, Conference Proceedings of IEEE Engineering in Medicine and Biology Society 1 (2006) 1161–1164. [2] N. Xu, X.R. Gao, B. Hong, X.B. Miao, S.K. Gao, F.S. Yang, BCI competition 2003 – data set IIb: enhancing P300 wave detection using ICA-based subspace projections for BCI applications, IEEE Transactions on Biomedical Engineering 51 (6) (2004) 1067–1072. [3] L.A. Farwell, E. Donchin, Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials, Electroencephalography and Clinical Neurophysiology 70 (6) (1988) 510–523. [4] W. Klimesch, P. Sauseng, S. Hanslmayr, W. Gruber, R. Freunberger, Event-related phase reorganization may explain evoked neural dynamics, Neuroscience and Biobehavioral Reviews 31 (7) (2007) 1003–1016. [5] A.C. Merzagora, M. Butti, R. Polikar, M. Izzetoglu, S. Bunce, S. Cerutti, A.M. Bianchi, B. Onaral, Model comparison for automatic characterization and classification of average ERPs using visual oddball paradigm, Clinical Neurophysiology 120 (2) (2009) 264–274. [6] C. Kranczioch, S. Debener, C.S. Herrmann, A.K. Engel, EEG gamma-band activity in rapid serial visual presentation, Experimental Brain Research 169 (2) (2006) 246–254. [7] O. Fehér, R. Schnell, Visual event related potentials the origin of wave P300 a computer model, Acta Biologica Hungarica 53 (4) (2002) 445–464. [8] M. Saito, T. Ishida, Cognitive resource model for the information-processing of task-irrelevant visual stimuli, Psychiatry & Clinical Neurosciences 56 (2) (2002) 145–151. [9] S. Hanslmayr, W. Klimesch, P. Sauseng, W. Gruber, M. Dopplemayr, R. Freunberger, T. Pecherstorfer, N. Birbaumer, Alpha phase reset contributes to the generation of ERPs, Cerebral Cortex 17 (1) (2007) 1–8. [10] S. Makeig, M. Westerfield, T.P. Jung, S. Enghoff, J. Townsend, E. Courchesne, T.J. Sejnowski, Dynamic brain sources of visual evoked responses, Science 295 (5555) (2002) 690–694. [11] P. Sauseng, W. Klimesch, W. Gruber, S. Hanslmayr, R. Freunberger, M. Doppelmayr, Are event-related potential components generated by phase resetting of brain oscillations? A critical discussion, Neuroscience 146 (4) (2007) 1435–1444.
D. Ming et al. / Biomedical Signal Processing and Control 5 (2010) 243–251 [12] R.J. Barry, Evoked activity and EEG phase resetting in the genesis of auditory Go/NoGo ERPs, Biological Psychology 80 (3) (2009) 292–299. [13] S.J. Luck (Ed.), An Introduction to the Event-Related Potential Technique, The MIT Press, 2005, pp. 28–36. [14] F. Nijboer, E.W. Sellers, J. Mellinger, M.A. Jordan, T. Matuz, A. Furdea, S. Halder, U. Mochty, D.J. Krusienski, T.M. Vaughan, J.R. Wolpaw, N. Birbaumer, A. Kübler, A P300-based brain–computer interface for people with amyotrophic lateral sclerosis, Clinical Neurophysiology 119 (8) (2008) 1909–1916. [15] P.K. Ackles, J.R. Jennings, M.G.H. Coles (Eds.), Advances in Psychophysiology, JAI Press, 1987, pp. 1–79. [16] F. Sharbrough, G.E. Chatrian, R. Lesser, H. Lüders, M. Nuwer, T.W. Picton, American electroencephalographic society guidelines for standard electrode position nomenclature, Journal of Clinical Neurophysiology 8 (2) (1991) 200–202.
251
[17] H. Aapo, E. Oja, Independent component analysis: algorithms and applications, Neural Networks 13 (4–5) (2000) 411–430. [18] L.I. Fuentemilla, J. Marco-Pallarés, C. Grau, Modulation of spectral power and of phase resetting of EEG contributes differentially to the generation of auditory event-related potentials, NeuroImage 30 (3) (2006) 909–916. [19] M. Valencia, M. Alegre, J. Artieda, Phase measures in the study of brain responses, Conference Proceedings of IEEE Engineering in Medicine and Biology Society 3 (2003) 2467–2470. [20] A. Delorme, S. Makeig, EEGLAB: an open source toolbox for analysis of singletrial EEG dynamics including independent component analysis, Journal of Neuroscience Methods 134 (1) (2004) 9–21.