- Email: [email protected]
Brain Computer Interfaced Single Key Omni Directional Pointing and Command System: a Screen Pointing Interface for Differently-abled Person
Available online at www.sciencedirect.com Available Available online online at at www.sciencedirect.com www.sciencedirect.com
Available online at www.sciencedirect.com
ScienceDirect
Procedia Computer Science 00 (2018) 000–000 Procedia Computer Science (2018) 000–000 Procedia Computer Science 13300 (2018) 161–168 Procedia Computer Science 00 (2018) 000–000
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
International Conference on Robotics and Smart Manufacturing (RoSMa2018) International International Conference Conference on on Robotics Robotics and and Smart Smart Manufacturing Manufacturing (RoSMa2018) (RoSMa2018)
Brain Brain Computer Computer Interfaced Interfaced Single Single Key Key Omni Omni Directional Directional Pointing Pointing and Command System: a Screen Pointing Interface for and Command System: a Screen Pointing Interface for Differently-abled Differently-abled Person Person Rahul Rajaaa , Suman Debb,∗ , Paritosh Bhattacharyac b,∗ b,∗, Paritosh Bhattacharyacc Rahul Raj , Suman Deb Rahul Raj , Suman Deb , Paritosh Bhattacharya a MCA, Department of Computer Science and Engineering, NIT Agartala, Agartala 799046, India a MCA, Department of Computer Science and Engineering, NIT Agartala, Agartala 799046, India a MCA, b Department Department of Computer Science Engineering, Agartala, Agartala 799046, India of Computer Science andand Engineering, NITNIT Agartala, Agartala 799046, India b Department of Computer Science and Engineering, NIT Agartala, Agartala 799046, India b Department c Department of Computer and Engineering, NITAgartala Agartala, Agartala 799046, India ofScience Mathematics, NIT Agartala, 799046, India c Department of Mathematics, NIT Agartala, Agartala 799046, India c Department of Mathematics, NIT Agartala, Agartala 799046, India
Abstract Abstract Abstract Biomedical signals have got very incredible reach to the people outside clinical domain as devices are developed that can be Biomedical signals got incredible reach to the outside clinical domain devices are that can Biomedical signals have got very veryand incredible reach to booming the people peoplearea outside clinicalnowadays. domain as as An devices are developed developed that Human can be be controlled using one’shave imagination this is the very of research effective way of using controlled using imagination is very area of nowadays. way of controlled using one’s one’s(HCI) imagination and this is the the very booming booming area(BCI) of research research nowadays. An An effective effective waywhich of using using Human Computer Interaction is usingand thethis Brain Computer Interaction and Electroencephalogram (EEG) canHuman extend Computer Interaction is the Interaction (BCI) and (EEG) which Computer Interaction (HCI) is using usingspecifically the Brain Brain Computer Computer Interaction (BCI) Generally, and Electroencephalogram Electroencephalogram (EEG) which whichiscan can extend the people’s access to (HCI) use machines for specially abled persons. EEG is a costly matter notextend every the people’s access to use machines specifically for specially abled persons. Generally, EEG is a costly matter which is the access to use machines specifically persons. Generally, EEGthe is possibility a costly matter whichchannel is not not every every timepeople’s affordable by any common person. Here infor thisspecially proposedabled system, we tried to explore of single EEG time affordable any person. in system, we to the channel EEG time affordable by any common common person. Here Here in this this proposed proposed system, we tried tried to explore explore the possibility possibility of single channel EEG through which aby person can communicate to computer according to his/her neurosignal excitement which of aresingle classified as Attenthrough which person can communicate to according to neurosignal excitement which classified Attenthrough which aa and person canstrength communicate to computer computer accordinginto to his/her his/her neurosignal excitement which are are classified as Attention, Meditation Blink and then can be interpreted effective Human Machine Interaction (HMI). The as proposed tion, Meditation and Blink and can into Human Machine Interaction (HMI). The tion, Meditation and brainwaves Blink strength strength and then thenblinks can be be interpreted intoineffective effective Humanusing Machine Interaction (HMI).Mobile The proposed proposed algorithm here uses (voluntary as interpreted a control tool BCI) detected Neurosky Mindwave headset algorithm here uses brainwaves (voluntary blinks as a control tool in BCI) detected using Neurosky Mindwave Mobile algorithm here uses brainwaves (voluntary blinks as a control tool in BCI) detected using Neurosky Mindwave Mobile headset from the frontal lobe (because it offers EEG clarity since this is the forehead area with minimal hair). This proposed systemheadset can be from the lobe (because it is area with proposed can from the frontal frontal lobe gaming, (becauseentertainment it offers offers EEG EEG clarity clarity since this is the the forehead areapurposes. with minimal minimal hair). This proposedassystem system can be be extended for typing, as wellsince as forthis daily lifeforehead interaction It canhair). also This be extended a non-verbal extended for typing, gaming, entertainment as well as for daily life interaction purposes. It can also be extended as a non-verbal extended for typing, gaming, entertainment well asThe for daily life interaction purposes. It proposed can also be extended as a non-verbal communication medium for specially abled as persons. experiment conducted using the system revealed significant communication medium specially abled persons. The experiment conducted using proposed revealed significant communication medium for for specially abled persons. Thetried experiment conducted using the the proposed system revealed significant development in accessing computer and typing skill. We to explore scan keyboard which can besystem operated by neural signals development in accessing computer typing skill. tried to keyboard development in Neurosky accessing Mindwave computer and and typing skill. We We tried to explore explore scan keyboard which which can can be be operated operated by by neural neural signals signals captured by the Mobile Headset Single channel EEGscan Headset. captured captured by by the the Neurosky Neurosky Mindwave Mindwave Mobile Mobile Headset Headset Single Single channel channel EEG EEG Headset. Headset. c 2018 2018 The The Authors. Authors. Published Published by by Elsevier Elsevier Ltd. Ltd. © 2018 The Authors. Authors. Published by by Elsevier Ltd. Ltd. cc 2018 The This is an license (https://creativecommons.org/licenses/by-nc-nd/4.0/) (https://creativecommons.org/licenses/by-nc-nd/4.0/). open access Published article under Elsevier the CC BY-NC-ND license This is an an open open access article under under the scientific CC BY-NC-ND BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/). This is access article the CC license (https://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the committee of the International Conference on Robotics and Smart Manufacturing. Keywords: BCI, Eye-Blink, Neurosignal, EEG, Machine Interaction, Assistive typing, Scan Keyboard Keywords: BCI, Keywords: BCI, Eye-Blink, Eye-Blink, Neurosignal, Neurosignal, EEG, EEG, Machine Machine Interaction, Interaction, Assistive Assistive typing, typing, Scan Scan Keyboard Keyboard
1. Introduction 1. 1. Introduction Introduction Development of technology can be a savior for people in their daily life. In many stages of human life there may of can be savior for in daily many human may Development of technology technology canmay be aaoccur saviorwhere for people people in their their daily life. life. In In many stages of human life there may be Development accidental or genetic disorders the brain is functioning well but stages motor of nerves arelife notthere working. be accidental or genetic disorders may occur where the brain is functioning well but motor nerves are not working. be accidental or person geneticfaces disorders occur where the brain is functioning well butlike motor nerves are In that case the lots ofmay problems accessing the present times machines computer. We not are working. trying to In In that that case case the the person person faces faces lots lots of of problems problems accessing accessing the the present present times times machines machines like like computer. computer. We We are are trying trying to to ∗ ∗ ∗
Corresponding author. Tel.: +91-9436459622 ; fax: +91-0381-2346-630. Corresponding Tel.: Corresponding author. Tel.: +91-9436459622 +91-9436459622 ;; fax: fax: +91-0381-2346-630. +91-0381-2346-630. E-mail address:author. [email protected] E-mail E-mail address: address: [email protected] [email protected] c 2018 The Authors. Published by Elsevier Ltd. 1877-0509 cc 2018 1877-0509 The Authors. Published by Ltd. This is an open access article under the CC BY-NC-ND license 1877-0509 2018 The Authors. Published by Elsevier Elsevier Ltd. 1877-0509 © 2018 The Authors. Published by Elsevier Ltd.(https://creativecommons.org/licenses/by-nc-nd/4.0/). This is an open access article under the BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/). This isisan article under the CC CC licenselicense (https://creativecommons.org/licenses/by-nc-nd/4.0/). This anopen openaccess access article under the BY-NC-ND CC BY-NC-ND (https://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the scientific committee of the International Conference on Robotics and Smart Manufacturing. 10.1016/j.procs.2018.07.020
162 2
Rahul Raj et al. / Procedia Computer Science 133 (2018) 161–168 Suman Deb / Procedia Computer Science 00 (2018) 000–000
Fig. 1. System Module Interconnection & EEG Setup
enable them by accessing brain signals whose motor nerve is not working (in case of amyotrophic lateral sclerosis (ALS), progressive bulbar palsy, primary lateral sclerosis (PLS)). 1.1. Problem Statement We want specially augmented access mechanism designed particularly for those people whose brain is working but rest of the motor nerves are non-functioning. This is also for brain functional activity analysis, brain function improvement and learning disorder improvement where it can be used for improving the concentration as well as proficiency of the work. 1.2. Proposed Solution Communication is a basic need for people to interact and express their thoughts. In this proposal, we tried to find out the EEG signals which can be interpreted through a customized series of algorithms to interpret signals to operate scanning keyboard. In an onscreen pointer movement and keyboard pressing, the primary requirement is what to press and when to press. These two questions are answered in a binary EEG signal trigger mechanism. The main goal is to develop a BCI system that is easy to use, cost effective and can be used in homes. The accuracy of signal received and processing has been given utmost care. 1.3. Relation of Brain Computer Interface (BCI) with Keyboard Control Human brain is very complex and is made up of about 100 billion neurons. There are many types of neurons in our brain such as motor neurons, sensory neurons etc. Every kind of neurons as and when required generate responses for some stimuli and in this way generating electrical impulses that can be detected by the electrodes and can be used to control and operate several devices after processing. BCIs thus directly uses brain signal and does not require any muscular activity as a control signal. With the help of BCI [4], a direct link between human brain and another physical device is achieved. In this proposal we have used Non-Invasive EEG technique for data collection to achieve the transfer of technology from clinic to outside world and to provide accuracy and ease of use by the means of Virtual Scanning Keyboard which works on the detection of intentional eye blink of the user. 2. Related Works Eye blinking is a phenomenon of closing the eyelid and is a partially self-governing action. Most of researches use eye blinks as an additional element in eye-tracking systems [1]. Sometimes eye blink detection can be used to emulate the behavior of mouse click [2]. There can be some alternate methods like, if the person is not able to fixate and cannot use traditional eye tracking system then a switch can be emulated to behave on eye blinks to enter and correct text [5]. Each kind of text entry, even using a single input as blinks, can be improved (increase of the text entry speed)
et al. / Computer Procedia Computer (2018) 161–168 SumanRahul Deb /Raj Procedia Science 00Science (2018) 133 000–000
163 3
Fig. 2. TGAM Chip Block Diagram & Plot of Brain Signals
implementing words prediction [6]. Most of the researches done in eye blink detection and uses are in some form of image processing mechanisms [17]. In the recent years, a noticeable number of BCI systems have been developed to provide an alternative communication tool for people with severe neuromuscular disorders. Hundreds of BCI research articles were published [18]. A lot of implemented BCI applications are associated with different areas, i.e.: mental speller [23], mouse control [24], robot arm control [25], drowsiness detection system [9], etc. EEG applied to control a virtual keyboard gave results from 0:85 up to CPM = 3:38 (Characters Per Minute) in error-free writing [26]. It has been noticed that the EEG and EMG data can be used in order to improve the accuracy of user input at the same time when using the biometric signal to the computer interface [5]. In this work, we have focused on the correctness of the typed letters per minute and it was found that the average accuracy of all the users have improved from 40% error rate in first round to 10.5% error rate in the fourth round of the trials. The novelty of this work is that we have considered not only the typing of letters but also made the control token move faster if the attention is high for some time, in that way attentive user gets the facility to type in a faster pace. In gaze direction control, user is not able to type faster even if he/she is very attentive. So, we have applied heuristic way to make the character selection faster.
3. Methodology The accomplished work comprises of four major modules as depicted in Fig. 1 (a), and is described in the following sections. 3.1. EEG Data Collection Neurons in the brain when get fired generates electrical signals that can be interpreted in terms of small voltages which comes in terms of millivolts which has a very high efficiency towards getting interfered by external signals. Due to this property of easily interference by outside signals or electrostatic noise, the recorded EEG signals are filtered and often recorded from many lobes of the brain. Now a days most of the BCI recording is done using multi-channel devices such as 17 channel, 32 channel, 128 channel that are depicted in Fig 1 (b) [20][21]. In this proposed work we have used single channel EEG device (Neurosky Mindwave Mobile Headset) to collect user’s EEG data even by knowing the fact that a single channel EEG measurement cannot indicate the area in which EEG is generated and to which direction the EEG spreads [22] because of the following reasons:
Rahul Raj et al. / Procedia Computer Science 133 (2018) 161–168 Suman Deb / Procedia Computer Science 00 (2018) 000–000
164 4
• We are interested in the Attention level and Blink Strength of the person to send command to the application program, not source localization of the signals. • Multi-channel BCIs are very costly and cannot be afforded by everyone. • Device used is easy to use and mobile, so home based application is easily possible. 3.1.1. Neurosky Mindwave Neuroskys MindWave is a low-cost consumer wireless EEG headset that has a dry electrode to record the signal from the person’s scalp and a reference electrode for ground voltage. Various data values are obtained by using this device, such as, raw EEG data, magnitude of frequency bands (Alpha, Beta, Delta, and Gamma), signal quality etc. The headset gets employed with ThinkGear ASIC Module (TGAM) chip (block diagram in fig. 2 (a)) [11] that preprocesses the data within the headset itself.
3.1.2. EEG Frequency Bands Research shows that brain states get changed due to different pattern of neural interactions in brain [3]. EEG is generally described in terms of its frequency band. The variation of amplitude and frequency of the wave represent various brain states [7], which depends on external stimulation and internal mental states [8]. The most familiar classification uses EEG waveform frequency (e.g., Alpha, Beta, Theta, and Delta) [9]. Table I shows how each brainwave frequency type correlated with mental states and conditions. By analyzing these EEG brainwave pattern, it can be used to interpret other mental states such as meditation and attention level of a subject, or even mental disorder such as epilepsy and attention deficit/hyperactivity disorder (ADHD) [10]. Table 1. EEG Related Brain States[9] BrainWave
Frequency
Mental States
Delta Theta Alpha Low Beta Midrange Beta High Beta Gamma
0.1Hz to 3Hz 4Hz to 7Hz 8Hz to 12Hz 12Hz to 15Hz 16Hz to 20Hz 21Hz to 30Hz 30Hz to 100Hz
Non-REM Sleep, Unconscious Fantasy, imaginary, dream Relaxed, tranquil, conscious Relaxed yet focused, integrated Thinking, aware of self and surroundings Alertness, Agitation Higher Mental Activity
In this project work, we are only interested in the integral value of Attention Signal (0-100) and Blink Strength (0255) signal that is provided by the Neurosky MindWave Mobile Headset. ThinkGear ASIC Module (TGAM) sensor of MindWave headset ensures that only Intentional Eye Blinks are captured and Attention values are almost accurate [11]. A plot of real time brain signals is depicted in Fig. 2 (b) [19].
3.2. Signal Processing and Feature Extraction Eye blink play a critical role to people suffering from motor neuron diseases. It helps to control many applications [12] [13] [14]. Every eye blink has certain distinct features such as frequency of operation, amplitude and time elapsed between closing and opening of the eyes. The data that were gathered in real-time was checked using the Java program. If the signal is a Blink, then type the character currently under control token and if the signal shows that attention is above 50 then move the control token faster. Faster movement of the control token facilitates the attentive user to type faster but if the attention is below 50 then control token goes over each cell after 1 second. TGAM chip block diagram in Fig. 2 (a) depicts that both the raw brainwaves and the eSense meters (Attention, Meditation) are calculated on the ThinkGear chip itself. The calculated values are output by the ThinkGear chip, through the headset, to the application program. Data are collected each second and is analyzed in the time domain to detect and correct artifacts as much as possible, without losing the accuracy of the original signal, using Neurosky’s proprietary algorithm.
Rahul Raj et al. / Procedia Computer Science 133 (2018) 161–168 Suman Deb / Procedia Computer Science 00 (2018) 000–000
165 5
3.3. Virtual Scanning Keyboard Prototype A virtual Keyboard comprising of only alphabets and two controls characters namely Space and Delete was developed using Java Applet technology as depicted in Fig. 4 (a). There are 30 cells in which each alphabet occupies 1 cell and control characters occupy 2 cells. The cells are organized in Row-Column fashion and arranged alphabetically. There is a control token that determines which letter to print. As the program starts, the control token starts moving from one token to another sequentially starting from cell A in a circular loop fashion. The system after startup, checks the signal for every second. We are using this system to check the signals after 1 second so as to remove any congestion in the serial communication or Bluetooth channel should remain clear to pass the next data. When the Eye-Blink is detected, the letter in the cell under control token is printed. By default, the control token waits for 1 second at each
Fig. 3. Work Flow of the System
cell and then moves forward. If the signals detected show that Attention level is growing i.e. user is more attentive then the control token automatically starts moving faster over the cells. The outline of the working procedure of the system is depicted by Algorithm 1 and Fig. 3. When the system starts, the Neurosky device needs to be paired with the system to send data over the Bluetooth channel. The received data from the electrode is first filtered out by the TGAM chip which contains some proprietary algorithms of ThinkGear to reduce noise and sends the filtered data. After receiving of the data, the strength of the blink is detected and then accordingly the control token moves and typing takes place. An additional check for the received EEG signal is applied to check the correctness of the data packet received. If the checksum calculated does not match with the checksum received, then the data packet received is discarded. At the startup time, the movement of control token is 1 second/ character of the keyboard. But if the program gets an attention value greater than 50 i.e., the user is attentive, the thread dealing the movement of control token is sent a signal to make the movement faster. 3.3.1. Users for Experiment Four users (male, ages: 21-24) participated in this experiment who were in good health, non-reported cognition illness and brain surgery, not taking medicines or drugs that affects the nervous system. The brain dynamics were recorded through NeuroSky MindWave Mobile in which single electrode was placed on forehead area. The EEG data were wirelessly acquired from FP1 (with reference to ear electrode) with 128 Hz sampling rate as shows in fig. 4 (b) [15]. Users were asked to remain in comfortable sitting posture throughout EEG recording session. Body movements were kept at minimum to reduce the amount of artifacts in EEG. There were four rounds of trials, each of 60 seconds. 60 letters were given to type using eye blink and the letters in a round were same for each user. The data that we recorded was correct typed letters versus incorrect typed letters in the given time constraint. No training was given to check the ease of adaptability of the system for a normal user. The data thus collected are displayed in Fig. 5.
Rahul Raj et al. / Procedia Computer Science 133 (2018) 161–168 Suman Deb / Procedia Computer Science 00 (2018) 000–000
166 6
Algorithm 1 Algorithm to Filter EEG Signal and Pass Control 1: procedure GetSignalPassControl 2: signalRead ← type of signal 3: signalStrength ← strength of signalRead 4: filter the signalRead 5: extract the feature of signalRead using neurosky standard values 6: if signalRead = Blink then 7: print the character under control token 8: wait for 1 second 9: if signalRead = Attention then 10: if signalS trength > 50 then 11: move the control token fast over the cells 12: wait for 1 second
Fig. 4. Keyboard UI & Test Set-up in the Lab
4. Performance Evaluation and Discussion
Table 2. ANOVA Test on the result Anova Single Factor Alpha
0.05
Groups Column1 Column2 Column3 Column4
Count 4 4 4 4
Sum 110 128 157 198
Mean 27.5 32 39.25 49.5
Variance 35.6667 22 14.25 27.6667
Source of Variation Between Groups Within Groups Total
SS 1106.1875 298.75 1404.9375
df 3 12 15
MS 368.729 24.8958
F 14.8109
To achieve the aim of this research, the following objectives were formulated:
P-value 0.00025
critical 3.49029
Rahul Raj et al. / Procedia Computer Science 133 (2018) 161–168 Suman Deb / Procedia Computer Science 00 (2018) 000–000
167 7
Fig. 5. Data Obtained during test
• Capture frontal EEG activities by a single-channel mobile EEG system. It was performed using Neurosky MindWave Mobile headset. To setup it over head is very easy and the feedback from the user about wearing experience was good. • Extract the Blink Strength and Attention signals to perform the typing. It was done using Java Interface and the GUI application. The signal is passed into the application from the Neurosky headset with a delay of 1 second to allow user to blink over the letter he/she wants within the time frame. • Evaluate the performance of the system by analyzing the accuracy of the system. Based upon the data obtained shown in fig. 5 at the time of testing of the application with respect to Correct and Incorrect typed letters in a time bounded environment. Fig. 6(a), depicts the error percentages by each user in each trial. From the figure, two types of results is obtained: i) Average Error rate of Per User in all the trials ii) Average Error rate of All the Users Per trial and these results are displayed in Fig. 6(b). The average error percentages of all the users in first, second, third and fourth trials was 54%, 46%, 34%, 17.5% respectively, and the average error percentages of first, second, third, and fourth user a. Error Percentage in all trials b. Error Average Plot in each trial was 35%, 40%, 47%, 28% respectively. It was found that in each subsequent sesFig. 6. Error Percentages in the trails. sion of test, the performance was getting better. In the first session, there was no training given about the system to the subjects, but in the later sessions the performance was improved that shows that it may take sometime to get proficient with the system and use it efficiently. As detection accuracies of BCIs are around 80% [16], error handling is an important issue for designing BCI applications in general. We have analyzed for nullifiability of the data from Fig. 8 using ANOVA- Single Factor procedure and the results is displayed in Table 2. The f-ratio value is 14.81088 and the p-value is .000245. The result is significant at p<.05. When a probability value is below the α (0.05 in this case) level, the effect is statistically significant and the null hypothesis is rejected. So we can conclude that there is relationship among the data. That means it can be concluded that every new user needs some time to get used to with the system and use it in its full potential. Training sessions can make the user adaptability faster. The results during the experiment revealed the significant operation of the proposal not only for assistive scanning mechanism but also it can be extended for gaming, entertainment etc. 5. Conclusion Brain Computer Interfaced Scanning Keyboard developed here has a long-term potential for effective areas like learning, training, brain stimulation, clinical purposes etc. This is one of its kind which can give an extensive ability to physically challenged people and neuromotor impaired patients. The keyboard interface is also significant for
168 8
Rahul Raj et al. / Procedia Computer Science 133 (2018) 161–168 Suman Deb / Procedia Computer Science 00 (2018) 000–000
expressing people’s thought in terms of written words which can be transitively converted to an action sequence for controlling daily life equipment’s to life saving instructions. The experiments with the prototype accomplish significant improvements in people’s non-verbal interaction ability with machine. This work is envisioned for effective interaction with software control action sequence for combating the high cost devices and complexity of operations. References [1] Andrew Duchowski, “Eye tracking methodology: Theory and practice”, Springer (2017) [2] Luka KrapicKristijan LenacSandi Ljubic, “Integrating blink click interaction into a head tracking system: implementation and usability issues”, Universal Access in the Information Society 14(2), 247-264 (2015) [3] E. Jeong, B. Moon and Y. Lee, “A Platform for Real Time Brain-Waves Analysis System. In Grid and Distributed Computing”, Springer, Heidelberg, vol. 261. pp. 431 437, 2011 [4] Erik Andreas Larsen, “Classification of EEG Signals in a Brain Computer Interface System”, NTNU [5] I. Scott MacKenzie, Behrooz Ashtiani, “Blinkwrite: efficient text entry using eye blinks”, Universal Access in the Information Society 10(1), 69-80 (2011) [6] Alexandru Pasarica, Radu Gabriel Bozomitu, Vlad Cehan, “Eye blinking detection to perform selection for an eye tracking system used in assistive technology”, Design and Technology in Electronic Packaging (SIITME), 2016 IEEE 22nd International Symposium for. pp. 213-216. IEEE (2016) [7] D. Kim, H. Han, S. Cho and U. Chong, “Detection of drowsiness with eyes open using EEG-based power spectrum analysis”, Strategic Technology (IFOST), pp. 1-4, 2012 [8] Krzysztof Dobosz, Klaudiusz Stawski, “Touchless Virtual Keyboard Controlled by Eye Blinking and EEG Signals”, International Conference on ManMachine Interactions ICMMI 2017: Man-Machine Interactions 5 pp 52-61 [9] H. Qin and J. Lin, “Detecting driver drowsiness and distraction via FFT”, Signal Processing, Communications and Computing, pp. 1-3, 2011 [10] NeuroSky Inc., “Brain Wave Signal (EEG) of NeuroSky, Inc.” 15 December 2009. [Online]. Available: http://frontiernerds.com/files/neuroskyvs-medical-eeg.pdf. [Accessed 30 January 2018] [11] https://store.neurosky.com/products/eeg-tgam [Accessed 30 January 2018] [12] Sravanth Kumar, Vivek Kumary and Bharat Guptaz, “Feature Extraction from EEG Signal through One Electrode Device for Medical Application”, 2015 1st International Conference on Next Generation Computing Technologies (NGCT-2015) Dehradun, India, 4-5 September 2015 [13] Wojciech SALABUN, Szczecin, “Processing and spectral analysis of the raw EEG signal from the MindWave” [14] Susmita Das, Sayan Kumar Swar, Shrisom Laha, Subhankar Mahindar, Suchetana Halder, KoushikHati, Sandipan Deb, “Design approach of Eye Tracking and Mind Operated Motorized System”, International Journal of Innovative Research in Science, Engineering and Technology (An ISO 3297: 2007 Certified Organization) Vol. 5, Issue 8, August 2016 [15] H. Jasper, “Ten-twenty electrode system of the international federation”, Electroenceph. Clin. Neurophysiol, Vol. 10, pp. 371-375, 1958 [16] B. Reuderink, “Games and Brain-Computer Interfaces: The State of the Art”, Internal Report, 2008 [17] Aleksandra Krolak, Pawel Strumillo, “Eye-blink detection system for human-computer interaction”, Universal Access in the Information Society pp. 1-11 (2012) [18] Han-Jeong Hwang, Soyoun Kim, Soobeom Choi, Chang-Hwan Im, “Eeg-based brain-computer interfaces: a thorough literature survey”, International Journal of Human-Computer Interaction 29(12), 814-826 (2013) [19] Prathibha R, Swetha L, Dr Shobha K R, “Brain Computer Interface: Design and Development of a Smart Robotic Gripper for a Prosthesis Environment” at 2017 International Conference on Networks & Advances in Computational Technologies (NetACT) —20-22 July 2017— Trivandrum 978-1-5090-6590-5/17/$31.00 2017 IEEE [20] http://openbci.com/ [21] http://neurologiclabs.com/ [22] Pulkit Grover, “Fundamental limits on source-localization accuracy of EEG-based neural sensing” at 2016 IEEE International Symposium on Information Theory 978-1-5090-1806-2/16/$31.00 2016 IEEE pp. 1794-1798 [23] Hubert Cecotti, “A self-paced and calibration-less ssvep-based brain-computer interface speller”, IEEE Transactions on Neural Systems and Rehabilitation Engineering 18(2), 1-133 (2010) [24] McFarland DJ, Krusienski DJ, Sarnacki WA, Wolpaw JR., “Emulation of computer mouse control with a noninvasive brain-computer interface”, Journal of neural engineering 5(2), 101 (2008) [25] Chapin JK, Moxon KA, Markowitz RS, Nicolelis MA, “Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex”, Nature neuroscience 2(7), 664-670 (1999) [26] R. Scherer, G.R. Muller, C. Neuper, B. Graimann, G. Pfurtscheller, “An asynchronously controlled eeg-based virtual keyboard: improvement of the spelling rate”, IEEE Transactions on Biomedical Engineering 51(6), 979-984 (2004)
Available online at www.sciencedirect.com
ScienceDirect
Procedia Computer Science 00 (2018) 000–000 Procedia Computer Science (2018) 000–000 Procedia Computer Science 13300 (2018) 161–168 Procedia Computer Science 00 (2018) 000–000
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
International Conference on Robotics and Smart Manufacturing (RoSMa2018) International International Conference Conference on on Robotics Robotics and and Smart Smart Manufacturing Manufacturing (RoSMa2018) (RoSMa2018)
Brain Brain Computer Computer Interfaced Interfaced Single Single Key Key Omni Omni Directional Directional Pointing Pointing and Command System: a Screen Pointing Interface for and Command System: a Screen Pointing Interface for Differently-abled Differently-abled Person Person Rahul Rajaaa , Suman Debb,∗ , Paritosh Bhattacharyac b,∗ b,∗, Paritosh Bhattacharyacc Rahul Raj , Suman Deb Rahul Raj , Suman Deb , Paritosh Bhattacharya a MCA, Department of Computer Science and Engineering, NIT Agartala, Agartala 799046, India a MCA, Department of Computer Science and Engineering, NIT Agartala, Agartala 799046, India a MCA, b Department Department of Computer Science Engineering, Agartala, Agartala 799046, India of Computer Science andand Engineering, NITNIT Agartala, Agartala 799046, India b Department of Computer Science and Engineering, NIT Agartala, Agartala 799046, India b Department c Department of Computer and Engineering, NITAgartala Agartala, Agartala 799046, India ofScience Mathematics, NIT Agartala, 799046, India c Department of Mathematics, NIT Agartala, Agartala 799046, India c Department of Mathematics, NIT Agartala, Agartala 799046, India
Abstract Abstract Abstract Biomedical signals have got very incredible reach to the people outside clinical domain as devices are developed that can be Biomedical signals got incredible reach to the outside clinical domain devices are that can Biomedical signals have got very veryand incredible reach to booming the people peoplearea outside clinicalnowadays. domain as as An devices are developed developed that Human can be be controlled using one’shave imagination this is the very of research effective way of using controlled using imagination is very area of nowadays. way of controlled using one’s one’s(HCI) imagination and this is the the very booming booming area(BCI) of research research nowadays. An An effective effective waywhich of using using Human Computer Interaction is usingand thethis Brain Computer Interaction and Electroencephalogram (EEG) canHuman extend Computer Interaction is the Interaction (BCI) and (EEG) which Computer Interaction (HCI) is using usingspecifically the Brain Brain Computer Computer Interaction (BCI) Generally, and Electroencephalogram Electroencephalogram (EEG) which whichiscan can extend the people’s access to (HCI) use machines for specially abled persons. EEG is a costly matter notextend every the people’s access to use machines specifically for specially abled persons. Generally, EEG is a costly matter which is the access to use machines specifically persons. Generally, EEGthe is possibility a costly matter whichchannel is not not every every timepeople’s affordable by any common person. Here infor thisspecially proposedabled system, we tried to explore of single EEG time affordable any person. in system, we to the channel EEG time affordable by any common common person. Here Here in this this proposed proposed system, we tried tried to explore explore the possibility possibility of single channel EEG through which aby person can communicate to computer according to his/her neurosignal excitement which of aresingle classified as Attenthrough which person can communicate to according to neurosignal excitement which classified Attenthrough which aa and person canstrength communicate to computer computer accordinginto to his/her his/her neurosignal excitement which are are classified as Attention, Meditation Blink and then can be interpreted effective Human Machine Interaction (HMI). The as proposed tion, Meditation and Blink and can into Human Machine Interaction (HMI). The tion, Meditation and brainwaves Blink strength strength and then thenblinks can be be interpreted intoineffective effective Humanusing Machine Interaction (HMI).Mobile The proposed proposed algorithm here uses (voluntary as interpreted a control tool BCI) detected Neurosky Mindwave headset algorithm here uses brainwaves (voluntary blinks as a control tool in BCI) detected using Neurosky Mindwave Mobile algorithm here uses brainwaves (voluntary blinks as a control tool in BCI) detected using Neurosky Mindwave Mobile headset from the frontal lobe (because it offers EEG clarity since this is the forehead area with minimal hair). This proposed systemheadset can be from the lobe (because it is area with proposed can from the frontal frontal lobe gaming, (becauseentertainment it offers offers EEG EEG clarity clarity since this is the the forehead areapurposes. with minimal minimal hair). This proposedassystem system can be be extended for typing, as wellsince as forthis daily lifeforehead interaction It canhair). also This be extended a non-verbal extended for typing, gaming, entertainment as well as for daily life interaction purposes. It can also be extended as a non-verbal extended for typing, gaming, entertainment well asThe for daily life interaction purposes. It proposed can also be extended as a non-verbal communication medium for specially abled as persons. experiment conducted using the system revealed significant communication medium specially abled persons. The experiment conducted using proposed revealed significant communication medium for for specially abled persons. Thetried experiment conducted using the the proposed system revealed significant development in accessing computer and typing skill. We to explore scan keyboard which can besystem operated by neural signals development in accessing computer typing skill. tried to keyboard development in Neurosky accessing Mindwave computer and and typing skill. We We tried to explore explore scan keyboard which which can can be be operated operated by by neural neural signals signals captured by the Mobile Headset Single channel EEGscan Headset. captured captured by by the the Neurosky Neurosky Mindwave Mindwave Mobile Mobile Headset Headset Single Single channel channel EEG EEG Headset. Headset. c 2018 2018 The The Authors. Authors. Published Published by by Elsevier Elsevier Ltd. Ltd. © 2018 The Authors. Authors. Published by by Elsevier Ltd. Ltd. cc 2018 The This is an license (https://creativecommons.org/licenses/by-nc-nd/4.0/) (https://creativecommons.org/licenses/by-nc-nd/4.0/). open access Published article under Elsevier the CC BY-NC-ND license This is an an open open access article under under the scientific CC BY-NC-ND BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/). This is access article the CC license (https://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the committee of the International Conference on Robotics and Smart Manufacturing. Keywords: BCI, Eye-Blink, Neurosignal, EEG, Machine Interaction, Assistive typing, Scan Keyboard Keywords: BCI, Keywords: BCI, Eye-Blink, Eye-Blink, Neurosignal, Neurosignal, EEG, EEG, Machine Machine Interaction, Interaction, Assistive Assistive typing, typing, Scan Scan Keyboard Keyboard
1. Introduction 1. 1. Introduction Introduction Development of technology can be a savior for people in their daily life. In many stages of human life there may of can be savior for in daily many human may Development of technology technology canmay be aaoccur saviorwhere for people people in their their daily life. life. In In many stages of human life there may be Development accidental or genetic disorders the brain is functioning well but stages motor of nerves arelife notthere working. be accidental or genetic disorders may occur where the brain is functioning well but motor nerves are not working. be accidental or person geneticfaces disorders occur where the brain is functioning well butlike motor nerves are In that case the lots ofmay problems accessing the present times machines computer. We not are working. trying to In In that that case case the the person person faces faces lots lots of of problems problems accessing accessing the the present present times times machines machines like like computer. computer. We We are are trying trying to to ∗ ∗ ∗
Corresponding author. Tel.: +91-9436459622 ; fax: +91-0381-2346-630. Corresponding Tel.: Corresponding author. Tel.: +91-9436459622 +91-9436459622 ;; fax: fax: +91-0381-2346-630. +91-0381-2346-630. E-mail address:author. [email protected] E-mail E-mail address: address: [email protected] [email protected] c 2018 The Authors. Published by Elsevier Ltd. 1877-0509 cc 2018 1877-0509 The Authors. Published by Ltd. This is an open access article under the CC BY-NC-ND license 1877-0509 2018 The Authors. Published by Elsevier Elsevier Ltd. 1877-0509 © 2018 The Authors. Published by Elsevier Ltd.(https://creativecommons.org/licenses/by-nc-nd/4.0/). This is an open access article under the BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/). This isisan article under the CC CC licenselicense (https://creativecommons.org/licenses/by-nc-nd/4.0/). This anopen openaccess access article under the BY-NC-ND CC BY-NC-ND (https://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the scientific committee of the International Conference on Robotics and Smart Manufacturing. 10.1016/j.procs.2018.07.020
162 2
Rahul Raj et al. / Procedia Computer Science 133 (2018) 161–168 Suman Deb / Procedia Computer Science 00 (2018) 000–000
Fig. 1. System Module Interconnection & EEG Setup
enable them by accessing brain signals whose motor nerve is not working (in case of amyotrophic lateral sclerosis (ALS), progressive bulbar palsy, primary lateral sclerosis (PLS)). 1.1. Problem Statement We want specially augmented access mechanism designed particularly for those people whose brain is working but rest of the motor nerves are non-functioning. This is also for brain functional activity analysis, brain function improvement and learning disorder improvement where it can be used for improving the concentration as well as proficiency of the work. 1.2. Proposed Solution Communication is a basic need for people to interact and express their thoughts. In this proposal, we tried to find out the EEG signals which can be interpreted through a customized series of algorithms to interpret signals to operate scanning keyboard. In an onscreen pointer movement and keyboard pressing, the primary requirement is what to press and when to press. These two questions are answered in a binary EEG signal trigger mechanism. The main goal is to develop a BCI system that is easy to use, cost effective and can be used in homes. The accuracy of signal received and processing has been given utmost care. 1.3. Relation of Brain Computer Interface (BCI) with Keyboard Control Human brain is very complex and is made up of about 100 billion neurons. There are many types of neurons in our brain such as motor neurons, sensory neurons etc. Every kind of neurons as and when required generate responses for some stimuli and in this way generating electrical impulses that can be detected by the electrodes and can be used to control and operate several devices after processing. BCIs thus directly uses brain signal and does not require any muscular activity as a control signal. With the help of BCI [4], a direct link between human brain and another physical device is achieved. In this proposal we have used Non-Invasive EEG technique for data collection to achieve the transfer of technology from clinic to outside world and to provide accuracy and ease of use by the means of Virtual Scanning Keyboard which works on the detection of intentional eye blink of the user. 2. Related Works Eye blinking is a phenomenon of closing the eyelid and is a partially self-governing action. Most of researches use eye blinks as an additional element in eye-tracking systems [1]. Sometimes eye blink detection can be used to emulate the behavior of mouse click [2]. There can be some alternate methods like, if the person is not able to fixate and cannot use traditional eye tracking system then a switch can be emulated to behave on eye blinks to enter and correct text [5]. Each kind of text entry, even using a single input as blinks, can be improved (increase of the text entry speed)
et al. / Computer Procedia Computer (2018) 161–168 SumanRahul Deb /Raj Procedia Science 00Science (2018) 133 000–000
163 3
Fig. 2. TGAM Chip Block Diagram & Plot of Brain Signals
implementing words prediction [6]. Most of the researches done in eye blink detection and uses are in some form of image processing mechanisms [17]. In the recent years, a noticeable number of BCI systems have been developed to provide an alternative communication tool for people with severe neuromuscular disorders. Hundreds of BCI research articles were published [18]. A lot of implemented BCI applications are associated with different areas, i.e.: mental speller [23], mouse control [24], robot arm control [25], drowsiness detection system [9], etc. EEG applied to control a virtual keyboard gave results from 0:85 up to CPM = 3:38 (Characters Per Minute) in error-free writing [26]. It has been noticed that the EEG and EMG data can be used in order to improve the accuracy of user input at the same time when using the biometric signal to the computer interface [5]. In this work, we have focused on the correctness of the typed letters per minute and it was found that the average accuracy of all the users have improved from 40% error rate in first round to 10.5% error rate in the fourth round of the trials. The novelty of this work is that we have considered not only the typing of letters but also made the control token move faster if the attention is high for some time, in that way attentive user gets the facility to type in a faster pace. In gaze direction control, user is not able to type faster even if he/she is very attentive. So, we have applied heuristic way to make the character selection faster.
3. Methodology The accomplished work comprises of four major modules as depicted in Fig. 1 (a), and is described in the following sections. 3.1. EEG Data Collection Neurons in the brain when get fired generates electrical signals that can be interpreted in terms of small voltages which comes in terms of millivolts which has a very high efficiency towards getting interfered by external signals. Due to this property of easily interference by outside signals or electrostatic noise, the recorded EEG signals are filtered and often recorded from many lobes of the brain. Now a days most of the BCI recording is done using multi-channel devices such as 17 channel, 32 channel, 128 channel that are depicted in Fig 1 (b) [20][21]. In this proposed work we have used single channel EEG device (Neurosky Mindwave Mobile Headset) to collect user’s EEG data even by knowing the fact that a single channel EEG measurement cannot indicate the area in which EEG is generated and to which direction the EEG spreads [22] because of the following reasons:
Rahul Raj et al. / Procedia Computer Science 133 (2018) 161–168 Suman Deb / Procedia Computer Science 00 (2018) 000–000
164 4
• We are interested in the Attention level and Blink Strength of the person to send command to the application program, not source localization of the signals. • Multi-channel BCIs are very costly and cannot be afforded by everyone. • Device used is easy to use and mobile, so home based application is easily possible. 3.1.1. Neurosky Mindwave Neuroskys MindWave is a low-cost consumer wireless EEG headset that has a dry electrode to record the signal from the person’s scalp and a reference electrode for ground voltage. Various data values are obtained by using this device, such as, raw EEG data, magnitude of frequency bands (Alpha, Beta, Delta, and Gamma), signal quality etc. The headset gets employed with ThinkGear ASIC Module (TGAM) chip (block diagram in fig. 2 (a)) [11] that preprocesses the data within the headset itself.
3.1.2. EEG Frequency Bands Research shows that brain states get changed due to different pattern of neural interactions in brain [3]. EEG is generally described in terms of its frequency band. The variation of amplitude and frequency of the wave represent various brain states [7], which depends on external stimulation and internal mental states [8]. The most familiar classification uses EEG waveform frequency (e.g., Alpha, Beta, Theta, and Delta) [9]. Table I shows how each brainwave frequency type correlated with mental states and conditions. By analyzing these EEG brainwave pattern, it can be used to interpret other mental states such as meditation and attention level of a subject, or even mental disorder such as epilepsy and attention deficit/hyperactivity disorder (ADHD) [10]. Table 1. EEG Related Brain States[9] BrainWave
Frequency
Mental States
Delta Theta Alpha Low Beta Midrange Beta High Beta Gamma
0.1Hz to 3Hz 4Hz to 7Hz 8Hz to 12Hz 12Hz to 15Hz 16Hz to 20Hz 21Hz to 30Hz 30Hz to 100Hz
Non-REM Sleep, Unconscious Fantasy, imaginary, dream Relaxed, tranquil, conscious Relaxed yet focused, integrated Thinking, aware of self and surroundings Alertness, Agitation Higher Mental Activity
In this project work, we are only interested in the integral value of Attention Signal (0-100) and Blink Strength (0255) signal that is provided by the Neurosky MindWave Mobile Headset. ThinkGear ASIC Module (TGAM) sensor of MindWave headset ensures that only Intentional Eye Blinks are captured and Attention values are almost accurate [11]. A plot of real time brain signals is depicted in Fig. 2 (b) [19].
3.2. Signal Processing and Feature Extraction Eye blink play a critical role to people suffering from motor neuron diseases. It helps to control many applications [12] [13] [14]. Every eye blink has certain distinct features such as frequency of operation, amplitude and time elapsed between closing and opening of the eyes. The data that were gathered in real-time was checked using the Java program. If the signal is a Blink, then type the character currently under control token and if the signal shows that attention is above 50 then move the control token faster. Faster movement of the control token facilitates the attentive user to type faster but if the attention is below 50 then control token goes over each cell after 1 second. TGAM chip block diagram in Fig. 2 (a) depicts that both the raw brainwaves and the eSense meters (Attention, Meditation) are calculated on the ThinkGear chip itself. The calculated values are output by the ThinkGear chip, through the headset, to the application program. Data are collected each second and is analyzed in the time domain to detect and correct artifacts as much as possible, without losing the accuracy of the original signal, using Neurosky’s proprietary algorithm.
Rahul Raj et al. / Procedia Computer Science 133 (2018) 161–168 Suman Deb / Procedia Computer Science 00 (2018) 000–000
165 5
3.3. Virtual Scanning Keyboard Prototype A virtual Keyboard comprising of only alphabets and two controls characters namely Space and Delete was developed using Java Applet technology as depicted in Fig. 4 (a). There are 30 cells in which each alphabet occupies 1 cell and control characters occupy 2 cells. The cells are organized in Row-Column fashion and arranged alphabetically. There is a control token that determines which letter to print. As the program starts, the control token starts moving from one token to another sequentially starting from cell A in a circular loop fashion. The system after startup, checks the signal for every second. We are using this system to check the signals after 1 second so as to remove any congestion in the serial communication or Bluetooth channel should remain clear to pass the next data. When the Eye-Blink is detected, the letter in the cell under control token is printed. By default, the control token waits for 1 second at each
Fig. 3. Work Flow of the System
cell and then moves forward. If the signals detected show that Attention level is growing i.e. user is more attentive then the control token automatically starts moving faster over the cells. The outline of the working procedure of the system is depicted by Algorithm 1 and Fig. 3. When the system starts, the Neurosky device needs to be paired with the system to send data over the Bluetooth channel. The received data from the electrode is first filtered out by the TGAM chip which contains some proprietary algorithms of ThinkGear to reduce noise and sends the filtered data. After receiving of the data, the strength of the blink is detected and then accordingly the control token moves and typing takes place. An additional check for the received EEG signal is applied to check the correctness of the data packet received. If the checksum calculated does not match with the checksum received, then the data packet received is discarded. At the startup time, the movement of control token is 1 second/ character of the keyboard. But if the program gets an attention value greater than 50 i.e., the user is attentive, the thread dealing the movement of control token is sent a signal to make the movement faster. 3.3.1. Users for Experiment Four users (male, ages: 21-24) participated in this experiment who were in good health, non-reported cognition illness and brain surgery, not taking medicines or drugs that affects the nervous system. The brain dynamics were recorded through NeuroSky MindWave Mobile in which single electrode was placed on forehead area. The EEG data were wirelessly acquired from FP1 (with reference to ear electrode) with 128 Hz sampling rate as shows in fig. 4 (b) [15]. Users were asked to remain in comfortable sitting posture throughout EEG recording session. Body movements were kept at minimum to reduce the amount of artifacts in EEG. There were four rounds of trials, each of 60 seconds. 60 letters were given to type using eye blink and the letters in a round were same for each user. The data that we recorded was correct typed letters versus incorrect typed letters in the given time constraint. No training was given to check the ease of adaptability of the system for a normal user. The data thus collected are displayed in Fig. 5.
Rahul Raj et al. / Procedia Computer Science 133 (2018) 161–168 Suman Deb / Procedia Computer Science 00 (2018) 000–000
166 6
Algorithm 1 Algorithm to Filter EEG Signal and Pass Control 1: procedure GetSignalPassControl 2: signalRead ← type of signal 3: signalStrength ← strength of signalRead 4: filter the signalRead 5: extract the feature of signalRead using neurosky standard values 6: if signalRead = Blink then 7: print the character under control token 8: wait for 1 second 9: if signalRead = Attention then 10: if signalS trength > 50 then 11: move the control token fast over the cells 12: wait for 1 second
Fig. 4. Keyboard UI & Test Set-up in the Lab
4. Performance Evaluation and Discussion
Table 2. ANOVA Test on the result Anova Single Factor Alpha
0.05
Groups Column1 Column2 Column3 Column4
Count 4 4 4 4
Sum 110 128 157 198
Mean 27.5 32 39.25 49.5
Variance 35.6667 22 14.25 27.6667
Source of Variation Between Groups Within Groups Total
SS 1106.1875 298.75 1404.9375
df 3 12 15
MS 368.729 24.8958
F 14.8109
To achieve the aim of this research, the following objectives were formulated:
P-value 0.00025
critical 3.49029
Rahul Raj et al. / Procedia Computer Science 133 (2018) 161–168 Suman Deb / Procedia Computer Science 00 (2018) 000–000
167 7
Fig. 5. Data Obtained during test
• Capture frontal EEG activities by a single-channel mobile EEG system. It was performed using Neurosky MindWave Mobile headset. To setup it over head is very easy and the feedback from the user about wearing experience was good. • Extract the Blink Strength and Attention signals to perform the typing. It was done using Java Interface and the GUI application. The signal is passed into the application from the Neurosky headset with a delay of 1 second to allow user to blink over the letter he/she wants within the time frame. • Evaluate the performance of the system by analyzing the accuracy of the system. Based upon the data obtained shown in fig. 5 at the time of testing of the application with respect to Correct and Incorrect typed letters in a time bounded environment. Fig. 6(a), depicts the error percentages by each user in each trial. From the figure, two types of results is obtained: i) Average Error rate of Per User in all the trials ii) Average Error rate of All the Users Per trial and these results are displayed in Fig. 6(b). The average error percentages of all the users in first, second, third and fourth trials was 54%, 46%, 34%, 17.5% respectively, and the average error percentages of first, second, third, and fourth user a. Error Percentage in all trials b. Error Average Plot in each trial was 35%, 40%, 47%, 28% respectively. It was found that in each subsequent sesFig. 6. Error Percentages in the trails. sion of test, the performance was getting better. In the first session, there was no training given about the system to the subjects, but in the later sessions the performance was improved that shows that it may take sometime to get proficient with the system and use it efficiently. As detection accuracies of BCIs are around 80% [16], error handling is an important issue for designing BCI applications in general. We have analyzed for nullifiability of the data from Fig. 8 using ANOVA- Single Factor procedure and the results is displayed in Table 2. The f-ratio value is 14.81088 and the p-value is .000245. The result is significant at p<.05. When a probability value is below the α (0.05 in this case) level, the effect is statistically significant and the null hypothesis is rejected. So we can conclude that there is relationship among the data. That means it can be concluded that every new user needs some time to get used to with the system and use it in its full potential. Training sessions can make the user adaptability faster. The results during the experiment revealed the significant operation of the proposal not only for assistive scanning mechanism but also it can be extended for gaming, entertainment etc. 5. Conclusion Brain Computer Interfaced Scanning Keyboard developed here has a long-term potential for effective areas like learning, training, brain stimulation, clinical purposes etc. This is one of its kind which can give an extensive ability to physically challenged people and neuromotor impaired patients. The keyboard interface is also significant for
168 8
Rahul Raj et al. / Procedia Computer Science 133 (2018) 161–168 Suman Deb / Procedia Computer Science 00 (2018) 000–000
expressing people’s thought in terms of written words which can be transitively converted to an action sequence for controlling daily life equipment’s to life saving instructions. The experiments with the prototype accomplish significant improvements in people’s non-verbal interaction ability with machine. This work is envisioned for effective interaction with software control action sequence for combating the high cost devices and complexity of operations. References [1] Andrew Duchowski, “Eye tracking methodology: Theory and practice”, Springer (2017) [2] Luka KrapicKristijan LenacSandi Ljubic, “Integrating blink click interaction into a head tracking system: implementation and usability issues”, Universal Access in the Information Society 14(2), 247-264 (2015) [3] E. Jeong, B. Moon and Y. Lee, “A Platform for Real Time Brain-Waves Analysis System. In Grid and Distributed Computing”, Springer, Heidelberg, vol. 261. pp. 431 437, 2011 [4] Erik Andreas Larsen, “Classification of EEG Signals in a Brain Computer Interface System”, NTNU [5] I. Scott MacKenzie, Behrooz Ashtiani, “Blinkwrite: efficient text entry using eye blinks”, Universal Access in the Information Society 10(1), 69-80 (2011) [6] Alexandru Pasarica, Radu Gabriel Bozomitu, Vlad Cehan, “Eye blinking detection to perform selection for an eye tracking system used in assistive technology”, Design and Technology in Electronic Packaging (SIITME), 2016 IEEE 22nd International Symposium for. pp. 213-216. IEEE (2016) [7] D. Kim, H. Han, S. Cho and U. Chong, “Detection of drowsiness with eyes open using EEG-based power spectrum analysis”, Strategic Technology (IFOST), pp. 1-4, 2012 [8] Krzysztof Dobosz, Klaudiusz Stawski, “Touchless Virtual Keyboard Controlled by Eye Blinking and EEG Signals”, International Conference on ManMachine Interactions ICMMI 2017: Man-Machine Interactions 5 pp 52-61 [9] H. Qin and J. Lin, “Detecting driver drowsiness and distraction via FFT”, Signal Processing, Communications and Computing, pp. 1-3, 2011 [10] NeuroSky Inc., “Brain Wave Signal (EEG) of NeuroSky, Inc.” 15 December 2009. [Online]. Available: http://frontiernerds.com/files/neuroskyvs-medical-eeg.pdf. [Accessed 30 January 2018] [11] https://store.neurosky.com/products/eeg-tgam [Accessed 30 January 2018] [12] Sravanth Kumar, Vivek Kumary and Bharat Guptaz, “Feature Extraction from EEG Signal through One Electrode Device for Medical Application”, 2015 1st International Conference on Next Generation Computing Technologies (NGCT-2015) Dehradun, India, 4-5 September 2015 [13] Wojciech SALABUN, Szczecin, “Processing and spectral analysis of the raw EEG signal from the MindWave” [14] Susmita Das, Sayan Kumar Swar, Shrisom Laha, Subhankar Mahindar, Suchetana Halder, KoushikHati, Sandipan Deb, “Design approach of Eye Tracking and Mind Operated Motorized System”, International Journal of Innovative Research in Science, Engineering and Technology (An ISO 3297: 2007 Certified Organization) Vol. 5, Issue 8, August 2016 [15] H. Jasper, “Ten-twenty electrode system of the international federation”, Electroenceph. Clin. Neurophysiol, Vol. 10, pp. 371-375, 1958 [16] B. Reuderink, “Games and Brain-Computer Interfaces: The State of the Art”, Internal Report, 2008 [17] Aleksandra Krolak, Pawel Strumillo, “Eye-blink detection system for human-computer interaction”, Universal Access in the Information Society pp. 1-11 (2012) [18] Han-Jeong Hwang, Soyoun Kim, Soobeom Choi, Chang-Hwan Im, “Eeg-based brain-computer interfaces: a thorough literature survey”, International Journal of Human-Computer Interaction 29(12), 814-826 (2013) [19] Prathibha R, Swetha L, Dr Shobha K R, “Brain Computer Interface: Design and Development of a Smart Robotic Gripper for a Prosthesis Environment” at 2017 International Conference on Networks & Advances in Computational Technologies (NetACT) —20-22 July 2017— Trivandrum 978-1-5090-6590-5/17/$31.00 2017 IEEE [20] http://openbci.com/ [21] http://neurologiclabs.com/ [22] Pulkit Grover, “Fundamental limits on source-localization accuracy of EEG-based neural sensing” at 2016 IEEE International Symposium on Information Theory 978-1-5090-1806-2/16/$31.00 2016 IEEE pp. 1794-1798 [23] Hubert Cecotti, “A self-paced and calibration-less ssvep-based brain-computer interface speller”, IEEE Transactions on Neural Systems and Rehabilitation Engineering 18(2), 1-133 (2010) [24] McFarland DJ, Krusienski DJ, Sarnacki WA, Wolpaw JR., “Emulation of computer mouse control with a noninvasive brain-computer interface”, Journal of neural engineering 5(2), 101 (2008) [25] Chapin JK, Moxon KA, Markowitz RS, Nicolelis MA, “Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex”, Nature neuroscience 2(7), 664-670 (1999) [26] R. Scherer, G.R. Muller, C. Neuper, B. Graimann, G. Pfurtscheller, “An asynchronously controlled eeg-based virtual keyboard: improvement of the spelling rate”, IEEE Transactions on Biomedical Engineering 51(6), 979-984 (2004)