- Email: [email protected]
Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis
G Model
REHAB 1135 1–7 Annals of Physical and Rehabilitation Medicine xxx (2017) xxx–xxx
Available online at
ScienceDirect www.sciencedirect.com
1 2 3 4 5 6 7 8 9
Original article
Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis Guy a,c, Marie-He´le`ne Soriani a,c, Mariane Bruno a,c, The´odore Papadopoulo b,c, Claude Desnuelle a,c,*, Maureen Clerc b,c
Q1 Violaine
a
Centre de re´fe´rence SLA, hoˆpital Pasteur 2, CHU de Nice, 30, avenue Voie-Romaine, 06001 Nice, France E´quipe projet Athena, Inria Sophia Antipolis-Me´diterrane´e, 2004, route des Lucioles, 06902 Sophia Antipolis, France Q2 Universite´ de Coˆte d’Azur, Coˆte d’Azur, France b c
A R T I C L E I N F O
A B S T R A C T
Article history: Received 29 May 2017 Accepted 19 September 2017
Objectives: Amyotrophic lateral sclerosis (ALS), a progressive neurodegenerative disease, restricts patients’ communication capacity a few years after onset. A proof-of-concept of brain–computer interface (BCI) has shown promise in ALS and ‘‘locked-in’’ patients, mostly in pre-clinical studies or with only a few patients, but performance was estimated not high enough to support adoption by people with physical limitation of speech. Here, we evaluated a visual BCI device in a clinical study to determine whether disabled people with multiple deficiencies related to ALS would be able to use BCI to communicate in a daily environment. Methods: After clinical evaluation of physical, cognitive and language capacities, 20 patients with ALS were included. The P300 speller BCI system consisted of electroencephalography acquisition connected to real-time processing software and separate keyboard-display control software. It was equipped with original features such as optimal stopping of flashes and word prediction. The study consisted of two 3-block sessions (copy spelling, free spelling and free use) with the system in several modes of operation to evaluate its usability in terms of effectiveness, efficiency and satisfaction. Results: The system was effective in that all participants successfully achieved all spelling tasks and was efficient in that 65% of participants selected more than 95% of the correct symbols. The mean number of correct symbols selected per minute ranged from 3.6 (without word prediction) to 5.04 (with word prediction). Participants expressed satisfaction: the mean score was 8.7 on a 10-point visual analog scale assessing comfort, ease of use and utility. Patients quickly learned how to operate the system, which did not require much learning effort. Conclusion: With its word prediction and optimal stopping of flashes, which improves information transfer rate, the BCI system may be competitive with alternative communication systems such as eyetrackers. Remaining requirements to improve the device for suitable ergonomic use are in progress. C 2017 Published by Elsevier Masson SAS.
Keywords: Brain–computer interface Amyotrophic lateral sclerosis P300 speller Augmentative and alternative communication
10 11
1. Introduction
12 13 14 15 16 17
Amyotrophic lateral sclerosis (ALS) is a rare, rapidly progressive and devastating fatal neurodegenerative motor neuron disease affecting mostly older people; the disease is clinically characterized by a combination of lower and upper motor neuron degeneration symptoms, with widespread distribution in bulbar, cervical, thoracic, and lumbosacral regions. It commonly starts
* Corresponding author. Centre de re´fe´rence SLA, poˆle neurosciences cliniques, hoˆpital Pasteur 2, CHU de Nice, 30, avenue Voie-Romaine, CS 51069, 06001 Nice cedex 1, France. Fax: +33 492 03 83 26. E-mail address: [email protected] (C. Desnuelle).
with focal onset and spreads inexorably to other anatomical areas 18 [1], leading in a few years to a very severe condition of muscle 19 weakness including limbs, thoracic and bulbar functions. 20 The disease incidence is about 2 to 3/100,000 in Western 21 countries and prevalence about 4 to 6/100,000 (2). The creation of Q322 multidisciplinary ALS centers has greatly improved clinical care in 23 the past decade and enhanced the survival and quality of life of 24 patients. 25 In 25% to 30% of affected individuals in the early stage of ALS, 26 dysarthria occurs as a first or predominant sign [2]. It affects up to 27 70% of patients with limb-onset disease, who gradually lose the 28 ability to communicate orally or by writing, as do all patients with 29 bulbar-onset disease when limbs eventually become affected. We 30
https://doi.org/10.1016/j.rehab.2017.09.004 C 2017 Published by Elsevier Masson SAS. 1877-0657/
Please cite this article in press as: Guy V, et al. Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis. Ann Phys Rehabil Med (2017), https://doi.org/10.1016/j.rehab.2017.09.004
G Model
REHAB 1135 1–7 2
V. Guy et al. / Annals of Physical and Rehabilitation Medicine xxx (2017) xxx–xxx
31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54
have no hard evidence (only grade IV evidence) regarding speech and language management strategies for patients with ALS. The European Federation of the Neurological Societies–ALS guidelines [3] suggest, as a good practice point, assessing communication every 3 to 6 months and the use of appropriate communication support systems. Various oral communication devices and nonverbal strategies used consist of low-tech and high-tech augmentative and alternative communication (AAC) systems. In this study, we investigated the usability of a brain computer interface (BCI) system for typing text for people with ALS. The principle of a BCI is to interpret the electric signals of the brain and translate them into commands. The feasibility of BCI communication has been reported in the past few years for individuals with ALS [4–10]. We evaluated BCI communication in a population of 20 severely disabled patients followed in the ALS Center of Nice University Hospital. The BCI system consisted of a virtual keyboard called the P300 speller [11,12] equipped with optimal stopping of flashes and word prediction, which are expected to improve performance in terms of information transfer rate (ITR). All patients underwent 2 sessions including 3 operating modes of progressively increasing complexity to investigate the usability of the system in terms of effectiveness, efficiency and satisfaction as recommended by the International Organization for Standardization (ISO 9241-1998) [13,14].
55
2. Materials and methods
56
2.1. Population studied
57 58 59 60 61 62 63 64 65 66 67 68 69 70 71
This was a non-invasive non-randomized prospective singlecenter study promoted by Nice University Hospital in accordance with the legal national regulations (approval by the local ethics committee CPP Sud Me´diterrane´e III [ref.2013.01.03 ter] and registered at ClinicalTrials.gov [NCT01897818]). After detailed information about the P300 speller and study, 20 patients who were routinely followed from disease onset in our center were included. Participants meeting the inclusion criteria and not the non-inclusion criteria in Appendix A were included. Oral communication disability was not considered critical to select patients because the goal was not to test BCI as an AAC in a target population but to test whether disabled people with multiple deficiencies related to ALS would be able, in a non-specific environment, to use BCI to communicate according to the concept of usability defined previously.
72
2.2. Experimental design
73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
At inclusion, patients underwent assessment that included general neurological examination, ALS Functional Rating Scale– Revised (FRS-R) [15] and modified Norris bulbar scale [16]. Global cognitive impairment was measured with the mini mental state examination (MMSE) and the frontal assessment battery. Specific psychometric tests were administered to evaluate executive functions (Wisconsin card sorting test, phonemic verbal fluency, Symbol Digit Modalities Test and Trail Making Test A and B), attention (Symbol Digit Modalities Test) and language (French naming test DO80). Mood was evaluated by the State Trait Anxiety Inventory scale and depression by the Montgomery-Asberg Depression Rating Scale. The initial use of the BCI device took place within 2 weeks after the initial assessment. An occupational therapist set up the system and provided explanations to the patient. All stages of the study were performed in a standard room. Participants sat in a comfortable chair or in their own wheelchair 90 cm from the LCD monitor.
Each patient participated in two identical P300 speller sessions 2 to 4 weeks apart. Each session lasted 60 to 90 min and consisted of 3 blocks: copy spelling (block 1), free spelling (block 2), and free use (block 3). At the beginning of each session, participants viewed a short audiovisual explanation about the subsequent experiments while they were wearing an EEG cap (ANT Neuro WaveguardTM, with active electrodes), from which 12 electrodes (Fz, Cz, C3, C4, Pz, P3, P4, Oz, O1, O2, P7, P8, grounded to AFz, average reference) were connected to a Refa8 amplifier (256-Hz sampling rate). Conductive gel was applied to each electrode, with impedances < 10 kOhms. Ability to stare at a screen and execute the instructions to use the device and evocation of a reliable visual P300 response were tested at that time. During calibration, participants successively focused on 10 letters, flashing 20 times. The recorded calibration data was used to train spatial filters and the linear discriminant analysis (LDA) classifier [17]. During the copy-spelling task (block 1 of each session), participants used the P300 speller to type two 10-letter words they overtly chose from a list (Appendix B, Table A1); while typing, participants were provided with cues (each keyboard letter to type briefly highlighted in blue) and feedback (letter highlighted in green if correctly selected by the P300 speller and in red otherwise). During typing, the word was progressively displayed under the keyboard, including possible typing errors. In this task, participants were instructed not to correct possible errors, so that the accuracy metrics were homogeneous. During the free-spelling task (block 2 of each session), participants used the P300 speller to type two 5-letter words (or one 10-letter word) they covertly chose from a list (Appendix B, Table A2); they did not receive any cues or feedback and were again instructed not to correct possible errors. The free-use task (block 3) was optional, depending on the patient’s fatigue and motivation. Participants who chose to perform this block could use the P300 speller freely to type words and sentences of their choice, including punctuation marks, and were also offered the possibility to use word prediction. In word prediction mode, participants could spell character-by-character or select full words suggested on the screen. Error correction was possible by using a backspace key. After each session, patients were asked to answer a questionnaire about satisfaction with the usefulness, comfort, and ease of use of the system on a 10-point visual analog scale (VAS).
91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132
2.3. P300 speller system
133
The P300 speller system was designed at Inria Sophia Antipolis [17]. It consisted of an EEG acquisition and real-time processing software using the OpenViBE platform [18] and a separate keyboard-display control software, both software running on a Windows 7 laptop with a One Intel Quad Core processor i7-3740QM (2.70 GHz, 6 MB cache). The laptop monitor was used to monitor the EEG signal quality, and a separate 2200 1680 1050 LCD monitor was used to display the keyboard. The keyboard had 43 symbols including punctuation marks and a backspace key. A choice of AZERTY (French layout) or ABCDE layout was available. As for all P300 speller keyboards, characters flashed in order to elicit a P300 response for the attended character. Here, the flashing consisted of briefly covering the character with a ‘‘smiley face’’, as this has been shown to elicit stronger P300 responses than simply highlighting the character [19]. The flash duration was 116.7 ms, and the inter-stimulus interval was 183.3 ms. Instead of row-column flashing, characters were flashed in pseudo-random groups, designed to minimize the consecutive flashing of the same characters, and the simultaneous flashing of neighbor characters [20].
134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154
Please cite this article in press as: Guy V, et al. Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis. Ann Phys Rehabil Med (2017), https://doi.org/10.1016/j.rehab.2017.09.004
G Model
REHAB 1135 1–7 V. Guy et al. / Annals of Physical and Rehabilitation Medicine xxx (2017) xxx–xxx
155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186
The timing of each flash was digitally conveyed to the EEG amplifier via a hardware connection. The stream of EEG data was segmented into epochs starting at the flash onset and lasting 500 ms. A real-time signal processing pipeline allowed to detect whether each EEG epoch contained a P300 response (target epoch) or not (non-target epoch). For this, a spatial filter was used (X-DAWN) followed by an LDA classifier [21]. During calibration and copy-spelling modes only, the target and non-target epochs were averaged across trials and displayed as 2 curves on the laptop monitor, because a sanity check of target and non-target epochs had different averaged time courses. The channels selected for display were Pz during calibration and the output of the spatial filter during copy spelling. Fig. 1 depicts the P300 speller system. In our system, the number of flashes per selection varied across sessions, contrary to previous P300 speller studies in ALS, in which the number of flashes per symbol was fixed [22] or adapted to the patient after calibration but fixed across the BCI sessions [8,23,24]. The system had an early stopping procedure [25] based on symbol-based evidence accumulation, in which the keyboard stopped flashing and the selected character was displayed as soon as its evidence exceeded all others by at least 90% (100% evidence indicating that according to its observations, the classifier has 100% confidence that this character is the one being attended). The P300 speller application also had word prediction ability based on the Presage library (http://www.presage.sourceforge.net/ ). This feature allowed for word completion and next word prediction by displaying a list of the 10 most probable words on the right side of the keyboard (Fig. 1c). The same flashing principle was used for full words as for characters, to allow their selection via the P300 detection: ‘‘smiley faces’’ were briefly overlayed on each word. Each full word also included a following blank space.
3
2.4. Statistical analysis
187
Assuming a 75%-point estimate for efficiency and effectiveness scores, for the cohort of more than 250 ALS patients routinely followed in the ALS Center of Nice University Hospital, we deemed it sufficient to include 20 patients in this study. Effectiveness was calculated as the proportion of patients who completed the copy and free spelling (blocks 1 and 2) in both sessions, whatever the performance. The 95% confidence interval (CI) of this score was calculated by the Poisson distribution (fitted to rare events). Efficiency was assessed with data from the second session. The Wilcoxon signed rank test was used for paired comparisons. The ITR was calculated with the Wolpaw formula [26].
188 189 190 191 192 193 194 195 196 197 198 199
3. Results
200
Among 21 participants selected, results for 20 (10 males) were analyzed: one was fatigued during the initial test. Table 1 gives a description of the participants. The mean age was 62, mean ALS FRS-R score 25/48 (range 8–37), and mean modified Norris bulbar score 22/39 (range 1–39). The mean elapsed time since symptoms onset was 6 years (range 1–15). All participants had normal gaze control on clinical examination. At the time of the study, 8 patients were using AAC: an alphabet board or a voice synthesizer with a keyboard or linked to a computer. Five patients had experienced an eye-tracker AAC device before the study, but it was considered inappropriate because of lack of ability to maintain a correct position. For 7 participants, all neuropsychological test results were in the normal ranges. The remaining 13 were unable to perform the entire neuropsychological evaluation because of severe dysarthria and/or writing disability. Despite this, for 10 of
201 202 203 204 205 206 207 208 209 210 211 212 213 214 215
Fig. 1. P300 speller system overview. In our P300 speller system (a), the user was provided with a keyboard displayed on a monitor connected to a single laptop running EEG signal acquisition, processing, classification and keyboard management. The laptop monitor was used to control the EEG signals and (b) the separability of target and nontarget evoked responses; c: the keyboard displayed to the user was equipped with a backspace character for correcting mistakes in free-use mode; c: example of a sentence typed during free use.
Please cite this article in press as: Guy V, et al. Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis. Ann Phys Rehabil Med (2017), https://doi.org/10.1016/j.rehab.2017.09.004
G Model
REHAB 1135 1–7 V. Guy et al. / Annals of Physical and Rehabilitation Medicine xxx (2017) xxx–xxx
4 Table 1
Q4 Demographic and clinical characteristics of participants with amyotrophic lateral sclerosis (ALS). Patient no.
Age (years)
Disease duration (years)
ALS FRS-R score/48
Norris score/39
El Escorial-R diagnostic criteria at inclusion
Trunk hypotonia
Alternative communication device
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
40 63 63 59 42 55 62 37 71 54 65 62 53 89 50 90 66 53 74 66
4 9 9 8 9 15 4 7 5 11 2 4 4 8 13 1 3 2 2 1
12 16 27 21 26 22 35 11 27 8 28 31 34 28 21 32 30 25 29 37
4 23 39 22 35 23 18 13 33 8 1 36 29 29 16 21 39 34 20 17
Definite Definite Probable Definite Definite Definite Definite Definite Definite Definite Definite Definite Probable Probable Definite Definite Probable Definite Definite Definite
Y Y N Y N N N N N Y N Y N N Y N N N N N
VS N N N N N VS VS N AB VS N N N VS N N N VS VS
Trunk hypotonia: present = Y, absent = N. Augmentative and alternative communication (AAC) device: AB: alphabet board; N: no AAC device used; VS: voice synthesizer. FRS-R: functional rating scale–revised.
216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234
these 13 participants, the uncompleted results of the performed tests were assessed as normal. One patient showed altered mental flexibility according to the Trail Making Test score, combined with impaired working memory (MMSE), which suggested dysexecutive syndrome. The 17 participants who underwent the naming test (DO80) showed no semantic memory impairment. According to the tests described, no chronic depression was evidenced. During the sessions, the mean installation time for the system was 13 min (range 6–30, variability in terms of the time needed to achieve correct impedance), and the mean calibration phase time was 7 min (range 6–9). In assessing usability, all 20 participants succeeded in completing blocks 1 and 2 for both sessions, for an effectiveness score of 100% (95% CI 82–100). No significant change was observed between the 2 sessions, 2 weeks apart (P = 0.27 for block 1, and P = 0.74 for block 2), so the system was mastered during the first session. In block 1 (‘‘copy spelling’’), more than 90% of symbols were correctly selected by all participants (Fig. 2). Table 2 gives individual accuracies for each of the 3 blocks.
Regarding efficiency, the 20 participants showed a mean of 10 or fewer flashes per selection and for 14, a mean of 5 or fewer (Fig. 3). The spelling accuracy was lower in block 2 (‘‘free spelling’’) than block 1 (‘‘copy spelling’’) probably because in block 2, the participants had to locate on their own the letter/symbol position on the keyboard, no longer guided by the cue received during copy spelling. However, 95% of the participants still achieved > 75% correct symbol selection (Fig. 4), and 65% of the participants, > 95% correct symbol selection. Two participants showed poor free-spelling accuracy (50% or below) (Table 2): for patient 9, because of technical problems with the display, which was not flashing correctly, the system had to be reset and recalibrated after block 2, and for patient 16, who became confused when looking for the letters on the keyboard screen. All 20 participants chose to perform the optional block 3 (‘‘free use’’), in which they could type any sentence of their choice. This block was particularly motivating because they could experience that the P300 speller indeed elicited free communication. The
Table 2 Participant accuracy (correct symbol selection percentage) in the 3-block sessions.
Fig. 2. Copy-spelling efficiency. Spelling accuracy histogram for block 1 (‘‘copy spelling’’). In this task the participants typed two 10-letter words out of a list (Appendix B), and were cued on the letters to select. The proportion of correct letters was above 90% for all participants, and reached 100% for 60% of participants.
Patient no.
Block 1 (copy spelling), %
Block 2 (free spelling), %
Block 3 (free use – no word prediction), %
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
100 95 95 50 95 90 100 100 100 100 100 100 100 90 95 95 95 100 100 100
80 100 95 100 100 75 95 75 45 100 90 90 80 95 90 50 100 100 85 100
75.8 74.5 59.3 94.6 90.9 56.5 93.1 63.2 100.0 100.0 72.1 84.2 100.0 85.7 100.0 94.6 77.8 45.5 85.4 51.3
Please cite this article in press as: Guy V, et al. Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis. Ann Phys Rehabil Med (2017), https://doi.org/10.1016/j.rehab.2017.09.004
235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251
G Model
REHAB 1135 1–7 V. Guy et al. / Annals of Physical and Rehabilitation Medicine xxx (2017) xxx–xxx
Fig. 3. Copy-spelling efficiency. Number of flashes per selection during copy spelling (block 1) determined online by an optimal stopping procedure. All participants experienced between 3 and 10 flashes, and 15 of 20 experienced 5 flashes or less per selection.
5
Fig. 4. Free-spelling efficiency. Spelling accuracy in free spelling (block 2). In this task the participants typed four 5-letter words they covertly chose from a list, without any cue on the letters to select. For 90% of the participants, the proportion of correct letters was over 75% and over 90% for 65% of the participants.
Fig. 5. Free-use effectiveness and efficiency. Spelling rate in block 3 (‘‘free use’’), for 12 participants who typed the same text with and without word prediction displaying the number of correct symbols per minute and showing the effect of word prediction. The information transfer rate (ITR), reported for individual subjects, without word prediction, achieved a mean value of 17.72 bits/min.
252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270
mean number of correct symbols per minute was 5.04 (SD 1.66) and 3.58 (SD 1.64) with and without word prediction. Among the 20 participants, 12 typed the same text with and without word prediction (see Appendix C). The median improvement with word prediction for these 12 patients was 1.18 (SD 0.84) additional correct symbols per minute (Wilcoxon signed rank test P = 0.016). Individual improvement per participant is reported in Fig. 5. Note that the two participants with low spelling accuracy in block 2 (patients 9 and 16) nevertheless showed 100% spelling accuracy in block 3. As an explanation for this improvement, for patient 9, the system was reset, and patient 16 figured out how to find the letters or full words to select on the screen. Patient satisfaction was rated on a VAS for 3 criteria. The mean score for comfort of use, from very uncomfortable to very comfortable (0 to 10) was 8.1 (SD 1.4); ease of use, from very difficult to very easy (0 to 10), 8.4 (SD 1.0); and appreciation of the utility of the system, from totally useless to very useful (0 to 10), 9.75 (SD 0.5). We could not compare the system to use of other high-tech AAC devices because no patient was routinely equipped with such a tool.
271
4. Discussion
272 273
Altogether, the results of our study demonstrate the usability of the P300 speller by disabled patients with ALS. The effectiveness of
the system was 100% because all patients accomplished blocks 1 and 2 in the 2 sessions; for efficiency, 95% of participants achieved up to 75% correct symbol selection (up to 95% for 65% of participants), and the mean satisfaction score was 8.7/10 for usefulness, ease of use and comfort of use. The study was conducted with arelatively largegroup of 20 patients with ALS. Most existing studies involved only a few patients or similar cohorts of 20 to 25 patients [5,23,24,27]. The study by Huggins et al. [28], with 61 patients, was only a survey of the opinion of people with ALS regarding the BCI design, which did not include testing the system. As reported in a meta-analysis [29], few studies have tested BCI in semi-realistic settings. The present study departs from a laboratory setting because the system was set up by an occupational therapist with no prior experience in EEG or BCI, in a regular office area of the hospital facilities. Moreover, the patients used a realistic keyboard, which allowed for free spelling with use of a backspace key to correct errors, and word prediction to accelerate throughput. Behavioral performances have been reported to affect P300 detection [23,30], which could be a source of concern for using the P300 speller, especially because for some patients, speech or writing deficiencies prevented a complete cognitive evaluation. However, a P300 wave was present in all patients, and the system was therefore usable for all of them. BCI was proposed as alternative cognitive assessment tool for non-verbal patients [31].
Please cite this article in press as: Guy V, et al. Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis. Ann Phys Rehabil Med (2017), https://doi.org/10.1016/j.rehab.2017.09.004
274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297
G Model
REHAB 1135 1–7 6
298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363
V. Guy et al. / Annals of Physical and Rehabilitation Medicine xxx (2017) xxx–xxx
To favor high spelling throughput, our system included optimal stopping of flashes and word prediction. Optimal stopping of flashes has been proposed previously in a research context, and in a clinical context, it has been studied offline, from data for which the number of flashes was fixed beforehand [32]. In this study, investigating optimal stopping of flashes in the core of the system, the optimal stopping procedure increased the throughput for all patients. In some other studies, an adaptation stage was used to select the number of flashes. For instance, in the protocol used by McCane et al. [23], symbols flashed between 10 and 30 times before selection (which they estimated offline could have been reduced to between 4 and 22 without detriment to the accuracy). Pasqualotto et al. [24] set the number of flashes to between 5 and 8. Our online optimal stopping procedure was able to determine the optimal number of flashes online, at each individual selection, which helps increase motivation [33]. The average number of flashes per selection ranged from 3 to 10 among our participants (Fig. 3). All participants decided to try word prediction in block 3, and for the 12 who typed out the same text with and without word prediction, the spelling rate (in symbols per minute) was improved with word prediction. For a few patients, word prediction decreased the spelling rate, which could be attributed to their difficulty in scanning the proposed words fast enough and in switching between typing strategies, as can be observed in non-BCI settings with word prediction [34]. Globally, word prediction brings a benefit but may require some extra training to be fully effective. However, the word prediction engine can be trained, and its dictionary and grammar rules updated with the users’ writing, which would increase performance. The participants were all able to operate the system under conditions that compared favorably with the literature. However, our experimental study raised a number of technical and practical issues to be addressed for regular daily use of the system. Our system was designed to be set up by caregivers (and not by EEG or computer science experts), so it should be further simplified to allow anyone to set it up easily [35]. Especially, the installation time – 13 min, on average – should be reduced to < 5 min to be practical. With the current electrode technology, reducing the number of electrodes (we used 12 + 1 ground) would contribute to this goal. Relevant electrode positions should be selected to provide both good signals and good comfort even for patients who need a support for holding their head [8,36]. Alternatively, use of gel-free electrodes would both improve setup time and be viable for long-term daily use, seven days a week. Hence, comfortable and aesthetic headsets with gel-free electrodes should be designed. Wires were used to connect the amplifier to the EEG cap and to a computer running the P300 speller software. For a daily ambulatory use, the EEG system should be lightweight and portable and wirelessly connected to the computer. A comparison by Pasqualotto et al. [24] of BCI and an eye-tracker AAC in ALS favored the eye-tracker because the BCI demanded more effort and was more time-consuming. Their reported ITR was 8.67 and 12.87 bits/min for BCI and the eyetracker. Our study produced a mean ITR of 17.72 bits/min in free use without word prediction (Fig. 5, table on the right). The ITR formula is impossible to apply with word prediction, where letters or entire words of different lengths can be selected. Note that our keyboard has fewer symbols (43 symbols vs. 49 symbols in [24]), which mechanically lowers the ITR. Despite its ergonomic limitations, BCI has the advantage of not imposing strict positioning in front of the screen as with eye-tracker devices, a point of great importance in the context of severely disabled patients with ALS. Regarding the P300 speller, each new user session started with a calibration stage lasting about 7 min. This calibration should be
much faster or should occur in the background, without needing the user’s cooperation. There are many ways to further adapt the system to the user and improve the speed (natural language statistics, online error correction). The tested P300 speller is a standalone keyboard. To be fully useful, it should interact with other applications (Web browsing, gaming, robotics or neuroprosthetics), taking into account the ergonomic constraints involved in the P300 speller paradigm [13,14]. By replacing letters with pictograms, the P300 speller principle can be used for environmental control. The keyboard could also include features such as a ‘‘pause’’ and voice synthesis. Future development will include addressing the above software-related issues, and developing a commercial low-cost EEG headset.
364 365 366 367 368 369 370 371 372 373 374 375 376 377
5. Conclusion
378
In the present study, involving 20 patients with ALS, the P300 speller BCI was effective. It may be considered a reliable input method allowing patients with ALS to communicate with their environment. Compared to previous studies, the key features of our system, such as word prediction and optimal stopping of flashes, allowed for significant gains in efficiency; hence, visual P300-based systems are a possible communication alternative for disabled patients, provided that some practical aspects of the EEG system setup are simplified to improve the satisfaction in use. Additional developments can be made at the hardware and software level, such as in the electrode detection and wire-free transmission of signals, to further enhance performance and usability.
379 380 381 382 383 384 385 386 387 388 389 390
Disclosure of interest
391
The authors declare that they have no competing interest.
392
Acknowledgements
393
This work received financial support from the French ALS Association (ARSLA), ANR CO-ADAPT (ANR09 EMER002) and assistance from the Research Department of Nice University Hospital. The authors thank all the patients who participated in the study and their caregivers for their involvement. Helpful advice from E. Maby and J. Mattout is acknowledged, as is technical help from D. Devlaminck and L. Mahe´. The authors thank A. Metelkina and C. Sakarovitch for statistical analysis and the staff of the Nice ALS Centre for technical help in clinical evaluations.
394 395 396 397 398 399 400 401 402
Appendix A. Supplementary data
403
Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.rehab.2017.09. 004.
404 405 406
References
407
[1] Pradat PF, Kabashi E, Desnuelle C. Deciphering spreading mechanisms in amyotrophic lateral sclerosis: clinical evidence and potential molecular processes. Curr Opin Neurol 2015;28:455–61. [2] Traynor BJ, Codd MB, Corr B, Forde C, Frost E, Hardiman O. Clinical features of amyotrophic lateral sclerosis according to the El Escorial and Airlie House diagnostic criteria: a population-based study. Arch Neurol 2000;57:1171–6. [3] Andersen PM, Borasio GD, Dengler R, Hardiman O, Kollewe K, Leigh PN, et al. EFNS task force on management of amyotrophic lateral sclerosis: guidelines for diagnosing and clinical care of patients and relatives. Eur J Neurol 2005;12: 921–38. [4] Nijboer F, Sellers EW, Mellinger J, Jordan MA, Matuz T, Furdea A, et al. A P300based brain-computer interface for people with amyotrophic lateral sclerosis. Clin Neurophysiol 2008;119:1909–16.
Please cite this article in press as: Guy V, et al. Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis. Ann Phys Rehabil Med (2017), https://doi.org/10.1016/j.rehab.2017.09.004
408 409 410 411 412 413 414 415 416 417 418 419 420
G Model
REHAB 1135 1–7 V. Guy et al. / Annals of Physical and Rehabilitation Medicine xxx (2017) xxx–xxx
421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472
[5] Mak JN, McFarland DJ, Vaughan TM, McCane LM, Tsui PZ, Zeitlin DJ, et al. EEG correlates of P300-based brain–computer interface (BCI) performance in people with amyotrophic lateral sclerosis. J Neural Eng 2012;9:026014. [6] Cipresso P, Carelli L, Solca F, Meazzi D, Meriggi P, Poletti B, et al. The use of P300-based BCIs in amyotrophic lateral sclerosis: from augmentative and alternative communication to cognitive assessment. Brain Behav 2012;4: 479–98. [7] Silvoni S, Cavinato M, Volpato C, Ruf CA, Birbaumer N, Piccione F. Amyotrophic lateral sclerosis progression and stability of brain computer interface communication. Amyotroph Lateral Scler Frontotemporal Degener 2013;56:390–6. [8] McCane LM, Heckman SM, McFarland DJ, Townsend G, Mak JN, Sellers EW, et al. P300-based brain-computer interface (BCI) event-related potentials (ERPs): people with amyotrophic lateral sclerosis (ALS) vs. age-matched controls. Clin Neurophysiol 2015;126:2124–31. [9] Geronimo A, Stephens HE, Schiff SJ, Simmons Z. Acceptance of brain-computer interfaces in amyotrophic lateral sclerosis. Amyotroph Lateral Scler Frontotemporal Degener 2015;3-4:258–64. [10] Riccio A, Holz EM, Arico P, Leotta F, Aloise F, Desideri L, et al. Hybrid P300based brain-computer interface to improve usability for people with severe motor disability: electromyographic signals for error correction during a spelling task. Arch Phys Med Rehabil 2015;96S3:S54–61. [11] Farwell LA, Donchin E. Talking off the top of your head: a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol 1988;70:510–23. [12] Clerc M, Mattout J, Devlaminck D, Papadopoulo T, Guy V, Desnuelle C. Verbal communication through brain computer interfaces. Interspeech; 2013 [14th Annual Conference of the international speech communication association]. [13] Schettini F, Riccio A, Simione L, Liberati G, Caruso M, Frasca V, et al. Assistive device with conventional, alternative and Brain-Computer Interface inputs to enhance interaction with the environment for people with amyotrophic lateral sclerosis: a feasibility and usability study. Arch Phys Med Rehab 2015;96:S46–53. [14] Zickler C, Riccio A, Leotta F, Hillian-Tress S, Halder S, Holz E, et al. A braincomputer interface as input channel for a standard assistive technology interface. Clin EEG Neurosci 2011;42:236–44. [15] Cedarbaum JM, Stambler N, Malta E, Fuller C, Hilt D, Thurmond B, et al. The ALSFRS-R: a revised ALS functional rating scale that incorporates assessments of respiratory function. BDNF ALS Study Group (Phase III). J Neurol Sci 1999;169:13–21. [16] Norris Jr FH, Calenchini PR, Fallat RJ, Panchari S, Jewett B. The administration of guanidine in amyotrophic lateral sclerosis. Neurology 1974;24:721–8. [17] Thomas E, Dauce´ E, Devlaminck D, et al. CoAdapt P300 speller: optimized flashing sequences and online learning, 6th international brain computer interface conference; 2014. [18] Renard Y, Lotte F, Gibert G, Congedo M, Maby E, Delannoy V, et al. OpenViBE: an open-source software platform to design test and use brain-computer interfaces in real and virtual environments Presence: teleoperators and virtual environments, 19. MIT Press; 2010. p. 35–53. [19] Jin J, Allison BZ, Kofmann T, Ku¨bler A, Zhang Y, Wang X, et al. The changing face of P300 BCIs: a comparison of stimulus changes in a P300 BCI involving faces, emotion and movement. PloS One 2012;7:e49688.
7
[20] Thomas E, Clerc M, Carpentier A, Munos R. Optimizing P300-speller sequences by RIP-ping groups apart. International IEEE/EMBS conference on neural engineering; 2013. [21] Rivet B, Souloumiac A, Attina V, Gibert G. xDAWN algorithm to enhance evoked potentials: applications to brain-computer interface. IEEE Trans Biomed Engin 2009;56:2035–43. [22] Geronimo A, Simmons Z, Schiff SJ. Performance predictors of brain-computer interfaces in patients with amyotrophic lateral sclerosis. J Neural Eng 2016; 13:026002. [23] McCane L, Sellers EW, McFarland DJ, Mak JN, Carmack CS, Zeltin D, et al. Braincomputer interface (BCI) evaluation in people with amyotrophic lateral sclerosis. Amyotroph Lateral Scler Frontotemporal Degener 2014;15:207–15. [24] Pasqualotto E, Matuz T, Federici S, Ruf CA, Bartl M, Belardinelli MO, et al. Usability and workload of access technology for people with severe motor impairment: a comparison of brain-computer interfacing and eye tracking. Neurorehabil Neural Repair 2015;10:950–7. [25] Mainsah BO, Collins LM, Colwell KA, Sellers EW, Ryan DB, Caves K, et al. Increasing BCI rates with dynamic stopping towards more practical use: an ALS study. J Neurol Eng 2015;12:016013. [26] Wolpaw JR, Ramoser H, McFarland D, Pfurtscheller G. EEG-based communication: improved accuracy by response verification. IEEE Trans Rehabil Eng 1998;3:326–33. [27] Silvoni S, Volpato C, Cavinato M, Merchetti M, Priftis K, Merico A, et al. P300based brain-computer interface communication: evaluation and follow-up in amyotrophic lateral sclerosis. Front Neurosci 2009;3:60–72. [28] Huggins JE, Wren PA, Gruis KL. What would brain-computer interface users want? Opinions priorities of potential users with amyotrophic lateral sclerosis. Amyotroph Lateral Scler 2011;5:318–24. [29] Marchetti M, Priftis K. Brain-computer interfaces in amyotrophic lateral sclerosis: a metanalysis. Clin Neurophysiol 2015;126:1255–63. [30] Seer C, Fu¨rkotter S, Vogts MB, Lange F, Abdulla S, Dengler R, et al. Executive dysfunctions and event-related brain potential in patients with amyotrophic lateral sclerosis. Front Aging Neurosci 2015;7:225–42. [31] Poletti B, Carelli L, Solca F, Lafronza A, Pedroli E, Faini A, et al. Cognitive assessment in amyotrophic lateral sclerosis by means of P300-brain computer interface: a preliminary study. Amyotroph lateral Scler Frontotemporal Degener 2016;12:1–9. [32] Aloise F, Schettini F, Arico` P, Salinari S, Guger C, Rinsma J, et al. Asynchronous P300-based brain-computer interface to control a virtual environment: initial tests on end users. Clin EEG Neurosci 2011;4:219–24. [33] Mattout J, Perrin M, Bertrand O, Maby E. Improving BCI performance through co-adaptation: applications to the P300-speller. Ann Phys Rehabil Med 2015;58:23–8. [34] Koster HH, Levine S. Learning and performance of able-bodied individuals using scanning systems with and without word prediction. Assist Technol 1994;6:42–53. [35] Liberati G, Pizzimenti A, Simione L, Riccio A, Schettini F, et al. Developing brain-computer interfaces from a user-centered perspective: assessing the needs of persons with amyotrophic lateral sclerosis, caregivers, and professionals. Appl Ergon 2015;50:139–46. [36] Krusienski DJ, Sellers EW, McFarland DJ, Vaughan TM, Wolpaw JR. Toward enhanced P300 speller performance. J Neurosci Methods 2008;167:15–21.
Please cite this article in press as: Guy V, et al. Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis. Ann Phys Rehabil Med (2017), https://doi.org/10.1016/j.rehab.2017.09.004
473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524
REHAB 1135 1–7 Annals of Physical and Rehabilitation Medicine xxx (2017) xxx–xxx
Available online at
ScienceDirect www.sciencedirect.com
1 2 3 4 5 6 7 8 9
Original article
Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis Guy a,c, Marie-He´le`ne Soriani a,c, Mariane Bruno a,c, The´odore Papadopoulo b,c, Claude Desnuelle a,c,*, Maureen Clerc b,c
Q1 Violaine
a
Centre de re´fe´rence SLA, hoˆpital Pasteur 2, CHU de Nice, 30, avenue Voie-Romaine, 06001 Nice, France E´quipe projet Athena, Inria Sophia Antipolis-Me´diterrane´e, 2004, route des Lucioles, 06902 Sophia Antipolis, France Q2 Universite´ de Coˆte d’Azur, Coˆte d’Azur, France b c
A R T I C L E I N F O
A B S T R A C T
Article history: Received 29 May 2017 Accepted 19 September 2017
Objectives: Amyotrophic lateral sclerosis (ALS), a progressive neurodegenerative disease, restricts patients’ communication capacity a few years after onset. A proof-of-concept of brain–computer interface (BCI) has shown promise in ALS and ‘‘locked-in’’ patients, mostly in pre-clinical studies or with only a few patients, but performance was estimated not high enough to support adoption by people with physical limitation of speech. Here, we evaluated a visual BCI device in a clinical study to determine whether disabled people with multiple deficiencies related to ALS would be able to use BCI to communicate in a daily environment. Methods: After clinical evaluation of physical, cognitive and language capacities, 20 patients with ALS were included. The P300 speller BCI system consisted of electroencephalography acquisition connected to real-time processing software and separate keyboard-display control software. It was equipped with original features such as optimal stopping of flashes and word prediction. The study consisted of two 3-block sessions (copy spelling, free spelling and free use) with the system in several modes of operation to evaluate its usability in terms of effectiveness, efficiency and satisfaction. Results: The system was effective in that all participants successfully achieved all spelling tasks and was efficient in that 65% of participants selected more than 95% of the correct symbols. The mean number of correct symbols selected per minute ranged from 3.6 (without word prediction) to 5.04 (with word prediction). Participants expressed satisfaction: the mean score was 8.7 on a 10-point visual analog scale assessing comfort, ease of use and utility. Patients quickly learned how to operate the system, which did not require much learning effort. Conclusion: With its word prediction and optimal stopping of flashes, which improves information transfer rate, the BCI system may be competitive with alternative communication systems such as eyetrackers. Remaining requirements to improve the device for suitable ergonomic use are in progress. C 2017 Published by Elsevier Masson SAS.
Keywords: Brain–computer interface Amyotrophic lateral sclerosis P300 speller Augmentative and alternative communication
10 11
1. Introduction
12 13 14 15 16 17
Amyotrophic lateral sclerosis (ALS) is a rare, rapidly progressive and devastating fatal neurodegenerative motor neuron disease affecting mostly older people; the disease is clinically characterized by a combination of lower and upper motor neuron degeneration symptoms, with widespread distribution in bulbar, cervical, thoracic, and lumbosacral regions. It commonly starts
* Corresponding author. Centre de re´fe´rence SLA, poˆle neurosciences cliniques, hoˆpital Pasteur 2, CHU de Nice, 30, avenue Voie-Romaine, CS 51069, 06001 Nice cedex 1, France. Fax: +33 492 03 83 26. E-mail address: [email protected] (C. Desnuelle).
with focal onset and spreads inexorably to other anatomical areas 18 [1], leading in a few years to a very severe condition of muscle 19 weakness including limbs, thoracic and bulbar functions. 20 The disease incidence is about 2 to 3/100,000 in Western 21 countries and prevalence about 4 to 6/100,000 (2). The creation of Q322 multidisciplinary ALS centers has greatly improved clinical care in 23 the past decade and enhanced the survival and quality of life of 24 patients. 25 In 25% to 30% of affected individuals in the early stage of ALS, 26 dysarthria occurs as a first or predominant sign [2]. It affects up to 27 70% of patients with limb-onset disease, who gradually lose the 28 ability to communicate orally or by writing, as do all patients with 29 bulbar-onset disease when limbs eventually become affected. We 30
https://doi.org/10.1016/j.rehab.2017.09.004 C 2017 Published by Elsevier Masson SAS. 1877-0657/
Please cite this article in press as: Guy V, et al. Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis. Ann Phys Rehabil Med (2017), https://doi.org/10.1016/j.rehab.2017.09.004
G Model
REHAB 1135 1–7 2
V. Guy et al. / Annals of Physical and Rehabilitation Medicine xxx (2017) xxx–xxx
31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54
have no hard evidence (only grade IV evidence) regarding speech and language management strategies for patients with ALS. The European Federation of the Neurological Societies–ALS guidelines [3] suggest, as a good practice point, assessing communication every 3 to 6 months and the use of appropriate communication support systems. Various oral communication devices and nonverbal strategies used consist of low-tech and high-tech augmentative and alternative communication (AAC) systems. In this study, we investigated the usability of a brain computer interface (BCI) system for typing text for people with ALS. The principle of a BCI is to interpret the electric signals of the brain and translate them into commands. The feasibility of BCI communication has been reported in the past few years for individuals with ALS [4–10]. We evaluated BCI communication in a population of 20 severely disabled patients followed in the ALS Center of Nice University Hospital. The BCI system consisted of a virtual keyboard called the P300 speller [11,12] equipped with optimal stopping of flashes and word prediction, which are expected to improve performance in terms of information transfer rate (ITR). All patients underwent 2 sessions including 3 operating modes of progressively increasing complexity to investigate the usability of the system in terms of effectiveness, efficiency and satisfaction as recommended by the International Organization for Standardization (ISO 9241-1998) [13,14].
55
2. Materials and methods
56
2.1. Population studied
57 58 59 60 61 62 63 64 65 66 67 68 69 70 71
This was a non-invasive non-randomized prospective singlecenter study promoted by Nice University Hospital in accordance with the legal national regulations (approval by the local ethics committee CPP Sud Me´diterrane´e III [ref.2013.01.03 ter] and registered at ClinicalTrials.gov [NCT01897818]). After detailed information about the P300 speller and study, 20 patients who were routinely followed from disease onset in our center were included. Participants meeting the inclusion criteria and not the non-inclusion criteria in Appendix A were included. Oral communication disability was not considered critical to select patients because the goal was not to test BCI as an AAC in a target population but to test whether disabled people with multiple deficiencies related to ALS would be able, in a non-specific environment, to use BCI to communicate according to the concept of usability defined previously.
72
2.2. Experimental design
73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
At inclusion, patients underwent assessment that included general neurological examination, ALS Functional Rating Scale– Revised (FRS-R) [15] and modified Norris bulbar scale [16]. Global cognitive impairment was measured with the mini mental state examination (MMSE) and the frontal assessment battery. Specific psychometric tests were administered to evaluate executive functions (Wisconsin card sorting test, phonemic verbal fluency, Symbol Digit Modalities Test and Trail Making Test A and B), attention (Symbol Digit Modalities Test) and language (French naming test DO80). Mood was evaluated by the State Trait Anxiety Inventory scale and depression by the Montgomery-Asberg Depression Rating Scale. The initial use of the BCI device took place within 2 weeks after the initial assessment. An occupational therapist set up the system and provided explanations to the patient. All stages of the study were performed in a standard room. Participants sat in a comfortable chair or in their own wheelchair 90 cm from the LCD monitor.
Each patient participated in two identical P300 speller sessions 2 to 4 weeks apart. Each session lasted 60 to 90 min and consisted of 3 blocks: copy spelling (block 1), free spelling (block 2), and free use (block 3). At the beginning of each session, participants viewed a short audiovisual explanation about the subsequent experiments while they were wearing an EEG cap (ANT Neuro WaveguardTM, with active electrodes), from which 12 electrodes (Fz, Cz, C3, C4, Pz, P3, P4, Oz, O1, O2, P7, P8, grounded to AFz, average reference) were connected to a Refa8 amplifier (256-Hz sampling rate). Conductive gel was applied to each electrode, with impedances < 10 kOhms. Ability to stare at a screen and execute the instructions to use the device and evocation of a reliable visual P300 response were tested at that time. During calibration, participants successively focused on 10 letters, flashing 20 times. The recorded calibration data was used to train spatial filters and the linear discriminant analysis (LDA) classifier [17]. During the copy-spelling task (block 1 of each session), participants used the P300 speller to type two 10-letter words they overtly chose from a list (Appendix B, Table A1); while typing, participants were provided with cues (each keyboard letter to type briefly highlighted in blue) and feedback (letter highlighted in green if correctly selected by the P300 speller and in red otherwise). During typing, the word was progressively displayed under the keyboard, including possible typing errors. In this task, participants were instructed not to correct possible errors, so that the accuracy metrics were homogeneous. During the free-spelling task (block 2 of each session), participants used the P300 speller to type two 5-letter words (or one 10-letter word) they covertly chose from a list (Appendix B, Table A2); they did not receive any cues or feedback and were again instructed not to correct possible errors. The free-use task (block 3) was optional, depending on the patient’s fatigue and motivation. Participants who chose to perform this block could use the P300 speller freely to type words and sentences of their choice, including punctuation marks, and were also offered the possibility to use word prediction. In word prediction mode, participants could spell character-by-character or select full words suggested on the screen. Error correction was possible by using a backspace key. After each session, patients were asked to answer a questionnaire about satisfaction with the usefulness, comfort, and ease of use of the system on a 10-point visual analog scale (VAS).
91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132
2.3. P300 speller system
133
The P300 speller system was designed at Inria Sophia Antipolis [17]. It consisted of an EEG acquisition and real-time processing software using the OpenViBE platform [18] and a separate keyboard-display control software, both software running on a Windows 7 laptop with a One Intel Quad Core processor i7-3740QM (2.70 GHz, 6 MB cache). The laptop monitor was used to monitor the EEG signal quality, and a separate 2200 1680 1050 LCD monitor was used to display the keyboard. The keyboard had 43 symbols including punctuation marks and a backspace key. A choice of AZERTY (French layout) or ABCDE layout was available. As for all P300 speller keyboards, characters flashed in order to elicit a P300 response for the attended character. Here, the flashing consisted of briefly covering the character with a ‘‘smiley face’’, as this has been shown to elicit stronger P300 responses than simply highlighting the character [19]. The flash duration was 116.7 ms, and the inter-stimulus interval was 183.3 ms. Instead of row-column flashing, characters were flashed in pseudo-random groups, designed to minimize the consecutive flashing of the same characters, and the simultaneous flashing of neighbor characters [20].
134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154
Please cite this article in press as: Guy V, et al. Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis. Ann Phys Rehabil Med (2017), https://doi.org/10.1016/j.rehab.2017.09.004
G Model
REHAB 1135 1–7 V. Guy et al. / Annals of Physical and Rehabilitation Medicine xxx (2017) xxx–xxx
155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186
The timing of each flash was digitally conveyed to the EEG amplifier via a hardware connection. The stream of EEG data was segmented into epochs starting at the flash onset and lasting 500 ms. A real-time signal processing pipeline allowed to detect whether each EEG epoch contained a P300 response (target epoch) or not (non-target epoch). For this, a spatial filter was used (X-DAWN) followed by an LDA classifier [21]. During calibration and copy-spelling modes only, the target and non-target epochs were averaged across trials and displayed as 2 curves on the laptop monitor, because a sanity check of target and non-target epochs had different averaged time courses. The channels selected for display were Pz during calibration and the output of the spatial filter during copy spelling. Fig. 1 depicts the P300 speller system. In our system, the number of flashes per selection varied across sessions, contrary to previous P300 speller studies in ALS, in which the number of flashes per symbol was fixed [22] or adapted to the patient after calibration but fixed across the BCI sessions [8,23,24]. The system had an early stopping procedure [25] based on symbol-based evidence accumulation, in which the keyboard stopped flashing and the selected character was displayed as soon as its evidence exceeded all others by at least 90% (100% evidence indicating that according to its observations, the classifier has 100% confidence that this character is the one being attended). The P300 speller application also had word prediction ability based on the Presage library (http://www.presage.sourceforge.net/ ). This feature allowed for word completion and next word prediction by displaying a list of the 10 most probable words on the right side of the keyboard (Fig. 1c). The same flashing principle was used for full words as for characters, to allow their selection via the P300 detection: ‘‘smiley faces’’ were briefly overlayed on each word. Each full word also included a following blank space.
3
2.4. Statistical analysis
187
Assuming a 75%-point estimate for efficiency and effectiveness scores, for the cohort of more than 250 ALS patients routinely followed in the ALS Center of Nice University Hospital, we deemed it sufficient to include 20 patients in this study. Effectiveness was calculated as the proportion of patients who completed the copy and free spelling (blocks 1 and 2) in both sessions, whatever the performance. The 95% confidence interval (CI) of this score was calculated by the Poisson distribution (fitted to rare events). Efficiency was assessed with data from the second session. The Wilcoxon signed rank test was used for paired comparisons. The ITR was calculated with the Wolpaw formula [26].
188 189 190 191 192 193 194 195 196 197 198 199
3. Results
200
Among 21 participants selected, results for 20 (10 males) were analyzed: one was fatigued during the initial test. Table 1 gives a description of the participants. The mean age was 62, mean ALS FRS-R score 25/48 (range 8–37), and mean modified Norris bulbar score 22/39 (range 1–39). The mean elapsed time since symptoms onset was 6 years (range 1–15). All participants had normal gaze control on clinical examination. At the time of the study, 8 patients were using AAC: an alphabet board or a voice synthesizer with a keyboard or linked to a computer. Five patients had experienced an eye-tracker AAC device before the study, but it was considered inappropriate because of lack of ability to maintain a correct position. For 7 participants, all neuropsychological test results were in the normal ranges. The remaining 13 were unable to perform the entire neuropsychological evaluation because of severe dysarthria and/or writing disability. Despite this, for 10 of
201 202 203 204 205 206 207 208 209 210 211 212 213 214 215
Fig. 1. P300 speller system overview. In our P300 speller system (a), the user was provided with a keyboard displayed on a monitor connected to a single laptop running EEG signal acquisition, processing, classification and keyboard management. The laptop monitor was used to control the EEG signals and (b) the separability of target and nontarget evoked responses; c: the keyboard displayed to the user was equipped with a backspace character for correcting mistakes in free-use mode; c: example of a sentence typed during free use.
Please cite this article in press as: Guy V, et al. Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis. Ann Phys Rehabil Med (2017), https://doi.org/10.1016/j.rehab.2017.09.004
G Model
REHAB 1135 1–7 V. Guy et al. / Annals of Physical and Rehabilitation Medicine xxx (2017) xxx–xxx
4 Table 1
Q4 Demographic and clinical characteristics of participants with amyotrophic lateral sclerosis (ALS). Patient no.
Age (years)
Disease duration (years)
ALS FRS-R score/48
Norris score/39
El Escorial-R diagnostic criteria at inclusion
Trunk hypotonia
Alternative communication device
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
40 63 63 59 42 55 62 37 71 54 65 62 53 89 50 90 66 53 74 66
4 9 9 8 9 15 4 7 5 11 2 4 4 8 13 1 3 2 2 1
12 16 27 21 26 22 35 11 27 8 28 31 34 28 21 32 30 25 29 37
4 23 39 22 35 23 18 13 33 8 1 36 29 29 16 21 39 34 20 17
Definite Definite Probable Definite Definite Definite Definite Definite Definite Definite Definite Definite Probable Probable Definite Definite Probable Definite Definite Definite
Y Y N Y N N N N N Y N Y N N Y N N N N N
VS N N N N N VS VS N AB VS N N N VS N N N VS VS
Trunk hypotonia: present = Y, absent = N. Augmentative and alternative communication (AAC) device: AB: alphabet board; N: no AAC device used; VS: voice synthesizer. FRS-R: functional rating scale–revised.
216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234
these 13 participants, the uncompleted results of the performed tests were assessed as normal. One patient showed altered mental flexibility according to the Trail Making Test score, combined with impaired working memory (MMSE), which suggested dysexecutive syndrome. The 17 participants who underwent the naming test (DO80) showed no semantic memory impairment. According to the tests described, no chronic depression was evidenced. During the sessions, the mean installation time for the system was 13 min (range 6–30, variability in terms of the time needed to achieve correct impedance), and the mean calibration phase time was 7 min (range 6–9). In assessing usability, all 20 participants succeeded in completing blocks 1 and 2 for both sessions, for an effectiveness score of 100% (95% CI 82–100). No significant change was observed between the 2 sessions, 2 weeks apart (P = 0.27 for block 1, and P = 0.74 for block 2), so the system was mastered during the first session. In block 1 (‘‘copy spelling’’), more than 90% of symbols were correctly selected by all participants (Fig. 2). Table 2 gives individual accuracies for each of the 3 blocks.
Regarding efficiency, the 20 participants showed a mean of 10 or fewer flashes per selection and for 14, a mean of 5 or fewer (Fig. 3). The spelling accuracy was lower in block 2 (‘‘free spelling’’) than block 1 (‘‘copy spelling’’) probably because in block 2, the participants had to locate on their own the letter/symbol position on the keyboard, no longer guided by the cue received during copy spelling. However, 95% of the participants still achieved > 75% correct symbol selection (Fig. 4), and 65% of the participants, > 95% correct symbol selection. Two participants showed poor free-spelling accuracy (50% or below) (Table 2): for patient 9, because of technical problems with the display, which was not flashing correctly, the system had to be reset and recalibrated after block 2, and for patient 16, who became confused when looking for the letters on the keyboard screen. All 20 participants chose to perform the optional block 3 (‘‘free use’’), in which they could type any sentence of their choice. This block was particularly motivating because they could experience that the P300 speller indeed elicited free communication. The
Table 2 Participant accuracy (correct symbol selection percentage) in the 3-block sessions.
Fig. 2. Copy-spelling efficiency. Spelling accuracy histogram for block 1 (‘‘copy spelling’’). In this task the participants typed two 10-letter words out of a list (Appendix B), and were cued on the letters to select. The proportion of correct letters was above 90% for all participants, and reached 100% for 60% of participants.
Patient no.
Block 1 (copy spelling), %
Block 2 (free spelling), %
Block 3 (free use – no word prediction), %
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
100 95 95 50 95 90 100 100 100 100 100 100 100 90 95 95 95 100 100 100
80 100 95 100 100 75 95 75 45 100 90 90 80 95 90 50 100 100 85 100
75.8 74.5 59.3 94.6 90.9 56.5 93.1 63.2 100.0 100.0 72.1 84.2 100.0 85.7 100.0 94.6 77.8 45.5 85.4 51.3
Please cite this article in press as: Guy V, et al. Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis. Ann Phys Rehabil Med (2017), https://doi.org/10.1016/j.rehab.2017.09.004
235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251
G Model
REHAB 1135 1–7 V. Guy et al. / Annals of Physical and Rehabilitation Medicine xxx (2017) xxx–xxx
Fig. 3. Copy-spelling efficiency. Number of flashes per selection during copy spelling (block 1) determined online by an optimal stopping procedure. All participants experienced between 3 and 10 flashes, and 15 of 20 experienced 5 flashes or less per selection.
5
Fig. 4. Free-spelling efficiency. Spelling accuracy in free spelling (block 2). In this task the participants typed four 5-letter words they covertly chose from a list, without any cue on the letters to select. For 90% of the participants, the proportion of correct letters was over 75% and over 90% for 65% of the participants.
Fig. 5. Free-use effectiveness and efficiency. Spelling rate in block 3 (‘‘free use’’), for 12 participants who typed the same text with and without word prediction displaying the number of correct symbols per minute and showing the effect of word prediction. The information transfer rate (ITR), reported for individual subjects, without word prediction, achieved a mean value of 17.72 bits/min.
252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270
mean number of correct symbols per minute was 5.04 (SD 1.66) and 3.58 (SD 1.64) with and without word prediction. Among the 20 participants, 12 typed the same text with and without word prediction (see Appendix C). The median improvement with word prediction for these 12 patients was 1.18 (SD 0.84) additional correct symbols per minute (Wilcoxon signed rank test P = 0.016). Individual improvement per participant is reported in Fig. 5. Note that the two participants with low spelling accuracy in block 2 (patients 9 and 16) nevertheless showed 100% spelling accuracy in block 3. As an explanation for this improvement, for patient 9, the system was reset, and patient 16 figured out how to find the letters or full words to select on the screen. Patient satisfaction was rated on a VAS for 3 criteria. The mean score for comfort of use, from very uncomfortable to very comfortable (0 to 10) was 8.1 (SD 1.4); ease of use, from very difficult to very easy (0 to 10), 8.4 (SD 1.0); and appreciation of the utility of the system, from totally useless to very useful (0 to 10), 9.75 (SD 0.5). We could not compare the system to use of other high-tech AAC devices because no patient was routinely equipped with such a tool.
271
4. Discussion
272 273
Altogether, the results of our study demonstrate the usability of the P300 speller by disabled patients with ALS. The effectiveness of
the system was 100% because all patients accomplished blocks 1 and 2 in the 2 sessions; for efficiency, 95% of participants achieved up to 75% correct symbol selection (up to 95% for 65% of participants), and the mean satisfaction score was 8.7/10 for usefulness, ease of use and comfort of use. The study was conducted with arelatively largegroup of 20 patients with ALS. Most existing studies involved only a few patients or similar cohorts of 20 to 25 patients [5,23,24,27]. The study by Huggins et al. [28], with 61 patients, was only a survey of the opinion of people with ALS regarding the BCI design, which did not include testing the system. As reported in a meta-analysis [29], few studies have tested BCI in semi-realistic settings. The present study departs from a laboratory setting because the system was set up by an occupational therapist with no prior experience in EEG or BCI, in a regular office area of the hospital facilities. Moreover, the patients used a realistic keyboard, which allowed for free spelling with use of a backspace key to correct errors, and word prediction to accelerate throughput. Behavioral performances have been reported to affect P300 detection [23,30], which could be a source of concern for using the P300 speller, especially because for some patients, speech or writing deficiencies prevented a complete cognitive evaluation. However, a P300 wave was present in all patients, and the system was therefore usable for all of them. BCI was proposed as alternative cognitive assessment tool for non-verbal patients [31].
Please cite this article in press as: Guy V, et al. Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis. Ann Phys Rehabil Med (2017), https://doi.org/10.1016/j.rehab.2017.09.004
274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297
G Model
REHAB 1135 1–7 6
298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363
V. Guy et al. / Annals of Physical and Rehabilitation Medicine xxx (2017) xxx–xxx
To favor high spelling throughput, our system included optimal stopping of flashes and word prediction. Optimal stopping of flashes has been proposed previously in a research context, and in a clinical context, it has been studied offline, from data for which the number of flashes was fixed beforehand [32]. In this study, investigating optimal stopping of flashes in the core of the system, the optimal stopping procedure increased the throughput for all patients. In some other studies, an adaptation stage was used to select the number of flashes. For instance, in the protocol used by McCane et al. [23], symbols flashed between 10 and 30 times before selection (which they estimated offline could have been reduced to between 4 and 22 without detriment to the accuracy). Pasqualotto et al. [24] set the number of flashes to between 5 and 8. Our online optimal stopping procedure was able to determine the optimal number of flashes online, at each individual selection, which helps increase motivation [33]. The average number of flashes per selection ranged from 3 to 10 among our participants (Fig. 3). All participants decided to try word prediction in block 3, and for the 12 who typed out the same text with and without word prediction, the spelling rate (in symbols per minute) was improved with word prediction. For a few patients, word prediction decreased the spelling rate, which could be attributed to their difficulty in scanning the proposed words fast enough and in switching between typing strategies, as can be observed in non-BCI settings with word prediction [34]. Globally, word prediction brings a benefit but may require some extra training to be fully effective. However, the word prediction engine can be trained, and its dictionary and grammar rules updated with the users’ writing, which would increase performance. The participants were all able to operate the system under conditions that compared favorably with the literature. However, our experimental study raised a number of technical and practical issues to be addressed for regular daily use of the system. Our system was designed to be set up by caregivers (and not by EEG or computer science experts), so it should be further simplified to allow anyone to set it up easily [35]. Especially, the installation time – 13 min, on average – should be reduced to < 5 min to be practical. With the current electrode technology, reducing the number of electrodes (we used 12 + 1 ground) would contribute to this goal. Relevant electrode positions should be selected to provide both good signals and good comfort even for patients who need a support for holding their head [8,36]. Alternatively, use of gel-free electrodes would both improve setup time and be viable for long-term daily use, seven days a week. Hence, comfortable and aesthetic headsets with gel-free electrodes should be designed. Wires were used to connect the amplifier to the EEG cap and to a computer running the P300 speller software. For a daily ambulatory use, the EEG system should be lightweight and portable and wirelessly connected to the computer. A comparison by Pasqualotto et al. [24] of BCI and an eye-tracker AAC in ALS favored the eye-tracker because the BCI demanded more effort and was more time-consuming. Their reported ITR was 8.67 and 12.87 bits/min for BCI and the eyetracker. Our study produced a mean ITR of 17.72 bits/min in free use without word prediction (Fig. 5, table on the right). The ITR formula is impossible to apply with word prediction, where letters or entire words of different lengths can be selected. Note that our keyboard has fewer symbols (43 symbols vs. 49 symbols in [24]), which mechanically lowers the ITR. Despite its ergonomic limitations, BCI has the advantage of not imposing strict positioning in front of the screen as with eye-tracker devices, a point of great importance in the context of severely disabled patients with ALS. Regarding the P300 speller, each new user session started with a calibration stage lasting about 7 min. This calibration should be
much faster or should occur in the background, without needing the user’s cooperation. There are many ways to further adapt the system to the user and improve the speed (natural language statistics, online error correction). The tested P300 speller is a standalone keyboard. To be fully useful, it should interact with other applications (Web browsing, gaming, robotics or neuroprosthetics), taking into account the ergonomic constraints involved in the P300 speller paradigm [13,14]. By replacing letters with pictograms, the P300 speller principle can be used for environmental control. The keyboard could also include features such as a ‘‘pause’’ and voice synthesis. Future development will include addressing the above software-related issues, and developing a commercial low-cost EEG headset.
364 365 366 367 368 369 370 371 372 373 374 375 376 377
5. Conclusion
378
In the present study, involving 20 patients with ALS, the P300 speller BCI was effective. It may be considered a reliable input method allowing patients with ALS to communicate with their environment. Compared to previous studies, the key features of our system, such as word prediction and optimal stopping of flashes, allowed for significant gains in efficiency; hence, visual P300-based systems are a possible communication alternative for disabled patients, provided that some practical aspects of the EEG system setup are simplified to improve the satisfaction in use. Additional developments can be made at the hardware and software level, such as in the electrode detection and wire-free transmission of signals, to further enhance performance and usability.
379 380 381 382 383 384 385 386 387 388 389 390
Disclosure of interest
391
The authors declare that they have no competing interest.
392
Acknowledgements
393
This work received financial support from the French ALS Association (ARSLA), ANR CO-ADAPT (ANR09 EMER002) and assistance from the Research Department of Nice University Hospital. The authors thank all the patients who participated in the study and their caregivers for their involvement. Helpful advice from E. Maby and J. Mattout is acknowledged, as is technical help from D. Devlaminck and L. Mahe´. The authors thank A. Metelkina and C. Sakarovitch for statistical analysis and the staff of the Nice ALS Centre for technical help in clinical evaluations.
394 395 396 397 398 399 400 401 402
Appendix A. Supplementary data
403
Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.rehab.2017.09. 004.
404 405 406
References
407
[1] Pradat PF, Kabashi E, Desnuelle C. Deciphering spreading mechanisms in amyotrophic lateral sclerosis: clinical evidence and potential molecular processes. Curr Opin Neurol 2015;28:455–61. [2] Traynor BJ, Codd MB, Corr B, Forde C, Frost E, Hardiman O. Clinical features of amyotrophic lateral sclerosis according to the El Escorial and Airlie House diagnostic criteria: a population-based study. Arch Neurol 2000;57:1171–6. [3] Andersen PM, Borasio GD, Dengler R, Hardiman O, Kollewe K, Leigh PN, et al. EFNS task force on management of amyotrophic lateral sclerosis: guidelines for diagnosing and clinical care of patients and relatives. Eur J Neurol 2005;12: 921–38. [4] Nijboer F, Sellers EW, Mellinger J, Jordan MA, Matuz T, Furdea A, et al. A P300based brain-computer interface for people with amyotrophic lateral sclerosis. Clin Neurophysiol 2008;119:1909–16.
Please cite this article in press as: Guy V, et al. Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis. Ann Phys Rehabil Med (2017), https://doi.org/10.1016/j.rehab.2017.09.004
408 409 410 411 412 413 414 415 416 417 418 419 420
G Model
REHAB 1135 1–7 V. Guy et al. / Annals of Physical and Rehabilitation Medicine xxx (2017) xxx–xxx
421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472
[5] Mak JN, McFarland DJ, Vaughan TM, McCane LM, Tsui PZ, Zeitlin DJ, et al. EEG correlates of P300-based brain–computer interface (BCI) performance in people with amyotrophic lateral sclerosis. J Neural Eng 2012;9:026014. [6] Cipresso P, Carelli L, Solca F, Meazzi D, Meriggi P, Poletti B, et al. The use of P300-based BCIs in amyotrophic lateral sclerosis: from augmentative and alternative communication to cognitive assessment. Brain Behav 2012;4: 479–98. [7] Silvoni S, Cavinato M, Volpato C, Ruf CA, Birbaumer N, Piccione F. Amyotrophic lateral sclerosis progression and stability of brain computer interface communication. Amyotroph Lateral Scler Frontotemporal Degener 2013;56:390–6. [8] McCane LM, Heckman SM, McFarland DJ, Townsend G, Mak JN, Sellers EW, et al. P300-based brain-computer interface (BCI) event-related potentials (ERPs): people with amyotrophic lateral sclerosis (ALS) vs. age-matched controls. Clin Neurophysiol 2015;126:2124–31. [9] Geronimo A, Stephens HE, Schiff SJ, Simmons Z. Acceptance of brain-computer interfaces in amyotrophic lateral sclerosis. Amyotroph Lateral Scler Frontotemporal Degener 2015;3-4:258–64. [10] Riccio A, Holz EM, Arico P, Leotta F, Aloise F, Desideri L, et al. Hybrid P300based brain-computer interface to improve usability for people with severe motor disability: electromyographic signals for error correction during a spelling task. Arch Phys Med Rehabil 2015;96S3:S54–61. [11] Farwell LA, Donchin E. Talking off the top of your head: a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol 1988;70:510–23. [12] Clerc M, Mattout J, Devlaminck D, Papadopoulo T, Guy V, Desnuelle C. Verbal communication through brain computer interfaces. Interspeech; 2013 [14th Annual Conference of the international speech communication association]. [13] Schettini F, Riccio A, Simione L, Liberati G, Caruso M, Frasca V, et al. Assistive device with conventional, alternative and Brain-Computer Interface inputs to enhance interaction with the environment for people with amyotrophic lateral sclerosis: a feasibility and usability study. Arch Phys Med Rehab 2015;96:S46–53. [14] Zickler C, Riccio A, Leotta F, Hillian-Tress S, Halder S, Holz E, et al. A braincomputer interface as input channel for a standard assistive technology interface. Clin EEG Neurosci 2011;42:236–44. [15] Cedarbaum JM, Stambler N, Malta E, Fuller C, Hilt D, Thurmond B, et al. The ALSFRS-R: a revised ALS functional rating scale that incorporates assessments of respiratory function. BDNF ALS Study Group (Phase III). J Neurol Sci 1999;169:13–21. [16] Norris Jr FH, Calenchini PR, Fallat RJ, Panchari S, Jewett B. The administration of guanidine in amyotrophic lateral sclerosis. Neurology 1974;24:721–8. [17] Thomas E, Dauce´ E, Devlaminck D, et al. CoAdapt P300 speller: optimized flashing sequences and online learning, 6th international brain computer interface conference; 2014. [18] Renard Y, Lotte F, Gibert G, Congedo M, Maby E, Delannoy V, et al. OpenViBE: an open-source software platform to design test and use brain-computer interfaces in real and virtual environments Presence: teleoperators and virtual environments, 19. MIT Press; 2010. p. 35–53. [19] Jin J, Allison BZ, Kofmann T, Ku¨bler A, Zhang Y, Wang X, et al. The changing face of P300 BCIs: a comparison of stimulus changes in a P300 BCI involving faces, emotion and movement. PloS One 2012;7:e49688.
7
[20] Thomas E, Clerc M, Carpentier A, Munos R. Optimizing P300-speller sequences by RIP-ping groups apart. International IEEE/EMBS conference on neural engineering; 2013. [21] Rivet B, Souloumiac A, Attina V, Gibert G. xDAWN algorithm to enhance evoked potentials: applications to brain-computer interface. IEEE Trans Biomed Engin 2009;56:2035–43. [22] Geronimo A, Simmons Z, Schiff SJ. Performance predictors of brain-computer interfaces in patients with amyotrophic lateral sclerosis. J Neural Eng 2016; 13:026002. [23] McCane L, Sellers EW, McFarland DJ, Mak JN, Carmack CS, Zeltin D, et al. Braincomputer interface (BCI) evaluation in people with amyotrophic lateral sclerosis. Amyotroph Lateral Scler Frontotemporal Degener 2014;15:207–15. [24] Pasqualotto E, Matuz T, Federici S, Ruf CA, Bartl M, Belardinelli MO, et al. Usability and workload of access technology for people with severe motor impairment: a comparison of brain-computer interfacing and eye tracking. Neurorehabil Neural Repair 2015;10:950–7. [25] Mainsah BO, Collins LM, Colwell KA, Sellers EW, Ryan DB, Caves K, et al. Increasing BCI rates with dynamic stopping towards more practical use: an ALS study. J Neurol Eng 2015;12:016013. [26] Wolpaw JR, Ramoser H, McFarland D, Pfurtscheller G. EEG-based communication: improved accuracy by response verification. IEEE Trans Rehabil Eng 1998;3:326–33. [27] Silvoni S, Volpato C, Cavinato M, Merchetti M, Priftis K, Merico A, et al. P300based brain-computer interface communication: evaluation and follow-up in amyotrophic lateral sclerosis. Front Neurosci 2009;3:60–72. [28] Huggins JE, Wren PA, Gruis KL. What would brain-computer interface users want? Opinions priorities of potential users with amyotrophic lateral sclerosis. Amyotroph Lateral Scler 2011;5:318–24. [29] Marchetti M, Priftis K. Brain-computer interfaces in amyotrophic lateral sclerosis: a metanalysis. Clin Neurophysiol 2015;126:1255–63. [30] Seer C, Fu¨rkotter S, Vogts MB, Lange F, Abdulla S, Dengler R, et al. Executive dysfunctions and event-related brain potential in patients with amyotrophic lateral sclerosis. Front Aging Neurosci 2015;7:225–42. [31] Poletti B, Carelli L, Solca F, Lafronza A, Pedroli E, Faini A, et al. Cognitive assessment in amyotrophic lateral sclerosis by means of P300-brain computer interface: a preliminary study. Amyotroph lateral Scler Frontotemporal Degener 2016;12:1–9. [32] Aloise F, Schettini F, Arico` P, Salinari S, Guger C, Rinsma J, et al. Asynchronous P300-based brain-computer interface to control a virtual environment: initial tests on end users. Clin EEG Neurosci 2011;4:219–24. [33] Mattout J, Perrin M, Bertrand O, Maby E. Improving BCI performance through co-adaptation: applications to the P300-speller. Ann Phys Rehabil Med 2015;58:23–8. [34] Koster HH, Levine S. Learning and performance of able-bodied individuals using scanning systems with and without word prediction. Assist Technol 1994;6:42–53. [35] Liberati G, Pizzimenti A, Simione L, Riccio A, Schettini F, et al. Developing brain-computer interfaces from a user-centered perspective: assessing the needs of persons with amyotrophic lateral sclerosis, caregivers, and professionals. Appl Ergon 2015;50:139–46. [36] Krusienski DJ, Sellers EW, McFarland DJ, Vaughan TM, Wolpaw JR. Toward enhanced P300 speller performance. J Neurosci Methods 2008;167:15–21.
Please cite this article in press as: Guy V, et al. Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis. Ann Phys Rehabil Med (2017), https://doi.org/10.1016/j.rehab.2017.09.004
473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524