Noninvasive Brain–Computer Interfaces

C H A P T E R

26 Noninvasive Brain–Computer Interfaces Gerwin Schalk1,2,3, Brendan Z. Allison1,4 1National

Center for Adaptive Neurotechnologies, Albany, NY, United States; 2Albany Medical College, Albany, NY, United States; 3State University of New York at Albany, Albany, NY, United States; 4University of California San Diego, La Jolla, CA, United States O U T L I N E

Introduction357 Overview of This Chapter 357 Electroencephalography358 Metabolic Activity 360 Brain–Computer Interfaces to Replace Function 361 Introduction361 Communication Functions 361 Simple Communication Functions Complex Communication Functions

Control Functions Computer Functions Worn Robotic Devices Mobile Robotic Devices

Future Directions

361 361

362 362 362 362

363

Brain–Computer Interfaces to Restore Function 363 Introduction363 Devices That Produce Limb Movements 363 Functional Electrical Stimulation 363 Orthoses363

Brain–Computer Interfaces for Restoration Upper Limb Lower Limb

363 363 364

Brain–Computer Interfaces to Enhance Function 365 Introduction365 User State 365 Error Detection 366 Sleep367 Image Recognition 367 Neuromarketing367 Brain–Computer Interfaces to Improve Function 368 Introduction368 Improvements to Motor Function 368 Improvements to Other Functions 370 Summary of the Current State of Noninvasive Brain–Computer Interfaces 371 Scientific and Technical Basis 371 Translating Brain–Computer Interfaces From Scientific Endeavors Into Clinically and Commercially Successful Technologies 371 Commercialization Potential of Various Noninvasive Brain–Computer Interface Technologies 372 Conclusions372 Acknowledgments372 References372

INTRODUCTION Overview of This Chapter Brain–computer interfaces (BCIs) measure brain activity, extract features from that activity, and convert those features into outputs that replace, restore, enhance, supplement, or improve human functions.

Neuromodulation, Second Edition http://dx.doi.org/10.1016/B978-0-12-805353-9.00026-7

BCIs may replace lost functions, such as speaking or moving. They may restore the ability to control the body, such as by stimulating nerves or muscles that move the hand. BCIs have also been used to improve functions, such as training users to improve the remaining function of damaged pathways required to grasp. BCIs can also enhance function, like warning a

357

© 2018 Elsevier Ltd. All rights reserved.

358 (B)

0.4

2 0.2 r

10 cm

(C)

(D) 0.6

20

0.4

10

0.0

r2

(A)

AMPLITUDE (µV)

26.  NONINVASIVE BRAIN–COMPUTER INTERFACES

0.2 0

0

20 40 FREQUENCY (Hz)

60

0.0 0

20 40 FREQUENCY (Hz)

60

FIGURE 26.1  (A and B) The changes in mu activity centered around 12 Hz for (A) actual and (B) imagined right-hand movements. The colors reflect the proportion of the signal variance accounted for by the task. These two images show that imagined movements produce changes that are less pronounced than those resulting from actual movements, but show a similar topographical distribution. (C) EEG power over site C3 for a different subject who rested (dashed line) or performed right-hand movement (solid line). The mu activity at about 12 Hz and its harmonic around 24 Hz are both greatly reduced by movement. (D) The resulting r2 correlations for rest versus movement. This image also shows that movement primarily affects power in the mu frequency bands and its harmonics. From Schalk, G., McFarland, D.J., Hinterberger, T., Birbaumer, N., Wolpaw, J.R., 2004. BCI2000: a general-purpose brain–computer interface (BCI) system. IEEE Trans. Biomed. Eng. 51 (6), 1034–1043.

sleepy driver to wake up. Finally, a BCI might supplement the body’s natural outputs, such as through a third hand. Different techniques are used to measure brain activity for BCIs. Most BCIs have used electrical signals that are detected using electrodes placed invasively within or on the surface of the cortex, or noninvasively on the surface of the scalp [electroencephalography (EEG)]. Some BCIs have been based on metabolic activity that is measured noninvasively, such as through functional magnetic resonance imaging (fMRI). This chapter is focused on providing an overview of noninvasive BCIs. After a brief review of the relevant aspects of EEG and fMRI, each of the subsequent sections is dedicated to one of the four different purposes that a BCI may serve and that have been realized as of this writing.

Electroencephalography EEG sensors detect the coordinated activity of large groups of neurons—the electrical signature of individual or only a few neurons is not detectable by electrodes outside the skull. EEG sensors are usually placed in an electrode cap that is designed to position the electrodes over specific brain regions. Some work has presented EEG electrodes in headbands, headphones, glasses, or other less obtrusive headwear. For many years, EEG electrodes were usually composed of silver/silver chloride rings that were housed in a plastic disk. Electrode gel was needed to establish an electrical connection between the scalp’s surface and each electrode. Work has validated dry electrodes that eliminate the time and inconvenience of gel (Guger et al., 2012; Fridman et al., 2016), but to what extent dry electrodes provide stable EEG, in particular in uncontrolled environments and when used by nonexperts, is still unclear.

Different types of features can be detected in the EEG and may serve as the basis for BCIs. One of the most important of these features is oscillatory activity in different frequency bands: delta (less than 4 Hz), theta (4–8 Hz), alpha (8–12 Hz), beta (18–25 Hz), and gamma (greater than 30 Hz). While the origin of oscillatory activity is still debated, oscillations probably reflect interactions between the cortex and the thalamus or other subcortical structures. Delta activity is most prominent during deep sleep when high-amplitude delta waves can be prevalent over many areas. Theta activity is prevalent during light sleep and meditation. Alpha activity increases over occipital areas when people rest with their eyes closed and during light sleep, and (along with theta and beta) may be used in BCIs to indicate workload or concentration. The phenomenon of “alpha blocking” refers to the decrease in alpha activity that occurs when a person is asked to open the eyes and perform a complex task. Because this is one of the most obvious changes in the EEG that people can easily produce, users are often asked to alternate between eyes-closed relaxation and eyes-open concentration to confirm that their EEG system is working properly. The changes in EEG activity during sleep are driven largely by activity in the pons, thalamus, and occipital regions. Activity in the same alpha frequency range, but detected over sensorimotor instead of visual areas, is called the mu rhythm. The mu rhythm is modulated by expected, actual, observed, or imagined motor movements or associated sensations. These changes in mu activity have been called event-related (de-)synchronization or ERD/S (see Fig. 26.1), and have been widely used in BCIs. Beta and gamma activity is most apparent during concentration and can also include harmonics of mu activity (Pfurtscheller, 1981; Pfurtscheller et al., 1997). These frequency bands have been used in BCIs to detect concentration or information overload. Both bands are often

V.  BRAIN, COMPUTER AND MACHINE INTERFACING

359

Introduction

divided into high and low, and low and high bands can reflect more details of the brain dynamics underlying cognition and emotion. While the source and purpose of the brain’s different oscillatory activities are not fully understood, they seem to generally reflect thalamocortical interactions (primarily through layers 4 and 5 of the cortex) to coordinate activity across different regions and neural populations (Pfurtscheller and Lopes da Silva, 1999). In addition to oscillatory activity that is detected in the frequency domain, electrophysiological activity in the time domain also reveals useful information. When activity is time locked to a stimulus, activity changes following the stimulus are called event-related potentials (ERPs). Because ERPs that result from only a single stimulus are usually too noisy to be detected, both researchers and BCI systems typically repeat the task and associated stimulus several times to acquire several ERPs that can be averaged together, resulting in a clearer signal. ERPs are often named according to their electrical valence (positive or negative) and time in milliseconds from the relevant event. For example, the P300 ERP reflects a positive change in voltage of about 300 ms after an event, and reflects cognitive processing of that event. The P300 has different subcomponents, notably the P3a and P3b, that each reflect different aspects of task processing. The frontally prominent P3a is largest when processing novel stimuli, and reflects attentional alerting and the need to update working memory. The more parietal P3b reflects memory updating and planning a response, such as pressing a button or counting. Concordantly, the sizes of a person’s frontal and parietal areas are correlated with the amplitude of the P3a and P3b, respectively. The P300 reflects contributions from other cortical and subcortical regions as well, including the hippocampus, anterior cingulate, and medial temporal lobes. Earlier components, such as the P100 and N170, instead convey early perceptual processing, and show activity in earlier processing areas such as V1 (primary visual cortex) (Polich, 2004). Time domain activity may also be detected prior to an anticipated event, such as a button press. Before a voluntary movement, the readiness potential (RP; also called Bereitschaftspotential or BP in German) will change across two stages (see Fig. 26.2). About 1.5 s prior to the movement, the supplementary motor area (SMA) and related motor preparation areas exhibit a slow bilateral negative change in voltage. About half a second prior to movement, a much sharper change is apparent contralateral to the movement in the SMA and the primary motor cortex (M1). These two stages seem to reflect movement planning and execution, respectively. The RP is one type of movement-related cortical

MRCP

(A) Bereitschaftspotential

Negative Slope

(B) (C)

EMG

Pre-BP

BP

NS

5µV 200ms

FIGURE 26.2  Different components of the readiness potential, also called the Bereitschaftspotential (BP), are shown. The rightmost vertical line reflects the onset of a voluntary movement. In this image, the voluntary movement was self-paced tapping of the right index finger. The BP phase shows a slowly developing negativity from about 1.5 to 0.5 s prior to the voluntary movement, which becomes more pronounced during the period 0.5 s prior to the movement (Castermans et al., 2013). MRCP, movement-related cortical potential; NS, negative slope.

potential (MRCP), a family of signals that can index movement speed, force, effort, precision, training, complexity, concentration, and other factors (Shibasaki and Hallett, 2006; Xu et al., 2016a,b). Another time domain EEG phenomenon is the contingent negative variation (CNV). The CNV is a bilateral negative change that is prominent over the top of the scalp, and primarily reflects activity from frontal areas. The CNV reflects slow changes, on the order of a few seconds, that can occur between a warning stimulus (which informs someone that a relevant stimulus will soon be shown) and an imperative stimulus (reflecting that someone needs to take action). The CNV was discovered over 50 years ago (Walter et al., 1964) and has been extensively studied. It can reflect a variety of factors, including emotional changes, focused attention, general arousal, and the stimuli’s expectancy and perceived relevance, probability, intensity, and timing. However, it has not been widely used in BCIs because other types of signals described here are generally more reliable, require less training, and allow higher bandwidth communication. Rapid presentation of visual stimuli (such as flickering LEDs or objects on a computer screen) can result in steady-state visual evoked potentials (SSVEPs). SSVEPs reflect the rapid firing of visual cortical areas, primarily V1. If the user focuses attention on one stimulus, EEG signals over visual areas increase in power at that frequency and its harmonics. This allows BCIs to detect which stimulus the user chose to attend to (see Fig. 26.3). If different stimuli are presented at the same frequency but different phases, a BCI may also infer the

V.  BRAIN, COMPUTER AND MACHINE INTERFACING

360

26.  NONINVASIVE BRAIN–COMPUTER INTERFACES

(B)

0.6

0.45

12.00 Hz

15.00 Hz

18.00 Hz

30.00 Hz

45.00 Hz

0.4

0.15

0.2 15

30

0 0

45

(C)

15

30

45

(D) 0.6

0.45

0.6 R2

AMPLITUDE (µV)

6.00 Hz

0.6

0.3 0 0

(E)

R2

AMPLITUDE (µV)

(A)

0.3

0.4

0.15 0 0

0.6

0.2 45 15 30 Frequency (Hz)

0 0

0.4

15 30 45 Frequency (Hz)

0.2 0

FIGURE 26.3  Steady-state visual evoked potential (SSVEP) activity elicited during selective attention to two oscillating checkerboards, each of which oscillated at 6 or 15 Hz. (A and C) Spectral power for one subject over site O1 (A) or O2 (C). The solid and dotted lines show activity elicited while the subject focused on the 15- or 6-Hz checkerboard, respectively. (B and D) The r2 values that reflect the correlation between different frequencies and the instruction to focus on either target stimulus. (E) A topographic map of these differences. It is shown that selective attention to a flickering stimulus increases power at the eliciting frequency and, to a lesser extent, the harmonics of that frequency. The SSVEP activity is much more pronounced over occipital areas than over other sites. From Allison, B.Z., McFarland, D.J., Schalk, G., Zheng, S.D., Jackson, M.M., Wolpaw, J.R., 2008. Towards an independent brain–computer interface using steady state visual evoked potentials. Clin. Neurophysiol. 119 (2), 399–408.

attended stimulus based on phase measurements in the EEG (Nakanishi et al., 2014) or their autocorrelation with an m-sequence in a variant of SSVEPs called code-based VEPs or c-VEPs (Bin et al., 2011). Vibrotactile stimuli can elicit steady-state somatosensory evoked potentials (SSSEPs), and thereby may provide the basis for BCIs for persons without vision (Nam et al., 2013). Steady-state auditory evoked potentials (SSAEPs) have also been studied. Consistent with other somatosensory evoked potentials (SEPs), SSSEPs and SSAEPs involve activity in the corresponding primary cortical sensory area in tandem with higher sensory areas and relevant thalamic nuclei (lateral geniculate, visual; medial geniculate, auditory; ventral posterolateral, somatosensory signals from the body). SEPs have many clinical and research applications, primarily exploring lower-level sensory processes. SEP research has also been used to study schizophrenia, depression, attentional deficits, epilepsy, and other conditions (Norcia et al., 2015).

FIGURE 26.4  These fMRI images show how a person with atten-

Metabolic Activity

tion deficit hyperactivity disorder (ADHD) exhibits different activity compared to a healthy control. Moreover, they show how fMRI can reveal correlates of brain function well below the surface of the cortex, which are difficult or impossible to detect with most other methods. However, as of this writing, fMRI systems are practical only in hospital settings. From ucdmc.ucdavis.edu (2014).

Techniques that measure metabolic activity detect changes in blood oxygenation or other indirect measurements of neuronal activity. Unlike electrical changes that immediately reflect the activity of neuronal populations, metabolic changes typically occur a few seconds after neuronal activity changes. Despite this inherent lag, some BCIs have used metabolic changes in successful demonstrations.

The two most common imaging techniques that can detect metabolic activity are fMRI and positron emission tomography (PET). FMRI and PET are volumetric imaging techniques, i.e., they can detect changes deep in the brain that are invisible to most electrical methods (see Fig. 26.4). At the same time, they require expensive and heavy equipment and they each incur other

V.  BRAIN, COMPUTER AND MACHINE INTERFACING

Brain–Computer Interfaces to Replace Function

361

practical challenges: fMRI requires a very powerful magnetic field that is unsafe for some patients, and PET requires the injection of radioactive tracers. Functional near-infrared spectroscopy (fNIRS) also detects changes in blood flow and does not have these disadvantages. It is safe, portable, and relatively inexpensive, although, like EEG, it is limited to the detection of activity near the brain’s surface. FNIRS requires placing a device on the surface of the scalp that includes an emitter and several detectors. The emitter shines light through the scalp; this light is reflected off of the cortex, and reflection parameters are changed depending on local cortical activity.

patients, even basic communication can confirm conscious awareness (Lesenfants et al., 2016; Ortner et al., 2016). For example, if they can reliably answer yes or no to questions regarding their city of birth or a parent’s name, then doctors and family members have objective proof of the ability and will to communicate. BCIs designed for patients with DOC are designed to interact with patients through auditory and/or tactile stimuli since these patients may be unable to use visual stimuli. These systems often use EEG-based measures of the P300 or motor imagery (Mueller-Putz et al., 2013; Schnuerer et al., 2015; Bauernfeind et al., 2015), though an fNIRS-based system was also demonstrated in 2016 (Hwang et al., 2016).

BRAIN–COMPUTER INTERFACES TO REPLACE FUNCTION

Complex Communication Functions BCIs for spelling often rely on the P300, a positive deflection in the ERP that is dominant over parietal areas and develops about 300 ms after stimuli that convey relevant information and are relatively rare (Polich, 2004; Krusienski et al., 2008). In the first P300 speller (Farwell and Donchin, 1988), healthy users observed a 6 × 6 matrix with letters and other characters, and were asked to silently count each time a target letter flashed. Next, each row or column of the matrix flashed sequentially. Because the users counted only the row flash and column flash that contained the target character, only those two flashes generated a P300. The BCI system could thus identify the target character by analyzing the ERPs generated by each flash. Alternatives to the method of flashing a row or column include the single character, checkerboard, and splotch spellers (Jin et al., 2012; Townsend et al., 2010; Guger et al., 2009). The P300 BCIs work reliably for nearly all healthy people and even ALS patients (Guger et al., 2009; Kaufmann et al., 2013). The P300 BCIs were validated with ALS patients in 2006 (Sellers et al., 2006; Vaughan et al., 2006). Since then, noteworthy advances include brain painting, noted below; the face speller, in which characters change to faces instead of flashing (Jin et al., 2012; Kaufmann et al., 2012, 2013); and nonvisual implementations of similar P300-based systems that can use auditory or tactile stimuli for patients without adequate vision (Severens et al., 2014; Furdea et al., 2009). Cheng et al. (2002) presented an SSVEP BCI system with 12 boxes that each flickered at a different frequency. The numbers 1 through 10 and two special characters were overlaid on the boxes. By focusing on one box, the user could transmit a cell phone number and call that phone. Later work from the same group demonstrated improved performance using a c-VEP approach (Bin et al., 2011), and other work showed that phase information can also improve performance (Nakanishi et al., 2014). SSVEP BCIs work for nearly

Introduction BCIs for replacing lost functions have been explored primarily to help persons with conditions that impair most or all voluntary movements, including persons with late-stage amyotrophic lateral sclerosis (ALS) or tetraplegia. For individuals struck by these conditions, BCIs may replace lost functions (such as communication or movement control) by using brain activity to control an artificial effector (such as a robotic arm or a communication system). The following sections give an overview of BCIs for communication or control that have been developed as of this writing.

Communication Functions Simple Communication Functions The simplest type of communication system entails binary communication, such as answering “yes” or “no” or switching a device on or off. One early BCI system provided control of a switch or a ball on a monitor using EEG signals associated with right-finger movement. Data were acquired from six electrode pairs over frontal and central sites, and the system provided asynchronous operation (Mason and Birch, 2000). Another system allowed users to modulate motor imagery to direct a cursor to answer questions (Miner et al., 1998). In another early study, a group from the US Air Force trained subjects to use SSVEP activity to bank an aircraft or to perform other tasks (Middendorf et al., 2000). BCIs for switch control have continued to develop, with switches based on MRCPs or hybrid fNIRS–EEG activity for wheelchair control (Cao et al., 2014; Koo et al., 2015). BCIs for very basic yes/no communication have gained more attention as tools for persons diagnosed with a disorder of consciousness (DOC). For these

V.  BRAIN, COMPUTER AND MACHINE INTERFACING

362

26.  NONINVASIVE BRAIN–COMPUTER INTERFACES

all healthy adults (Guger et al., 2012), but have not been well explored with patients (but see Lim et al. 2013; Zoltan et al. 2016). One of the most prominent BCI research directions in the late 20th century relied on slow cortical potentials (SCPs). These are very slow drifts in the EEG that patients can learn to increase or decrease over months of training, prominent over central sites. Patients with no residual movement learned to modulate their SCPs to move a cursor to iteratively select letters or letter groups (Birbaumer et al., 1999). SCPs have not been widely used in BCIs for several years because of the long training time and low communication bandwidth. BCIs for spelling based on motor imagery gained attention after work showed that patients with ALS can use motor imagery to control a BCI (Kuebler et al., 2005). Several people, including a patient with tetraplegia, were able to use motor imagery to direct a cursor up or down toward different letters or letter groups on the right side of a monitor while the cursor moved steadily from left to right (Vaughan et al., 2006). In the Hex-OSpell approach (Blankertz et al., 2006), the user views a monitor with a hexagon surrounded by six other hexagons. The central hexagon contains an arrow, while the other hexagons each contain six letters or other characters. At the start of each trial, the arrow begins moving in a clockwise direction. When the arrow points to a hexagon containing the desired group of characters, the user can perform motor imagery (such as left hand grasping) to make the arrow longer until it reaches the desired hexagon. Next, the arrow returns to its starting point, while the other six hexagons’ contents each change to one of the six characters that the user just chose. The arrow begins moving again, and the user can choose one of the six characters in the same fashion. Thus, Hex-OSpell provides an intuitive two-level spelling interface, with clear trial timing and goals, based on simple binary motor control (Severens et al., 2014; Muller et al., 2008; Rohani et al., 2012).

Control Functions Computer Functions The first noninvasive BCI publication described an SSVEP-based BCI in which the user could direct a cursor up, left, down, or right by focusing on one of four oscillating boxes on a monitor (Vidal, 1973). Several groups have described noninvasive BCIs for one-, two-, or three-dimensional cursor control (Li et al., 2010; McFarland et al., 2010; Wolpaw et al., 2003; Muller et al., 2008; Coyle et al., 2011; Scherer et al., 2012; Long et al., 2015).

Cursor movement has been extended to a variety of tasks with noninvasive BCIs, including web browsing (Long et al., 2015; Mugler et al., 2010; Karim et al., 2006) and gaming/virtual navigation (Scherer et al., 2012; Coyle et al., 2011). BCI-based control of smart homes can also implement virtual navigation through a home environment (Edlinger et al., 2011; Carabalona et al., 2012). Another way that BCIs can replace lost functions is through providing a mechanism for creative expression. BCIs have been used to compose music based on EEG measures of emotion (Makeig et al., 2011). The Brain Painting system allows users to create paintings on a monitor through motor imagery or P300 activity (Halder et al., 2009; Muenssinger et al., 2010). Several ALS patients have posted their paintings online, and reported significant enjoyment using the system. Worn Robotic Devices BCIs have been validated for control of wearable robotic devices such as orthoses, prostheses, and exoskeletons. In Pfurtscheller et al. (2010) and Ortner et al. (2011), subjects used SSVEP activity to control a hand orthosis. In addition, in Pfurtscheller et al. (2010), the system also allowed subjects to use mu activity to activate or deactivate LEDs generating the SSVEP. This hybrid approach allowed users to reduce the annoyance caused by flickering stimuli. Related work with BCIs to control functional electrical stimulation, prostheses, and exoskeletons shows potential to both replace natural mobility and facilitate therapy (Angulo-Sherman et al., 2016; Rohm et al., 2014; Rupp, 2014). Mobile Robotic Devices Bell et al. (2008) demonstrated a P300-based BCI system that presented either four or six images that each corresponded to robot control commands. Data were recorded from 32 EEG channels while nine healthy subjects silently counted each time a target image flashed. The overall accuracy across subjects was 98.4%, yielding up to 24 bits/min. Each command could direct a mobile robot equipped with a camera to perform complex actions, relying on the robot’s software to perform the low-level actions needed to navigate around a room, get a glass, or perform other tasks. In Galan et al. (2008), two healthy subjects each participated in five sessions with 10 trials each, during which they used a 64-channel EEG system to drive real and simulated wheelchairs along a predefined path. The BCI allowed three commands based on mental tasks (left-hand imagery, turn left; rest, forward; word association, turn right). The two subjects attained 100% and 80% overall accuracy.

V.  BRAIN, COMPUTER AND MACHINE INTERFACING

Brain–Computer Interfaces to Restore Function

Future Directions Several promising directions for noninvasive BCIs could replace lost functions in different patient groups. For persons with DOC, noninvasive BCIs could go beyond basic assessment and communication to provide more detailed assessment of cognitive function, rehabilitation, outcome prediction, more complex communication such as spelling, and basic environmental control (Li et al., 2016). For example, detection of SSSEPs associated with tactile stimuli could allow several choices for patients who cannot see (Choi et al., 2015). Patients may benefit from BCI systems for bladder or bowel control (Collinger et al., 2013).

BRAIN–COMPUTER INTERFACES TO RESTORE FUNCTION Introduction

363

through transcutaneous or subcutaneous electrodes. Transcutaneous stimulation involves two or more conductive pads that are placed on the surface of the skin such that sending a current between the pads will trigger muscle contraction. This noninvasive approach avoids the cost, need for appropriate medical staff, and other concerns of more invasive approaches. However, because transcutaneous stimulators cannot be positioned with the same precision as invasive stimulators, transcutaneous stimulation is less effective for some muscle groups, and can cause relatively more discomfort and muscle fatigue. Like noninvasive neuroimaging systems, transcutaneous stimulation requires mounting the electrodes in the correct locations before each usage session, which takes several minutes. Invasive subcutaneous electrodes can be mounted permanently, while percutaneous electrodes are typically used for shorter durations.

BCIs may be used to restore a patient’s ability to control his or her body. This category of BCIs aims solely to help persons with disabilities. Unlike BCIs that replace function, which control external devices (such as a robotic arm), BCIs that restore function eventually move the body’s own limbs. The goal is to bypass damaged pathways that connect to functioning effectors, such as the patient’s arms and hands. This technology could benefit people with stroke, brain injury, spinal cord injury, and other conditions that damage the brain or spinal cord. A substantial volume of work has focused on restoring function to the arms and hands. This is a prevalent need for many patients and is a relatively safe research direction (restoring function to lower limbs adds the risk of falling), and stimulating nerves or muscles in the arm to initiate hand grasping may be simpler than producing the extremely intricate and coordinated muscle activations necessary for walking or speaking. The following section gives an overview of devices that have been used to produce limb movements.

Orthoses Orthoses are external, noninvasive devices that are attached to the body to facilitate movement in various ways. Simple orthoses include plastic braces that can be strapped to the foot and ankle that help restore some movement to persons with foot or ankle injuries. Simple orthoses like these have no moving parts, no degrees of freedom, and no need for a control mechanism. More complex orthoses may entail mechanical components that are designed to move the foot, arm, shoulder, or other body part. These systems often have more than 1 degree of freedom, and need some mechanism to control their operation. In principle, BCIs are an appealing control mechanism, as these users already have limited mobility and thus reduced options to control devices. With complex orthoses, BCIs may be used in combination with shared control so that users can simply imagine performing the desired movement (such as walking) and the details of the timing, location, intensity, and duration of muscle stimulation are managed by the BCI system (Ortner et al., 2011; Rohm et al., 2013; Millan et al., 2010).

Devices That Produce Limb Movements

Brain–Computer Interfaces for Restoration

Functional Electrical Stimulation A functional electrical stimulator (FES) is a device to stimulate the muscle groups that control specific movements, such as grasping, wrist dorsiflexion, or knee flexion. Thus, patients who have lost the ability to trigger muscle movement owing to damage to the central nervous system can use a BCI system to bypass these damaged pathways and an FES system to produce movements. FES devices may stimulate muscles

Upper Limb The Freehand neuroprosthetic is an implanted FES device. In relatively early work, it was combined with a 64-channel EEG-based BCI to restore grasp function (Lauer et al., 1999; Peckham and Knutson, 2005). Later work allowed a single patient to use the Graz motor imagery approach, which had already been validated for orthosis control (Pfurtscheller et al., 2000), with the Freehand system (Muller-Putz et al., 2005).

V.  BRAIN, COMPUTER AND MACHINE INTERFACING

364

26.  NONINVASIVE BRAIN–COMPUTER INTERFACES

FIGURE 26.5  (A) A patient using an electrode cap to detect movement imagery that causes a noninvasive (transcutaneous) functional electrical stimulator (FES) system to activate and trigger grasp function. (B) A different patient who instead uses an invasively implanted FES to control grasp. From Muller-Putz, G.R., Scherer, R., Pfurtscheller, G., Rupp, R., 2006. Brain–computer interfaces for control of neuroprostheses: from synchronous to asynchronous mode of operation. Biomed. Tech. (Berl.) 51 (2), 57–63.

FIGURE 26.6  The left (A, top) shows an orthosis that can be opened or closed by moving along the axes shown by white arrows via steadystate visual evoked potential activity elicited by either of the black LEDs (B, left middle). The remaining three images show the orthosis moving throughout three stages of opening and closing. The right shows a healthy volunteer using the orthosis to move a bottle with a white foam shield. From Ortner, R., Allison, B.Z., Korisek, G., Gaggl, H., Pfurtscheller, G., 2011. An SSVEP BCI to control a hand orthosis for persons with tetraplegia. IEEE Trans. Neural Syst. Rehabil. Eng. 19 (1), 1–5.

Fig. 26.5 presents examples of two patients using different types of FES systems with BCIs to restore grasp control. Many groups have explored other issues relating to functional electrical stimulation and upper limb movement. Other early work (Pfurtscheller et al., 2002) compared the efficacy of EEG- to EMG-based control of an orthosis. In one study exploring long-term use, a single C4 spinal cord injury (SCI) patient used EEG-based motor imagery measures to control both an FES and an orthosis to restore hand function. After a year of training, his motor imagery accuracy remained at only 70%, but he still found the device useful (Rohm et al., 2013). For patients who cannot attain good accuracy with motor imagery, SSVEP activity was validated for orthosis control. Seven healthy volunteers without prior training were told to focus on one of two flickering LEDs placed on an orthosis to either open or close it (see

Fig. 26.6). Six volunteers attained good accuracy in an asynchronous control paradigm, although the system produced a high rate of false positives, and some participants did not like the flickering lights (Ortner et al., 2011). Related work addressed these problems with a hybrid motor imagery–SSVEP system for orthosis control. This BCI system allowed subjects to use motor imagery as a “brain switch” to switch the LEDs on or off, thus enabling users to eliminate both problems. Four of six healthy participants attained good accuracy (Pfurtscheller et al., 2010). Ramos-Murguialday et al. (2012) found that proprioceptive feedback led to improved motor imagery control of a robotic exoskeleton (compared to no feedback). Lower Limb BCIs to restore control of the lower limbs often use EEG-based imagination of walking, as this provides a

V.  BRAIN, COMPUTER AND MACHINE INTERFACING

Brain–Computer Interfaces to Enhance Function

365

FIGURE 26.7  An EEG-based system to control a lower-limb exoskeleton being used by a healthy volunteer (left) and a spinal cord injury patient (right). The work with the patient requires safety rails and nearby staff because of the risk of falling. From Lopez-Larraz, E., Trincado-Alonso, F., Rajasekaran, V., Perez-Nombela, S., Del-Ama, A.J., Aranda, J., Minguez, J., Gil-Agudo, A., Montesano, L., August 3, 2016. Control of an ambulatory exoskeleton with a brain–machine interface for spinal cord injury gait rehabilitation. Front. Neurosci. 10, 359. http://dx.doi.org/10.3389/fnins.2016.00359. eCollection 2016.

natural and intuitive mapping between thought and action and is relatively easy to detect with trained users. One patient with paraplegia resulting from SCI learned to either relax or imagine walking while EEG measures of these activities were used to drive a lower-limb orthosis (Do et al., 2013). One study with healthy volunteers validated BCIs to start or stop an exoskeleton by thinking about gait initiation or termination (Hortal et al., 2016a,b). In a proof-of-concept study, three healthy people used EEG-based measures of gait initiation to direct an exoskeleton (LopezLarraz et al., 2016) (see Fig. 26.7). Luu et al. (2016) classified different types of lower-limb movements (hip, knee, and ankle) as people trained to control a virtual avatar’s gait over 8 days. The users’ attention during gait was also explored using different EEG frequency bands, which could influence feedback during system operation (Costa et al., 2016).

BRAIN–COMPUTER INTERFACES TO ENHANCE FUNCTION Introduction The BCIs described so far focus on people with disabilities who seek to replace or restore lost functions. BCIs may eventually also benefit healthy persons by enhancing the capabilities of the central nervous system to enable people to perform tasks better, faster, or more easily. BCIs to enhance function are generally “passive,” meaning that the user does not actively have to perform mental activities devoted to controlling the BCI (such as imagining movements or counting flashes or tones). Instead, passive systems measure activity while the user is performing other tasks. Thus, BCIs to enhance function generally aim to provide additional capabilities without disturbing or distracting the user.

User State A substantial body of work since 2007 has shown that the EEG offers indicators of arousal, engagement, workload, emotional valence, and other factors. Accurate real-time detection of these indicators could be used to modify a user’s ongoing interaction with a computer or another external device. This direction has been explored for decades, but is still largely confined to the laboratory. Jung et al. (1997) detected alpha and theta changes over electrode sites Cz and Pz/Oz while sonar operators performed a simulated sonar detection task, and could predict periods of poor task performance. Lin et al. (2008) demonstrated a wireless four-channel EEG system that could detect alertness lapses and sound a warning. In a simulated driving environment with 11 healthy participants, Wang et al. (2014) discriminated effective from ineffective warning tones using EEG spectral activity. The authors later extended this approach (Huang et al., 2016), which might lead to other mitigation measures for drivers or others who do not notice warning stimuli. In these examples, the BCI enhances natural function by reducing the risk of disaster resulting from alertness lapses. BCIs could also enhance users’ ongoing interaction with software. This approach could reduce errors or keep users engaged, such as by increasing difficulty if users are bored (Millan et al., 2010; Gevins et al., 1995; Zander and Jatzev, 2012; Fairclough, 2009; Nijholt et al., 2009). For example, research from NASA explored real-time “adaptive automation” using EEG-based measures while people performed tasks from a multiattribute test battery at different difficulty levels, with the goal of adapting the user’s workload to reduce overload (Prinzel et al., 2000, 2002). The Defense Advanced Research Projects Agency (DARPA)-funded Augmented Cognition Program also aimed to learn more about system operators’ mental states based on EEG and other

V.  BRAIN, COMPUTER AND MACHINE INTERFACING

366

26.  NONINVASIVE BRAIN–COMPUTER INTERFACES

FIGURE 26.8  The EEG-enhanced World of Warcraft system from Nijholt et al. (2009). The left shows how EEG-based measures of stress can change the elf character into a bear. The right shows a person playing the game.

+ H

scalp projection

3 µV 500 msec +3.9

Average

µV –0.1 END OF TRIAL

0.1s–0.2s

FIGURE 26.9  The left and middle show error-related activity from Schalk et al. (2000). The left image shows event-related potentials over site Cz, and the middle image shows the scalp projections for 40 ms around the positive peak for each subject. The right shows error activity from Parra et al. (2003) from 100 to 200 ms after feedback presentation. These images indicate prominent central activity 100–200 ms after feedback presentation.

measures to adapt the system’s interaction with that user (Schmorrow et al., 2006; St. John et al., 2004). Automatic adaptation to the state of a user has been further explored in aviation (Brookings et al., 1996; Sterman and Mann, 1995). Kirchner et al. (2016) presented a system that could adapt ongoing interaction with a P300 BCI system based on EEG measures of task load. BCIs have also been used to enhance games and creative applications. In Nijholt et al. (2009), healthy people played World of Warcraft through conventional means (keyboard and mouse). The system changed the player’s game character between an elf and a bear based on EEGbased evaluation of stress (see Fig. 26.8). Reissland and Zander (2009) and Zander and Jatzev (2012) described EEG-based systems to detect bluffing in a game environment and perceived loss of control. Makeig et al. (2011) described a BCI that can produce music in real time based on EEG measures of user emotion.

Error Detection People inevitably make mistakes while using BCIs or other technologies. Errors may go undetected or uncorrected, or users need to perform some corrective action

that requires time and attention. EEG measures can reveal error potentials, such as event-related negativity (ERN), if people believe they just made a mistake. Realtime detection of ERN or similar EEG features could be used to correct errors or other goals (Hoffmann and Falkenstein, 2012; Wessel, 2012). Schalk et al. (2000) recorded activity from 64 EEG channels while four subjects used mu and beta activity to direct a cursor to the word “yes” or “no,” and established the difference in EEG activity recorded during the trials in which the subjects did or did not succeed in controlling the cursor to the correct target. One hundred eighty milliseconds after subjects received feedback indicating an erroneous selection, they exhibited a strong positive peak, followed by a negative peak, which was most prominent over Cz (Fig. 26.9, left). The authors estimated that, if error activity were used for error correction within their yes/no BCI system, the system’s information transfer rate would improve by 0% to 21% across the four subjects. A different group also explored error activity recorded from 64 EEG channels and implications for online error correction (Parra et al., 2003). Seven healthy subjects performed a button-press task in a visual discrimination paradigm. ERPs also showed

V.  BRAIN, COMPUTER AND MACHINE INTERFACING

367

Brain–Computer Interfaces to Enhance Function Composition

Emotion people no people

–1

0

1

1

1

2

3

3 0

50

100

150

200

Time (msec)

250

300

µV

0

2

emotional neutral

–1

0 µV

µV

Content

figure/ground scene

–1

2 3 0

50

100

150

200

250

300

0

50

Time (msec)

100

150

200

250

300

Time (msec)

FIGURE 26.10  Event-related potential activity elicited by images that varied in composition, content, and emotion. From Loew, A., Bradley, M.M., Lang, P.J., 2013. Perceptual processing of natural scenes at rapid rates: effects of complexity, content, and emotional arousal. Cogn. Affect. Behav. Neurosci. 13 (4), 860–868. Available from: http://dx.doi.org/10.3758/s13415-013-0179-1.

prominent frontocentral activity from 100 to 200 ms after erroneous responses only (Fig. 26.9, right), which could reduce errors by −6% to 49% across the seven subjects. The P300 can also indicate that the user received feedback that was unexpected or erroneous (Polich, 2004), which could be used in adaptive BCIs (Bayliss et al., 2004). Tong et al. (2016) used multiple features for error detection while healthy people used a P300 BCI. In addition to modifying users’ interactions with BCIbased communication systems, real-time error monitoring could also lead to adaptations with conventional software, such as real-time error correction when a typist makes a mistake using a word processor (Allison, 2009).

Sleep Millions of people in the United States alone have serious trouble sleeping, including falling asleep, remaining asleep, and having nightmares. BCIs may be used to enhance sleep in different ways. Sleep stages, including sleep onset, could be identified and classified via EEG, EMG, electrooculogram (EOG), heart rate (HR), and other signals (Beniczky et al., 2013; Goldstein and Chervin, 2016; Lin et al., 2008; Silverman et al., 1968). These signals have been used to modulate a user’s environment, such as relaxing music to foster sleep onset, or influence the timing of an alarm clock.

Image Recognition One approach to BCI could enhance the natural ability to convey the recognition of specific images. People employed in image recognition, such as in security, research, or product review, often must view many thousands of images with the goal of detecting one or more “target” images, then pressing a button or taking other action to convey this target detection to a computer. BCIs may improve this process by automatically determining when the user detects a target image, eliminating the need to respond and allowing faster image presentation. In the rapid serial visualization paradigm

(RSVP), a sequence of single images is presented very quickly while EEG activity is used to distinguish target from nontarget images. Gerson et al. (2005) showed that nine subjects who had to press a button after each target stimulus exhibited frontal EEG activity that developed 200 ms before the button press. Pohlmeyer et al. (2011) extended this direction with a real-time system to reorder an image database. EEG measures were combined with a semisupervised artificial visual system to estimate users’ interest in each image during RSVP and change its position in the image sequence accordingly. In Loew et al. (2013), 27 participants observed 128 color images that varied in composition (figure/ground or scene), content (people or none), and valence (arousing or neutral). The images were presented serially for 330 ms each with no delay between images while 128 EEG channels were recorded. Fig. 26.10 shows how averaged ERPs differ across these three categories. Results could be used to order stimuli based on richer information about each image, although extending results to single-trial applications may be challenging. Several articles have addressed single-trial RSVP algorithms and performance, as this application is not well suited to averaged data (Fuhrmann Alpert et al., 2014; Manor and Geva, 2015; Lin et al., 2015; Bigdely-Shamlo et al., 2008; Marathe et al., 2016).

Neuromarketing The years since 2007 have seen considerable work on neuromarketing, in which EEG, fMRI, magnetoencephalography, and other physiological signals are used to help marketing researchers learn more about people’s decisions and reactions relating to products and services. This type of research may not fall under the conventional definition of a BCI (because it does not provide real-time feedback to the user). At the same time, this approach could adapt to users and/or modify the ongoing interaction with users, such as by presenting additional ads that are similar to an ad that a user likes, to better identify which components of that ad are appealing. In a typical paradigm, healthy participants view ads, movie trailers, or other short clips while activity is monitored,

V.  BRAIN, COMPUTER AND MACHINE INTERFACING

368

26.  NONINVASIVE BRAIN–COMPUTER INTERFACES

possibly after a brief period of adapting classification parameters to each user. BCIs may provide more detailed information than conventional methods such as questionnaires and focus groups, and do not require interrupting people, because they simply rely on signals produced by the user during his or her natural interaction. For example, subconscious aspects of whether someone likes or dislikes a car stimulus were evaluated with ERPs (Wriessnegger et al., 2015). Several studies from one group used EEG, sometimes in combination with galvanic skin responses and HR measurements, to assess memorization and attention when people viewed advertisements (Astolfi et al., 2008; Vecchiato et al., 2011, 2012). Other work has evaluated EEG-based measures associated with lighting and color in a marketing context (Bercik et al., 2016).

BRAIN–COMPUTER INTERFACES TO IMPROVE FUNCTION Introduction This section addresses BCIs that are designed to produce a lasting, perhaps permanent, improvement in nervous system function to alleviate a particular condition. Thus, BCIs to improve function differ from the other BCI approaches, which focus primarily on providing benefits during BCI use. For example, BCIs to restore motor function may involve the same hardware and similar software as BCIs to improve motor function, but only the latter category focuses on training protocols that provide lasting beneficial changes to the nervous system, and thereby complement rehabilitation therapy. Successful development of BCIs to improve function would address an enormous problem: in addition to the personal impact on the lives of patients and those who care for them, movement and other disabilities can entail many other costs. Patients may have limited or no ability to work, create, or care for themselves or others. Assistive technologies, managed care facilities, rehabilitation systems and therapies, and other ancillary costs are tremendous (Demaerschalk et al., 2010; Howard-Wilsher et al., 2016). Therapy may require numerous sessions, spread across weeks or months, which can require being driven to a hospital or rehabilitation center. Thus, new devices, methods, or concepts to improve the efficacy and effectiveness of rehabilitation therapy are sorely needed.

Improvements to Motor Function Millions of people in the United States alone have difficulty moving because of disease or injury affecting the brain and/or spinal cord. Major causes include stroke, traumatic brain injury, tumors, and SCI. Conventional

FIGURE 26.11  This image shows how EEG activity, processed through a laptop, could drive conventional stroke feedback mechanisms such as a functional electrical stimulator (FES) or an avatar. From Sabathiel, N., Irimia, D.C., Allison, B.Z., Guger, C., Edlinger, G., 2016. Paired associative stimulation with brain–computer interfaces: a new paradigm for stroke rehabilitation. In: International Conference on Augmented Cognition. Springer, pp. 261–272.

approaches to improve motor control include therapy, several robotic assistive devices, functional electrical stimulation, virtual reality, and other means. Since 2007, many researchers have presented realtime brain imaging, sometimes with direct electrical or magnetic brain stimulation, as a complement to existing movement therapy tools and methods. The expectation is that real-time measures of motor imagery could influence devices that present feedback (Fig. 26.11). Thus, feedback such as FES activation, avatar movement, brain stimulation, and/or rewarding sounds or images could occur only when the patient imagines the correct movement, perhaps influenced further by movement intensity or other parameters. This combination of approaches is based on the principle that improving the neural communication between two relevant groups of neurons relies on the coordinated activation between them, as per Hebbian learning and long-term potentiation. The goal of rehabilitation therapy is to strengthen the pathways connecting neurons in the cortex (such as neurons responsible for grasping) to neurons that directly trigger corresponding muscle activity. BCIs could add several contributions to this process. Real-time measures of movement imagery in the cortex can be used to automatically influence different types of feedback while providing helpful information to a therapist, including compliance monitoring. Electrical or magnetic devices can stimulate relevant cortical neurons to further increase activation when downstream neurons are active. Other neuroimaging advances could lead to more detailed and accurate diagnoses and treatments. Beyond improving movement, BCIs can produce other persistent changes to treat different patient groups.

V.  BRAIN, COMPUTER AND MACHINE INTERFACING

369

Brain–Computer Interfaces to Improve Function

(A)

(B)

Training strategy 1 Signal features

Device that assists movement

Signal features Impaired efferent signal

Impaired efferent signal

Impaired volitional movement

ly

CNS

n

ai

Br

sig na ls m ar or e e pr fu og nc re tio ss na ive l

CNS

sig ls

Practice of close-to-normal movement

e

ar l

ive ss

na

re

io ct

og

pr

n fu

ain

e

or

More normal afferent signal

Br

m

na

Direct training of cortical signal

Training strategy 2

ly

(C) Goal

Restored CNS Normal efferent signal

Normal volitional movement

More normal afferent signal

FIGURE 26.12  In (A), users directly train the central nervous system (CNS) to produce healthier signals, thereby leading to improved CNS function (C). In (B), users instead attain this goal by practicing movements using a brain–computer interface assisted by a device, improving sensory feedback and thus improving CNS function. From Daly, J.J., Wolpaw, J.R., 2008. Brain–computer interfaces in neurological rehabilitation. Lancet Neurol. 7 (11), 103243.

Pfurtscheller and Neuper (2006) presented future directions in BCI research, with prominent attention to improving functional recovery from stroke. They noted that detecting motor imagery is essential in effective rehabilitation, and addressed related issues that remain topical, such as virtual reality training, BCIs that manage real-time feedback based on motor imagery, first-person/kinesthetic motor imagery, and adapting parameters to each patient. Birbaumer and Cohen (2007) suggested the use of transcranial magnetic stimulation (TMS) or transcranial direct current stimulation (TDCS) to strengthen Hebbian learning and thus improve movement. Daly and Wolpaw (2008) presented two BCI approaches to improve function. In the first approach, users train to produce different patterns of brain activity through feedback. The second approach relies on repeated use of BCI to control a movement device to encourage activity-dependent plasticity through sensory feedback (Fig. 26.12). Other early work suggested TMS as a tool to accelerate rehabilitation of cognitive, motor, and language functions (Rossi and Rossini, 2004; Siebner et al., 1999). The former article showed that TMS could help patients with dystonia such as writer’s cramp, although improvements were only transient. Other early work with 15 chronic hemiparetic stroke patients showed that repetitive (r) TMS (unlike sham

stimulation) improved the learning of a finger movement sequence, and produced significantly larger muscle evoked potentials (MEPs) (Kim et al., 2006); see also Chang et al. (2015). Some authors have continued to explore TMS for motor rehabilitation. For example, Buetefisch et al. (2015) showed that rTMS of 0.1 Hz, unlike other frequencies or subthreshold stimulation, led to significant improvements in acceleration and MEP size. Du et al. (2016) found that TMS stimulation in first-time stroke patients was more effective at 1 Hz than at 3 Hz, both of which were more effective than sham. These and other studies using TMS to improve motor function support the notion that simultaneous activation of both cortical and associated downstream neurons increases motor learning. Furthermore, additional work to explore optimal stimulation frequencies and other parameters could lead to greater improvement. Mrachacz-Kersting et al. (2011) found that MRCPs can be used to trigger peripheral stimulation that improves excitability. Mrachacz-Kersting et al. (2016) used MRCPs to detect movement and thereby control feedback in a rehabilitation paradigm, and described significant improvements in functional and neuroimaging measures with only three training sessions. The authors noted that MRCPs can provide more precise synchrony between movement imagery and feedback, thus improving Hebbian learning.

V.  BRAIN, COMPUTER AND MACHINE INTERFACING

370

26.  NONINVASIVE BRAIN–COMPUTER INTERFACES

Another ongoing problem that is being addressed is our lack of a thorough understanding of how the brain relearns movement and other tasks, especially in the context of BCI-mediated rehabilitation. Straudi et al. (2016) used TDCS with a robotic system to treat persons with different types of strokes. The treatment was most effective on patients with chronic and subcortical stroke. Carey et al. (2005) and Fallani et al. (2013) also found that cortical changes resulting from rehabilitative training varied with stroke location or movement impairment. These and other results may help guide physicians, therapists, and BCI practitioners in deciding the best components and parameters of BCI for different patients. Young et al. (2014) used fMRI and several behavioral assessments to compare two groups of chronic stroke patients (mean 13.13 months after stroke onset) with persistent trouble with upper-limb movement. Eight of these patients participated in BCI therapy, while six had no therapy. Only the BCI group showed fMRI changes and improvements in behavioral performance consistent with improved movement. Most research has focused on upper-limb rehabilitation, because this is a common need of stroke patients and requires less extensive therapy systems. Rehabilitation of lower-limb movement disabilities, including gait, can also entail an exoskeleton, EMG and FES devices for the legs and hips, and significant safety measures to prevent falling. Despite these challenges, systems for BCI-based gait rehabilitation have been explored (Pons and Torricelli, 2014; Lopez-Larraz et al., 2016; BeldaLois et al., 2011). Jiang et al. (2015) found that MRCPs could help predict gait initiation in healthy subjects, as can ERD (Hortal et al., 2016b), which both groups noted could be beneficial in gait rehabilitation. MRCPs can also detect foot movements in other work focused on rehabilitation (Aliakbaryhosseinabadi et al., 2014). In addition to work with writer’s cramp, BCIs could be useful beyond helping stroke patients regain movement. Stroke patients also have trouble with spastic tremors, which BCIs could address (Pons and Torricelli, 2014). Persons with other movement disabilities might also benefit, such as those with spinal injuries, traumatic brain injury, Parkinson disease, or cerebral palsy (Birbaumer and Cohen, 2007; Lopez-Larraz et al., 2016; Pons and Torricelli, 2014; Rossi and Rossini, 2004). The increase in interest in using BCIs to improve motor function is apparent from an abundance of review articles. Several publications have summarized and commented upon the use of EEG-based BCIs for persistent functional improvement (Chaudhary et al., 2015; Soekadar et al., 2015; Kubis, 2016; McCrimmon et al., 2016; Paggiaro et al., 2016; Remsik et al., 2016; Riccio et al., 2016; Ushiba and Soekadar, 2016). These reviews generally find this direction to be promising, though more extensive clinical validation is clearly needed.

Improvements to Other Functions BCIs, often called bio- and neurofeedback in this domain, have been extensively explored and applied to improve the central nervous system by improving emotional and/or psychiatric factors (Marzbani et al., 2016). Neurofeedback has been used to facilitate relaxation in both healthy people and persons with stress-related conditions for decades, with increased recent interest in posttraumatic stress disorder (Gapen et al., 2016). In a typical paradigm, users try to modulate activity in particular EEG frequency bands over different regions, such as alpha and beta. Both healthy and disabled users, as well as elderly persons, have also long used neurofeedback with the goal of improving memory (Miralles et al., 2015; Gomez-Pilar et al., 2016; Kober et al., 2015). Related efforts involve neurofeedback to train vigilance, distractibility, and other attentional factors. While interest has grown in such BCIs to treat patients with attention deficit hyperactivity disorder or other conditions, healthy patients have also sought to improve their attention (Gomez-Pilar et al., 2016; Cortese et al., 2016; OrdikhaniSeyedlar et al., 2016; Bluschke et al., 2016). Several other patient groups have been explored. An fMRI-based BCI system was used to train psychopaths to learn fear conditioning (Birbaumer and Cohen, 2007; Sitaram et al., 2014). Patients with criminal psychopathy were trained to upregulate blood oxygen level-dependent (BOLD) activity in the left anterior insula, part of the brain’s fear circuitry. FMRI activation showed increased BOLD activity, suggesting that the psychopaths learned fear, although behavioral results were not reported. This approach could be extended to help persons with obsessive–compulsive disorder or other groups (Buyukturkoglu et al., 2015). Interestingly, Birbaumer and colleagues published earlier work showing that patients with epilepsy could reduce seizures by training SCP activity (Rockstroh et al., 1993), although this approach never became part of clinical practice. BCIs and related therapies to treat addiction could be applied to anorexia and other eating disorders (Lackner et al., 2016). Other BCIs aim to reduce the symptoms of autism (Kouijzer et al., 2013; Friedrich et al., 2014, 2015; Pineda et al., 2014) by improving mirror neuron system function, which appears to be impaired in persons with autism (Oberman et al., 2005). For example, in the “Social Mirroring” game, children with autism receive positive or negative feedback when they produce certain physiological signals while interacting with a virtual character (Friedrich et al., 2014). An important issue in this area is the ability to perform BCI rehabilitation in home settings (Sabathiel et al., 2016; Mastropietro et al., 2016; Miralles et al., 2015). Traveling to a hospital, therapy center, or other remote location for dozens of therapy sessions can be

V.  BRAIN, COMPUTER AND MACHINE INTERFACING

Summary of the Current State of Noninvasive Brain–Computer Interfaces

especially burdensome for patients who cannot drive or have special transportation needs, such as a van designed for persons with motor disabilities. Homebased BCI training might occur in tandem with remote therapy, thus reducing cost while still providing some contact with medical professionals. A caregiver in the patient’s home could be trained to mount an electrode cap or other sensors, prepare an FES system, launch software, review task instructions with the patient, or help in other ways. Therapists might also conduct “house call” rehabilitation sessions to provide these and more advanced services. BCIs to improve function are probably the most ambitious of the five approaches. They could render other types of BCIs unnecessary, heal a wide variety of conditions, and appeal to healthy users for many reasons. At the same time, side effects could be harmful, and thus, overall societal impact could be complex. This potentially broad impact means that adequate clinical validation and ethical considerations are particularly important.

SUMMARY OF THE CURRENT STATE OF NONINVASIVE BRAIN–COMPUTER INTERFACES Scientific and Technical Basis In accord with many other studies over the past decades, the studies described in this chapter have provided ample evidence that noninvasive brain signal modalities can be used to give information about the motoric, sensory, cognitive, or affective state of a user, and that such information can be decoded from the brain in real time, i.e., as the user engages in a particular task. This largely laboratory-based body of evidence provides the scientific basis for systems that can replace, restore, enhance, supplement, or improve human functions. The growing miniaturization of sensor, affector, and computing technologies, as well as the great increase in pervasiveness and affordability of computing power, provides the technical basis for devices that can be small and reasonably affordable.

Translating Brain–Computer Interfaces From Scientific Endeavors Into Clinically and Commercially Successful Technologies BCI technologies have a strong scientific and technical basis, receive modest and sustained support from funding agencies, and undeniably fascinate scientists, the media, and the public. However, only very few BCI technologies have been translated into new clinically and commercially successful diagnostic or treatment options.

371

This critical issue in “moving beyond demonstrations” has been recognized for many years—indeed, it was the theme for the 2002 BCI Meeting in Rensselaerville, New York. The numerous discussions in scientific articles or BCI workshops on this topic since then have usually centered on the necessity for further technical improvements (e.g., in bit rate, practicality of EEG headsets, and calibration or other signal processing techniques). The implicit notion is that improvement in one or most of those technical factors will result in commercial adoption of BCI technologies. While technical improvements of different kinds will always be welcome or may even be critical, it is important to recognize that perhaps the most important requirement for any successful new technology is that its benefit exceeds its cost. An assessment of benefit needs to consider not only the realistic benefit of the BCI technology relative to existing technologies, but also the number of individuals that are interested in using it. For example, a BCIbased communication device may support brain-based spelling at a rate of a few characters per minute. This capacity to spell using the brain may be inspiring and statistically well above chance, but it may still not provide a clinically relevant improvement over existing augmentative communication devices or may provide such improvement to only very few people. Likewise, the assessment of cost needs to consider not only the variable cost of the BCI device, but also the risks and fixed cost of regulatory approvals, intellectual property strategies and other legal activities, development of new sales/marketing channels, product development, support, and the complexities associated with the change to the BCI technologies from the traditional technologies that they replace (Allison, 2009; Millan et al., 2010; Miralles et al., 2015). If the aggregated benefit does not exceed the aggregated cost, commercialization will be unlikely to succeed. Finally, it is important to recognize that successful commercialization in the medical sector requires successful navigation and resolution of complex reimbursement, regulatory, and legal issues. The knowledge and people required for successful commercialization, as well as the medical, corporate, and legal arrangements necessary to support this process, have existed for decades for traditional medical devices and pharmaceuticals. This critical infrastructure is in a very early stage for BCI systems, and requires substantial cost and effort to set up. Hiring and training appropriate staff, developing procedures to follow different requirements, designing and manufacturing products accordingly, testing for compliance, developing proposals to certifying agencies and responding to their requests, validation with target users (especially clinical trials), and ethical approvals create substantial but necessary challenges when extending BCIs from the research into the commercial

V.  BRAIN, COMPUTER AND MACHINE INTERFACING

372

26.  NONINVASIVE BRAIN–COMPUTER INTERFACES

domain. Furthermore, these and other issues can vary substantially in different regions and with different patient groups.

Commercialization Potential of Various Noninvasive Brain–Computer Interface Technologies Noninvasive BCIs to replace, restore, or enhance functions, such as those that enable spelling, restore movements, or warn a sleepy driver, have been intensely studied for some time. The principal challenge facing these systems is that they need to derive an assessment of the user’s brain state very rapidly and accurately. Unfortunately, all current noninvasive sensors can only produce either unreliable measurements quickly or reliable measurements very slowly. Given this principal limitation, the number of users that can benefit from these types of BCI technologies using current noninvasive sensor technologies is small, and/or those users often already have access to non-BCI alternatives. For this reason, successful commercialization of these types of BCIs will probably require the development of asyet unknown sensor technologies that do not face these issues. Noninvasive BCIs to improve functions, such as those that rehabilitate people with chronic stroke or incomplete SCI, have attracted increasing interest over the past several years. They induce beneficial plasticity in specific central nervous system circuits and often have more relaxed requirements regarding the rapidity or accuracy of measurements. The number of users that can benefit from those BCI technologies is potentially very large, and those users often have no viable alternative. Hence, with additional development and validation, and with development of proper regulatory, reimbursement, and commercialization strategies, these types of BCIs appear to have a potential trajectory toward successful translation and commercialization. Indeed, several companies have begun to introduce products that rehabilitate function in chronic stroke survivors, including IpsiHand by Neurolutions (www.neurolutions.org), NBetter by NeuroStyle (www.neuro-style.com), and recoveriX by g.tec (www.recoverix.at).

Conclusions In conclusion, the information that can be extracted from the brain provides many opportunities to increase individual or societal performance. Full realization of these possibilities for noninvasive BCIs will probably require the development of completely novel sensors that can support rapid and reliable measurements of brain function.

Acknowledgments This work was supported by the NIH (EB00856, EB006356, and EB018783), the US Army Research Office (W911NF-08-1-0216, W911NF-12-1-0109, W911NF-14-1-0440), and Fondazione Neurone.

References Aliakbaryhosseinabadi, S., Jiang, N., Vuckovic, A., Lontis, R., Dremstrup, K., Farina, D., Mrachacz-Kersting, N., 2014. Detection of movement intention from movement-related cortical potentials with different paradigms. In: Replace, Repair, Restore, RelieveBridging Clinical and Engineering Solutions in Neurorehabilitation. Springer, pp. 237–244. Allison, B.Z., McFarland, D.J., Schalk, G., Zheng, S.D., Jackson, M.M., Wolpaw, J.R., 2008. Towards an independent brain-computer interface using steady state visual evoked potentials. Clin. Neurophysiol. 119 (2), 399–408. Allison, B.Z., 2009. Toward ubiquitous BCIs. In: Brain-Computer Interfaces. Springer, pp. 357–387. Angulo-Sherman, I., Costa-García, A., Monge-Pereira, E., SalazarVaras, R., Zerafa, R., 2016. BCI applied to neurorehabilitation. In: Emerging Therapies in Neurorehabilitation II. Springer, pp. 169–196. Astolfi, L., De Vico Fallani, F., Cincotti, F., Mattia, D., Bianchi, L., Marciani, M.G., Salinari, S., Colosimo, A., Tocci, A., Soranzo, R., Babiloni, F., 2008. Neural basis for brain responses to TV commercials: a high-resolution EEG study. IEEE Trans. Neural Syst. Rehabil. Eng. 16 (6), 522–531. Bauernfeind, G., Horki, P., Kurz, E.-M., Schippinger, W., Pichler, G., Mueller-Putz, G.R., 2015. Improved concept and first results of an auditory single-switch BCI for the future use in disorders of consciousness patients. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, pp. 1902–1905. Bayliss, J.D., Inverso, S.A., Tentler, A., 2004. Changing the P300 brain computer interface. Cyberpsychol. Behav. 7 (6), 694–704. Belda-Lois, J.-M., Mena-del Horno, S., Bermejo-Bosch, I., Moreno, J.C., Pons, J.L., Farina, D., Iosa, M., Molinari, M., Tamburella, F., Ramos, A., 2011. Rehabilitation of gait after stroke: a review towards a topdown approach. J. Neuroeng. Rehabil. 8 (1), 1. Bell, C.J., Shenoy, P., Chalodhorn, R., Rao, R.P., 2008. Control of a humanoid robot by a noninvasive brain-computer interface in humans. J. Neural Eng. 5 (2), 214–220. Beniczky, S., Aurlien, H., Brogger, J.C., Fuglsang-Frederiksen, A., Martins-da Silva, A., Trinka, E., Visser, G., Rubboli, G., Hjalgrim, H., Stefan, H., Rosen, I., Zarubova, J., Dobesberger, J., Alving, J., Andersen, K.V., Fabricius, M., Atkins, M.D., Neufeld, M., Plouin, P., Marusic, P., Pressler, R., Mameniskiene, R., Hopfengartner, R., van Emde Boas, W., Wolf, P., 2013. Standardized computer-based organized reporting of EEG: SCORE. Epilepsia (6), 1112–1124. Bercik, J., Horska, E., Wang, R.W., Chen, Y.C., 2016. The impact of parameters of store illumination on food shopper response. Appetite 106, 101–109. Bigdely-Shamlo, N., Vankov, A., Ramirez, R.R., Makeig, S., 2008. Brain activity-based image classification from rapid serial visual presentation. IEEE Trans. Neural Syst. Rehabil. Eng. 16 (5), 432–441. Bin, G., Gao, X., Wang, Y., Li, Y., Hong, B., Gao, S., 2011. A high-speed BCI based on code modulation VEP. J. Neural Eng. 8 (2), 025015. Birbaumer, N., Cohen, L.G., 2007. Brain-computer interfaces: communication and restoration of movement in paralysis. J. Physiol. 579 (Pt 3), 621–636. Birbaumer, N., Ghanayim, N., Hinterberger, T., Iversen, I., Kotchoubey, B., Kubler, A., Perelmouter, J., Taub, E., Flor, H., 1999. A spelling device for the paralysed. Nature 398 (6725), 297–298.

V.  BRAIN, COMPUTER AND MACHINE INTERFACING

References

Blankertz, B., Dornhege, G., Krauledat, M., Schröder, M., Williamson, J., Murray-Smith, R., Müller, K.-R., 2006. The berlin brain-computer interface presents the novel mental typewriter Hex-o-Spell. In: Proceedings of the 3rd International Brain-Computer Interface Workshop and Training Course, Graz, Austria, pp. 108–109. Bluschke, A., Broschwitz, F., Kohl, S., Roessner, V., Beste, C., 2016. The neuronal mechanisms underlying improvement of impulsivity in ADHD by theta/beta neurofeedback. Sci. Rep. 6, 31178. Brookings, J.B., Wilson, G.F., Swain, C.R., 1996. Psychophysiological responses to changes in workload during simulated air traffic control. Biol. Psychol. 42 (3), 361–377. Buetefisch, C.M., Howard, C., Korb, C., Haut, M.W., Shuster, L., Pergami, P., Smith, C., Hobbs, G., 2015. Conditions for enhancing the encoding of an elementary motor memory by rTMS. Clin. Neurophysiol. 126 (3), 581–593. Buyukturkoglu, K., Roettgers, H., Sommer, J., Rana, M., Dietzsch, L., Arikan, E.B., Veit, R., Malekshahi, R., Kircher, T., Birbaumer, N., Sitaram, R., Ruiz, S., 2015. Self-regulation of anterior insula with real-time fMRI and its behavioral effects in obsessive-compulsive disorder: a feasibility study. PLoS One 10 (8), e0135872. Cao, L., Li, J., Ji, H., Jiang, C., 2014. A hybrid brain computer interface system based on the neurophysiological protocol and brain-actuated switch for wheelchair control. J. Neurosci. Methods 229, 33–43. Carabalona, R., Grossi, F., Tessadri, A., Castiglioni, P., Caracciolo, A., de Munari, I., 2012. Light on! real world evaluation of a P300-based brain-computer interface (BCI) for environment control in a smart home. Ergonomics 54 (5), 552–563. Carey, L.M., Abbott, D.F., Egan, G.F., Bernhardt, J., Donnan, G.A., 2005. Motor impairment and recovery in the upper limb after stroke behavioral and neuroanatomical correlates. Stroke 36 (3), 625–629. Castermans, T., Duvinage, M., Cheron, G., Dutoit, T., 2013. Towards effective non-invasive brain-computer interfaces dedicated to gait rehabilitation systems. Brain Sci. 4 (1), 1–48. Chang, M.C., Kim, D.Y., Park, D.H., 2015. Enhancement of cortical excitability and lower limb motor function in patients with stroke by transcranial direct current stimulation. Brain Stimul. 8 (3), 561–566. Chaudhary, U., Birbaumer, N., Curado, M., 2015. Brain-machine interface (BMI) in paralysis. Ann. Physical Rehabil. Med. 58 (1), 9–13. Cheng, M., Gao, X., Gao, S., Xu, D., 2002. Design and implementation of a brain-computer interface with high transfer rates. IEEE Trans. Biomed. Eng. 49 (10), 1181–1186. Choi, I., Bond, K., Krusienski, D., Nam, C.S., 2015. Comparison of stimulation patterns to elicit steady-state somatosensory evoked potentials (SSSEPs): implications for hybrid and SSSEP-based BCIs. In: 2015 IEEE International Conference on. Systems, Man, and Cybernetics (SMC). IEEE, pp. 3122–3127. Collinger, J.L., Boninger, M.L., Bruns, T.M., Curley, K., Wang, W., Weber, D.J., 2013. Functional priorities, assistive technology, and brain-computer interfaces after spinal cord injury. J. Rehabil. Res. Dev. 50 (2), 145–160. Cortese, S., Ferrin, M., Brandeis, D., Holtmann, M., Aggensteiner, P., Daley, D., Santosh, P., Simonoff, E., Stevenson, J., Stringaris, A., Sonuga-Barke, E.J., June 2016. Neurofeedback for attention-deficit/ hyperactivity disorder: metaanalysis of clinical and neuropsychological outcomes from randomized controlled trials. J. Am. Acad. Child Adolesc. Psychiatry 55 (6), 444–455. http://dx.doi. org/10.1016/j.jaac.2016.03.007. Epub April 1, 2016. Costa, A., Ianez, E., Ubeda, A., Hortal, E., Del-Ama, A.J., Gil-Agudo, A., Azorin, J.M., 2016. Decoding the attentional demands of gait through EEG gamma band features. PLoS One 11 (4), e0154136. Coyle, D., Garcia, J., Satti, A.R., McGinnity, T.M., 2011. EEG-based continuous control of a game using a 3 channel motor imagery BCI: BCI game. In: 2011 IEEE Symposium on. Computational Intelligence, Cognitive Algorithms, Mind, and Brain (CCMB). IEEE, pp. 1–7.

373

Daly, J.J., Wolpaw, J.R., 2008. Brain-computer interfaces in neurological rehabilitation. Lancet Neurol. 7 (11), 103243. Demaerschalk, B.M., Hwang, H.M., Leung, G., 2010. US cost burden of ischemic stroke: a systematic literature review. Am. J. Manag. Care 16 (7), 525–533. Do, A.H., Wang, P.T., King, C.E., Chun, S.N., Nenadic, Z., December 2013. Brain-computer interface controlled robotic gait orthosis. J. Neuroeng. Rehabil. 9 (10), 111. http://dx.doi. org/10.1186/1743-0003-10-111. Du, J., Tian, L., Liu, W., Hu, J., Xu, G., Ma, M., Fan, X., Ye, R., Jiang, Y., Yin, Q., 2016. Effects of repetitive transcranial magnetic stimulation on motor recovery and motor cortex excitability in patients with stroke: a randomized controlled trial. Eur. J. Neurol. 23 (11), 1666–1672. Edlinger, G., Holzner, C., Guger, C., 2011. A hybrid brain-computer interface for smart home control. In: International Conference on Human-Computer Interaction. Springer, pp. 417–426. Fairclough, S.H., 2009. Fundamentals of physiological computing. Interact. Comput. 21 (1), 133–145. Fallani, F.D.V., Pichiorri, F., Morone, G., Molinari, M., Babiloni, F., Cincotti, F., Mattia, D., 2013. Multiscale topological properties of functional brain networks during motor imagery after stroke. NeuroImage 83, 438–449. Farwell, L.A., Donchin, E., 1988. Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr. Clin. Neurophysiol. 70 (6), 510–523. Fridman, I., Cordeiro, M., Rais-Bahrami, K., McDonald, N.J., Reese Jr., J.J., Massaro, A.N., Conry, J.A., Chang, T., Soussou, W., Tsuchida, T.N., 2016. Evaluation of dry sensors for neonatal EEG recordings. J. Clin. Neurophysiol. 33 (2), 149–155. Friedrich, E.V., Suttie, N., Sivanathan, A., Lim, T., Louchart, S., Pineda, J.A., 2014. Brain-computer interface game applications for combined neurofeedback and biofeedback treatment for children on the autism spectrum. Front. Neuroeng. 7, 21. Friedrich, E.V., Sivanathan, A., Lim, T., Suttie, N., Louchart, S., Pillen, S., Pineda, J.A., December 2015. An effective neurofeedback intervention to improve social interactions in children with autism spectrum disorder. J. Autism Dev. Disord. 45 (12), 4084–4100. http:// dx.doi.org/10.1007/s10803-015-2523-5. Fuhrmann Alpert, G., Manor, R., Spanier, A.B., Deouell, L.Y., Geva, A.B., 2014. Spatiotemporal representations of rapid visual target detection: a single-trial EEG classification algorithm. IEEE Trans. Biomed. Eng. 61 (8), 2290–2303. Furdea, A., Halder, S., Krusienski, D.J., Bross, D., Nijboer, F., Birbaumer, N., Kubler, A., 2009. An auditory oddball (P300) spelling system for brain-computer interfaces. Psychophysiology 46 (3), 617–625. Galan, F., Nuttin, M., Lew, E., Ferrez, P.W., Vanacker, G., Philips, J., Millan Jdel, R., 2008. A brain-actuated wheelchair: asynchronous and non-invasive brain-computer interfaces for continuous control of robots. Clin. Neurophysiol. 119 (9), 2159–2169. Gapen, M., van der Kolk, B.A., Hamlin, E., Hirshberg, L., Suvak, M., Spinazzola, J., 2016. A pilot study of neurofeedback for chronic PTSD. Appl. Psychophysiol. Biofeedback 41 (3), 251–261. Gerson, A.D., Parra, L.C., Sajda, P., 2005. Cortical origins of response time variability during rapid discrimination of visual objects. NeuroImage 28 (2), 342–353. Gevins, A., Leong, H., Du, R., Smith, M.E., Le, J., DuRousseau, D., Zhang, J., Libove, J., 1995. Towards measurement of brain function in operational environments. Biol. Psychol. 40 (1–2), 169–186. Goldstein, C., Chervin, R., 2016. Waking up to sleep research in 2015. Lancet Neurol. 15 (1), 15–17. Gomez-Pilar, J., Corralejo, R., Nicolas-Alonso, L.F., Alvarez, D., Hornero, R., 2016. Neurofeedback training with a motor imagerybased BCI: neurocognitive improvements and EEG changes in the elderly. Med. Biol. Eng. Comput. 54 (11), 1655–1666.

V.  BRAIN, COMPUTER AND MACHINE INTERFACING

374

26.  NONINVASIVE BRAIN–COMPUTER INTERFACES

Guger, C., Daban, S., Sellers, E., Holzner, C., Krausz, G., Carabalona, R., Gramatica, F., Edlinger, G., 2009. How many people are able to control a P300-based brain-computer interface (BCI)? Neurosci. Lett. 462 (1), 94–98. Guger, C., Krausz, G., Allison, B.Z., Edlinger, G., 2012. Comparison of dry and gel based electrodes for P300 brain-computer interfaces. Front. Neurosci. 6, 60. Halder, S., Furdea, A., Leeb, R., Müller-Putz, G., Hosle, A., Kübler, A., 2009. Implementation of SMR-based brain painting. In: Poster at COST Neuromath Workshop. Leuven, Belgium. Hoffmann, S., Falkenstein, M., 2012. Predictive information processing in the brain: errors and response monitoring. Int. J. Psychophysiol. 83 (2), 208–212. Hortal, E., Planelles, D., Iáñez, E., Costa, A., Úbeda, A., Azorín, J., 2016a. Detection of gait initiation through a ERD-based brain-computer interface. In: Advances in Neurotechnology, Electronics and Informatics. Springer, pp. 141–150. Hortal, E., Ubeda, A., Ianez, E., Azorin, J.M., Fernandez, E., 2016b. EEG-based detection of starting and stopping during gait cycle. Int. J. Neural Syst. 26 (7), 1650029. Howard-Wilsher, S., Irvine, L., Fan, H., Shakespeare, T., Suhrcke, M., Horton, S., Poland, F., Hooper, L., Song, F., 2016. Systematic overview of economic evaluations of health-related rehabilitation. Disabil. Health J. 9 (1), 11–25. Huang, K.C., Huang, T.Y., Chuang, C.H., King, J.T., Wang, Y.K., Lin, C.T., Jung, T.P., 2016. An EEG-based fatigue detection and mitigation system. Int. J. Neural Syst. 26 (4), 1650018. Hwang, H.-J., Choi, H., Kim, J.-Y., Chang, W.-D., Kim, D.-W., Kim, K., Jo, S., Im, C.-H., 2016. Toward more intuitive brain-computer interfacing: classification of binary covert intentions using functional near-infrared spectroscopy. J. Biomed. Opt. 21 (9), 091303. http:// dx.doi.org/10.1117/1.JBO.21.9.091303. Jiang, N., Gizzi, L., Mrachacz-Kersting, N., Dremstrup, K., Farina, D., 2015. A brain-computer interface for single- trial detection of gait initiation from movement related cortical potentials. Clin. Neurophysiol. 126 (1), 154159. Jin, J., Allison, B.Z., Kaufmann, T., Kubler, A., Zhang, Y., Wang, X., Cichocki, A., 2012. The changing face of P300 BCIs: a comparison of stimulus changes in a P300 BCI involving faces, emotion, and movement. PLoS One 7 (11), e49688. Jung, T.P., Makeig, S., Stensmo, M., Sejnowski, T.J., 1997. Estimating alertness from the EEG power spectrum. IEEE Trans. Biomed. Eng. 44 (1), 60–69. Karim, A.A., Hinterberger, T., Richter, J., Mellinger, J., Neumann, N., Flor, H., Kübler, A., Birbaumer, N., 2006. Neural internet: web surfing with brain potentials for the completely paralyzed. Neurorehabil. Neural Repair 20 (4), 508–515. Kaufmann, T., Schulz, S.M., Grunzinger, C., Kubler, A., 2012. Flashing characters with famous faces improves ERP-based brain-computer interface performance. J. Neural Eng. 8 (5), 056016. Kaufmann, T., Schulz, S.M., Kueblitz, A., Renner, G., Wessig, C., Kuebler, A., 2013. Face stimuli effectively prevent brain-computer interface inefficiency in patients with neurodegenerative disease. Clin. Neurophysiol. 124 (5), 893–900. Kim, Y.-H., You, S.H., Ko, M.-H., Park, J.-W., Lee, K.H., Jang, S.H., Yoo, W.-K., Hallett, M., 2006. Repetitive transcranial magnetic stimulation-induced corticomotor excitability and associated motor skill acquisition in chronic stroke. Stroke 37 (6), 1471–1476. Kirchner, E.A., Kim, S.K., Tabie, M., Wohrle, H., Maurus, M., Kirchner, F., June 2016. An intelligent man-machine interface-multi-robot control adapted for task engagement based on single-trial detectability of P300. Front. Hum. Neurosci. 21 (10), 291. http://dx.doi. org/10.3389/fnhum.2016.00291. eCollection 2016. Kober, S.E., Schweiger, D., Witte, M., Reichert, J.L., Grieshofer, P., Neuper, C., Wood, G., December 2015. Specific effects of EEG-based neurofeedback training on memory functions in post-stroke victims. J. Neuroeng. Rehabil. 1 (12), 107. http://dx.doi.org/10.1186/s12984-015-0105-6.

Koo, B., Lee, H.G., Nam, Y., Kang, H., Koh, C.S., Shin, H.C., Choi, S., 2015. A hybrid NIRS-EEG system for self-paced brain computer interface with online motor imagery. J. Neurosci. Methods 244, 26–32. Kouijzer, M.E., van Schie, H.T., Gerrits, B.J., Buitelaar, J.K., de Moor, J.M., 2013. Is EEG-biofeedback an effective treatment in autism spectrum disorders? A randomized controlled trial. Appl. Psychophysiol. Biofeedback 38 (1), 17–28. Krusienski, D.J., Sellers, E.W., McFarland, D.J., Vaughan, T.M., Wolpaw, J.R., 2008. Toward enhanced P300speller performance. J. Neurosci. Methods 167 (1), 15–21. Kubis, N., 2016. Non-invasive brain stimulation to enhance post-stroke recovery. Front. Neural Circuits 10. Kuebler, A., Nijboer, F., Mellinger, J., Vaughan, T.M., Pawelzik, H., Schalk, G., McFarland, D.J., Birbaumer, N., Wolpaw, J.R., 2005. Patients with ALS can use sensorimotor rhythms to operate a braincomputer interface. Neurology 64 (10), 1775–1777. Lackner, N., Unterrainer, H.F., Skliris, D., Shaheen, S., Dunitz-Scheer, M., Wood, G., Scheer, P.J., Wallner-Liebmann, S.J., Neuper, C., July– September, 2016. EEG neurofeedback effects in the treatment of adolescent anorexia nervosa. Eat. Disord. 24 (4), 354–374. http:// dx.doi.org/10.1080/10640266.2016.1160705. Epub March 30, 2016. Lauer, R.T., Peckham, P.H., Kilgore, K.L., 1999. EEG-based control of a hand grasp neuroprosthesis. Neuroreport 10 (8), 1767–1771. Lesenfants, D., Chatelle, C., Saab, J., Laureys, S., Noirhomme, Q., 2016. Neurotechnological communication with patients with disorders of consciousness. In: Neurotechnology and Direct Brain Communication: New Insights and Responsibilities Concerning Speechless but Communicative Subjects, p. 85 (Chapter 6). Li, Y., Long, J., Yu, T., Yu, Z., Wang, C., Zhang, H., Guan, C., 2010. An EEG-based BCI system for 2-D cursor control by combining mu/ beta rhythm and P300 potential. IEEE Trans. Biomed. Eng. 57 (10), 2495–2505. Li, Y., Pan, J., Long, J., Yu, T., Wang, F., Yu, Z., Wu, W., 2016. Multimodal BCIs: target detection, multidimensional control, and awareness evaluation in patients with disorder of consciousness. Proc. IEEE 104 (2), 332–352. Lim, J.-H., Hwang, H.-J., Han, C.-H., Jung, K.-Y., Im, C.-H., 2013. Classification of binary intentions for individuals with impaired oculomotor function: “eyes-closed” SSVEP-based brain-computer interface (BCI). J. Neural Eng. 10 (2), 026021. Lin, C.T., Chen, Y.C., Huang, T.Y., Chiu, T.T., Ko, L.W., Liang, S.F., Hsieh, H.Y., Hsu, S.H., Duann, J.R., 2008. Development of wireless brain computer interface with embedded multitask scheduling and its application on real-time driver’s drowsiness detection and warning. IEEE Trans. Biomed. Eng. 55 (5), 1582–1591. Lin, C.-T., Wang, Y.-K., Fang, C.-N., Yu, Y.-H., King, J.-T., 2015. Extracting patterns of single-trial EEG using an adaptive learning algorithm. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, pp. 6642–6645. Loew, A., Bradley, M.M., Lang, P.J., 2013. Perceptual processing of natural scenes at rapid rates: effects of complexity, content, and emotional arousal. Cogn. Affect. Behav. Neurosci. 13 (4), 860–868. Available from: http://dx.doi.org/10.3758/s13415-013-0179-1. Long, C., Zhongpeng, W., Feng, H., Jiajia, Y., Hongzhi, Q., Peng, Z., Baikun, W., Dong, M., 2015. An online hybrid brain-computer interface combining multiple physiological signals for webpage browse. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2015, 1152–1155. Lopez-Larraz, E., Trincado-Alonso, F., Rajasekaran, V., PerezNombela, S., Del-Ama, A.J., Aranda, J., Minguez, J., Gil-Agudo, A., Montesano, L., August 3, 2016. Control of an ambulatory exoskeleton with a brain-machine interface for spinal cord injury gait rehabilitation. Front. Neurosci. 10, 359. http://dx.doi.org/10.3389/ fnins.2016.00359. eCollection 2016. Luu, T.P., He, Y., Brown, S., Nakagame, S., Contreras-Vidal, J.L., 2016. Gait adaptation to visual kinematic perturbations using a real-time closed-loop brain-computer interface to a virtual reality avatar. J. Neural Eng. 13 (3), 036006.

V.  BRAIN, COMPUTER AND MACHINE INTERFACING

References

Makeig, S., Leslie, G., Mullen, T., Sarma, D., Bigdely-Shamlo, N., Kothe, C., 2011. First demonstration of a musical emotion BCI. In: Affective Computing and Intelligent Interaction. Springer, pp. 487–496. Manor, R., Geva, A.B., 2015. Convolutional neural network for multicategory rapid serial visual presentation BCI. Front. Comput. Neurosci. 9, 146. Marathe, A.R., Lawhern, V.J., Wu, D., Slayback, D., Lance, B.J., 2016. Improved neural signal classification in a rapid serial visual presentation task using active learning. IEEE Trans. Neural Syst. Rehabil. Eng. 24 (3), 333–343. Marzbani, H., Marateb, H.R., Mansourian, M., April 2016. Neurofeedback: a comprehensive review on system design, methodology and clinical applications. Basic Clin. Neurosci. 7 (2), 143–158. Available from: http://www.ncbi.nlm.nih.gov/ pubmed/27303609. Mason, S.G., Birch, G.E., 2000. A brain-controlled switch for asynchronous control applications. IEEE Trans. Biomed. Eng. 47 (10), 1297–1307. Mastropietro, A., Arlati, S., Mrakic-Sposta, S., Fontana, L., Franchin, C., Malosio, M., Pittaccio, S., Gramigna, C., Molteni, F., Sacco, M., et al., 2016. Quantitative EEG and virtual reality to support post-stroke rehabilitation at home. In: Innovation in Medicine and Healthcare 2016. Springer, pp. 147–157. McCrimmon, C.M., Wang, P.T., Nenadic, Z., Do, A.H., 2016. BCI-based neuroprostheses and physiotherapies for stroke motor rehabilitation. In: Neurorehabilitation Technology. Springer, pp. 617–627. McFarland, D.J., Sarnacki, W.A., Wolpaw, J.R., 2010. Electroencephalographic (EEG) control of three-dimensional movement. J. Neural Eng. 7 (3), 036007. Middendorf, M., McMillan, G., Calhoun, G., Jones, K.S., 2000. Braincomputer interfaces based on the steady-state visual-evoked response. IEEE Trans. Rehabil. Eng. 8 (2), 211–214. Millan, J.D., Rupp, R., Muller-Putz, G.R., Murray-Smith, R., Giugliemma, C., Tangermann, M., Vidaurre, C., Cincotti, F., Kubler, A., Leeb, R., Neuper, C., Muller, K.R., Mattia, D., 2010. Combining brain-computer interfaces and assistive technologies: state-of-theart and challenges. Front. Neurosci. 4. Miner, L.A., McFarland, D.J., Wolpaw, J.R., 1998. Answering questions with an electroencephalogram-based brain- computer interface. Arch. Phys. Med. Rehabil. 79 (9), 1029–1033. Miralles, F., Vargiu, E., Dauwalder, S., Sola, M., Muller-Putz, G., Wriessnegger, S.C., Pinegger, A., Kubler, A., Halder, S., Kathner, I., Martin, S., Daly, J., Armstrong, E., Guger, C., Hintermuller, C., Lowish, H., 2015. Brain computer interface on track to home. Sci. World J. 2015, 623896. Mrachacz-Kersting, N., Jiang, N., Niazi, I.K., Farina, D., 2011. Peripheral electrical stimulation triggered by movement related cortical potentials enhances cortical excitability. In: Front. Comput. Neurosci. Conference Abstract: BC11: Computational Neuroscience & Neurotechnology Bernstein Conference & Neurex Annual Meeting. Mrachacz-Kersting, N., Jiang, N., Stevenson, A.J.T., Niazi, I.K., Kostic, V., Pavlovic, A., Radovanovic, S., Djuric- Jovicic, M., Agosta, F., Dremstrup, K., 2016. Efficient neuroplasticity induction in chronic stroke patients by an associative brain-computer interface. J. Neurophysiol. 115 (3), 1410–1421. Mueller-Putz, G.R., Pokorny, C., Klobassa, D.S., Horki, P., 2013. A single-switch BCI based on passive and imagined movements: toward restoring communication in minimally conscious patients. Int. J. Neural Syst. 23 (02), 1250037. Muenssinger, J.I., Halder, S., Kleih, S.C., Furdea, A., Raco, V., Hoesle, A., Kubler, A., 2010. Brain painting: first evaluation of a new braincomputer interface application with ALS-patients and healthy volunteers. Front. Neurosci. 4, 182. Mugler, E.M., Ruf, C.A., Halder, S., Bensch, M., Kubler, A., 2010. Design and implementation of a p300-based brain-computer interface for controlling an internet browser. IEEE Trans. Neural Syst. Rehabil. Eng. 18 (6), 599–609.

375

Muller, K.R., Tangermann, M., Dornhege, G., Krauledat, M., Curio, G., Blankertz, B., 2008. Machine learning for real-time single-trial EEGanalysis: from brain-computer interfacing to mental state monitoring. J. Neurosci. Methods 167 (1), 82–90. Muller-Putz, G.R., Scherer, R., Pfurtscheller, G., Rupp, R., 2005. EEGbased neuroprosthesis control: a step towards clinical practice. Neurosci. Lett. 382 (1–2), 169–174. Muller-Putz, G.R., Scherer, R., Pfurtscheller, G., Rupp, R., 2006. Braincomputer interfaces for control of neuroprostheses: from synchronous to asynchronous mode of operation. Biomed. Tech. (Berl.) 51 (2), 57–63. Nakanishi, M., Wang, Y., Wang, Y.T., Mitsukura, Y., Jung, T.P., 2014. A high-speed brain speller using steady-state visual evoked potentials. Int. J. Neural Syst. 24 (6), 1450019. Nam, Y., Cichocki, A., Choi, S., 2013. Common spatial patterns for steady-state somatosensory evoked potentials. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2013, 2255–2258. Nijholt, A., Bos, D.P.-O., Reuderink, B., 2009. Turning shortcomings into challenges: brain-computer interfaces for games. Entertain. Comput. 1 (2), 85–94. Norcia, A.M., Appelbaum, L.G., Ales, J.M., Cottereau, B.R., Rossion, B., 2015. The steady-state visual evoked potential in vision research: a review. J. Vis. 15 (6), 4. Oberman, L.M., Hubbard, E.M., McCleery, J.P., Altschuler, E.L., Ramachandran, V.S., Pineda, J.A., July 2005. EEG evidence for mirror neuron dysfunction in autism spectrum disorders. Brain Res. Cogn. Brain Res. 24 (2), 190–198. Ordikhani-Seyedlar, M., Lebedev, M.A., Sorensen, H.B., Puthusserypady, S., 2016. Neurofeedback therapy for enhancing visual attention: state-of-the-art and challenges. Front. Neurosci. 10, 352. Ortner, R., Allison, B.Z., Korisek, G., Gaggl, H., Pfurtscheller, G., 2011. An SSVEP BCI to control a hand orthosis for persons with tetraplegia. IEEE Trans. Neural Syst. Rehabil. Eng. 19 (1), 1–5. Ortner, R., Annen, J., von Oertzen, T., Espinosa, A., Rodriguez, J., Allison, B.Z., Edlinger, G., Laureys, S., Hamberger, M., Kammerhofer, A., et al., 2016. BCIs for DOC patients: assessment, communication, and new directions. In: International Conference on Universal Access in Human-Computer Interaction. Springer, pp. 62–71. Paggiaro, A., Birbaumer, N., Cavinato, M., Turco, C., Formaggio, E., Del Felice, A., Masiero, S., Piccione, F., 2016. Magnetoencephalography in stroke recovery and rehabilitation. Front. Neurol. 7. Parra, L.C., Spence, C.D., Gerson, A.D., Sajda, P., 2003. Response error correction-a demonstration of improved human-machine performance using real-time EEG monitoring. IEEE Trans. Neural Syst. Rehabil. Eng. 11 (2), 173–177. Peckham, P.H., Knutson, J.S., 2005. Functional electrical stimulation for neuromuscular applications. Annu. Rev. Biomed. Eng. 7, 327–360. Pfurtscheller, G., Lopes da Silva, F.H., 1999. Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin. Neurophysiol. 110 (11), 1842–1857. Pfurtscheller, G., Neuper, C., 2006. Future prospects of ERD/ERS in the context of brain-computer interface (BCI) developments. Prog. Brain Res. 159, 433–437. Pfurtscheller, G., Stancak Jr., A., Edlinger, G., 1997. On the existence of different types of central beta rhythms below 30 Hz. Electroencephalogr. Clin. Neurophysiol. 102 (4), 316–325. Pfurtscheller, G., Guger, C., Mller, G., Krausz, G., Neuper, C., 2000. Brain oscillations control hand orthosis in a tetraplegic. Neurosci. Lett. 292 (3), 211–214. Pfurtscheller, G., Muller, G., Korisek, G., February 2002. Mental activity hand orthosis control using the EEG: a case study. Rehabil. (Stuttg.) 41 (1), 48–52. Pfurtscheller, G., Solis-Escalante, T., Ortner, R., Linortner, P., MullerPutz, G.R., 2010. Self-paced operation of an SSVEP-based orthosis with and without an imagery-based “brain switch:” a feasibility study towards a hybrid BCI. IEEE Trans. Neural Syst. Rehabil. Eng. 18 (4), 409–414.

V.  BRAIN, COMPUTER AND MACHINE INTERFACING

376

26.  NONINVASIVE BRAIN–COMPUTER INTERFACES

Pfurtscheller, G., 1981. Central beta rhythm during sensorimotor activities in man. Electroencephalogr. Clin. Neurophysiol. 51 (3), 253–264. Pineda, J.A., Carrasco, K., Datko, M., Pillen, S., Schalles, M., April 28, 2014. Neurofeedback training produces normalization in behavioural and electrophysiological measures of high-functioning autism. Philos. Trans. R. Soc. Lond. B Biol. Sci. 369 (1644), 20130183. http://dx.doi.org/10.1098/rstb.2013.0183. Print 2014. Pohlmeyer, E.A., Wang, J., Jangraw, D.C., Lou, B., Chang, S.F., Sajda, P., 2011. Closing the loop in cortically-coupled computer vision: a brain-computer interface for searching image databases. J. Neural Eng. 8 (3), 036025. Polich, J., 2004. Clinical application of the P300 event-related brain potential. Phys. Med. Rehabil. Clin. N. Am. 15 (1), 133–161. Pons, J., Torricelli, D., 2014. Emerging Therapies in Neurorehabilitation. Springer. Prinzel, L.J., Freeman, F.G., Scerbo, M.W., Mikulka, P.J., Pope, A.T., 2000. A closed-loop system for examining psychophysiological measures for adaptive task allocation. Int. J. Aviat. Psychol. 10 (4), 393–410. Prinzel III, L.J., Pope, A.T., Freeman, F.G., 2002. Physiological selfregulation and adaptive automation. Int. J. Aviat. Psychol. 12 (2), 179–196. Ramos-Murguialday, A., Schurholz, M., Caggiano, V., Wildgruber, M., Caria, A., Hammer, E.M., Halder, S., Birbaumer, N., 2012. Proprioceptive feedback and brain computer interface (BCI) based neuroprostheses. PLoS One 7 (10), e47048. http://dx.doi. org/10.1371/journal.pone.0047048. Epub October 5, 2012. Reissland, J., Zander, T.O., 2009. Automated detection of bluffing in a game — revealing a complex covert user state with a passive BCI. In: Proceedings of the Human Factors and Ergonomics Society Europe Chapter Annual Meeting, Linkoeping, Sweden. Remsik, A., Young, B., Vermilyea, R., Kiekhoefer, L., Abrams, J., Evander Elmore, S., Schultz, P., Nair, V., Edwards, D., Williams, J., Prabhakaran, V., 2016. A review of the progression and future implications of brain-computer interface therapies for restoration of distal upper extremity motor function after stroke. Expert Rev. Med. Devices 13 (5), 445–454. Riccio, A., Pichiorri, F., Schettini, F., Toppi, J., Risetti, M., Formisano, R., Molinari, M., Astolfi, L., Cincotti, F., Mattia, D., 2016. Interfacing brain with computer to improve communication and rehabilitation after brain damage. Prog. Brain Res. 228, 357–387. Rockstroh, B., Elbert, T., Birbaumer, N., Wolf, P., Duchting-Roth, A., Reker, M., Daum, I., Lutzenberger, W., Dich-gans, J., 1993. Cortical self-regulation in patients with epilepsies. Epilepsy Res. 14 (1), 63–72. Rohani, D.A., Henning, W.S., Thomsen, C.E., Kjaer, T.W., Puthusserypady, S., Sorensen, H.B., 2012. BCI using imaginary movements: the simulator. Comput. Methods Prog. Biomed. 111 (2), 300–307. Rohm, M., Schneiders, M., Muller, C., Kreilinger, A., Kaiser, V., MullerPutz, G.R., Rupp, R., October 2013. Hybrid brain-computer interfaces and hybrid neuroprostheses for restoration of upper limb functions in individuals with highlevel spinal cord injury. Artif. Intell. Med. 59 (2), 133–142. http://dx.doi.org/10.1016/j.artmed.2013.07.004. Epub September 13, 2013. Rohm, M., Schneiders, M., Muller, C., Kreilinger, A., Kaiser, V., MullerPutz, G.R., Rupp, R., 2014. Hybrid brain- computer interfaces and hybrid neuroprostheses for restoration of upper limb functions in individuals with highlevel spinal cord injury. Artif. Intell. Med. 59 (2), 133–142. Rossi, S., Rossini, P.M., 2004. TMS in cognitive plasticity and the potential for rehabilitation. Trends Cogn. Sci. 8 (6), 273–279. Rupp, R., 2014. Challenges in clinical applications of brain computer interfaces in individuals with spinal cord injury. Front. Neuroeng. 7, 38. Sabathiel, N., Irimia, D.C., Allison, B.Z., Guger, C., Edlinger, G., 2016. Paired associative stimulation with brain- computer interfaces: a new paradigm for stroke rehabilitation. In: International Conference on Augmented Cognition. Springer, pp. 261–272.

Schalk, G., Wolpaw, J.R., McFarland, D.J., Pfurtscheller, G., 2000. EEG-based communication: presence of an error potential. Clin. Neurophysiol. 111 (12), 2138–2144. Schalk, G., McFarland, D.J., Hinterberger, T., Birbaumer, N., Wolpaw, J.R., 2004. BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE Trans. Biomed. Eng. 51 (6), 1034–1043. Scherer, R., Proell, M., Allison, B., Mueller-Putz, G.R., 2012. New input modalities for modern game design and virtual embodiment. In: 2012 IEEE Virtual Reality Workshops (VRW). IEEE, pp. 163–164. Schmorrow, D., Stanney, K.M., Wilson, G., Young, P., 2006. Augmented cognition in human-system interaction. In: Handbook of Human Factors and Ergonomics, third ed., pp. 1364–1383. Schnuerer, A., Espinosa, A., Guger, C., 2015. mindBEAGLE: a BCI for communication and assessment of consciousness for patients with disorders of consciousness. Clin. Neurophysiol. 126 (8), e152–e153. Sellers, E.W., Kubler, A., Donchin, E., 2006. Brain-computer interface research at the University of South Florida cognitive psychophysiology laboratory: the P300 speller. IEEE Trans. Neural Syst. Rehabil. Eng. 14 (2), 221–224. Severens, M., Van der Waal, M., Farquhar, J., Desain, P., 2014. Comparing tactile and visual gaze-independent brain-computer interfaces in patients with amyotrophic lateral sclerosis and healthy users. Clin. Neurophysiol. 125 (11), 2297–2304. Shibasaki, H., Hallett, M., 2006. What is the Bereitschaftspotential? Clin. Neurophysiol. 117 (11), 2341–2356. Siebner, H.R., Tormos, J.M., Ceballos-Baumann, A.O., Auer, C., Catala, M.D., Conrad, B., Pascual-Leone, A., 1999. Low-frequency repetitive transcranial magnetic stimulation of the motor cortex in writer’s cramp. Neurology 52 (3), 529–537. Silverman, D., Masland, W.S., Rodin, E.A., 1968. Electroencephalography. Prog. Neurol. Psychiatry 23, 303–337. Sitaram, R., Caria, A., Veit, R., Gaber, T., Ruiz, S., Birbaumer, N., 2014. Volitional control of the anterior insula in criminal psychopaths using real-time fMRI neurofeedback: a pilot study. Front. Behav. Neurosci. 8, 344. Soekadar, S.R., Birbaumer, N., Slutzky, M.W., Cohen, L.G., 2015. Brainmachine interfaces in neurorehabilitation of stroke. Neurobiol. Dis. 83, 172–179. St. John, M., Kobus, D.A., Morrison, J.G., Schmorrow, D., 2004. Overview of the DARPA augmented cognition technical integration experiment. Int. J. Hum. Comput. Interact. 17 (2), 131–149. Sterman, M.B., Mann, C.A., 1995. Concepts and applications of EEG analysis in aviation performance evaluation. Biol. Psychol. 40 (1–2), 115–130. Straudi, S., Fregni, F., Martinuzzi, C., Pavarelli, C., Salvioli, S., Basaglia, N., 2016. tDCS and robotics on upper limb stroke rehabilitation: effect modification by stroke duration and type of stroke. Biomed. Res. Int. 2016. Tong, J., Lin, Q., Xiao, R., Ding, L., 2016. Combining multiple features for error detection and its application in brain-computer interface. Biomed. Eng. Online 15, 17. Townsend, G., LaPallo, B.K., Boulay, C.B., Krusienski, D.J., Frye, G.E., Hauser, C.K., Schwartz, N.E., Vaughan, T.M., Wolpaw, J.R., Sellers, E.W., 2010. A novel P300-based brain-computer interface stimulus presentation paradigm: moving beyond rows and columns. Clin. Neurophysiol. 121 (7), 1109–1120. Ushiba, J., Soekadar, S., 2016. Brain-machine interfaces for rehabilitation of poststroke hemiplegia. Prog. Brain Res. 228, 163–183. Vaughan, T.M., McFarland, D.J., Schalk, G., Sarnacki, W.A., Krusienski, D.J., Sellers, E.W., Wolpaw, J.R., 2006. The Wadsworth BCI research and development program: at home with BCI. IEEE Trans. Neural Syst. Rehabil. Eng. 14 (2), 229–233. Vecchiato, G., Astolfi, L., De Vico Fallani, F., Toppi, J., Aloise, F., Bez, F., Wei, D., Kong, W., Dai, J., Cincotti, F., Mattia, D., Babiloni, F., 2011. On the use of EEG or MEG brain imaging tools in neuromarketing research. Comput. Intell. Neurosci. 2011, 643489.

V.  BRAIN, COMPUTER AND MACHINE INTERFACING

References

Vecchiato, G., Kong, W., Maglione, A.G., Wei, D., 2012. Understanding the impact of TV commercials: electrical neuroimaging. IEEE Pulse 3 (3), 42–47. Vidal, J.J., 1973. Toward direct brain-computer communication. Annu. Rev. Biophys. Bioeng. 2, 157–180. Walter, W., Cooper, R., Aldridge, V., McCallum, W., Winter, A., 1964. Contingent negative variation: an electric sign of sensori-motor association and expectancy in the human brain. Nature 203, 380–384. Wang, Y.T., Huang, K.C., Wei, C.S., Huang, T.Y., Ko, L.W., Lin, C.T., Cheng, C.K., Jung, T.P., 2014. Developing an EEG-based on-line closed-loop lapse detection and mitigation system. Front. Neurosci. 8, 321. Wessel, J.R., 2012. Error awareness and the error-related negativity: evaluating the first decade of evidence. Front. Hum. Neurosci. 6, 88. Wolpaw, J.R., McFarland, D.J., Vaughan, T.M., Schalk, G., 2003. The Wadsworth Center brain-computer interface (BCI) research and development program. IEEE Trans. Neural Syst. Rehabil. Eng. 11 (2), 204–207. Wriessnegger, S.C., Hackhofer, D., Muller-Putz, G.R., 2015. Classification of unconscious like/dislike decisions: first results towards a novel application for BCI technology. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2015, 2331–2334.

377

Xu, R., Jiang, N., Dosen, S., Lin, C., Mrachacz-Kersting, N., Dremstrup, K., Farina, D., 2016a. Endogenous sensory discrimination and selection by a fast brain switch for a high transfer rate brain-computer interface. IEEE Trans. Neural Syst. Rehabil. Eng. 24 (8), 901–910 Xu, R., Jiang, N., Mrachacz-Kersting, N., Dremstrup, K., Farina, D., January 21, 2016b. Factors of influence on the performance of a short-latency non-invasive brain switch: evidence in healthy individuals and implication for motor function rehabilitation. Front. Neurosci. 9, 527. http://dx.doi.org/10.3389/fnins.2015.00527. eCollection 2015, 24 (8), 901–910. Young, B.M., Nigogosyan, Z., Remsik, A., Walton, L.M., Song, J., Nair, V.A., Grogan, S.W., Tyler, M.E., Edwards, D.F., Caldera, K., Sattin, J.A., Williams, J.C., Prabhakaran, V., 2014. Changes in functional connectivity correlate with behavioral gains in stroke patients after therapy using a brain-computer interface device. Front. Neuroeng. 7, 25. Zander, T.O., Jatzev, S., 2012. Context-aware brain-computer interfaces: exploring the information space of user, technical system and environment. J. Neural Eng. 9 (1), 016003. Zoltan, B., Brunner, P., Zeitlin, D.M., Adamovich-Zeitlin, R., Ruiz, N., Cannella, A., Paradiso, R., McFarland, D., Jian, W., Carmack, C., Wolpaw, J., Vaughan, T., 2016. Development of an eyes-closed SSVEP-based BCI for people with impaired vision. In: Society for Neuroscience Annual Meeting. Society for Neuroscience.

V.  BRAIN, COMPUTER AND MACHINE INTERFACING