The effects of visual distractors on cognitive load in a motor imagery brain-computer interface

Journal Pre-proof The Effects of Visual Distractors on Cognitive Load in a Motor Imagery Brain-Computer Interface Zahra Emami, Tom Chau

PII:

S0166-4328(19)30002-6

DOI:

https://doi.org/10.1016/j.bbr.2019.112240

Reference:

BBR 112240

To appear in:

Behavioural Brain Research

Received Date:

6 January 2019

Revised Date:

2 August 2019

Accepted Date:

13 September 2019

Please cite this article as: Emami Z, Chau T, The Effects of Visual Distractors on Cognitive Load in a Motor Imagery Brain-Computer Interface, Behavioural Brain Research (2019), doi: https://doi.org/10.1016/j.bbr.2019.112240

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. © 2019 Published by Elsevier.

The Effects of Visual Distractors on Cognitive Load in a Motor Imagery Brain-Computer Interface Zahra Emami2 and Tom Chau1, 2,* 1 Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada, 2 Hospital for Sick Children, Toronto, ON, Canada * Corresponding author. Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, 150 Kilgour Road, Toronto, Ontario M4G 1R8, Canada E-mail address: [email protected]

ro



Transient, visual distractors significantly increase a physiological index of cognitive load. A ratio of parietal theta to parietal alpha power is most sensitive to manipulations to task difficulty on a BCI interface. Perceived cognitive load has a strong, negative correlation with BCI classification accuracy for low-performing participants.

-p

 

of

Highlights

Abstract

re

A brain-computer interface (BCI) is a system that translates neural activity into a practical output. Its functionality, therefore, depends not only on the computer itself, but also on the cognitive

lP

system of the user. Distractors have the potential to capture attention, increase cognitive load, and may therefore impact BCI use. The purpose of the current study is to determine the effects of small visual distractors on the cognitive load of users of a motor imagery-BCI, and to examine whether

ur na

these distractor-mediated effects can be improved by modifying the task interface. Sixteen typically-developed participants completed two sessions of online motor imagery to control an EEG-BCI, under conditions of no distractors, visual distractors, and cognitive strategies (intended to mitigate cognitive load) amid distractors. Cognitive load for each session was assessed through both a ratio of theta to alpha power and the NASA-Task Load Index (NASA-TLX). Task-irrelevant

Jo

visual stimuli were found to significantly increase the objective measure of cognitive load, particularly for parietal channels. Subjective cognitive load as indexed by the NASA-TLX was predictive of a decrease in BCI performance for participants with below 0.75 classification accuracy (R2=0.32, p<0.001), which may indicate a differential susceptibility to changes in workload for “low”-performing participants. Quantifying and addressing the increased cognitive load imparted by distractors on BCI users can aid in the future applicability of the technology in real-world settings. Keywords: brain-computer interface, motor imagery, distractor, cognitive load, theta/alpha, electroencephalography 1

1. Introduction Distractors are task-irrelevant stimuli which have the potential to impart a cost on the processing of task-relevant information and on the current goal-driven behaviour. However, load theory suggests that whether distractors have an impact on information processing depends on the level of load on cognitive control functions, such as perceptual processing and working memory [1, 2]. Cognitive load represents the burden that performing a particular task imposes on the cognitive system [3, 4]. Based on load theory, a task with high perceptual load, containing large, complex sets of information, will necessitate most of the cognitive system’s processing

of

capacity, thus preventing the processing of task-irrelevant stimuli. However, when the task does not carry a great perceptual load, there will be leftover attentional resources that can ‘spill over’

ro

to the processing of distractors, resulting in greater distractor-related interference [1, 2]. Various studies have examined the influence of perceptual load on distractor processing, providing

-p

support for load theory (for a review see [5]). Working memory load, another aspect of task difficulty, also appears to have a significant influence on distraction effects, exerting reliable top-

re

down control on the involuntary orienting of attention [3]. Lavie (2004) showed that selective attention in a flanker task is more prone to interference when concurrent working memory load is

lP

high than when it is low [3]. When low demands are placed on cognitive control processes such as working memory, processing task-irrelevant information can be better inhibited, and cognitive control can be more effective [1]. Therefore, working memory load and perceptual load have

ur na

been described as being diametrically opposed: flanker interference is decreased by high perceptual load, and increased by high working memory load [3]. Furthermore, directing attention to task-related stimuli has been shown to negate the suppressive effects of distracting stimuli, and can enhance efficient information processing [6, 7], effectively reducing the load on the cognitive system. The load theory of selective attention corroborates the

Jo

idea that task-irrelevant information has the potential to constrain the availability of cognitive resources for the processing of relevant information, and by producing a distraction, can have a detrimental effect on task performance. The theory also suggests that manipulations of task difficulty, based on perceptual and working memory load, may restrict the effect of distractors on

2

the cognitive system. The relationship between the cognitive system and task load is depicted in

ro

of

Figure 1.

-p

FIGURE 1. A theoretical schematic of the relationship between the working memory load and perceptual load of a task on the cognitive system’s ability to selectively allocate attention to taskrelevant information.

re

Different methods have been used to assess cognitive load. Subjective rating scales, such as the NASA-Task Load Index (NASA-TLX), which takes into account the perceived level of

lP

mental effort the user experiences during a task, can be used to provide a measure of the accumulated cognitive load over the course of a task [8]. However, since these rating scales do not provide a continuous assessment of cognitive load and might be influenced by various factors

ur na

such as the user’s expectation [8], other measures can be used in conjunction. A continuous and more direct form of cognitive load assessment includes a physiological measure provided by electroencephalography (EEG), which captures cortical electrical activity. The alpha rhythm (713 Hz), measured over the parietal cortex, is associated with a relaxed wakefulness without attention, and is found to be suppressed during mental concentration and alert attention [8, 9].

Jo

Therefore, the alpha rhythm can be a useful index to measure mental effort [10]. A linear suppression, or event-related desynchronization (ERD), of the alpha rhythm has been found to correspond to increasing levels of working memory load [11-13]. Additionally, frontal theta activity (3-7 Hz) also appears to correlate with task difficulty, increasing in magnitude and manifesting as an event-related synchronization (ERS) in conditions of high cognitive load [14, 15]. The differential correlation of these two rhythms to task difficulty can be combined through determination of the ERS/ERD index of theta/alpha compared to baseline [8]. This index of 3

alpha and theta power has been shown to be sensitive to different levels of task difficulty and cognitive load [8, 11, 16]. Therefore, a measure of attention-related EEG rhythms may provide neural insight into the influence of distractors on the cognitive system. Research examining the effect of distractors often entail specific, controlled tasks that have little real-world relevance. Consequently, the effect of distractions on task performance are of little to no consequence for the individual in such laboratory conditions. However, in an everyday, real-world setting, the tasks that we take on are often very important, and distractions may be especially detrimental. One such case where a task is uniquely susceptible to the effect of distractions, and

of

sensitive to its consequences, is in a brain-computer interface (BCI). A BCI is a technology

affording a direct channel between the nervous system and the environment. Performing a certain

ro

mental task, such as the imagination of motor movement, or motor imagery, can produce distinct and reliable changes to brain patterns [17]. These changes in brain activity can be used as a signal

-p

to control a BCI.

The ease of use of a BCI is an important consideration for the effectiveness of this technology

re

[18, 19]. Distractors may enhance the cognitive burden of users. In turn, a high cognitive workload can produce stress and errors [20] and diminish a BCI user’s quality of experience.

lP

Understanding how to design a BCI interface that can reduce cognitive workload is crucial, as it is one of the prominent limiting factors facing the technology [17, 20]. It is thus essential to examine the effect and extent of distractions on the cognitive load of users of a BCI, and to consider

ur na

strategies to counter the potential compromising effects of distractors in order to construct a BCI system suitable for use in real-world situations. In a previous study, we have shown that visual distractors can disrupt motor imagery-related patterns in alpha and beta ERS/ERD during BCI control [21]. The results of the previous study indicated that BCI performance can remain robust to distractor-mediated changes in brain activity, but the potential for real-world applicability of the

Jo

technology needs to be further examined via a complementary investigation of the impact of distractors on the user. As an extension to our previous findings, the following study examines the effect of task-irrelevant information on the cognitive load of the BCI users, employing both objective and subjective measures of cognitive load. While there have been previous studies focusing on improving user perceived effort during neurofeedback, these attempts are mainly limited to improving the BCI signal processing algorithm [22, 23]. However, changes to the BCI algorithm may not necessarily account for changes in brain activity outside of the BCI control signal, which may influence perceived user ease and effort. The current study proposes a user4

oriented and simple addition to the BCI task interface to minimize the influence of environmental distractors on cognitive load, based on load theory.

2. Methodology 2.1 Subjects Sixteen healthy participants were recruited for the study (13 females, 13 right-handed, mean age of 24 ± 3 years) after providing written informed consent. The participants were selected

of

based on whether they could fulfill two-class online BCI control above chance levels using motor imagery, according to the screening procedures outlined in Emami & Chau (2018).

ro

Participants had normal or corrected-to-normal vision, without any neurological or motor

disorders, according to self-report. As well, participants refrained from consuming caffeine

-p

during the 2 hours prior to each session. The experimental protocols were approved by the ethics committee of the University of Toronto and Holland Bloorview Kids Rehabilitation Hospital.

re

2.2 Instrumentation Cortical electrical signals were recorded using a

lP

BrainAmp DC amplifier [24]. Twenty-five active electrodes

were placed on the scalp, in accordance with the International 10-20 system [25], over the primary sensorimotor area, the

ur na

supplementary motor area, frontal and parietal regions (Figure 2). The AFz location was used as ground, and all signals were referenced to the linked mastoids. Electrodes over the sensorimotor cortex, including the FC3, FC1, FCz, FC2, FC4, C3, C1, Cz, C2, C4, CP3, CP1, CPz, CP2, and

Jo

CP4, were used for capturing the brain-computer interface control signal. All of the electrodes, including the remaining electrodes over the frontal and parietal brain regions, were used for post-hoc analysis of cognitive load. Electrode impedance was kept below 20 kΩ. During each session, participants were seated in a comfortable chair facing a computer screen approximately 1 meter away for displaying task cues and feedback. 5

FIGURE 2. Electrode placement. The dashed line indicates those channels used for the BCI, while the dotted line indicates those used for post-hoc analyses of cognitive load.

2.3 Task Protocol Figure 3 depicts the online BCI task interface that was used. Each participant completed two sessions. Each session began with 4 “offline” blocks for the purpose of providing training data for the classifier, followed by 8 “online” blocks that included trial-by-trial feedback. The blocks were separated by user-determined breaks. Each block consisted of 18 motor imagery trials, 9 trials per class of motor imagery. Therefore, each session consisted of

ur na

lP

re

-p

ro

of

72 offline trials and 144 online trials (72 online trials of each class).

Jo

FIGURE 3. The three conditions in the experimental sessions. The no-distractor condition involved only the cues pertaining to the motor imagery task, the distractor condition included the presentation of a task-irrelevant stimulus during the motor imagery period, and the cognitive strategy conditions involved the implementation of cognitive strategies along with the distractor. The cognitive strategies consisted of either (1) re-displaying the task cue after distractor offset, or (2) enhancing the perceptual complexity of the task interface for the entire length of the trial.

During the first 2 seconds of each 9-10 second-long trial, a fixation cross was presented, where the participant was instructed to relax and avoid any motion. Following the 2 second fixation, a visual cue was presented on the screen for 500 milliseconds indicating the start of the 6

motor imagery period, and the type of imagery to be performed. A white arrow pointing to the left indicated left-hand kinesthetic motor imagery, and an arrow pointing to the right indicated right-hand kinesthetic motor imagery. Participants were instructed to choose imagined finger movements that were continuous during the period of the motor imagery trial. An example of repetitive, sequential finger curling was provided to every participant, but each participant was permitted the freedom to select their own personal strategy that best suited their ease and perceived performance. The participants were instructed to perform the appropriate imagery following the cue while a fixation cross was presented on the screen (in order to prevent eye

of

movements), until a second arrow was presented. For the offline trials, the second cue was

always a black arrow signaling the end of the motor imagery period. For the online trials, the

ro

second arrow provided feedback based on the performance of the current trial: a grey arrow indicated unsuccessful classification while a black arrow indicated successful performance.

-p

Each trial was separated by a blank screen for 1-2 seconds. During this period, the participant was asked to remain relaxed while waiting for the next task. All task stimuli were presented on

re

the center of the screen to prevent saccadic eye movements.

Task-irrelevant visual stimuli were presented in approximately 30% of the 144 online trials in

lP

each of the experimental sessions. A single task-irrelevant stimulus, or distractor, was presented for 500 milliseconds, at a random onset of 0.5-2.5 seconds following the motor imagery cue and during the motor imagery period. The distractors were black and white visual stimuli, ranging

ur na

from simple shapes to complex images (see Appendix A for a list of the distractors). The distractors were displayed at the center of the screen and on a similar scale as the motor imagery cues and fixation cross, in order to prevent saccadic eye movements. There were 24 unique distractors used in the study, and they were pseudo-randomly presented with a constraint of 3 consecutive runs as the most proximal reoccurrence in order to minimize the risk of diminishing

Jo

the effect of any one distractor. Approximately 30% of the distractor trials also included one of two cognitive strategies. The proposed cognitive strategies were based on altering the perceptual and working memory

load of the task interface, in order to attempt to account for the potential involuntary orienting of attention to the task-irrelevant distractor. One of these strategies was intended to reduce the working memory load during the task by briefly re-displaying the motor imagery cue (the arrow) on the screen 500 milliseconds following distractor offset. The second strategy was intended to enhance the perceptual load of the task interface to prevent involuntary orienting away from the 7

task and to reduce distractor salience. This was done by replacing the blank background with one simulating visual noise, for the entire length of the trial. (Figure 3) 2.4 BCI Performance The BCI performance was determined based on the accuracy of classification of right-hand and left-hand motor imagery in the online trials, using the EEG data captured over the sensorimotor cortex. Data processing was performed using the EEGLAB toolbox [26] in Matlab as well as in-house scripts. The EEG signal was pre-processed by band-passing the

of

signal between 7 and 30 Hz. In order to enhance separability between right- and left-hand motor imagery, a common spatial pattern (CSP) algorithm was applied to the task-related EEG

ro

components [27, 28] to reduce the number of channels to the top 3. Relevant features, including log-variance and alpha and beta band power, were extracted from the channel-selected data.

-p

The combined pool of features was then further reduced to a set of 5 most discriminative features using a fast correlation-based filter (FCBF) [29]. Classification of the selected pool of

re

features was performed based on linear discriminant analysis. Accuracy values could range from 0 to 1, with 0.56 being the adjusted chance level, based on the number of trials in the session [30]. The EEG data has been previously analyzed with respect to BCI performance and

lP

distractor-related changes to motor imagery brain activation; the BCI algorithm and results are reported in detail in Emami & Chau (2018).

ur na

2.5 NASA-Task Load Index

In order to ascertain the accumulated perceived load, the NASA-Task Load Index (NASATLX) was administered. The NASA-TLX is a widely-used, multi-dimensional 21-point Likert scale with six subscale ratings covering aspects of task workload, including mental, temporal and physical demand, perceived performance, effort, and frustration [31, 32]. The questionnaire

Jo

was administered using the paper and pencil version immediately after 5 selected blocks during the course of the experimental sessions: following the training blocks (blocks 1-4), and after the 6th, 8th, 10th, and 12th blocks, to provide a qualitative measure of changes in subjective workload over the entire session [31, 32]. Therefore, for each participant’s session, 5 values for subjective cognitive load were acquired from the raw NASA-TLX score, which was computed as the total of the six subscale ratings. (Refer to Appendix B for the version of the NASA-TLX used in the study) 8

2.6 Analysis of Objective Load The EEG signal of the online trials was down-sampled from the acquisition sampling rate of 1000 Hz to 256 Hz, and a Butterworth bandpass filter was applied with low and high cut-off frequencies of 3 Hz and 30 Hz, respectively. This frequency band encompasses the theta and alpha rhythms. All trials were epoched to include the 2 second baseline and the entire 5 second motor imagery period, providing 144 trials in total. The trials were split into those without distractors, those with distractors, and those with distractors and cognitive strategies, providing 96, 32, and 16 trials per condition and per session, respectively. A short-time Fourier transform

of

(STFT) was applied to the signal to extract theta and alpha band power of the motor imagery period, which was normalized against the corresponding band power of the baseline period. The

ro

ratio of frontal theta power to central-parietal alpha power was computed for each trial and for each condition.

-p

A 3-way ANOVA was conducted to determine a statistical difference in cognitive load

distractors, cognitive strategies).

re

between subjects, sessions (online blocks in session 1 & 2) and conditions (no distractors,

To determine whether the distractors increased cognitive load, a one-sided Wilcoxon rank-

lP

sum test was used to examine the difference in power ratio between the no- distractor trials and the distractor trials. A two-sided test of significance was also used to determine the difference in power ratio between distractor trials and cognitive strategy trials. Finally, a two-sided rank sum

ur na

test was applied to determine whether the cognitive strategies differed significantly from the nodistractor trials. These tests were applied to both experimental sessions for each participant. In order to confirm the suitability of the frontal theta to parietal alpha power measure, the specific channel sensitivity to the conditions was investigated. A two-sided Wilcoxon rank sum test was also performed for the ratio between a single channel’s theta power and a single

Jo

channel’s alpha power, for all pairs of channels. The number of times each channel pair provided a significant p value (p<0.05), across all three aforementioned tests, sessions, and participants, was determined.

2.7 Analysis of Subjective Load A robust linear regression was performed on the NASA-TLX scores and the BCI classification accuracies. The scores and accuracies were divided into the 4 online blocks. Each participant completed 2 sessions of 4 online blocks, for a total of 8 blocks of online accuracies and NASA9

TLX scores for each participant. The data were then separated into two groups: high performers, where accuracies equaled or exceeded 0.75, and low performers, where accuracies fell below 0.75. This partition was based on the recommended 70% classification for a practical BCI [33] and a post-hoc analysis of the mean performance of the participants in the two experimental sessions. The grouping of participants into low- and high-performing participants was intended to identify whether cognitive load differentially influenced participants based on their performance. For each group, a robust linear regression was applied to estimate the strength of

of

the outlier-resistant regression fit to the grouped data.

3. Results

ro

3.1 Objective Cognitive Load

There was a significant main effect of subject (F15,4482=125.81, p<0.001) and condition

-p

(F2,4482=48.10, p<0.001) on cognitive load, but no significant effect for the session number. There was also a significant interaction effect between subject and session (F30,4482=9.02, p<0.001) and

re

between subject and condition (F30,4482=2.13, p<0.01). No significant 3-way interaction between the factors was found. The effect of distractors and cognitive strategies on the frontal theta to

lP

parietal alpha index of cognitive load is displayed in Figure 4. Out of 16 participants, 10 showed a significant increase in the theta/alpha index between no-distractor trials and distractor trials, for at least 1 session, based on the one-sided rank sum tests (p<0.025). Out of all sessions across

ur na

participants, 37.5% exhibited this effect. Out of 16 participants, 11 participants showed a significant increase in the theta/alpha index between no-distractor trials and the cognitive strategy trials, for at least 1 session, for a total of 40.6% of all sessions across participants. In session 1, for 8 of the 16 participants, the cognitive strategy trials resulted in a significantly increased theta/alpha value as compared to the distractor trials. This difference was not significant for any

Jo

participants in session 2. Four main observations across participants and sessions are noteworthy: a)

In 20% of all of the sessions across participants, the distractor trials did not result in a significantly higher cognitive load index than the no-distractor trials, whereas the cognitive strategy trials did result in a significantly larger cognitive load index as compared to the nodistractor and distractor trials (Figure 5a).

b) In 25% of all sessions across participants, the distractor and cognitive strategy trials exhibited an increased cognitive load index as compared to the no-distractor trials 10

(Figure 5b). c)

For participants 8 and 15’s second session, a significant difference between nodistractor trials and distractor trials was observed, with no significant difference between cognitive strategy trials and the no-distractor trials (Figure 5c).

d)

No effect of condition was observed for 34% of all sessions across participants. This

Jo

ur na

lP

re

-p

ro

of

percentage is mainly attributed to 5 out of 16 participants (Figure 5d).

FIGURE 4.

Hinton plots of the left-tailed significant differences between no-distractor trials vs. distractor trials (left), no-distractor trials vs. cognitive strategy trials (middle), and distractor trials vs. cognitive strategy trials (right), for every participant and session.

11

of

3.2 Channel Sensitivity to Objective Load

-p

ro

FIGURE 5. The frontal theta to central-parietal alpha power for each condition for (a) participant 5 and (b) participant 13’s first session, and (c) participant 15 and (d) participant 16’s second session. Bars indicate a significant difference between the two conditions, p<0.05.

re

The rank sum test on each channel pairing revealed that a combination of parietal channels for alpha power and parietal channels for theta power resulted in the greatest number of

lP

significant differences between the three experimental conditions (no-distractor, distractor, cognitive strategy) across all participants and sessions. While the pairing of parietal theta power and parietal alpha power offers the greatest sensitivity to changes in task condition, the

ur na

combination of frontal theta power and central alpha power was least affected by the

Jo

conditions. These results are displayed in Figure 6.

12

of ro -p

lP

re

FIGURE 6. The number of significant comparisons (p<0.05) between the three conditions, using the theta/alpha index calculated from a single channel’s theta power over a single channel’s alpha power, for all channel pairings, and across all participants and sessions.

Using the parietal channels for the ratio of theta power to alpha power, approximately 60% of the sessions showed a significant increase in the ratio for distractor trials compared to no-

ur na

distractor trials. Approximately 31% of session showed an increase in the index for the cognitive strategy trials compared to the distractor trials. And, 62.5% of the sessions exhibited an increase of the ratio for the cognitive strategy trials compared to the no-distractor trials. Using the most sensitive channels for the ratio of theta to alpha power, instead of the conventional frontal theta to central-parietal alpha ratio, resulted in an increased sensitivity in the three tests of significance

Jo

(no-distractor vs. distractor trials, distractor vs. cognitive strategy trials, no-distractor vs. cognitive strategy trials) of 22.5%, 6 %, and 21.9%, respectively. 3.3 Subjective Cognitive Load Across participants, the mean total workload as a percentage of the possible score was 36%, which is between a low and moderate workload. There was no significant difference between total or subdomain NASA-TLX scores between the two experimental sessions. The mean subdomain scores, across participants and sessions, as a percentage of the total 13

workload score indicate that the BCI task was relatively high in effort (25.3%) and mental demand (23.5%) but low in frustration (12.8%) and physical demand (9.3%). The relationship between subjective cognitive load and BCI performance in the online trials was examined across all users. A linear model was fitted for the combined group, and a significant regression equation was found (F(126)=6.6, p=0.011), with an R2=0.05. As a post-hoc investigation of this relationship, the sample was demarcated according to BCI accuracy with a threshold of 0.75. The relationship between subjective cognitive load, as measured by the NASA-TLX, and the online BCI performance, indexed by

of

classification accuracy, for low- and high- performing groups is displayed in Figure 7. A

model was fitted only to the low-performing group (F(37)=17.2, p<0.0002, R2=0.32) with

ro

classification accuracies at or below 0.75, and another was fitted exclusively to the highperforming group (F(87)=1.21, p>0.05, R2=0.014). The strength of the linear model to

-p

predict performance based on cognitive load was greatest for the low-performing group, where an increase in subjective cognitive load was predictive of a decrease in BCI

Jo

ur na

lP

re

performance.

FIGURE 7. The NASA-TLX scores and BCI classification accuracies per online block and per session, for each participant. The filled dot represents data corresponding to high performers, while the white dot corresponds to those with an accuracy below 0.75. A robust linear regression line is fitted for the high-performing group and for the lowperforming group. 14

4. Discussion As expected, the distractors resulted in a higher objective cognitive load in most participants. While traditionally, frontal theta power is used as a measure of cognitive load [8], parietal theta power was found to be more sensitive to the condition manipulations in this study. There is, however, a strong coherence between frontal and parietal theta [34]. Frontal theta power is considered a source of the central executive, whereas parietal theta is associated with working memory and visual processing [34-37]. For a BCI that provides visual feedback, such as the one

of

used in this study, activation of visual processing areas like the parietal cortex is expected. As well, Cooper (2015) demonstrated that paradigms which rely on reactive control exhibit a fronto-

ro

parietal oscillatory synchronization of the theta band [34], which is associated with interference control and an increased need for allocation of mental resources [35, 36]. This is consistent with

-p

current theories of selective attention [1, 3]. The successful completion of the motor imagery task in the presence of distractors requires the actions of the central executive in managing interference

re

and appropriately assigning cognitive resources [38, 39]. The sensitivity of parietal alpha power to manipulations of cognitive load is also consistent with the literature. Alpha networks are not as

lP

extensive as those for theta [34]. Task switching, which entails executive control of selective attention, has been found to be associated with a reduction in alpha power. It is also associated with an increase in theta coherence, but not alpha coherence, across fronto-parietal networks [37,

ur na

40]. Based on the results of this study, we suggest that a more comprehensive measure of objective cognitive load can include a ratio of frontal-parietal theta power to parietal alpha power, particularly for visual tasks involving interference control. The motor imagery task itself may have played a role in the effect of distractors on cognitive load and on the effectiveness of the cognitive strategies. Performing motor imagery

Jo

likely imparted a relatively mild load on cognitive functions such as working memory, although to differing degrees across participants, since the users reported low to moderate effort scores on the NASA-TLX. As well, because the participants were allowed the freedom to select their own personal strategy for kinesthetic motor imagery over the course of their training, we can expect that the task was of relatively low personal load, as compared to that of a prescribed motor imagery movement that may have been difficult for certain individuals. Based on load theory, a task with low working memory load allows individuals to better account for interference effects [1, 5]. However, the BCI interface used in this study was of 15

fairly low perceptual load; at any given time, a blank screen was presented with a single neutral, black-white visual stimulus, even during distractor presentation. Based on load theory, the fact that the standard motor imagery task involved a perceptually simple interface may have contributed to the observed increase in cognitive load upon presentation of distractors, and even with the cognitive strategies. The similarity between the cognitive strategies and the distractors is demonstrated by the fact that, while many participants displayed a significant difference between the distractor trials and cognitive strategy trials in session 1, no such difference was observed in session 2 for any participant. It is then plausible to hypothesize that

of

the cognitive strategies may have acted as distractors, especially as participants became

habituated to their novelty. Still, the second session for participants 8 and 15 seemed to follow

ro

the initially anticipated results, whereby distractors resulted in an increase in cognitive load and the cognitive strategies eliminated this significance against the no-distractor trials. For

-p

these two participants, the cognitive strategies successfully diminished distractor-related increases in cognitive load.

re

Further investigation of the proposed cognitive strategies is required. It is possible that if all of the trials included the proposed strategies, the cognitive strategies would no longer act as

lP

distractors themselves, but as a part of the interface, and thus increase the perceptual load of the task. In this case, the strategies are expected to reduce cognitive load, and their effectiveness in accounting for distractor effects can be better determined. It is also possible that the

ur na

electrophysiological measure of cognitive load also encompasses the perceptual load of the task, contributing to the high measure observed in the cognitive strategy trials. After all, the cognitive strategies were meant to increase perceptual load in order to limit the “spill-over” effect of attention to the distractors, based on load theory. Therefore, differences in visual processing between these strategies and the no-distractor and distractor trials was expected. A measure of

Jo

cognitive load in other domains, distinct from perceptual load, may better elucidate the mechanism by which the cognitive strategies increase perceptual load and perhaps ultimately reduce the attentional load of the distractors. The presentation of visual distractors in the current study was intended to introduce

spontaneous task-irrelevant information that may resemble certain real-world environments. While many current BCIs require the user to visually attend to a screen, there has been increasing interest in real-world BCIs that allow users to interact without the obstruction of a traditional visual display. As such, augmented reality and BCI combinations have been proposed [41, 42]. In 16

such a combination, the user would wear a heads-up display through which he or she views the real-world while selectively attending to BCI cues. In this manner of set-up, real world distractors appear in the same location as the BCI cue. As an alternative to a centrally-located distractor, a flanking paradigm may be implemented, with careful consideration for ocular artefacts. Moreover, the effectiveness of modifying the perceptual and working memory load of the task interface may be further examined for other manipulations to load, such as with simultaneous tasks like dichotic listening [43] or verbal n-backs [44]. While the undertaking of these concurrent tasks is not necessarily directly representative of real-world BCI use, it is

of

possible that the more significant cognitive burden of multitasking may be more conducive to alleviation from cognitive strategies than small, user-independent environmental stimuli.

ro

The subjective cognitive load findings underscore the potentially deleterious effect of high

cognitive load on a BCI. The subjective data also suggest that some users may be particularly

-p

vulnerable to increases in perceived cognitive load. It is possible that the greater working memory demand for the BCI task in some participants compromised cognitive control functions underlying

re

selective attention (for a review, [3]). Consequently, these participants may experience increased cognitive load and reduced performance in the distractor-laden experimental sessions. These

lP

participants are then at risk of both not meeting BCI use requirements, as well as having a less positive experience with the technology, which compounded together, contribute to discontinued use.

ur na

Future studies should examine the relationship between objective and subjective measures of cognitive load, and the possibility that these measures capture different aspects of information processing. Given the findings of significant effects of even small and transient visual distractors, the examination of the effects of large, sustained visual distractors, as well as auditory distractors, on the cognitive load of BCI users is warranted. Ultimately, minimizing environmental influences

Jo

on real-world BCI use can be best managed not only with improving the robustness and adaptability of the BCI (for example, [45]), but also in considering strategies to minimize the user’s cognitive load.

5. Conclusions We found that transient, task-irrelevant visual distractors can increase theta/alpha power, particularly over the parietal cortex, as a physiological, objective measure of cognitive load. It is important to account for distractor effects on cognitive load, as some BCI users may be 17

particularly susceptible to changes in workload. The proposed cognitive strategies seem to have the unintended effect of heightening cognitive load. Future research should investigate alternative strategies for mitigating increases in cognitive load as a result of exposure to visual distractors in motor imagery BCIs.

Conflict of Interest Statement

Jo

ur na

lP

re

-p

ro

of

The authors declare no conflict of interest.

18

References

Jo

ur na

lP

re

-p

ro

of

[1] Lavie, N., "Attention, distraction, and cognitive control under load," Current Directions in Psychological Science, vol. 19, no. 3, pp. 143-279, 2010. [2] Lavie, N., & Tsal, Y., "Perceptual load as a major determinant of the locus of selection in visual attention," Perception & Psychophysics, vol. 56, pp. 339-354, 1994. [3] Lavie, N., Hirst, A., de Fockert, J.W., & Viding, E., "Load theory of selective attention and cognitive control," Journal of Experimental Psychology: General, vol. 133, no. 3, pp. 339-354, 2004. [4] Sweller, J., Ayres, P., & Kalyuga, S., Cognitive Load Theory, New York, NY: Springer, 2011. [5] Lavie, N., "Distracted and confused?: Selective attention under load," TRENDS in Cognitive Sciences, vol. 9, no. 2, pp. 75-82, 2005. [6] Kastner, S., De Weerd, P., Desimone, R., & Ungerleider, L.G., "Mechanisms of directed attention in the human extrastriate cortex as revealed by functional MRI," Science, vol. 282, pp. 108-111, 1998. [7] Kastner, S., De Weerd, P., Desimone, R., & Ungerleider, L.G., "Modulation of sensory suppression: Implications for receptive field sizes in the human visual cortex," Journal of Neurophysiology, vol. 86, pp. 1398-1411, 2001. [8] Antonenko, P., Paas, F., Grabner, R., & van Gog, T., "Using electroencephalography to measure cognitive load," Educational Psychology Review, vol. 22, no. 4, pp. 425-438, 2010. [9] Sanei, S., & Chambers, J.A., EEG signal processing, John Wiley & Sons, 2013. [10] Venables, L., & Fairclough, S., "The influence of performance feedback on goal-setting and mental effort regulation," Motiv. Emotion, vol. 33, pp. 63-74, 2009. [11] Stipacek, A., Grabner, R. H., Neuper, C., Fink, A., & Neubauer, A. C., "Sensitivity of EEG alpha band desynchronization to different memory components and increasing levels of memory load," Neuroscience Letters, vol. 353, pp. 193-196, 2003. [12] Khader, P. H., Jost, K., Ranganath, C., & Rosler, F., "Theta and alpha oscillations during working- memory maintenance predict successful long-term memory encoding," Neuroscience Letters, vol. 468, pp. 339-343, 2010. [13] Gevins, A., & Smith, M.E. "Neurophysiological measures of cognitive workload during human–computer interaction," Theoretical Issues in Ergonomics Science, vol. 4, no. 1-2, pp. 113-131, 2003. [14] Gevins, A., Smith, M.E., McEvoy, L., & Yu, D. "High-resolution EEG mapping of cortical activation related to working memory: effects of task difficulty, type of processing, and practice," Cerebral Cortex, vol. 7, pp. 374-385, 1997. [15] Gevins, A., & Smith, M.E., "Neurophysiological measures of working memory and individual differences in cognitive ability and cognitive style," Cerebral Cortex, vol. 10, pp. 829-839, 2000. [16] Neubauer, A. C., Fink, A., Grabner, R. H., Christa, N., & Wolfgang, K., "Sensitivity of alpha band ERD to individual differences in cognition," Progress in Brain Research, vol. 159, pp. 167-178, 2006. [17] Curran, E.A., & Stokes, M.J., "Learning to control brain activity: A review of the production and control of EEG components for driving brain–computer interface (BCI) systems," Brain and Cognition, vol. 51, pp. 326-336, 2003. [18] van Gerven, M., Farquhar, J., Schaefer, R., Vlek, R., Geuze, J. et al., "The brain-computer interface cycle," Journal of Neuroengineering, vol. 6, no. 4, p. 041001, 2009. [19] Mulvenna, M., Lightbody, G., Thomson, E., McCullagh, P., Ware, M., et al., "Realistic expectations with brain computer interfaces," Journal of Assistive Technologies, vol. 6, pp. 233-244, 2012. [20] Riccio, A., Leotta, F., Bianchi, L., Aloise, F., Zickler, C., et al., "Workload measurement in a communication application operated through a P300-based brain-computer interface," Journal of Neural Engineering, vol. 8, no. 2, pp. 1741-2552, 2011. [21] Emami, Z., & Chau, T. "Investigating the Effects of Visual Distractors on the Performance of a Motor Imagery Brain-Computer Interface," Clinical Neurophysiology, vol. 129, no. 6, pp. 1268-1275, 2018. [22] Bauer, R., Fels, M., Royter, V., Raco, V., & Gharabaghi, A. “Closed-loop adaptation of neurofeedback based on mental effort facilitates reinforcement learning of brain self-regulation,” Clinical Neurophysiology, vol. 127, no. 9, pp. 3156– 3164, 2016. https://doi.org/10.1016/j.clinph.2016.06.020 [23] Dagaev, N., K. Volkova and A. Ossadtchi, "Latent variable method for automatic adaptation to background states in motor imagery BCI," Journal of Neural Engineering, vol. 15, no. 1, 016004, 2017. [24] GmbH Brain Products, "BrainAmp DC," [Online]. Available: http://www.brainproducts.com/productdetails.php?id=2. [Accessed 28 10 2014]. [25] Homan, R.W., Herman, J., & Purdy, P., "Cerebral location of international 10-20 system electrode placement," Electroencephalog. Clin. Neurophysiol., vol. 66, pp. 376-382, 1987. [26] Makeig, S., Westerfield, W., Enghoff, S., Jung, T-P., Townsend, J., Courchesne, E., & Sejnowski, “Dynamic brain sources of visual evoked responses,” Science, vol. 295, no. 5555, pp. 690-694, 2002. [27] Kee, C.-Y., Ponnambalam, S.G., and Loo, C.-K. (2015). Multi-objective genetic algorithm as channel selection method for P300 and motor imagery data set. Neurocomputing, 161, 120-131. [28] Pang, Y., Yuan, Y., and Wang, K. (2012). Learning optimal spatial filters by discriminant analysis for brain computer interface. Neurocomputing, 77, 20-27 19

[29] Yu, L., and Liu, H. (2003). Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution. Proc. Twentieth Int Conf Mach Learn, ICML, 3, 856-863. [30] Müller-Putz, G.R., Scherer, R., Brunner, C., Leeb, R., & Pfurtscheller, G., "Better than random? A closer look on BCI results," International Journal of Bioelectromagnetism, vol. 10, no. 1, pp. 52-55, 2008. [31] Hart, S. G. & Staveland, L. E., "Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research," in Human Mental Workload, Amsterdam, North Holland Press. [32] Hart, S.G., "NASA-Task Load Index (NASA-TLX); 20 years later," in Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting, Santa Monica: HFES, 2006.

Jo

ur na

lP

re

-p

ro

of

[33] Kübler, A., Kotchoubey, B., Kaiser, J., and Wolpaw, J.R. “Brain-computer communication: Unlocking the locked in,” Psychol Bull, vol. 123, no. 3, pp. 358-375, 2001. [34] Cooper, P.S., Wong, A.S.W., Fulham, W.R., Thienel, R., Mansfield, E., et al., "Theta frontoparietal connectivity associated with proactive and reactive cognitive control processes," NeuroImage, vol. 108, pp. 354-363, 2015. [35] Mizuhara, H., & Yamaguchi, Y., "Human cortical circuits for central executive function emerge by theta," NeuroImage, vol. 36, pp. 232-244, 2007. [36] Sauseng, P., Hoppe, J., Klimesch, W., Gerloff, C., and Hummel, F.C., "Dissociation of sustained attention from central executive functions: local activity and interregional connectivity in the theta range," European Journal of Neuroscience, vol. 25, pp. 587-593, 2007. [37] Sauseng, P., Klimesch, W., Freunberger, R., Pecherstorfer, T., Hanslmayr, S. & Doppelmayr, M., "Relevance of EEG alpha and theta oscillations during task switching," Exp. Brain Res., vol. 170, pp. 295-301, 2006. [38] Baddeley, A. "Working memory," Science, vol. 255, no. 5044, pp. 556-559, 1992. [39] Hasher, L. & Zacks, R.T., "Working memory, comprehension, and aging: A review and a new view," Psychology of Learning and Motivation, vol. 22, pp. 193-225, 1988. [40] Käthner, I., Wriessnegger, S. C., Müller-Putz, G. R., Kübler, A., & Halder, S. “Effects of mental workload and fatigue on the P300, alpha and theta band power during operation of an ERP (P300) brain-computer interface,” Biological Psychology, vol. 102, pp. 118–129, 2014. https://doi.org/10.1016/j.biopsycho.2014.07.014 [41] Takano, K., Hata, N., & Kansaku, K. “Towards intelligent environments: an augmented reality–brain–machine interface operated with a see-through head-mount display,” Frontiers in Neuroscience, vol. 5, no. 60, 2011. [42] Gang, P., Hui, J., Stirenko, S., Gordienko, Y., Shemsedinov, T., Alienin, O., ... & González, E. A. (2018, April). User-driven intelligent interface on the basis of multimodal augmented reality and brain-computer interaction for people with functional disabilities. In Future of Information and Communication Conference (pp. 612-631). Springer, Cham. [43] Zhao, Y., Tang, J., Cao, Y., Jiao, X., Xu, M., Zhou, P., et al. “Effects of Distracting Task with Different Mental Workload on Steady-State Visual Evoked Potential Based Brain Computer Interfaces—an Offline Study,” Frontiers in Neuroscience, vol 12, pp. 79, 2018. https://doi.org/10.3389/fnins.2018.00079 [44] Cunillera, T., Fuentemilla, L., Periañez, J., Marco-Pallarès, J., Krämer, U. et al., "Brain oscillatory activity associated with task switching and feedback processing," Cognitive, Affective, & Behavioral Neuroscience, vol. 12, no. 1, pp. 16-33, 2012. [45] Myrden, A. and T. Chau, "Towards psychologically adaptive brain–computer interfaces," Journal of Neural Engineering, vol. 13, no. 6, pp. 066022, 2016.

20

Jo

ur na

lP

re

-p

ro

of

Appendix A

FIGURE 1. The task-irrelevant visual stimuli (distractors) presented in the experimental sessions

21

Appendix B NASA- Task Load Index: To be completed during the session Hart and Staveland’s NASA Task Load Index (TLX) method assesses work load on five 7-point scales. Increments of high, medium and low estimates for each point result in 21 gradations on the scales.

Mental Demand

Task

Date

How mentally demanding was the task?

How physically demanding was the task?

-p

Physical Demand

Very High

ro

Very Low

Perfect

Failure

How hard did you have to work to accomplish your level of performance?

Jo

Effort

Very Low

Frustration

Very High

How successful were you in accomplishing the motor imagery?

ur na

Performance

How hurried or rushed was the pace of the task?

lP

Very Low

Very High

re

Very Low

Temporal Demand

of

Participant ID

Very High

How insecure, discouraged, irritated, stressed, and annoyed were you?

Very Low

Very High

22