alpha neurofeedback training on imitating brain activity patterns in visual artists

Biomedical Signal Processing and Control 56 (2020) 101661

Contents lists available at ScienceDirect

Biomedical Signal Processing and Control journal homepage: www.elsevier.com/locate/bspc

The effect of beta/alpha neurofeedback training on imitating brain activity patterns in visual artists Nasrin Sho’ouri a,b,∗ , Mohammad Firoozabadi c , Kambiz Badie d a

Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran Faculty of Technology and Engineering, Central Tehran Branch, Islamic Azad University, Tehran, Iran Faculty of Medical Sciences, Tarbiat Modares University, Tehran, Iran d Research Institute for ICT, Tehran, Iran b c

a r t i c l e

i n f o

Article history: Received 7 February 2019 Received in revised form 30 June 2019 Accepted 17 August 2019 Keywords: EEG Neurofeedback Beta/alpha training Fuzzy adaptive neurofeedback training Mental imagery Visual artist

a b s t r a c t Finding differences between the brain signals of professional visual artists and non-artists during mental imagery of four paintings brought up the idea of using neurofeedback to resemble the patterns of the brain activity of non-artists to those of professional visual artists. To this end, the protocol for increasing the beta band activity and inhibiting the alpha band activity in channel T5, was utilized to train 12 nonartist subjects (7 subjects in the experimental group and 5 subjects in the sham group). To train the brain activity of the subjects, the fuzzy adaptive neurofeedback training procedure was used in which a variable scoring index and a mental fatigue index were defined to determine the success rate of the subjects and to discontinue training if the subjects be influenced by mental fatigue; respectively. Auditory feedback and visual feedback were employed to encourage the subjects. After ten neurofeedback training sessions, the relative low beta power in the experimental group during the visualization of four paintings significantly increased although the relative alpha power declined in this group. No variations were also observed in the beta and alpha band activities in the sham group. Besides, there was no significant changes in the eyes open baseline EEG signals in subjects in both study groups. Considering the variations in the brain activity of the subjects in the experimental group during mental imagery, it was likely that the use of the proposed training procedure could improve mental imagery skills and enhance the performance of novice visual artists. © 2019 Published by Elsevier Ltd.

1. Introduction Being endowed with expertise in a specific field can help out a person to practice skills with more precision and higher quality, as well as less mental effort than non-skilled or novice individuals [1–3]. In fact, prior knowledge and more training can make a professional do a skill with more efficiency. In this respect, previous studies have revealed that acquiring professional skills can bring about changes in the patterns of brain activity of individuals; thus, it is assumed that the distinctive performance of professionals is due to these variations [4–6]. Accordingly, finding the difference between the EEG signals of skilled and non-skilled individuals created the idea of using neurofeedback among researchers to enhance

∗ Corresponding author at: Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran. E-mail addresses: [email protected] (N. Sho’ouri), [email protected] (M. Firoozabadi), k [email protected] (K. Badie). https://doi.org/10.1016/j.bspc.2019.101661 1746-8094/© 2019 Published by Elsevier Ltd.

the performance of novice subjects and also help them acquire skills in a faster manner [7–19]. In this domain, neurofeedback means providing feedback for neuronal responses or EEG signals to a person in order to train them in terms of controlling electrical activity of the brain. Neurofeedback is also considered as a training process in which the brain learns to self-regulate itself. During this training process, the brain activity is controlled at both conscious and unconsciousness levels. In this regard, conscious learning occurs when a person finds the relationship between their mental status and applied feedback. Besides, the major part of learning takes place at the unconscious level, wherein the brain can gradually control the feedback directly and automatically. New skills acquired consciously and unconsciously are also automatically transferred to daily activities. Likewise, the skills learned by the brain during the neurofeedback can be long-lasting [2,20,21]. Thus far, neurofeedback has been able to cure many cognitive diseases. For example; illnesses such as anxiety [22–24], depression [22], hyperactivity [25,26], learning disability [27], epilepsy

2

N. Sho’ouri, M. Firoozabadi and K. Badie / Biomedical Signal Processing and Control 56 (2020) 101661

[28–30], and sleep disorders [31,32] have been successfully treated via neurofeedback. In addition, this method has been utilized to enhance attention [9,33,34], memory [35,36], dancing [7,37], sports skills [38–40], and musical performance [8,18,41,42], and also its effectiveness has been confirmed. However, improvement of the skill-related performance in visual arts has not been taken into consideration. For this reason, the present study focused on boosting mental imagery skills through neurofeedback. To this end, the protocol of increasing low beta power and decreasing relative alpha power at channel T5 proposed by Sho’ouri et al. specifically to enhance mental imagery performance was used. The given protocol was suggested based on examining the brain signals of visual artists and non-artists during mental imagery and determining the most effective discriminative feature and its related electrode location. Therefore, the research hypothesis addressed was that training subjects with this protocol could resemble the patterns of brain activity of non-artists to those of professional visual artists during mental imagery [43]. In the course of neurofeedback training, one or two electrodes are often placed on the scalp. These electrodes can record electrical activity in a subject’s brain and also display the brain waves in a simulated form within a computer such as a video game or an audio feedback. In this mode, a video playback or a computer game navigation is performed without using hands and solely by the brain waves. In this way, a person can discover favorable or unfavorable conditions of one’s brain waves by monitoring progress or stoppage of the game and getting rewards or losing scores or even changes occurring in the audio or video playback and thus try to correct the production of brain waves by controlling the game or the video [2]. For example, if the subject intends to increase one’s alpha band, the alpha power is continuously calculated in a moving time window of the brain signals and later compared with a threshold. Whenever the brain signal alpha power in the subject exceeds the threshold, the subject is also encouraged through video and/or audio feedback and their scores are increased. It should be noted that the type of feedback to encourage, the threshold selection method, and the score increment method can play important roles in the success rate of the subject. If an appropriate method is not chosen to encourage the subjects, they will not be well-guided and may not achieve the best results from their training [44]. Audio or video feedback is often used to encourage subjects. In this regard, Vernon et al. showed that the use of auditory and visual feedback combinations rather than using one type of feedback was more effective in giving information about one’s brain activity to a subject [2]. So, in this study, auditory feedback and visual feedback were employed to encourage the subjects. There are also several ways to choose the threshold. One of the common approaches to thresholding is a manual selection method. In this way, the therapist manually selects a threshold for the whole session of a subject. If the value of the training EEG feature of the subject is higher than the selected threshold in a time window, one point is added to the subject’s scores and a visual or auditory feedback is applied to them [45,46]. An issue raised within this method is that the threshold is chosen by the therapist and it does not adapt to the subject’s brain activity during the training. To deal with this problem, some other researchers make use of adaptive thresholding method in which the threshold is automatically selected adaptive to the brain activity of the subject and whenever the value of the training EEG feature of the subject is higher than the set threshold, one point is added to their scores. To determine the threshold, the training EEG feature values are checked in a time window and the new threshold for the next time interval is considered equal to the value, the 60–85% of the previous time window, in which the value of the training EEG feature of the subject is higher than that value [12,37,39,47–54]. The positive point about this method is that the amount of threshold adapts exactly with the

brain activity of the subject and the therapist is not at all involved in the choice of the threshold. But the drawback to this method is that the subject may feel confused in evaluating one’s success and they may not be guided properly. Continuous matching of the threshold can thus reduce the threshold when the subject’s performance is weaker. As a result, the likelihood of gaining scores by the subject is increased and the subject mistakenly thinks that they had a better performance than ever before. When the subject’s performance improves, the new threshold increases and it is harder to get more points. As a result, the subject may receive fewer points and mistakenly conclude a weaker performance than the previous one [44]. To deal with this problem, Sho’ouri et al. suggested fuzzy adaptive neurofeedback training (FNFT) procedure. In this method, the threshold is selected adaptive to the brain activity of the subject and also variable scoring index is used to encourage the subject. That is, the scoring index (score increment rate) is not constant and it is determined adaptive to a subject’s brain activity. The process of scoring in this method is that if the threshold of a training EEG feature is exceeded within a time interval and if the subject succeeds, more points are added to them. In contrast, if the threshold of a training EEG feature goes down due to poor performance, fewer points are added for each success at a new time interval. Thus, getting false points by subjects is prevented and the probability of the subject’s success rate is increased [44]. Thus, the FNFT method was used for training the brain activity of the subjects in the present research. The purpose of this study was to train 12 non-artists to resemble them to professional visual artists in terms of the patterns of brain activity during their mental imagery. The protocol used to this end was to enhance the low beta band activity (15–18 Hz) and to inhibit the alpha band activity (8–12 Hz) in channel T5 [43]. Moreover, the FNFT method was employed to train these individuals. Ultimately, the study results were to answer the question whether neurofeedback training had changed the patterns of brain activity of ordinary subject during mental imagery or not. Previous research studies related to this domain were also reviewed below. Then, the details of the proposed method were delineated and the results were illustrated. Finally, the results were discussed. 2. Related works 2.1. EEG analysis of experts Research studies have confirmed a difference between patterns in the brain activity of skilled and non-skilled individuals, in various domains, while performing activities related to the desired skills. For example, Hatfield et al. have found that expert marksmen exhibit higher alpha wave activity during the preparatory aiming period [38]. It has been found that EEG synchronization particularly in delta frequency band increases for chess experts as compared to novices while solving chess problems [55]. Deeny et al. observed that EEG coherence decreases for expert marksmen as compared to novices during aiming period [56]. Crews et al. found that an increase in right-hemisphere alpha wave activity is related to decreased errors for professional golfers [57]. An overall increase in alpha wave activity has been observed for karate experts while breaking wooden boards [58]. Fink et al. examined the EEG signals of expert dancers and nondancers in terms of alpha wave activity. They observed that expert dancers exhibit more right-hemispheric alpha synchronization as compared to novices during mental imagery of an improvisational dance [59]. Orgs et al. have also found that low beta wave activity significantly decrease for dancers while watching dance movements [60]. It has been reported that alpha wave activity increases for musicians as compared to non-musicians while passively listening to music [61,62]. Petsche et al. have shown that musicians and non-

N. Sho’ouri, M. Firoozabadi and K. Badie / Biomedical Signal Processing and Control 56 (2020) 101661

musicians are different in the levels of EEG coherence. Moreover, it has been found that beta wave activity plays a major role in the music processing [63,64]. A higher significant increased phase synchrony in gamma frequency band has been reported over distributed cortical areas for musicians as compared to non-musicians while listening to music [65]. It has been observed that blood flow increased in right-posterior parietal region of the brain for a portrait artist and a non-artist while they were asked to draw a series of faces. However, the artist showed lower activation as compared to the non-artist. In addition, artist showed higher brain activity in the right frontal area of the brain as compared to non-artist, suggesting that a visual artist uses higher order cognitive functions while viewing and drawing a face [66]. Pang et al. have observed that artistic expertise is correlated negatively with the amplitude of the ERP responses to visual stimuli [6]. Bhattacharya et al. have shown that visual artists exhibit significantly higher delta synchronization and alpha band desynchronization as compared to non-artists while they were asked to mentally compose a drawing [67]. It has also been observed that EEG phase synchrony is significantly higher for visual artists in beta and gamma wave activities as compared to non-artists while viewing a painting [68]. It has been reported that visual artists and non-artist are distinguishable using scaling exponents during visual perception and mental imagery of some paintings [69–71]. A decreased alpha wave activity has been observed for visual artists as compared to non-artists during visual perception and mental imagery [5]. Artists also showed increased approximate entropy as compared to non-artists during the performances of the two cognitive tasks [72,73]. A significant rising trend has been also observed in the beta and delta band activities during visualization and mental imagery of four paintings in professional visual artists compared with non-artist subjects [73]. Moreover, it has been reported that the two groups are distinguishable using cepstrum and wavelet coefficients [74,75]. Differences observed between EEG signals of skilled and nonskilled or novices gave rise to the idea of using neurofeedback among researchers in order to make changes in patterns of brain activity, enhance performance in novices; and consequently, assist them to act in a faster manner in their acquisition. Up to now, results associated with examining differences in the brain signals of musicians, athletes, as well as professional dancers have been utilized to design a neurofeedback training protocol in order to enhance subjects’ performance. But no research study was found in which the patterns of the brain activity of non-artists had been trained via neurofeedback to resemble them to those of professional visual artists. For this reason, the present study focused on visual arts. 2.2. Performance enhancement using neurofeedback There are research studies using neurofeedback training to change the patterns of the brain activity of novice subjects and increase their performance. For example, Egner et al. found a positive correlation between success on the theta/alpha enhancement and improved musical performance [9,76]. Landers et al. examined the effect of alpha enhancement in left-temporal region to improve archery performance, and also its effectiveness has been confirmed [77]. Ros et al. have shown that SMR/theta neurofeedback training led to improved surgical technique and reducing total surgery time [11]. Rostami et al. examined the effect of SMR (13–15 Hz) enhancement at C3, alpha/theta (8–12 Hz/4–8 Hz) at Pz, and high beta (20–30 Hz) inhibition in a group of rifle shooters. They observed that the trained subjects showed improved shooting performance [12]. The effectiveness of increased theta/alpha at Pz on improvement of novice dancers has also been reported [53]. Performance enhancements provided by alpha/theta training have also been reported for novice singers [13–16].

3

According to the review of the related literature, there was insufficient research in terms of implementing a neurofeedback training course to change the patterns of the brain activity of non-artist subjects and enhance their performance associated with visual arts. Therefore, a neurofeedback training course was implemented for non-artists in the present study using the results of previous research [43,44,73], so that the patterns of their brain activity could resemble those of professional visual artists and also their access to skills related to visual arts could be facilitated. 2.3. Thresholding and reward method Research studies have previously applied various thresholding and reward methods. For example, Raymond et al. have trained novice dancers to decrease alpha/theta at Pz. When the subjects’ theta wave activities were higher than their alpha wave activities, the sound of crashing waves was heard. Otherwise, a babbling brook sound was heard. A threshold was also considered for each frequency band. These thresholds were set manually by a therapist and updated to theta and alpha amplitudes surpassing 60% of the training time. Supra threshold bursts of alpha or theta were rewarded by a high or low-pitched gong sound, respectively [53]. A similar alpha/theta training procedure was employed to change mood and personality [54]. Gruzelier et al. have also employed a similar reward method for alpha/theta (alpha: 7–10 Hz and theta: 4–7 Hz) training of novice musicians [16,42]. Rostami et al. have trained rifle shooters using two neurofeedback protocols for performance enhancement. The first trial of each session consisted of an SMR protocol (SMR increase (13–15 Hz) and high beta (20–30 Hz) inhabitation) at C3 and C4, and the second trial of each session consisted of an alph/theta protocol at Pz. The subjects were rewarded if they could keep SMR activities above the baseline level for 80% (60% for alpha theta protocol) of the training time (at least for 0.5 s) and decreased high beta activities for 20% of the training time (at least for 0.5 s). Threshold levels were manually updated when subjects received rewards 90% (at least for 0.5 s) of the training duration [12]. Lee et al. studied the potential of neurofeedback training (beta increase and theta decrease at C3 and C4) on the treatment of ADHD children. Subjects were rewarded if they could keep beta levels above the threshold 20% of the treatment time, and decrease theta wave activities 70% of the time. The threshold was manually set by the therapist depending on the subject’s performance [46]. Ghoshooni et al. examined the effect of increased SMR (12–15 Hz) at Fz and Oz to improve cognitive performance. The individual SMR band of each subject was obtained from their recorded EEG baseline. The threshold was set manually so that the SMR activity amplitude was over the threshold 60% of the training time window. The subjects were rewarded when their SMR activity was higher than the threshold [78]. Khodakarami et al. and Sajadi et al. have also used a similar reward method for gamma training of healthy female students and alpha/theta training of students with learning disability, respectively [51,52]. Azarpaikan et al. have examined the effect of increased SMR (12–15 Hz) and decreased theta activity (4–7 Hz) at O1 and O2 to improve physical balance in Parkinson’s patients. The level of brainwaves was determined depending on an EEG baseline recorded from CZ for each session. Then, the patients played three video games on the computer screen. The video game was stopped when the brain activity changed in an incorrect frequency band [79]. 3. Proposed method The present study was comprised of two stages. At the first stage, the non-artists received neurofeedback training; and at the second stage, the EEG signals recorded before, during, and after training

4

N. Sho’ouri, M. Firoozabadi and K. Badie / Biomedical Signal Processing and Control 56 (2020) 101661

Fig. 1. The alpha/beta ratio histograms of visual artists and non-artists [44].

were processed. The details of the various stages were presented as follows. 3.1. Neurofeedback training method designed to enhance mental imagery performance of subjects To design a neurofeedback training course; training protocol, threshold method, scoring method, and type of feedback are supposed to be specified. In addition, a user-friendly software environment should be designed to train subjects. Here are some explanations provided for the above mentioned issues. 3.1.1. Choosing a neurofeedback training protocol to improve mental imagery performance of subjects To enhance the mental imagery performance of the subjects via neurofeedback, the protocol suggested by Sho’ouri et al. was used. In this respect, they processed the recorded 19-channel EEG signals from visual artists and non-artists during their mental imagery and calculated the relative power of different frequency bands from them. Then, the most effective distinguishing features and their related location for electrode placement were determined using the genetic algorithm, and ultimately suggested the low beta (18-15 Hz) activity increasement and the alpha wave (8–12 Hz) activity inhabitation to mental imagery performance enhancement of novice visual artists [43]. 3.1.2. Determining a thresholding method suitable for neurofeedback training In this research, the fuzzy adaptive neurofeedback training (FNFT) method was used to train the subjects. In this method, during neurofeedback training, the new threshold in each time window is considered to be equal to a percentage higher than the mean value of the training EEG feature in the previous time interval. To determine what percentage the threshold should be higher than the average of the training feature in the previous time interval, a test was taken. This means that several subjects were trained with neurofeedback. Then, in a few experiments, numbers such as 10%, 20%, and so on were tested as a percentage increase in the mean low beta power for selecting a new threshold, and the speed of gaining scores among the subjects were compared in different states. Eventually, it seemed that the ability to quickly get a good score was boosted through selecting a new threshold equal to the value of 0.25% higher than the average low beta. At the beginning of the training, the value of the initial threshold was considered by 0.1 and it was updated in each 15-second time window [44]. 3.1.3. Choosing an appropriate scoring method for neurofeedback training In the FNFT method, the value of the scoring index is variable and it is determined proportionally based on the brain activity of the subject. In this respect, the mean value of the EEG training feature

is calculated in a time window during the training, and its value is given to a designed fuzzy system. This system determines the appropriate value for the scoring index for the next time window. A number of fuzzy membership functions are considered during the minimum and maximum values of training EEG feature to design the fuzzy system. Then, each membership function is assigned an appropriate value for the scoring index [44]. In this study, the main EEG feature for training was the relative low beta power. Alpha/beta ratio was also used to determine the value of scoring index. Therefore, during training, when the relative low beta power of the subject was exceeded by the threshold, their scores were also increased, and the alpha/beta ratio of the subject could determine the points added to their scores [44]. The alpha/beta values for both non-artist and visual artist groups were calculated using the EEG signals recorded in the study by Bhattacharya et al. [70,80]. Accordingly, the number of membership functions, their centers and their consequences were determined [44]. Fig. 1 shows the histogram of the alpha/beta power ratio of both visual artist and non-artist groups in channel T5. According to Fig. 1, the lowest and the highest values for the alpha/beta ratio of visual artists were 0.88 and 6.11; respectively. Likewise, the lowest value for the alpha/beta ratio of non-artists was 1.57 and the highest value was equal to 32.19. So, the range related to the alpha/beta ratio of both groups consisted of three general areas. The first area was related to the ratio of 6.11 to 32.19 which had less value because it was limited to non-artists; therefore, increase in scores could be less for it. The second area was the common one between the two groups i.e. a range from 1.57 to 6.11. This area had a higher score than the first one. The third area was at the range of 1.57 or lower. Since this area was exclusively associated with visual artists, it was much more important than the other two areas. The given ranges could be also divided into smaller areas and a scoring index value could be apportioned to each range according to its importance [44]. In this study, the minimum value for the scoring index was 1 and the maximum value was 10. Fig. 2 shows the 21 considered membership functions assigned to the alpha/beta ratio. The centers of the membership functions and the corresponding amount of the increase in scores were illustrated in Table 1. To calculate the scoring index, the membership values of the mean alpha/beta ratio of the subject’s EEG signal to the membership functions was measured, and accordingly the scoring index was calculated as follows:

M SI =

i=1 M

yi i (SIS)

i=1

i (SIS)

(1)

In the above relation; SI is the scoring index and SIS represents the scoring index value setting feature which is considered as the alpha/beta ratio in this study. As well, i and yi show the member-

N. Sho’ouri, M. Firoozabadi and K. Badie / Biomedical Signal Processing and Control 56 (2020) 101661

5

Fig. 2. The membership functions for the alpha/beta in the range of 0.88–32. The distances between the centers are considered different [44].

Table 1 The centers and their corresponding consequences of the designed fuzzy rule base [44]. Centers Consequences

32 1

26 1.10

22 1.2

18 1.3

14 1.4

12 1.6

10 1.8

8 2.2

6 2.4

ship value of the SIS feature mean to the ith membership function, and the consequence of the ith fuzzy rule, respectively [44]. 3.1.4. Determining of mental fatigue index In the FNFT method, a mental fatigue index should be selected to notify the therapist about the subject’s mental fatigue during training and stop the training process. In this respect, Azarnoosh et al. reported that Katz fractal dimension and the correlation dimension could significantly decrease in the case of mental fatigue [81]. On the other hand, no significant difference was observed between the two study groups in terms of the Katz fractal dimension and the correlation dimension [73]. Since the calculation of the Katz fractional dimension was simpler than the correlation dimension and its calculation time was shorter, this feature was selected as an indicator of fatigue during training [44]. In this research, the fatigue index was calculated in each 15-second time window simultaneously with the determination of the threshold and if its value in the subject’s EEG signal had significantly decreased, the therapist was warned. 3.1.5. EEG signal recording device To record the signals and to train the subjects, the FlexComp Infiniti System manufactured by Thought Technology (serial number: A2068, Model: SA7550 M, made in Canada) was used. This device had 10 channels and used for purposes such as biofeedback, neurofeedback, as well as signal recording. To record the EEG, sampling frequency was set on 256 Hz. An active electrode (EEGZ T., Model: SA9305, Z5417) was placed at T5 (according to 10–20 standard electrode placement system) and common reference electrode was placed at the left earlobe and a ground site was placed on the right ear lobe [44]. The electrode impedance was kept below 1 k at active electrode. The signal recording was also performed on a computer with a CPU at 3 GHz and a 1 GB RAM. 3.1.6. Designing a user-friendly interface for neurofeedback training To train the subjects, a simulation-based interface compatible with the FlexComp Infiniti System was designed. During training, 21 clips from the collection of images were also used as visual feedback and auditory feedback to encourage the subjects. In the designed interface, two monitors were similarly considered for the user and the subject receiving the training. In the monitor placed in front of the subject; there was a movie display box, a box to display the score, and an LED for warning. If the relative lower beta power in the subject’s brain signal exceeds by the threshold, the image in

5 2.8

4 3.2

3.5 3.8

3 4.4

2.5 5

2 5.6

1.8 6.4

1.6 7.2

1.4 8

1.2 8.2

1 9

0.88 10

front of the subject is changed and the auditory feedback is heard. In addition, an subject’s score is increased based on the SI and the LED in front of the subject is on for a moment [44]. The length and width of the movie display box and the intensity of the auditory feedback were variable and they could change in correspondence with the SI. As a result, the subject could realize the value of one’s score by observing the resizing of the image box and changing the intensity of the auditory feedback. In fact, the subject not only tried to get a point, but also made attempts to see the movie in the larger box. In this respect, the following relation was used to set the auditory feedback and the image box size: x =  (s − smin ) x/ (smax − smin ) + x0

(2)

In the above relation, x is the parameters appointed (length, width, and intensity of sound), x0 represents the lowest value of parameter x, x refers to the difference between the minimum and the maximum values of parameter x, s is the value of SI, smax shows the highest value of scoring index, and smin is the lowest value of SI. In this study, smax = 10 and smin = 1. Fig. 3 shows the two monitors for the two different thresholds and SI (9.45 vs. 4.11). The threshold was increased in Fig. 3b and, therefore, the length and the width of the video frame and SI value was increased as compared to Fig. 3a. In the monitor in front of the therapist, it is possible to enter the training time, the time interval for re-setting the threshold (in seconds), as well as the initial threshold. The relative power limit and the frequency range for high and low artifacts can be also changed. The video and audio of the auditory feedback can be also selected by the therapist. As the output, the last time since the onset of training, the threshold value, the score, the value of SI, and the mean value of the training feature were displayed. In addition, the mental fatigue index was calculated and displayed. If the value of this index showed the subject’s fatigue, the therapist was warned by an LED. In this case, the therapist could stop the training [44]. After the training was stopped; the raw signal, signal filtered with notch filter, and an array of subjects’ scores could be stored. The operation of the designed interface was in this way that the sample signals were initially received from the encoder of the device; then, every second of the signal was filtered using a 6thorder elliptic filter to eliminate the 50 Hz power supply noise, and the training feature value was calculated in the filtered signal. If the value of the training feature was higher than the related threshold value and the relative power of the artifacts was lower than the threshold of their own, the subject’s score was increased by SI points. As the scores were added, the LED in front of the subject is turned on for a moment and the sound of the auditory feedback

6

N. Sho’ouri, M. Firoozabadi and K. Badie / Biomedical Signal Processing and Control 56 (2020) 101661

Fig. 3. The two monitors of the designed NFT interface (right: monitor of subject and left: monitor of therapist) for two different SI values of (a) 9.45 and (b) 4.11.

is heard. Additionally, several video frames were displayed for the subject as a visual feedback. When it is the time of the threshold change, new threshold is determined based on the average of training feature in the prior time window. In addition, the average SIS feature is given to the fuzzy systems responsible for SI calculation and the mental fatigue index is also simultaneously calculated. 3.2. Neurofeedback training of non-artists to create or imitate brain activity of Visual Artists In this research, neurofeedback training for each subject involved five steps: selection of participants, introduction session, first visual perception and mental imagery test, neurofeedback training of subjects, and implementation of the second visual perception and mental imagery test (repetition of the first test). Each subject also attended in the laboratory for a total of 13 sessions. Furthermore, the block diagram of the subject’s neurofeedback training steps was illustrated in Fig. 4. 3.2.1. Participants A total number of 12 female students enrolled in Bachelor’s, Master’s, and PhD courses (Biomedical Engineering and Computer Engineering) participated in our experiment. To this end, 7 of them were randomly assigned to the experimental group (with real feedback) and 5 of them were placed in the sham group (with fake feedback). The subjects also had normal or corrected vision. Initially, an informed consent form and a demographic characteristics form were given to the subjects. The participants affected with one of the mental disorders (epilepsy, major depression, acute anxiety,

forgetfulness, etc.) were excluded from the study. The demographic characteristics of the subjects were shown in Table 2, and the difference between the two study groups was measured by MannWhitney U test. According to the results of this test, no significant difference was observed between the given characteristics of both groups. 3.2.2. Introduction session for training process and laboratory environment The first session was dedicated to familiarity with the laboratory, how to record a signal, first-session test, neurofeedback training process, and administration of an IQ test (Jensen test, 2003). At the beginning of the session, the subject completed the informed consent form and the demographic characteristics form. Then, they were provided with explanations about the purpose of the research and the training method. Since the first and the second visual perception and the mental imagery tests were the same, it was likely that the novelty of the images for the subjects in the first test and their repetition in the second test influence the results. For this reason, in the introduction session, the images used in the first test were shown respectively to each subject and the EEG signals were recorded after viewing and visualization of one or two images by the subjects. This could lead the duplication of images for the subjects in both tests. Then, the neurofeedback training monitor was placed in front of the subjects and they experienced the training for a short time. 3.2.3. The first test Before the test, the questionnaires were given to the subjects. After completing the questionnaires, the subjects were required to

N. Sho’ouri, M. Firoozabadi and K. Badie / Biomedical Signal Processing and Control 56 (2020) 101661

7

Fig. 4. The block diagram of the subject’s neurofeedback training steps.

Table 2 Demographic characteristics of the subjects. Group

IQ

BMI

Head size (cm)

Height (cm)

Weight (kg)

Age (year)

Experimental Sham

108.4 ± 7.2 109.7 ± 11.2

0.0022 ± 0.0002 0.0020 ± 0.0003

56.4 ± 1.4 55.8 ± 1.3

164.4 ± 5.3 162.8 ± 5.1

59.1 ± 7.4 52.4 ± 9.5

25.2 ± 3.6 24 ± 2.7

Fig. 5. The block diagram of the first test.

perform four cognitive tasks of visual perception and four cognitive tasks of mental imagery. To this end, four painting (a drawing of Food Festival by Jordean, a black-pen drawing from Rembrand, an abstract painting by Kandinsky, and a portrait by Holbein) were separately displayed to each subject (the selected images were the same ones shown to the visual artists and non-artists in the test by Bhattacharya et al. and the ones by which Sho’ouri et al. proposed the training protocol used in this study based on recorded signals during their mental imagery [43,70,80]). Besides, a 17-inch monitor was used to display the images. The subjects were also asked to look at the images displayed for two minutes. During watching each picture, the subjects’ EEG signals from the channel T5 were recorded for two minutes. Then, the subject was asked to look at the wall and visualize the image seen for two minutes. During the mental imagery, the EEG signals were also recorded. In addition, the subjects’ eyes open baseline signals for two minutes were recorded. The block diagram of the first test was illustrated in Fig. 5. 3.2.4. Rey–Osterrieth complex figure test In this test, the subject is asked to copy a shape. After copy, the subject is asked after 3 min to transfer the shape from one’s memory on a paper. The given shape has no clear concept and it is easy to draw. In this test, a subject’s perception is specified through reviewing the copy method. In addition, recall the shape can provide information about subject’s memory performance. To evaluate this test, the copy trial (the subjects started from the general components such as a large rectangle and after that drew the details or the subjects drew the details one after the other beside each other), the copy time, and the authenticity of the picture (presence of all components) were examined. In both cases of copying and restoring the image, drawing the general components and their details were assigned with more points than starting with details. In both cases of making a copy or restoration, taking less time could add more points to the subjects’ scores. Regarding the accuracy of the information, the subject’s score was reduced for each element that was missing or incorrectly drawn. To calculate the subjects’ scores, there were several tables in which a percentage of 10 to 100 as the final score were assigned for the copy and restoration time, drawing manner, and the number of components drawn correctly [82,83]. 3.2.5. Neurofeedback training of subjects After the first test, the subjects received neurofeedback training for ten sessions. At the beginning of each training session, the

subjects’ eyes open baseline signals were recorded. Then, the brain activity of the subjects was trained via neurofeedback during three 10-min trials. From the fourth session on, a 2-min trial was added at the end of the session. The difference between the fourth trial and the three other ones was that only auditory feedback was applied to the subject in the fourth trial. In fact; the given subjects were asked to look at the wall, visualize the images seen during training, and try to get more points according to the given auditory feedback. The reason to add the fourth trial was to make the subjects try to get scores during mental imagery and enhance their visualization performance. Upon completion of all the trials, the subjects’ eyes open baseline signals were re-recorded. The stages of training in each session were shown in the block diagram of Fig. 6.

3.3. Processing the recorded signals The training features were calculated by the MATLAB and the statistical analyses were carried out through the SPSS. In total, the following signals were analyzed:

- Eyes open baseline signals recorded in the first test, - Signals related to visualization of the four images in the first test, - Eyes open baseline signals recorded before training for all sessions, - Signals recorded during training within the first three trials for all sessions, - Signals recorded during training within the fourth trials for all sessions, - Eyes open baseline signals recorded after training for all sessions, - Eyes open baseline signals recorded in the second test, and - Signals related to visualization of the four paintings in the second test.

3.3.1. Data pre-processing and processing The signals recorded to remove artifacts caused by muscles and blinking were filtered with a band-pass Butterworth filter with a bandwidth from 0.3 Hz to 44 Hz. In addition, parts of the signal in which the effects of the artifact could be seen after filtering were excluded. To normalize the data, the absolute power value in the different frequency bands (delta (2–4 Hz), theta (4–8 Hz), alpha (8–13 Hz), sensorimotor rhythm (13–15 Hz), and low beta

8

N. Sho’ouri, M. Firoozabadi and K. Badie / Biomedical Signal Processing and Control 56 (2020) 101661

Fig. 6. The block diagram of stages of training in each session.

Table 3 Relative power of traditional frequency bands of eyes open baseline EEG signals in the experimental group (*: p < 0.05, **: p < 0.01 and ***: p < 0.001).

Test1 Test2

Delta

Theta

Alpha

SMR

Low Beta

Group

Delta

Theta

Alpha

SMR

Low Beta

0.29 ± 0.15 0.25 ± 0.18

0.18 ± 0.06 0.19 ± 0.08

0.38 ± 0.16 0.42 ± 0.20

0.10 ± 0.04 0.08 ± 0.04

0.12 ± 0.09 0.11 ± 0.09

Experimental Sham

0.29 ± 0.13 0.28 ± 0.12

0.18 ± 0.06 0.22 ± 0.05

0.38 ± 0.16 0.49 ± 0.18

0.10 ± 0.03 0.05 ± 0.01

0.12 ± 0.03 0.06 ± 0.01

Table 4 Relative power of traditional frequency bands of eyes open baseline EEG signals in the sham group (*: p < 0.05, **: p < 0.01 and ***: p < 0.001).

Test1 Test2

Table 6 Relative power of traditional frequency bands of eyes open baseline EEG signals of the two groups after training (*: p < 0.05, **: p < 0.01 and ***: p < 0.001).

Delta

Theta

Alpha

SMR

Low Beta

Group

Delta

Theta

Alpha

SMR

Low Beta

0.28 ± 0.14 0.21 ± 0.11

0.22 ± 0.07 0.26 ± 0.09

0.49 ± 0.15 0.54 ± 0.18

0.05 ± 0.02 0.05 ± 0.01

0.06 ± 0.03 0.06 ± 0.05

Experimental Sham

0.25 ± 0.16 0.21 ± 0.08

0.19 ± 0.08 0.26 ± 0.10

0.42 ± 0.20 0.54 ± 0.18

0.08 ± 0.02 0.05 ± 0.02

0.11 ± 0.03 0.06 ± 0.03

(15–18 Hz)) as follows were divided into the absolute power value in the frequency range of 2–20 Hz. pr =

Table 5 Relative power of traditional frequency bands of eyes open baseline EEG signals of the two groups before training (*: p < 0.05, **: p < 0.01 and ***: p < 0.001).

pa PaT

According to Tables 3 and 4, no significant difference was found between the baseline signals of the first and the second tests in both experiential and groups in the frequency bands.

(3)

In the above relation, Pa is the absolute power in a particular frequency band and PaT refers to absolute power in the range of 2–20 Hz. 3.3.2. Statistical analyses To check the normality of the distribution of the calculated features from the recorded signals, Kolmogorov-Smirnov test was used. If the features’ distributions were normal and the samples were independent, the t-test was employed to examine the difference between the two groups. But, if the samples were dependent, the paired t-test could be used to examine the differences. The nonparametric form of the independent t-test and the paired t-test are Mann-Whitney U test and Wilcoxon signed-rank test, respectively. The Friedman test was also used to examine the difference between several dependent samples with non-normal distribution. 4. Results of neurofeedback training to imitate the brain activity of visual artists 4.1. Results of examining the recorded signals during the first and the second tests 4.1.1. Results of comparing eyes open baseline EEG signals recorded of each of the two groups during the first and the second tests The eyes open baseline signals in both experimental and sham groups were individually compared. Considering the normality and non-normality of the feature distribution of the data, paired t-test or Wilcoxon signed-rank test were used to compare the subjects. Tables 3 and 4 show the mean relative power of traditional frequency bands of baseline signals in the experimental and sham groups in the states of eyes open as well as the first and the second tests; respectively.

4.1.2. Results of comparing baseline signals recorded for both groups during the first and the second tests Baseline signals of both experimental and sham groups were compared with each other in order to find the differences before and after training. For this purpose, depending on the normal and non-normal distribution of the features, independent t-test or Mann-Whitney U test were used. Tables 5 and 6 show the results of the comparison of baseline signals in both study groups before and after training, respectively. Considering the baseline signals of the first and the second tests, no significant difference was observed between the relative powers of various bands between the two study groups. 4.1.3. Results of comparing recorded EEG signals related during the mental imagery for each group before and after training The difference in each group before and after training was investigated in the relative power of different frequency as follows. To do this, the Wilcoxon signed-rank test was used due to the non-normal distribution of the feature distribution (Fig. 7). As can be seen, the relative alpha power was significantly decreased in the experimental group after the training (p < 0.05). A significant increased relative low beta power was also observed for experimental group after the training (p < 0.05). As a result, subjects in the experimental group were successful in increasing the relative beta wave activity and reducing the alpha wave activity. The relative delta wave activity of these subjects was also increased after training although such an increase was not significant. The same comparison was performed for the sham group. In Fig. 8, the relative power of low beta, alpha, and theta in the sham group before and after training were compared with each other. As can be seen, only the relative theta power increased significantly in the sham group after training (p < 0.05). As a result, the subjects in the sham group did not succeed in increasing the relative beta power and reducing the relative alpha power, which

N. Sho’ouri, M. Firoozabadi and K. Badie / Biomedical Signal Processing and Control 56 (2020) 101661

9

Fig. 7. Relative power of traditional frequency bands of EEG signals during the mental imagery before and after training for experimental group (*: p < 0.05, **: p < 0.01 and ***: p < 0.001).

Fig. 8. Relative power of traditional frequency bands of EEG signals during the mental imagery before and after training for sham group (*: p < 0.05, **: p < 0.01 and ***: p < 0.001).

10

N. Sho’ouri, M. Firoozabadi and K. Badie / Biomedical Signal Processing and Control 56 (2020) 101661

Fig. 9. The trend of the changes in the relative power of different bands of the first three trials for all training sessions (the three first trials) in experimental group (*: p < 0.05, **: p < 0.01 and ***: p < 0.001).

Table 7 The scores of Rey–Osterrieth complex figure test before and after training for the two groups (*: p < 0.05, **: p < 0.01 and ***: p < 0.001). Group Experimental Sham

Copy

Recall

87.14 ± 15.05 96.42 ± 6.55 86.66 ± 17.28 89.66 ± 28.58

81.19 ± 12.38 94.04 ± 5.25* 64.66 ± 15.56 85.66 ± 17.66*

was the target of the training. No significant difference was also observed between the relative power of the rest of frequency bands between the mental imagery signals in the sham group before and after training. 4.1.4. Results of Rey–Osterrieth complex figure test The scores of the copy trial and Andre Rey’s restoration of geometric images for each group before and after training as well as those of both groups were compared. Regarding the non-normal distribution of the scores, the Wilcoxon signed-rank test was used to compare the scores of each group before and after training. Besides, the scores of both groups were compared with each other using the Mann-Whitney U test. The results were presented in Table 7. As can be seen, the scores of recall trial in both groups significantly increased (p < 0.05). But, no significant difference was observed between the scores assigned to copying images in both groups before and after training. Furthermore, no significant difference was found between the scores of copying and restoring images in both groups before and after training.

of the calculated features, the Friedman test was used to examine the changes during training. The trend of the changes in the relative power of different bands of the first three trials for all training sessions was shown in Fig. 9. According to Fig. 9, the relative theta power of the subjects during the training could significantly decrease (p < 0.01). However, the SMR power (p < 0.01) and low beta power (p < 0.05) increased significantly. No significant difference was also observed in the rest of the bands. 4.2.2. Results of examining EEG signals recorded during the fourth trial The change trend in low beta power of the fourth trial of all training sessions was examined similar to that of the first three trials. Considering the fourth trial, a significant difference was observed during training only in low beta relative power (p < 0.01). As an example, the change trend of relative low beta and alpha power for the fourth trial was illustrated in Fig. 10.

4.2. Results of examining EEG signals recorded during neurofeedback training

4.2.3. Results of changes in the relative low Beta power for each subject during neurofeedback training The trend of the changes in the relative low beta power was examined for each subject. To this end, EEG signals related to the first three trials of all training sessions for each subject were placed next to each other in order. Then, all the signals were divided into time windows lasting 5 s. Finally, using a line, the data related to each subject were fitted, and then the significance of the change trend was measured via analysis of variance (ANOVA). The results were illustrated in Fig. 11. According to Fig. 11, all the subjects were successful in increasing the low beta during training with the exception of subject No. 3 and subject No. 5.

4.2.1. Results of examining EEG signals recorded during the three first trials The trend of the changes in the relative power of different bands of all the subjects during the three first trials of all neurofeedback training sessions was examined. For this purpose, the relative power of various bands of all the signals recorded during training sessions was calculated. Due to the non-normal distribution

4.2.4. Examining change trend of baseline signals during neurofeedback training The relative power values of different bands of all baseline signals recorded before and after the training session were calculated for both groups. Then, the significance of trend changes of calculated power was evaluated during the sessions by Friedman test. In total, no significant changes were observed in the EEG feature

N. Sho’ouri, M. Firoozabadi and K. Badie / Biomedical Signal Processing and Control 56 (2020) 101661

11

Fig. 10. The trend of the changes in the relative power of different bands of the fourth trial for all training sessions in experimental group (*: p < 0.05, **: p < 0.01 and ***: p < 0.001).

examined in the baseline signals before training in the experimental group. As an example, the change trends of relative low beta and alpha power of baseline signals before and after training in the experimental group are illustrated in Fig. 12. The same comparison was done for the baseline signals in the sham group before and after training. In the case of this group, no significant changes were found between the calculated EEG features during training in both cases. The results were presented in Fig. 13.

5. Discussion 5.1. Innovation aspects of the research In this study, there were attempts during the neurofeedback training to resemble the patterns of the brain activity of non-artists to those of professional visual artists during mental imagery of some images. The protocol used in this study (increasing the beta band activity and inhibiting the alpha band activity in channel T5), was specifically designed to enhance mental imagery skills of the given subjects [43]. The reason for this choice was that the efficiency and effectiveness of the training course would be further enhanced if the neurofeedback training protocol had been specifically designed based on the differences of the brain signals in the target group and the training one. In most research studies in the field of neurofeedback, a constant scoring method had been employed [7,12,47,51–55,78,79,84]. Accordingly, one point could be added to the total scores obtained by subjects based on their success rates and they could be encouraged with the same audio or video feedback. The subject’s performance also had no effect on the intensity of the feedback received as well as the score obtained for each success; therefore, the subject could fail to evaluate oneself in detail. Moreover, the criterion of encouraging the subject was to increase or decrease their brain activity towards the threshold which was usually determined adaptively to the brain activity of the subject. If the subject’s performance was poor within a time interval, the threshold would decrease for the next time interval. At the subsequent time interval, increase in the training feature of the brain signal relative to the threshold would also become simpler. As a result, the subject might receive more points in the newer time interval and could also evaluate oneself more successfully compared with the previous time interval. It should be noted that the given increase in scores was only attributable to the lower threshold. When the subject’s performance improves, the new threshold increases and it is harder to get more points. As a result, the subject may receive fewer points and mistakenly conclude a weaker performance than the previous one [44].

In the present study, fuzzy adaptive neurofeedback training (FNFT) was used for training purposes in which a variable scoring method was employed. In this method, a scoring index could be defined whose value was determined with regard to the success rate of the subject [44]. So; the better the performance of a subject during training, the higher the scoring index and also the more the points added to their total scores for each success. If the threshold had declined within a time interval, the scoring index could also drop; resulting in adding fewer points to the total scores of the subject for each success within the next time interval. Accordingly, the subject could realize one’s poor performance and did not get it wrong in evaluating oneself. In addition, the intensity of audio and video feedback could change with respect to the scoring index to help in better evaluation of the subject. In view of that, the proposed method was able to act more successfully than other methods used in previous research studies in terms of guiding the subjects during training. Research studies have revealed that the amount of mental fatigue can also have effects on performance of subjects and their success rates during neurofeedback training [85]. However, mental fatigue during training has been ignored in such studies [7–19,37–42,45–47,51–55,78,79,84]. In this study, a fatigue index was defined which could determine the amount of mental fatigue in each subject. Within each time interval during training, the amount of fatigue index could be calculated and the therapist would be informed to discontinue the training process if there was a significant change. Besides, this could avoid frustration in the subjects as well as waste of time and money. 5.2. Effect of learning art on cognition Learning art can have effects on the mechanisms related to attention. During the exercises in learning art, subjects can enhance their attention-related mechanisms. Therefore, improving the given mechanisms can eventually lead to changes in subjects’ cognitive processes. Motivation and encouragement can also help with reinforcing attention. It has been shown that reinforced attention in the face of divergent problems, in the presence of feedback from subject’s performance, as well as their encouragement and increased motivation can result in changes in the cognitive processes among subjects [86]. 5.3. Use of neurofeedback to access patterns of brain activity in visual artists It should be noted that learning art can influence mechanisms related to attention. If this type of learning is accompanied by motivation and encouragement, it will have more effects [86]. In the present study, neurofeedback was used to boost the quality of men-

12

N. Sho’ouri, M. Firoozabadi and K. Badie / Biomedical Signal Processing and Control 56 (2020) 101661

Fig. 11. The trend of the changes in the relative low beta power was examined for each subject (*: p < 0.05, **: p < 0.01 and ***: p < 0.001).

tal imagery in non-artists. It should be noted that neurofeedback is a training method targeting the enhancement of subjects’ motivation. The protocol used was also to increase the relative low beta power and reduce the relative alpha power in channel T5. The given protocol has been specifically designed to enhance visual skills of non-artists and novice subjects. To design this protocol, the brain signals of artists and non-artists were processed during the mental imagery of several paintings and then the most effective EEG features distinguishing both groups along with the best location for placing the electrodes were selected. It was observed that the low beta power could significantly increase in artists all over the brain during mental imagery [43]. As mentioned, learning art can have effects on mechanisms related to attention, and consequently influence cognitive skills [86]. In this respect, Fries et al. found that synchronous activity of beta band could facilitate visual perception

process through enhancing attention [87]. Besides, Bhathacharya et al. showed that phase synchronization in beta and gamma bands in artists during mental imagery was higher [80]. In addition, beta activity was associated with thought, concentration, attention, perception, and cognition [88–91]. As a result, neurofeedback training with a relative low beta power increasement could contribute to reinforcing attention-related mechanisms. In addition, in this study, a set of various images with different styles were used for practice and encouragement of the subjects during training. Reinforcing attention through varied exercises could also help with changing a subject’s cognitive processes. This is the event that occurs during learning art. In addition, it should be noted that specific areas of the brain are devoted to each art. Visual arts are related to temporal and parietal areas [86]. The channel T5 is also located in the temporal region and it is related

N. Sho’ouri, M. Firoozabadi and K. Badie / Biomedical Signal Processing and Control 56 (2020) 101661

13

Fig. 12. The change trends of relative low beta and alpha power of baseline EEG signals before and after training in the experimental group (*: p < 0.05, **: p < 0.01 and ***: p < 0.001).

to memory and mental imagery. So, training subjects to increase beta power in the area of T5 to enhance visual skills may provide the necessary conditions for progress in the field of visual arts. 5.4. How to train subjects with neurofeedback In this study, the FNFT method was used to train brain activity of the subjects. To update the threshold, relative low beta power average in each time window was calculated and a new threshold of 25% above the average were obtained. If this percentage was small, the speed of getting scores was higher; thus, finding a rule to gain scores by the subjects was more difficult and training was more confusing. If this percentage was large, the speed of gaining scores could significantly get down and this could bring about frustration in subjects. As a result, a percentage must be selected to provide a reasonable and moderate scoring speed for the subjects. Given this issue, 25% seemed a suitable percentage [44]. The time window size for calculating the threshold was considered by 15 s. If the size of this window was considered larger, the scoring index could remain stable for a longer period and it could not reflect the moment-by-moment performance of the subjects well. If the size of this window was considered small, the speed of interpreting scoring index was high and the size of the box could change quickly. As a result, training may be also confusing for the

subject. Due to these issues, the time windows lasting 15 s seemed appropriate [44]. During the training, a set of images were shown as a clip to the subjects, and if the subject was successful, the image in front of them could change. The subject was also asked to gain scores only by seeing, remembering, and visualizing the images. In fact, the subject was not allowed in any way to increase their beta power during training, and they were only required to try to gain points with respect to the images and their remembrance. For example, the subjects were not allowed to do math calculations or think about various issues, recall memories, and so on to boost their beta band activity. In this study, the purpose of neurofeedback training was to change the patterns of brain activity of ordinary subjects during mental imagery in order to resemble such patterns to those in visual artists. Thus, beta band activity in subjects needs to be increased during mental imagery activity and their alpha band activity must be inhibited. Therefore, subjects should learn to enhance their beta activity and inhibit alpha band activity during observation, mental imagery, and remembrance of images. As a result, allowing subjects not to use different ways to get points could increase their chances of success in achieving the training goals. In this study, auditory feedback and visual feedback were used to encourage the subjects. According to Vernon et al., using a combination of auditory and visual feedback could increase the efficiency of neurofeedback training [2]. In addition, giving scores to the sub-

14

N. Sho’ouri, M. Firoozabadi and K. Badie / Biomedical Signal Processing and Control 56 (2020) 101661

Fig. 13. The change trends of relative low beta and alpha power of baseline EEG signals before and after training in the sham group (*: p < 0.05, **: p < 0.01 and ***: p < 0.001).

jects was used to encourage them. To increase the scores, a scoring index was defined whose value was determined according to the subject’s brain activity. Accordingly, better performance of the subject could lead to adding more scores to their total scores if they were successful. The length and the width of the image box and the intensity of the auditory feedback were also changed by the value of the scoring index, so that the subject could fully understand their condition and performance, and increase their success rate. To train subjects, three training trials along with auditory and visual feedback were used. In addition, the fourth trial was added from the fourth session for training by which only auditory feedback were applied to the subjects and they were asked to look at the wall and visualize the images seen and try to gain their scores and also listen to the sound of auditory feedback with more intensity. After adding the fourth trial, the subjects tried to increase their beta band activity and decrease their alpha band activity during their mental imagery and consequently increase the efficiency of their training. Gruzelier et al. observed that the subjects’ performance had reduced over time in a training session [85]. However, research studies related to the training of individuals with neurofeedback did not address the issue of examining mental fatigue during [7–19,37–42,45–47,51–55,78,79,84]. While mental fatigue can reduce the performance of the subjects, the effectiveness of training, and also cause disappointment. Therefore, monitoring mental fatigue during training and stopping training in the event of fatigue

can prevent the reduction of the subject’s efficiency. For this reason, there were attempts to investigate mental fatigue during training in this study. To this end, according to the results obtained by Azarnoush et al. [81], the Katz’s fractal dimension was selected as the mental fatigue index. The calculation of this feature was simple and no significant difference was seen between artists and non-artists in this respect. 5.5. Training results It has been observed that relative beta power of the visual artists during mental imagery significantly increased compared with that in non-artists and their alpha power reduced. In addition, the delta power in artists during mental imagery was significantly higher than that in non-artists [43]. In this study, subjects in the experimental group could successfully increase their low beta power in a significant manner and reduce their alpha power during mental imagery. However, the increase in their delta power was not significant. Comparing the mean of relative beta power after training in subjects and artists showed that subjects receiving neurofeedback training had produced higher relative beta power during mental imagery compared to visual artists [43]. Since the relative power was considered as the training criterion, the more increased relative beta power in the experimental group compared to those in visual artists led to their increased relative delta power was not sufficient and their difference was not significant. Consequently, in

N. Sho’ouri, M. Firoozabadi and K. Badie / Biomedical Signal Processing and Control 56 (2020) 101661

order to make all the necessary changes to the brain activity of the experimental group after training, the changes in the activity of the various bands were required to be controlled. This means that the relative power of a band should not exceed. Comparing the relative power of various bands in the sham group did not show any significant difference. As a result, the proposed method to train subjects had effects on changes in their brain activity. Of the 7 subjects in the experimental group, 5 subjects could increase their low beta band activity during training. Subjects No. 1, 2, 4, and 7 were able to successfully raise their relative low beta power from the very beginning and gain higher scores. During the training sessions, they also made progress in this regard. Subject No. 6 was still unable until the fifth session to obtain the decreased alpha/beta ratio. This subject received numerous scores but her scoring index was low, so the increase in her scores was low and also she watched the images in a smaller box. The small image box size and low-intensity sound of the auditory feedback also made this subject think about finding ways to increase their scores. At the sixth session, this subject was able to find a way to magnify the image box and maintain that status. Upon completion of the course of training, the highest level of brain activity control was observed in this subject. When this subject was asked to enlarge the image box, she immediately did this; and vice versa, this person could reduce the size of the image box as she wished. In terms of low beta power, subject No. 3 did not change significantly. She also failed to find the right way to raise her points and they adopted a steady stream in gaining scores. The scoring index for this subject was often low and this person did not show much motivation to find a way to change the score. Low beta band activity in subject No. 5 also fell significantly during training. Besides, scoring index during training sessions for this subject reduced and it reached its minimum in the final session. This subject could not find the way to raise points. Alpha band activity of this subject compared to the rest of the subjects was quite high. In some trials, the subject was allowed to try other ways to gain scores in addition to thinking about the images and remembering them. For example, she was permitted to do math calculations, to think about daily tasks, to review their good and bad memories, and so on; but no change was observed in her beta band activity. In fact, the best performance of this subject was seen during her first training session and this subject’s performance became weaker during the sessions. The subject was also asked about her mental state at the first training session. She said that she had felt nervous about an issue before training. She was also urged to think about that issue or any issues that had angered her. But again, a change in her scoring process was not achieved. Her beta band activity also significantly decreased and her alpha power was increased. The score for this subject’s IQ test was higher than the average, and there was a great deal of alpha band activity in this subject from the very beginning.

5.6. Further research In this study, the protocol for the enhancement of visual skills was implemented on non-artists. It is better to use the proposed training process for novice artists in future and then measure the performance of these subjects by professionals in visual arts, so it can be concluded whether changes in the brain activity have affected the success and performance of non-skilled artists or not. If this process is successful, the proposed training may play an effective role in reducing the learning time of novice visual artists and improving their performance. Additionally, various other EEG feature should be extracted from the brain signals of the subjects before and after training in order to determine the success of the

15

proposed protocol in helping the subjects to achieve the patterns of brain activity of visual artists with more confidence. 6. Conclusion In this study, 12 non-artists were trained through neurofeedback to achieve the brain activity of professional visual artists. To train brain activity of the subjects, FNFT procedure was used in which a variable scoring index and a mental fatigue index had been employed. After training, these subjects could significantly increase their low beta band power during mental imagery and reduce their alpha power. However, no significant change was observed in the brain activity of the sham group. As a result, the proposed neurofeedback training method was effective in training the subjects. Therefore, it was recommended to recruit novice artists as subjects and to measure their artistic performance by professionals in visual arts to determine the effectiveness of the protocol and the proposed training process within the art-related performance of novice art learners. Acknowledgments The authors would like to thank the Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University for all its support and the students who participated in the experiment. Declaration of Competing Interest The authors declare no conflicts of interest with respect to the research, authorship, and/or publication of this article. References [1] B. Abernethy, D.G. Russell, Expert-novice differences in an applied selective attention task, Sport Psychol. 9 (1987) 326–345. [2] D.J. Vernon, Can neurofeedback training enhance performance? An evaluation of the evidence with implications for future research, Appl. Psychophysiol. Biofeedback 30 (4) (2005) 347–364. [3] F. Allard, S. Graham, M.E. Paarsalu, Perception in sport: basketball, J. Sport. Psychol. 2 (1980) 14–21. [4] D.J. Vernon, Can neurofeedback training enhance performance an evaluation of the evidence with implications for future research, Appl. Psychophysiol. Biofeedback 30 (2005) 347–364. [5] N. Shourie, S.M.P. Firoozabadi, K. Badie, Investigation of EEG alpha rhythm of artists and nonartists during visual perception, mental imagery, and rest, J. Neurother.: Investig. Neuromodulation Neurofeedback Appl. Neurosci. 17 (3) (2013) 166–177. [6] C.Y. Pang, et al., Electrophysiological correlates of looking at paintings and its association with art expertise, Biol. Psychol. 93 (2013) 246–254. [7] J. Raymond, et al., Biofeedback and dance performance: a preliminary investigation, Appl. Psychophysiol. Biofeedback 30 (1) (2005) 65–73. [8] T. Egner, J. Gruzelier, Ecological validity of neurofeedback: modulation of slow wave EEG enhances musical performance, NeuroReport 14 (9) (2003) 1221–1224. [9] T. Egner, J.H. Gruzelier, EEG biofeedback of low beta band components: frequency-specific effects on variables of attention and event-related brain potentials, Clin. Neurophysiol. 115 (1) (2004) 131–139. [10] D.M. Landers, et al., The influence of electrocortical biofeedback on performance in pre-elite archers, Med. Sci. Sports Exerc. 23 (1) (1991) 123–129. [11] T. Ros, et al., Optimizing microsurgical skills with EEG neurofeedback, BMC Neurosci. 10 (1) (2009) 87. [12] R. Rostami, et al., The effects of neurofeedback on the improvement of rifle shooters’ performance, J. Neurother. 16 (4) (2012) 264–269. [13] B. Kleber, et al., Effects of EEG-biofeedback on professional singing performances, Rev. Esp. Psichol. 10 (2008), 77–61. [14] J. Leach, et al., Alpha theta versus SMR training for novice singers/advanced instrumentalists, Rev. Esp. Psichol. 10 (2008) 62. [15] J. Gruzelier, A theory of alpha/theta neurofeedback, creative performance enhancement, long distance functional connectivity and psychological integration, Cogn. Process. 10 (1) (2009) 101–109. [16] J.H. Gruzelier, et al., Replication of elite music performance enhancement following alpha/theta neurofeedback and application to novice performance and improvisation with SMR benefits, Biol. Psychol. 95 (2014) 96–107.

16

N. Sho’ouri, M. Firoozabadi and K. Badie / Biomedical Signal Processing and Control 56 (2020) 101661

[17] J.H. Gruzelier, EEG-neurofeedback for optimising performance. I: a review of cognitive and affective outcome in healthy participants, Neurosci. Biobehav. Rev. 44 (2014) 124–141. [18] J.H. Gruzelier, EEG-neurofeedback for optimising performance. II: creativity, the performing arts and ecological validity, Neurosci. Biobehav. Rev. 44 (2014) 142–158. [19] J.H. Gruzelier, EEG-neurofeedback for optimising performance. III: a review of methodological and theoretical considerations, Neurosci. Biobehav. Rev. 44 (2014) 159–182. [20] M. Thompson, L. Thompson, The Neurofeedback Book: An Introduction to Basic Concepts in Applied Psychophysiology, 2003. [21] J.D. Kropotov, Functional Neuromarkers for Psychiatry, 2016. [22] D.C. Hammond, Neurofeedback treatment of depression and anxiety, J. Adult Dev. 12 (2–3) (2005) 131–137. [23] D.C. Hammond, Neurofeedback with anxiety and affective disorders, Child Adolesc. Psychiatr. Clin. N. Am. 14 (1) (2005) 105–123. [24] N.C. Moore, A review of EEG biofeedback treatment of anxiety disorders, Clin. EEG Neurosci. 31 (1) (2000) 1–6. [25] N.J. Steiner, et al., Neurofeedback and cognitive attention training for children with attention-deficit hyperactivity disorder in schools, J. Dev. Behav. Pediatr. 35 (1) (2014) 18–27. [26] D. Vernon, A. Frick, J. Gruzelier, Neurofeedback as a treatment for ADHD: a methodological review with implications for future research, J. Neurother. 8 (2) (2004) 53–82. [27] M.A. Tansey, EEG sensorimotor rhythm biofeedback training: some effects on the neurologic precursors of learning disabilities, Int. J. Psychophysiol. 1 (2) (1984) 163–177. [28] M.B. Sterman, Basic concepts and clinical findings in the treatment of seizure disorders with EEG operant conditioning, Clin. EEG Neurosci. 31 (1) (2000) 45–55. [29] R.S. Monderer, D.M. Harrison, S.R. Haut, Neurofeedback and epilepsy, Epilepsy Behav. 3 (3) (2002) 214–218. [30] M.B. Sterman, T. Egner, Foundation and practice of neurofeedback for the treatment of epilepsy, Appl. Psychophysiol. Biofeedback 31 (1) (2006) 21–35. [31] A. Cortoos, et al., An exploratory study on the effects of tele-neurofeedback and tele-biofeedback on objective and subjective sleep in patients with primary insomnia, Appl. Psychophysiol. Biofeedback 35 (2) (2010) 125–134. [32] K. Hoedlmoser, et al., Instrumental conditioning of human sensorimotor rhythm (12–15 Hz) and its impact on sleep as well as declarative learning, Sleep 31 (10) (2008) 1401. [33] K.P. Thomas, A. Vinod, C. Guan, Enhancement of attention and cognitive skills using EEG based neurofeedback game, in: 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), IEEE, 2013. [34] T. Egner, J.H. Gruzelier, Learned self-regulation of EEG frequency components affects attention and event-related brain potentials in humans, Neuroreport 12 (18) (2001) 4155–4159. [35] D. Vernon, et al., The effect of training distinct neurofeedback protocols on aspects of cognitive performance, Int. J. Psychophysiol. 47 (1) (2003) 75–85. [36] A. Barnea, A. Rassis, E. Zaidel, Effect of neurofeedback on hemispheric word recognition, Brain Cogn. 59 (3) (2005) 314–321. [37] J. Gruzelier, et al., Application of alpha/theta neurofeedback and heart rate variability training to young contemporary dancers: state anxiety and creativity, Int. J. Psychophysiol. 93 (1) (2014) 105–111. [38] B.D. Hatfield, D.M. Landers, W.J. Ray, Cognitive processes during self paced motor performance: an electroencephalographic profile of skilled marksmen, J. Sport. Psychol. 6 (1984) 42–59. [39] D.C. Hammond, Neurofeedback for the enhancement of athletic performance and physical balance, J. Am. Board Sport Psychol. 1 (1) (2007) 1–9. [40] C. Ring, et al., Investigating the efficacy of neurofeedback training for expediting expertise and excellence in sport, Psychol. Sport Exerc. 16 (2015) 118–127. [41] J. Gruzelier, et al., Immediate effects of alpha/theta and SMR neurofeedback on music performance, Int. J. Psychophysiol. 93 (1) (2014) 96–104. [42] J. Gruzelier, et al., Beneficial outcome from EEG-neurofeedback on creative music performance, attention and well-being in school children, Biol. Psychol. 95 (2014) 86–95. [43] N. Shourie, M. Firoozabadi, K. Badie, Neurofeedback training protocols based on spectral EEG feature subset and channel selection for performance enhancement of novice visual artists, Biomed. Signal Process. Control 43 (2018) 117–129. [44] N. Shourie, M. Firoozabadi, K. Badie, Fuzzy adaptive neurofeedback training: an efficient neurofeedback training procedure providing a more accurate progress rate for trainee, Biomed. Signal Process. Control 44 (2018) 75–81. [45] D. Mroczkowska, J. Białkowska, A. Rakowska, Neurofeedback as supportive ˛ Psychiatrii Neurol. 23 (4) (2014) therapy after stroke. Case report, Postepy 190–201. [46] E.-J. Lee, C.-H. Jung, Additive effects of neurofeedback on the treatment of ADHD: a randomized controlled study, Asian J. Psychiatry 25 (2017) 16–21. [47] M. Ghoshuni, et al., The effect of sensorimotor rhythm enhancing neurofeedback on power of adjacent frequency band, Biomed. Eng. Appl. Basis Commun. 24 (2012) 307–312. [48] K. von Carlowitz-Ghori, et al., Voluntary control of corticomuscular coherence through neurofeedback: a proof-of-principle study in healthy subjects, Neuroscience 290 (2015) 243–254. [49] J.R. Evans, A. Abarbanel, Introduction to Quantitative EEG and Neurofeedback, Elsevier, 1999.

[50] M. Ninaus, et al., Brain volumetry and self-regulation of brain activity relevant for neurofeedback, Biol. Psychol. 110 (2015) 126–133. [51] Z. Khodakarami, M.P. Firoozabadi, Self-regulation of brain gamma band activity through neurofeedback and its effects on visual feature bindings in healthy female students, koomesh 16 (1) (2014) 36–45. [52] S.A. Sadjadi, A.M. Akhoundpour, P. Hashemian, Evaluation of Neurofeedback Therapy in Children With Mathematic Disorder in Third-grade Elementary School, 2014. [53] J. Raymond, et al., Biofeedback and dance performance: a preliminary investigation, Appl. Psychophysiol. Biofeedback 30 (1) (2005) 65–73. [54] J. Raymond, et al., The effects of alpha/theta neurofeedback on personality and mood, Cogn. Brain Res. 23 (2) (2005) 287–292. [55] H.-J. Volke, et al., On-coupling and off-coupling of neocortical areas in chess experts and novices as revealed by evoked EEG coherence measures and factor-based topological analysis—a pilot study, J. Psychophysiol. 16 (1) (2002) 23. [56] S.P. Deeny, et al., Cortico-cortical communication and superior performance in skilled marksmen: an EEG coherence analysis, J. Sport Exerc. Phychol. 25 (2003) 188–204. [57] D.J. Crews, D.M. Landers, Electroencephalographic measures of attentional patterns prior to the golf putt, Med. Sci. Sports Exerc. 25 (1993) 116–126. [58] D. Collins, G. Powell, I. Davies, An electroencephalographic study of hemispheric processing patterns during karate performance, J. Sport Exerc. Psychol. 12 (1990) 223–234. [59] A. Fink, B. Graif, A.C. Neubauer, Brain correlates underlying creative thinking: EEG alpha activity in professional vs. novice dancers, NeuroImage 46 (2009) 854–862. [60] G. Orgs, et al., Expertise in dance modulates alpha/beta event-related desynchronization during action observation, Eur. J. Neurosci. 27 (2008) 3380–3384. [61] M.J. Wagner, Effect of music and biofeedback on alpha brainwave rhythms and attentiveness of musicians and non-musicians, J. Res. Music Educ. 23 (1) (1975) 3–13. [62] M.J. Wagner, Brainwaves and biofeedback: a brief history—implications for music research, J. Music Ther. 12 (2) (1975) 46–58. [63] H. Petsche, et al., The EEG: an adequate method to concretize brain processes elicited by music, Music Percept. 6 (1988) 133–159. [64] H. Petsche, et al., EEG coherence and musical thinking, Music Percept. 11 (1993) 117–151. [65] J. Bhattacharya, H. Petsche, Phase synchrony analysis of EEG during music perception reveals changes in functional connectivity due to musical expertise, Signal Proc. 85 (11) (2005) 2161–2177. [66] R.L. Solso, Brain activities in a skilled versus a novice artist: an fMRI study, Leonardo 34 (1) (2001) 31–34. [67] J. Bhattacharya, H. Petsche, Drawing on mind’s canvas: Differences in cortical integration patterns between artists and non-artists, Hum. Brain Mapp. 26 (1) (2005) 1–14. [68] J. Bhattacharya, H. Petsche, Shadows of artistry: cortical synchrony during perception and imagery of visual art, Cogn. Brain Res. 13 (2) (2002) 179–186. [69] N. Shourie, M. Firoozabadi, K. Badie, Investigation and classification of EEG signals related to artists and nonartists during visual perception, mental imagery and rest using scaling exponent, Iranian J. Biomed. Eng. 7 (4) (2014). [70] S. Karkare, G. Saha, J. Bhattacharya, Investigating long-range correlation properties in EEG during complex cognitive tasks, Chaos Solitons Fractals 42 (2009) 2067–2073. [71] N. Shourie, S.M.P. Firoozabadi, K. Badie, Information evaluation and classification of scaling exponents of EEG signals corresponding to visual perception, mental imagery & mental rest for artists and non-artists, in: 2011 18th Iranian Conference of Biomedical Engineering (ICBME), IEEE, 2011. [72] N. Shourie, M. Firoozabadi, K. Badie, Analysis of EEG signals related to artists and nonartists during visual perception, mental imagery, and rest using approximate entropy, Biomed Res. Int. 2014 (2014). [73] N. Sho’ouri, M. Firoozabadi, K. Badie, Neurofeedback training protocols based on selecting distinctive features and identifying appropriate channels to enhance performance in novice visual artists, Biomed. Signal Process. Control 49 (2019) 308–321. [74] N. Shourie, S.M.P. Firoozabadi, K. Badie, A comparative investigation of wavelet families for analysis of EEG signals related to artists and non-artists during visual perception, mental imagery and rest, J. Neurother.: Investig. Neuromodulation Neurofeedback Appl. Neurosci. 17 (4) (2013) 248–257. [75] N. Shourie, Cepstral analysis of EEG during visual perception and mental imagery reveals influence of artistic expertise, J. Med. Signals Sens. 6 (4) (2016) 203–217. [76] T. Egner, J.H. Gruzelier, Ecological validity of neurofeedback: modulation of slow wave EEG enhances musical performance, Neuroreport 14 (9) (2003) 1221–1224. [77] D.M. Landers, et al., The influence of electrocortical biofeedback on performance in pre-elite archers, Med. Sci. Sports Exerc. 23 (1) (1991) 123–129. [78] M. Ghoshuni, et al., The effect of sensorimotor rhythm enhancing neurofeedback on power of adjacent frequency bands, Biomed. Eng. Appl. Basis Commun. 24 (04) (2012) 307–312. [79] A. Azarpaikan, H.T. Torbati, M. Sohrabi, Neurofeedback and physical balance in Parkinson’s patients, Gait Posture 40 (1) (2014) 177–181. [80] J. Bhattacharyaa, H. Petsche, Shadows of artistry: cortical synchrony during perception and imagery of visual art, Cogn. Brain Res. 13 (2) (2002) 179–186.

N. Sho’ouri, M. Firoozabadi and K. Badie / Biomedical Signal Processing and Control 56 (2020) 101661 [81] M. Azarnoosh, et al., Evaluating nonlinear variability of mental fatigue behavioral indices during long-term attentive task, Complexity 17 (6) (2012) 7–16. [82] A. Rey, L’examen psychologique dans les cas d’encéphalopathie traumatique (Les problems.), Arch. Psychol. 28 (1941) 215–285. [83] P.-A. Osterrieth, Filetest de copie d’une figure complex: Contribution a l’etude de la perception et de la memoire [The test of copying a complex figure: a contribution to the study of perception and memory], Arch. Psychol. 30 (1944) 286–356. [84] Y. Zandi Mehran, M. Firoozabadi, R. Rostami, Brain inconspicuous effect by local sinusoidal extremely low frequency magnetic exposure based on wavelet packet analysis: innovation in online passive neurofeedback therapy by the neuro-LSELF system, J. Neurother. 17 (4) (2013) 226–247. [85] J. Gruzelier, et al., Learned control of slow potential interhemispheric asymmetry in schizophrenia, Int. J. Psychophysiol. 34 (1999) 341–348.

17

[86] C.H. Asbury, et al., Learning, Arts, and the Brain: the Dana Consortium Report on Arts and Cognition, Dana Press, 2008. [87] P. Fries, et al., Modulation of oscillatory neuronal synchronization by selective visual attention, Science 291 (2001) 1560–1563. [88] N. Kopell, et al., Gamma rhythms and beta rhythms have different synchronization properties, Neurobiology 97 (4) (2000) 1867–1872. [89] A. Wróbel, Beta activity: a carrier for visual attention, Acta Neurobiol. Exp. 60 (2000) 247–260. [90] J. Baumeister, et al., Influence of phosphatidylserine on cognitive performance and cortical activity after induced stress, Nutr. Neurosci. 11 (3) (2008) 103–110. [91] A. Dietrich, R. Kanso, A review of EEG, ERP, and neuroimaging studies of creativity and insight, Psychol. Bull. 136 (5) (2010) 822–848.