- Email: [email protected]
Research on Comparison between Model Human Processor and Readiness Potential
Available online at www.sciencedirect.com Available online at www.sciencedirect.com
ScienceDirect ScienceDirect
Procedia Computer Science 00 (2019) 000–000 Procedia Computer Science 00 (2019) 000–000
Available online at www.sciencedirect.com
ScienceDirect
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
Procedia Computer Science 159 (2019) 2005–2014
23rd International Conference on Knowledge-Based and Intelligent Information & Engineering 23rd International Conference on Knowledge-Based Systems and Intelligent Information & Engineering Systems
Research on Comparison between Model Human Processor and Research on Comparison between Model Human Processor and Readiness Potential Readiness Potential Nanako Shimizuaa, Toshitaka Higashinobb, Masato Sogacc* Nanako Shimizu , Toshitaka Higashino , Masato Soga *
a Graduate School of System Engineering, Wakayama University, Sakaedani 930, Wakayama-city 640-8510, Japan a Graduate School of Information Science and Wakayama Technology,University, Osaka University, Yamadaoka 1-1, Suita-city 565-0871, Japan Graduate School of System Engineering, Sakaedani 930, Wakayama-city 640-8510, Japan a b Faculty Engineering, University, 930, Wakayama-city 640-8510, Japan Japan Graduate SchoolofofSystem Information ScienceWakayama and Technology, OsakaSakaedani University, Yamadaoka 1-1, Suita-city 565-0871, a Faculty of System Engineering, Wakayama University, Sakaedani 930, Wakayama-city 640-8510, Japan b
Abstract Abstract In thinking about human-computer interface, it is important to know how human beings recognize, judge and act. Therefore, Card In al. thinking about human-computer interface, it is important knowModel how human recognize, judge and act. Therefore, Card et devised a model of human cognitive processing processtocalled Humanbeings Processor (MHP). However, this MHP predicts et al. devised a model human cognitive process calledfrom Model (MHP). In However, thiswe MHP predicts processing time from of empirical rules, andprocessing is not much considered theHuman aspectProcessor of brain activity. this study, focused on processing time from empirical and is not much the MHP aspectfrom of brain activity. this activity study, we on readiness potential (RP), which rules, is a characteristic brainconsidered wave, andfrom verified the aspect ofIn brain by focused measuring readiness potential (RP), (EEG) which when is a characteristic brain user wave,performance. and verified Experimental MHP from theresults aspectshow of brain by be measuring Electroencephalography performing basic that activity MHP can roughly Electroencephalography performing basic user performance. explained from the aspect(EEG) of RP,when which is a characteristic brain activity. Experimental results show that MHP can be roughly explained from the aspect of RP, which is a characteristic brain activity. © 2019 The Author(s). Published by Elsevier B.V. © 2019 2019 The The Author(s). Authors. Published bybyElsevier B.V. © Published Elsevier B.V. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of KES International. Peer-review under responsibility of KES International. Peer-review under responsibility of KES International. Keywords: Model Human Processor; Readiness Potential; Basic User Performance; Electroencephalography; Human Computer Interface Keywords: Model Human Processor; Readiness Potential; Basic User Performance; Electroencephalography; Human Computer Interface
1. Introduction 1. Introduction There is the model of human cognitive processing process called Model Human Processor (MHP) modeled by Card is the model of human cognitive processing processtime called Human Processor by Card at alThere [1]. However, MHP is a model in which the processing is Model predicted empirically, and(MHP) has notmodeled been studied so at al [1]. However, MHP a model in which processingintime predicted empirically, and has not been studied so much from the aspect of isbrain activity [2][3].theTherefore, this is study, we focus on Electroencephalography (EEG) much the aspect of brain Therefore, thisverify study, we focus on aspect Electroencephalography called from readiness potential (RP), activity measure[2][3]. and analyze EEG,inand MHP from the of brain activity. (EEG) called readiness potential (RP), measure and analyze EEG, and verify MHP from the aspect of brain activity.
* Corresponding author. Tel.: +81-73-457-8457. E-mail address:author. [email protected] * Corresponding Tel.: +81-73-457-8457. E-mail address: [email protected] 1877-0509 © 2019 The Author(s). Published by Elsevier B.V. This is an open access underPublished the CC BY-NC-ND 1877-0509 © 2019 The article Author(s). by Elsevier license B.V. (https://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of KES International. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of KES International.
1877-0509 © 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of KES International. 10.1016/j.procs.2019.09.373
2
2006
Author name / Procedia Computer Science 00 (2019) 000–000
Nanako Shimizu et al. / Procedia Computer Science 159 (2019) 2005–2014
2. Purpose In MHP, the average time required for the perceptual process is 100ms, the average time required for the cognitive process is 70ms, and the time required for the motor process is 70ms (Fig. 1). So, using MHP, we can show the cognitive process of human’s four basic user performance: Simple Reaction, Physical Matches, Name Matches, and Class Matches as shown in Fig. 2. Specifically, the four basic user performance are as follows. Simple Reaction is a reaction performed when some information is presented. Physical Matches is a reaction performed when the presented information is the same as the information stored in advance. Name Matches is a reaction to be performed when the presented information can be represented by the same name as the name stored in advance. Class Matches is a reaction to be performed when the presented information can be represented by the same category as the category stored in advance. Also, RP is an EEG discovered by Kornhuber et al [4]. It is a small potential change that appears 550ms prior to voluntary movement, and the potential changes in the negative direction [5]. It is an EEG related to preparation for voluntary movement and is observed in the motor area. If we plot the time 550ms prior to voluntary movement by clicking a mouse in the Fig. 2, the time RP would appear will be shown in Fig. 3. In MHP, it is the Cognitive Processor section that prepares for movement in the brain. However, as shown in Fig. 3, the time when RP appears is before the time when Cognitive Processors work, which indicates that MHP and RP contradict each other. So, in this research, we construct an experiment corresponding to four models, measure the EEG in the experiment, and verify the contradiction point by confirming the appearance time of RP.
Fig. 1. Model Human Processor
Author Shimizu name / Procedia Computer Science 00 (2019) 000–000 Nanako et al. / Procedia Computer Science 159 (2019) 2005–2014
20073
Fig. 2. Four basic user performance
Fig. 3. Contradiction of MHP and RP
3. Method We constructed four experiments corresponding to MHP and measured the electroencephalogram of the subject in the experiment. In the experiment, a visual stimulus is presented to a subject wearing an electroencephalograph, and when the visual stimulus (target stimulus) taught in advance is presented, the left click of the mouse is performed, and when the other visual stimulus is presented, the right click is performed. In addition, the time from the presentation of the visual stimulus to the mouse click was recorded as the reaction time (RT).
2008 4
Nanako Shimizu et al. / Procedia Computer Science 159 (2019) 2005–2014 Author name / Procedia Computer Science 00 (2019) 000–000
Table 1 shows the visual stimulus, target stimulus and presentation rate of target stimulus in each experiment. In addition, BIOSEMI Active Two was used for EEG measurement. The sampling rate was 512Hz, and the electrodes were arranged based on the international 10-20 method (Fig. 4). In the visual stimulus presentation method, as shown in Fig. 5, the screen with the gaze point (+) presented and the screen with the visual stimulus presented alternately. The presentation time of the gaze point on the screen was shifted randomly between 1000 and 2000ms. This prevents the subject from remembering the timing at which the visual stimulus is presented. The visual stimulus screen was presented for 600ms. In each experiment, the visual stimulus were presented 30 times each. It is necessary for averaging EEG data. Table 1. Types of visual stimulus
Fig. 4. International 10-20 system
Fig. 5. How to present visual stimulus
Author name / Procedia Computer Science 00 (2019) 000–000
Nanako Shimizu et al. / Procedia Computer Science 159 (2019) 2005–2014
5
2009
4. Analysis Fig. 6 shows the process of deriving an averaged waveform for each subject from the acquired EEG data. By dividing the eye potential data from the EEG data, the noise due to eye movement and blinking was removed, and the noise due to phenomena other than the electroencephalogram was removed by applying a band pass filter of 1-30Hz. The noise-removed brain wave data was divided into intervals of -1000ms to 500ms from the trigger (the position of the mouse click). Baseline correction was performed for each of the divided data, using the average potential of 700ms to -500ms as a baseline with reference to the position of visual stimulus presentation. Then, data with RT exceeding the mean ± 2 SD (± 2 times the standard deviation) were excluded as outliers, and averaging was performed. Moreover, the total average waveform was derived by further averaging the average waveform of each subject.
Fig. 6. Process of EEG data shaping
2010 6
Nanako Shimizu et al. / Procedia Computer Science 159 (2019) 2005–2014 Author name / Procedia Computer Science 00 (2019) 000–000
5. Result Data from C3 electrodes of 10 healthy male and female subjects were used for analysis. The position of the C3 electrode is the motor area, and RP can be detected. 5.1. Comparison of RT in four models Table 2 shows the mean and variance of RT in four models. One-way analysis of variance was performed on the RT in four models at a significance level of 5%. As a result of multiple comparison, it was found that there was a significant difference among RTs of the four models, and RT became significantly longer in the order of Simple Reaction, Physical Matches, Name Matches, and Class Matches. Fig. 7 shows the mean value, standard error, and p value of RT in four models. In Fig. 7, the position of 0ms is the position of the mouse click, and the red line shows the timing of the visual stimulus presentation. The vertical axis is the amplitude (μV), and the horizontal axis is the time (sec). 5.2. Appearance position of RP Fig. 8 shows the total average waveform in four models and the position where RP began to appear (the position where the amplitude of the waveform began to swing significantly negatively). The light blue section is a section rejected at a significance level of 5% by performing a one-sample t-test, assuming that the null hypothesis is “normal distribution with zero mean and unknown variance”. Also, Fig. 9 shows the prediction process of cognitive processing in four models in MHP, the position of appearance of RP, and the position of mouse click. Note that the position of 0ms in Fig. 9 is the timing of visual stimulus presentation. From Fig. 9, it can be seen that, in Physical Matches, Name Matches, and Class Matches, the appearance position of RP while Cognitive Processor is working. However, in Simple reaction, RP appeared while Perceptual Processor is working. 5.3. Peak position of RP The position of the peak of the total average waveform and RP in the four models (the position where the amplitude of the waveform is significantly maximized in the interval of appearance of RP) is shown in Fig. 10. In Fig. 10, the position of 0ms is the position of the mouse click, and the red line shows the timing of the visual stimulus presentation. Also, Fig. 11 shows the prediction process of cognitive processing in four models of MHP, the position of the peak of RP, and the position of the mouse click. Note that the position of 0ms in Fig. 11 is the timing of visual stimulus presentation. It can be seen from Fig. 11 that in each reaction, the time from the visual stimulus presentation to the appearance of the RP peak extends in the order of Simple Reaction, Physical Matches, Name Matches, and Class Matches. This result is common to the fact that the number of Cognitive Processors in MHP increases in the order of Simple response, Physical Matches, Name Matches, and Class Matches. Table 2. Mean value and dispersion of RT
Nanako Shimizu et al. / Procedia Computer Science 159 (2019) 2005–2014 Author name / Procedia Computer Science 00 (2019) 000–000
Fig. 7. Mean value of RT, standard error, p-value
Fig. 8. Appearance position of RP
2011 7
8 2012
Author name / Procedia Computer Science 00 (2019) 000–000 Nanako Shimizu et al. / Procedia Computer Science 159 (2019) 2005–2014
Fig. 9. Prediction model in MHP and Appearance potion of RP
Fig. 10. Peak position of RP
Author name / Procedia Computer Science 00 (2019) 000–000 Nanako Shimizu et al. / Procedia Computer Science 159 (2019) 2005–2014
9 2013
Fig. 11. Prediction model in MHP and peak potion of RP
6. Consideration RT in the four models were significantly longer in the order of Simple Reaction, Physical Matches, Name Matches, and Class Matches. The prediction of RT of four models in MHP is shorter in the order of Simple Reaction, Physical Matches, Name Matches, and Class Matches. That is, the experiments of the four models performed this time were in agreement with the prediction of RT in MHP. Regarding the appearance position of RP, in Physical Matches, Name Matches, and Class Matches, it appeared in the section of Cognitive Processor. Therefore, in Physical Matches, Name Matches, and Class Matches, it can be said that MHP has no problem in terms of brain activity. On the other hand, in simple response, RP appeared while Perceptual Processor works. A simple response is a very simple response compared to the other three responses: mouse is clicked when a visual stimulus is presented. Therefore, it may be possible that the brain was ready to click the mouse before actually perceive the visual stimulus. However, even if the brain is ready for mouse click, the decision whether to actually perform mouse click may be considered after perception of the visual stimulus. Therefore, there is no mistake in the model of MHP which predicted the cognitive processing process of simple response as "perception → cognition (response determination) → movement". In addition, the position of the peak of RP is delayed in the order of Simple Reaction, Physical Matches, Name Matches, and Class Matches. This order is the same as the position order of reaction determination by Cognitive Processor in MHP. Therefore, it is considered that the position of the peak of RP corresponds to the position of reaction determination in MHP. As shown in Fig. 3, the following can be considered as the reason why RP did not appear 550 ms before exercise. The Libet Experiment revealed that RP appeared 550 ms before motion. However, the content of the experiment performed in the Libet Experiment is to have the subject press the button at any time, which is very different from the content performed in this experiment. Therefore, in this experiment, it is thought that RP appeared at a position different from the position shown in Fig. 3. From the above, it was found that MHP can be roughly explained also from the aspect of RP, which is a characteristic brain activity.
2014 10
Nanako Shimizu et al. / Procedia Computer Science 159 (2019) 2005–2014 Author name / Procedia Computer Science 00 (2019) 000–000
7. Summary In this study, we focused on RP, which is a characteristic EEG in order to verify whether MHP, which is an empirically derived model, is correct from the aspect of brain activity. Then, an experiment corresponding to four basic user performances was constructed, and the EEG in the experiment was measured. As a result of the experiment, RT was significantly shorter in the order of Simple Reaction, Physical Matches, Name Matches, and Class Matches, which was consistent with the tendency of RT prediction in MHP. The appearance position of RP is in a section of Cognitive Processor in Physical Matches, Name Matches, and Class Matches, and it is a section of Perceptual Processor in Simple reaction. In addition, it was found that the time from the presentation of visual stimulus to the appearance of the peak of RP increased in the order of Simple Reaction, Physical Matches, Name Matches, and Class Matches. This corresponds to the increase in the number of Cognitive Processors in MHP in the order of Simple Reaction, Physical Matches, Name Matches, and Class Matches. Therefore, MHP was found to be correct in Physical Matches, Name Matches, and Class Matches. On the other hand, in Simple response, RP appeared while Perceptual Processor was working. Since this is a very Simple response compared to the other three responses, it is possible that the brain was ready for movement before perceiving a visual stimulus. Therefore, MHP is not wrong even in Simple Reaction. From the above, it was found that MHP can be roughly explained also from the aspect of RP, which is a characteristic brain activity. 8. Acknowledgment This work was supported by JSPS KAKENHI Grant Number JP17H01996. References [1] Stuart K. Card, Thomas P. Moran, and Allen Newell (1983) “The Psychology of Human-Computer Interaction” Crc Press [2] Toshitaka Higashino, Yudai Asano, Masato Soga (2017) “Investigation of Model Human Processor by EEG Measurement” Procedia Computer Science 112: 2040-2047 [3] Kitty K. Lui, Michael D. Nunez, Jessica M. Cassidy, Joachim Vandekerckhove, Steven C. Cramer, and Ramesh Srinivasan (2018) “Timing of readiness potential reflect a decision-making process in the human brain” https://doi.org/10.1101/338806 [4] Hans H. Kornhuber, and Lüder Deecke (1965) “Hirnpotentialänderungen bei Willkürbewegungen und passive Bewegungen des Menschen: Bereitschaftspotential und refferente Potentiale” Pflüger’s Archiv für die gesamte Physiologie des Menschen und der Tiere 284: 1-17 [5] Benjamin Libet (1999) “Do We Have Free Will?” Journal of Consciousness Studies 6(8-9), 47-57
ScienceDirect ScienceDirect
Procedia Computer Science 00 (2019) 000–000 Procedia Computer Science 00 (2019) 000–000
Available online at www.sciencedirect.com
ScienceDirect
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
Procedia Computer Science 159 (2019) 2005–2014
23rd International Conference on Knowledge-Based and Intelligent Information & Engineering 23rd International Conference on Knowledge-Based Systems and Intelligent Information & Engineering Systems
Research on Comparison between Model Human Processor and Research on Comparison between Model Human Processor and Readiness Potential Readiness Potential Nanako Shimizuaa, Toshitaka Higashinobb, Masato Sogacc* Nanako Shimizu , Toshitaka Higashino , Masato Soga *
a Graduate School of System Engineering, Wakayama University, Sakaedani 930, Wakayama-city 640-8510, Japan a Graduate School of Information Science and Wakayama Technology,University, Osaka University, Yamadaoka 1-1, Suita-city 565-0871, Japan Graduate School of System Engineering, Sakaedani 930, Wakayama-city 640-8510, Japan a b Faculty Engineering, University, 930, Wakayama-city 640-8510, Japan Japan Graduate SchoolofofSystem Information ScienceWakayama and Technology, OsakaSakaedani University, Yamadaoka 1-1, Suita-city 565-0871, a Faculty of System Engineering, Wakayama University, Sakaedani 930, Wakayama-city 640-8510, Japan b
Abstract Abstract In thinking about human-computer interface, it is important to know how human beings recognize, judge and act. Therefore, Card In al. thinking about human-computer interface, it is important knowModel how human recognize, judge and act. Therefore, Card et devised a model of human cognitive processing processtocalled Humanbeings Processor (MHP). However, this MHP predicts et al. devised a model human cognitive process calledfrom Model (MHP). In However, thiswe MHP predicts processing time from of empirical rules, andprocessing is not much considered theHuman aspectProcessor of brain activity. this study, focused on processing time from empirical and is not much the MHP aspectfrom of brain activity. this activity study, we on readiness potential (RP), which rules, is a characteristic brainconsidered wave, andfrom verified the aspect ofIn brain by focused measuring readiness potential (RP), (EEG) which when is a characteristic brain user wave,performance. and verified Experimental MHP from theresults aspectshow of brain by be measuring Electroencephalography performing basic that activity MHP can roughly Electroencephalography performing basic user performance. explained from the aspect(EEG) of RP,when which is a characteristic brain activity. Experimental results show that MHP can be roughly explained from the aspect of RP, which is a characteristic brain activity. © 2019 The Author(s). Published by Elsevier B.V. © 2019 2019 The The Author(s). Authors. Published bybyElsevier B.V. © Published Elsevier B.V. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of KES International. Peer-review under responsibility of KES International. Peer-review under responsibility of KES International. Keywords: Model Human Processor; Readiness Potential; Basic User Performance; Electroencephalography; Human Computer Interface Keywords: Model Human Processor; Readiness Potential; Basic User Performance; Electroencephalography; Human Computer Interface
1. Introduction 1. Introduction There is the model of human cognitive processing process called Model Human Processor (MHP) modeled by Card is the model of human cognitive processing processtime called Human Processor by Card at alThere [1]. However, MHP is a model in which the processing is Model predicted empirically, and(MHP) has notmodeled been studied so at al [1]. However, MHP a model in which processingintime predicted empirically, and has not been studied so much from the aspect of isbrain activity [2][3].theTherefore, this is study, we focus on Electroencephalography (EEG) much the aspect of brain Therefore, thisverify study, we focus on aspect Electroencephalography called from readiness potential (RP), activity measure[2][3]. and analyze EEG,inand MHP from the of brain activity. (EEG) called readiness potential (RP), measure and analyze EEG, and verify MHP from the aspect of brain activity.
* Corresponding author. Tel.: +81-73-457-8457. E-mail address:author. [email protected] * Corresponding Tel.: +81-73-457-8457. E-mail address: [email protected] 1877-0509 © 2019 The Author(s). Published by Elsevier B.V. This is an open access underPublished the CC BY-NC-ND 1877-0509 © 2019 The article Author(s). by Elsevier license B.V. (https://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of KES International. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of KES International.
1877-0509 © 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of KES International. 10.1016/j.procs.2019.09.373
2
2006
Author name / Procedia Computer Science 00 (2019) 000–000
Nanako Shimizu et al. / Procedia Computer Science 159 (2019) 2005–2014
2. Purpose In MHP, the average time required for the perceptual process is 100ms, the average time required for the cognitive process is 70ms, and the time required for the motor process is 70ms (Fig. 1). So, using MHP, we can show the cognitive process of human’s four basic user performance: Simple Reaction, Physical Matches, Name Matches, and Class Matches as shown in Fig. 2. Specifically, the four basic user performance are as follows. Simple Reaction is a reaction performed when some information is presented. Physical Matches is a reaction performed when the presented information is the same as the information stored in advance. Name Matches is a reaction to be performed when the presented information can be represented by the same name as the name stored in advance. Class Matches is a reaction to be performed when the presented information can be represented by the same category as the category stored in advance. Also, RP is an EEG discovered by Kornhuber et al [4]. It is a small potential change that appears 550ms prior to voluntary movement, and the potential changes in the negative direction [5]. It is an EEG related to preparation for voluntary movement and is observed in the motor area. If we plot the time 550ms prior to voluntary movement by clicking a mouse in the Fig. 2, the time RP would appear will be shown in Fig. 3. In MHP, it is the Cognitive Processor section that prepares for movement in the brain. However, as shown in Fig. 3, the time when RP appears is before the time when Cognitive Processors work, which indicates that MHP and RP contradict each other. So, in this research, we construct an experiment corresponding to four models, measure the EEG in the experiment, and verify the contradiction point by confirming the appearance time of RP.
Fig. 1. Model Human Processor
Author Shimizu name / Procedia Computer Science 00 (2019) 000–000 Nanako et al. / Procedia Computer Science 159 (2019) 2005–2014
20073
Fig. 2. Four basic user performance
Fig. 3. Contradiction of MHP and RP
3. Method We constructed four experiments corresponding to MHP and measured the electroencephalogram of the subject in the experiment. In the experiment, a visual stimulus is presented to a subject wearing an electroencephalograph, and when the visual stimulus (target stimulus) taught in advance is presented, the left click of the mouse is performed, and when the other visual stimulus is presented, the right click is performed. In addition, the time from the presentation of the visual stimulus to the mouse click was recorded as the reaction time (RT).
2008 4
Nanako Shimizu et al. / Procedia Computer Science 159 (2019) 2005–2014 Author name / Procedia Computer Science 00 (2019) 000–000
Table 1 shows the visual stimulus, target stimulus and presentation rate of target stimulus in each experiment. In addition, BIOSEMI Active Two was used for EEG measurement. The sampling rate was 512Hz, and the electrodes were arranged based on the international 10-20 method (Fig. 4). In the visual stimulus presentation method, as shown in Fig. 5, the screen with the gaze point (+) presented and the screen with the visual stimulus presented alternately. The presentation time of the gaze point on the screen was shifted randomly between 1000 and 2000ms. This prevents the subject from remembering the timing at which the visual stimulus is presented. The visual stimulus screen was presented for 600ms. In each experiment, the visual stimulus were presented 30 times each. It is necessary for averaging EEG data. Table 1. Types of visual stimulus
Fig. 4. International 10-20 system
Fig. 5. How to present visual stimulus
Author name / Procedia Computer Science 00 (2019) 000–000
Nanako Shimizu et al. / Procedia Computer Science 159 (2019) 2005–2014
5
2009
4. Analysis Fig. 6 shows the process of deriving an averaged waveform for each subject from the acquired EEG data. By dividing the eye potential data from the EEG data, the noise due to eye movement and blinking was removed, and the noise due to phenomena other than the electroencephalogram was removed by applying a band pass filter of 1-30Hz. The noise-removed brain wave data was divided into intervals of -1000ms to 500ms from the trigger (the position of the mouse click). Baseline correction was performed for each of the divided data, using the average potential of 700ms to -500ms as a baseline with reference to the position of visual stimulus presentation. Then, data with RT exceeding the mean ± 2 SD (± 2 times the standard deviation) were excluded as outliers, and averaging was performed. Moreover, the total average waveform was derived by further averaging the average waveform of each subject.
Fig. 6. Process of EEG data shaping
2010 6
Nanako Shimizu et al. / Procedia Computer Science 159 (2019) 2005–2014 Author name / Procedia Computer Science 00 (2019) 000–000
5. Result Data from C3 electrodes of 10 healthy male and female subjects were used for analysis. The position of the C3 electrode is the motor area, and RP can be detected. 5.1. Comparison of RT in four models Table 2 shows the mean and variance of RT in four models. One-way analysis of variance was performed on the RT in four models at a significance level of 5%. As a result of multiple comparison, it was found that there was a significant difference among RTs of the four models, and RT became significantly longer in the order of Simple Reaction, Physical Matches, Name Matches, and Class Matches. Fig. 7 shows the mean value, standard error, and p value of RT in four models. In Fig. 7, the position of 0ms is the position of the mouse click, and the red line shows the timing of the visual stimulus presentation. The vertical axis is the amplitude (μV), and the horizontal axis is the time (sec). 5.2. Appearance position of RP Fig. 8 shows the total average waveform in four models and the position where RP began to appear (the position where the amplitude of the waveform began to swing significantly negatively). The light blue section is a section rejected at a significance level of 5% by performing a one-sample t-test, assuming that the null hypothesis is “normal distribution with zero mean and unknown variance”. Also, Fig. 9 shows the prediction process of cognitive processing in four models in MHP, the position of appearance of RP, and the position of mouse click. Note that the position of 0ms in Fig. 9 is the timing of visual stimulus presentation. From Fig. 9, it can be seen that, in Physical Matches, Name Matches, and Class Matches, the appearance position of RP while Cognitive Processor is working. However, in Simple reaction, RP appeared while Perceptual Processor is working. 5.3. Peak position of RP The position of the peak of the total average waveform and RP in the four models (the position where the amplitude of the waveform is significantly maximized in the interval of appearance of RP) is shown in Fig. 10. In Fig. 10, the position of 0ms is the position of the mouse click, and the red line shows the timing of the visual stimulus presentation. Also, Fig. 11 shows the prediction process of cognitive processing in four models of MHP, the position of the peak of RP, and the position of the mouse click. Note that the position of 0ms in Fig. 11 is the timing of visual stimulus presentation. It can be seen from Fig. 11 that in each reaction, the time from the visual stimulus presentation to the appearance of the RP peak extends in the order of Simple Reaction, Physical Matches, Name Matches, and Class Matches. This result is common to the fact that the number of Cognitive Processors in MHP increases in the order of Simple response, Physical Matches, Name Matches, and Class Matches. Table 2. Mean value and dispersion of RT
Nanako Shimizu et al. / Procedia Computer Science 159 (2019) 2005–2014 Author name / Procedia Computer Science 00 (2019) 000–000
Fig. 7. Mean value of RT, standard error, p-value
Fig. 8. Appearance position of RP
2011 7
8 2012
Author name / Procedia Computer Science 00 (2019) 000–000 Nanako Shimizu et al. / Procedia Computer Science 159 (2019) 2005–2014
Fig. 9. Prediction model in MHP and Appearance potion of RP
Fig. 10. Peak position of RP
Author name / Procedia Computer Science 00 (2019) 000–000 Nanako Shimizu et al. / Procedia Computer Science 159 (2019) 2005–2014
9 2013
Fig. 11. Prediction model in MHP and peak potion of RP
6. Consideration RT in the four models were significantly longer in the order of Simple Reaction, Physical Matches, Name Matches, and Class Matches. The prediction of RT of four models in MHP is shorter in the order of Simple Reaction, Physical Matches, Name Matches, and Class Matches. That is, the experiments of the four models performed this time were in agreement with the prediction of RT in MHP. Regarding the appearance position of RP, in Physical Matches, Name Matches, and Class Matches, it appeared in the section of Cognitive Processor. Therefore, in Physical Matches, Name Matches, and Class Matches, it can be said that MHP has no problem in terms of brain activity. On the other hand, in simple response, RP appeared while Perceptual Processor works. A simple response is a very simple response compared to the other three responses: mouse is clicked when a visual stimulus is presented. Therefore, it may be possible that the brain was ready to click the mouse before actually perceive the visual stimulus. However, even if the brain is ready for mouse click, the decision whether to actually perform mouse click may be considered after perception of the visual stimulus. Therefore, there is no mistake in the model of MHP which predicted the cognitive processing process of simple response as "perception → cognition (response determination) → movement". In addition, the position of the peak of RP is delayed in the order of Simple Reaction, Physical Matches, Name Matches, and Class Matches. This order is the same as the position order of reaction determination by Cognitive Processor in MHP. Therefore, it is considered that the position of the peak of RP corresponds to the position of reaction determination in MHP. As shown in Fig. 3, the following can be considered as the reason why RP did not appear 550 ms before exercise. The Libet Experiment revealed that RP appeared 550 ms before motion. However, the content of the experiment performed in the Libet Experiment is to have the subject press the button at any time, which is very different from the content performed in this experiment. Therefore, in this experiment, it is thought that RP appeared at a position different from the position shown in Fig. 3. From the above, it was found that MHP can be roughly explained also from the aspect of RP, which is a characteristic brain activity.
2014 10
Nanako Shimizu et al. / Procedia Computer Science 159 (2019) 2005–2014 Author name / Procedia Computer Science 00 (2019) 000–000
7. Summary In this study, we focused on RP, which is a characteristic EEG in order to verify whether MHP, which is an empirically derived model, is correct from the aspect of brain activity. Then, an experiment corresponding to four basic user performances was constructed, and the EEG in the experiment was measured. As a result of the experiment, RT was significantly shorter in the order of Simple Reaction, Physical Matches, Name Matches, and Class Matches, which was consistent with the tendency of RT prediction in MHP. The appearance position of RP is in a section of Cognitive Processor in Physical Matches, Name Matches, and Class Matches, and it is a section of Perceptual Processor in Simple reaction. In addition, it was found that the time from the presentation of visual stimulus to the appearance of the peak of RP increased in the order of Simple Reaction, Physical Matches, Name Matches, and Class Matches. This corresponds to the increase in the number of Cognitive Processors in MHP in the order of Simple Reaction, Physical Matches, Name Matches, and Class Matches. Therefore, MHP was found to be correct in Physical Matches, Name Matches, and Class Matches. On the other hand, in Simple response, RP appeared while Perceptual Processor was working. Since this is a very Simple response compared to the other three responses, it is possible that the brain was ready for movement before perceiving a visual stimulus. Therefore, MHP is not wrong even in Simple Reaction. From the above, it was found that MHP can be roughly explained also from the aspect of RP, which is a characteristic brain activity. 8. Acknowledgment This work was supported by JSPS KAKENHI Grant Number JP17H01996. References [1] Stuart K. Card, Thomas P. Moran, and Allen Newell (1983) “The Psychology of Human-Computer Interaction” Crc Press [2] Toshitaka Higashino, Yudai Asano, Masato Soga (2017) “Investigation of Model Human Processor by EEG Measurement” Procedia Computer Science 112: 2040-2047 [3] Kitty K. Lui, Michael D. Nunez, Jessica M. Cassidy, Joachim Vandekerckhove, Steven C. Cramer, and Ramesh Srinivasan (2018) “Timing of readiness potential reflect a decision-making process in the human brain” https://doi.org/10.1101/338806 [4] Hans H. Kornhuber, and Lüder Deecke (1965) “Hirnpotentialänderungen bei Willkürbewegungen und passive Bewegungen des Menschen: Bereitschaftspotential und refferente Potentiale” Pflüger’s Archiv für die gesamte Physiologie des Menschen und der Tiere 284: 1-17 [5] Benjamin Libet (1999) “Do We Have Free Will?” Journal of Consciousness Studies 6(8-9), 47-57