- Email: [email protected]
Using Bold-fMRI to detect cortical areas and visual fatigue related to stereoscopic vision
Displays 50 (2017) 14–20
Contents lists available at ScienceDirect
Displays journal homepage: www.elsevier.com/locate/displa
Using Bold-fMRI to detect cortical areas and visual fatigue related to stereoscopic vision q Chunxiao Chen a,⇑, Jing Wang a, Yupin Liu b, Xin Chen b a b
Department of Biomedical Engineering, Nanjing University of Aeronautics & Astronautics, Nanjing, China Department of Radiology, Guangdong Province Traditional Chinese Medical Hospital, Guangzhou, China
a r t i c l e
i n f o
Article history: Received 8 January 2016 Received in revised form 14 August 2017 Accepted 2 September 2017 Available online 5 September 2017 Keywords: fMRI Stereoscopic vision Block design Event-related design
a b s t r a c t Purpose: The location of cortical areas and visual fatigue related to stereoscopic vision were explored by two types of fMRI designs, namely block stimulation and event-related stimulation. The stimulations consist of 2D/3D images and different depths of the stereoscopic 3D images. Method: 20 normal subjects were randomly divided into the block group and the event-related group. Blood oxygenation level-dependent functional magnetic resonance imaging (Bold-fMRI) was performed in two groups. Functional data was preprocessed and statistically analyzed by SPM8. The result was reported by REST. Result: In the block stimulation group, compared to 2D image stimulation, 3D image stimulation results in more activated brain areas, including frontal lobes, occipital lobes and limbic lobes, especially in the frontal eye field (Brodmann Area 8, BA8) and middle occipital gyrus (BA18/19). In the event-related group, compared to 2D images, viewing 3D images causes significant activations in temporal lobe, mainly represented in BA19/13/31/37. Additionally, 3D image stimulation with the focus set at front depth can lead to the activation of more brain areas compared to the back depth, including inferior parietal lobule and posterior central gyrus. Conclusion: The formation of the stereoscopic vision requires the collaboration of more brain areas, so that viewing stereoscopic videos for a long period may result in visual fatigue; meanwhile, the front depth of field can contribute to more activated brain areas than the back depth of field. As a parameter of stereoscopic images, it is valid to state that the depth of field may affect visual fatigue. Ó 2017 Published by Elsevier B.V.
1. Introduction Recently, 3D technology has become one of the most popular research topics worldwide. Compared to traditional 2D videos, 3D videos are able to bring more realistic perceptions and stronger visual impact. As the 3DTV gradually becomes prevalent, potential health risks caused by watching 3D videos, including severe visual fatigue, nausea and headache, have drawn researchers’ attention [1]. According to the result of the subjective questionnaire, while watching 3D movies, 36% of the subjects experienced severe visual fatigue and 7% had to stop watching due to the visual fatigue [2]. Therefore, studying 3DTV’s influence on people’s health is extremely necessary. Since visual fatigue is the most prominent symptom of long period of watching 3DTV, many studies have been conducted on this q
This paper was recommended for publication by Richard H.Y. So.
⇑ Corresponding author.
E-mail addresses: [email protected] (C. Chen), [email protected] (J. Wang), [email protected] (Y. Liu), [email protected] (X. Chen). http://dx.doi.org/10.1016/j.displa.2017.09.003 0141-9382/Ó 2017 Published by Elsevier B.V.
topic. Lambooij [3] clarified the most pertinent factors contributing to visual fatigue, including temporally changing demand of accommodation-vergence linkage, three-dimensional artifacts and unnatural blur. Cho [4] studied visual discomfort caused by watching stereoscopic 3D content in the term of depth and proved that the visual fatigue is proportional to the depth of field. Yu [5] utilized the eye movement signal to detect the visual fatigue caused by 2D and 3D displays and found that compared with 2D videos, 3D videos would increase the blink frequency and scanning range, which are both proportional to visual fatigue. Kim [6] conducted research on the power spectrum of the electroencephalogram (EEG) and noticed the significant variations of the b wave in the occipital region after watching 3DTV. Functional Magnetic Resonance Imaging (fMRI) is widely used in the research of brain due to its noninvasiveness, high temporal and spatial resolution. There are two major designs for fMRI study—the block design and the event-related design, each with specific advantages and disadvantages. Block visual stimulation requires a long time to stable signals and has relatively stronger activation and larger activated regions. This paradigm suits for
C. Chen et al. / Displays 50 (2017) 14–20
the detection and location of regions of interest (ROI) during particular tasks. However, the block visual stimulation does not account for the transient responses at the beginning and end of task blocks. Event-related visual stimulation will be displayed alternately with short time for each image, which can detect Blood Oxygenation Level Dependent (BOLD) changes caused by instantaneous visual stimulation and will further our understanding of the dynamic nature of the neurophysiology of the human visual cortex. Thus, Event-related visual stimulation has a decrease of signal-tonoise, leading to less activity than block visual stimulation [7]. Jung [8] used fMRI to identify the cortical areas associated with the experience of visual discomfort in the viewing of stereoscopic images. Nakagawa [9] utilized block design fMRI to compare the variations of the brain activation between the first and last quarter during the attention task experiment and significant fatigueinduced deactivation was observed in frontal, temporal, occipital, and parietal cortices, cerebellum and midbrain. Gur [10] explored emotion processing deficits in patients with schizophrenia by using block BOLD fMRI and revealed that patients failed to activate limbic regions during emotion processing tasks. As for the application of block design fMRI in visual research, Dongchan [11] used fMRI to investigate the relationship between visual fatigue and binocular parallax. When the visual stimulation exceeds the comfort depth, neuron activity will be stronger in the frontal eye field (FEF). Additionally, he considered that the excessive binocular disparity stimulus may induce overload to the intraparietal sulcus [12]. Chen [13] revealed that 3D group showed more significant differences in brain activation compared with 2D group in occipital lobe and parietal lobe. However, during this experiment, 3D and 2D image stimulations were carried out alternately without interval and the influence of the hemodynamic response function (HRF) was ignored. Nevertheless, the block design averages the positive and negative responses occurred in a single block [14]. As a result, the urge to extract transient BOLD activity leads to the development of the event-related design. Marsman [15] used eye fixations as events in fMRI study to reveal cortical processing during the free exploration of visual images and found that fixations on different objects in different task contexts resulted in distinct cortical patterns of activation. Visual fatigue caused by watching 3DTV has been evaluated based on the subjective questionnaire in our previous work [16]. This paper incorporated block and event-related fMRI to study the cortex area related stereoscopic vision and its activation intensity under the stimulation with different stereoscopic depth of fields. This paper also discusses the brain functional mechanism that forms the stereoscopic vision, providing a reference to the objective evaluation of the visual fatigue caused by stereoscopic videos.
2. Method 2.1. Subjects A total of 20 healthy dextromanual subjects (10 males, 10 females, with the youngest to be 19 and oldest to be 24, and an average age of 22.3) participated in the study. All subjects have normal vision or corrected vision and normal stereoscopic sense. Eight 3D-movie clips with four levels of depth of field (1/2 in front of and 1/2 behind the focus point) were displayed to ensure that they had normal stereoscopic sensitivity. In the meantime, they do not have medical contraindications such as severe concomitant disease, alcoholism, drug abuse, as well as psychological or intellectual problems which are likely to limit compliance. Subjects lay flat inside the scanner with their heads immobilized, wearing sponge earplugs to avoid audio stimuli during the fMRI scans. All
15
participants have signed the informed consent form. Subjects were divided into two groups (block group and event-related group) randomly by statistical analysis and respectively received block and event-related stimulation. The local ethics committee has approved the study protocol. Each participant has signed an informed consent form and received ¥300 (CNY) in compensation for their participation. 2.2. Experimental device Subjects lay flat inside the scanner and watched the images projected on the optical reflector which fixed in the head coil of MRI. With the help of blue-and-red glasses, which replaced the metal joints with wood material, subjects could obtain the stereo view comfortably. The distance from subjects’ eyes to the reflector was 38 cm. All images had the same 1920 1080 pixel resolution and different depth. Front depth of field 3D images had the average depth of 487 mm and standard deviation of 143 mm, compared to a back depth average of 460 mm and standard deviation of 125 mm. 2.3. Method for the visual stimulation Block visual stimulation and event-related visual stimulation were performed to compare cortical activation between watching 3D and 2D images. 2.4. Block visual stimulation Each subject performed a visual attention task comprised of alternating blocks of two different visual demands (3D images [A] vs. 2D images [B]) and rest [C]. The task sequence was designed by E-prime2.0 and the block design (A-C-B-C) is shown in Fig. 1. The task sequence includes 3D images, 2D images and rest. Block A contains 6 3D images displayed subsequently by red and blue stereoscopy; block B contains 6 2D images with each corresponding to the 3D image; each image will be shown for 2s in the center of the display; block C indicates the rest status, during which subjects will watch the black screen for 12s, with a white cross in the center (‘+’) to minimize eyeball movements. Block A-C-B-C was repeated for 7 times and a 12s rest was arranged prior to the task sequence in order for the subjects to familiarize with the experiment environment. 2.4.1. Event-related visual stimulation The stimulation sequence was designed by E-prime2.0 and the event-related design shown in Fig. 2 was adopted as the visual stimulation. During the design of the visual stimulation, 30 3D and 30 2D images will be displayed alternately with 0.5s for each image. The interval between stimulations is set to be 6s and subject will look at a black screen with a white cross in the middle (‘+’) to minimize the eyeball movement during the rest. Subjects will familiarize with the experiment environment 10s before the test. To study the activation intensity under the stimulation of different stereoscopic depth of fields, front view and back view experiments were conducted separately in two days. Front depth of field and back depth of field 3D images were respectively adopted in each day’s experiment as the stereoscopic stimulation. 2.5. Data collection Functional and anatomical MR images were obtained with GESignaHDx3.0T Magnetic Resonance System in Guangdong Province Traditional Chinese Medical Hospital. The functional images were acquired with Gradient Echo-Echo Planar Imaging (GRE-EPI) sequence for whole brain images, which started slightly prior to
16
C. Chen et al. / Displays 50 (2017) 14–20
Fig. 1. Design and examples of stimuli presented to subjects while block visual task.
Fig. 2. Design and examples of stimuli presented to subjects while Event-related visual task.
the stimulation task until the end of the task. The scanning parameters are: repetition time (TR) = 2000 ms, echo time (TE) = 30 ms, flip angle (FA) = 90°, thickness/gap = 4/0 mm, matrix size of a single slice = 64 64, field of view (FOV) = 240 240 mm and 32 axial slices. Sagittal anatomical images were acquired with 3Dimensional T1-Weighted Gradient Echo Sequence and the scanning parameters are: TR = 7.796 ms, TE = 2.984 ms, FA = 12°, matrix size of a single slice = 256 256, thickness/gap = 2/0 mm, FOV = 256 256 mm and 196 sagittal images.
3. Results and discussion 3.1. Results for block Bold-fMRI Bold-fMRI data analysis was conducted using Statistical Parametric Mapping (SPM8) with post hoc tests of ANOVA method. To study the difference on brain activation areas during watching 3D and 2D images, statistical parametric images were displayed on the Montreal Neurological Institute (MNI) standard 3D brain template, and anatomical indicators were used to mark brain areas with significant differences. Sizes, brain areas, MNI coordinates and related intensity of statistically significant clusters were also
recorded. Fig. 3 shows the SPM results during watching 3D and 2D images from section view and render view; Table 1 displays each brain area’s location, with R and L representing the right hemisphere and left hemisphere respectively, BA referring to Brodmann Area and K meaning the activation cluster size. Larger K value shows that more brain voxels are significantly activated and the color bar represents the relationship between the color and T value in order to indicate activation intensity. Fig. 3 indicates that compared to viewing 2D images, significant differences will occur at occipital lobe, frontal lobe and limbic lobe while viewing 3D images. Occipital lobe includes associated cortex (BA18/BA19), frontal lobe includes middle frontal gyrus (BA9/ BA10), superior frontal gyrus (BA6/8), medial frontal gyrus (BA9) and limbic lobe includes cingulate gyrus (BA32), with BA8 (frontal eye movement area) and BA32 (somatosensory control zone) being the most significant. 3.2. Results for the event-related Bold-fMRI 3.2.1. Comparison of 3D and 2D images The analysis methods are similar to Section 3.1. SPM results between viewing 3D and 2D images are shown in Fig. 4; detailed brain area locations are shown in Table 2. According to Fig. 4 and
17
C. Chen et al. / Displays 50 (2017) 14–20
Fig. 3. Experiment results caused by viewing 3D images and 2D images in block design.
Table 1 The more strongly activated sites produced by viewing 3D images compared to 2D images under block design. Activation site (L/R)
Occipital lobe Frontal lobe
Limbic lobe
MNI coordinate (mm)
Lingual Gyrus (L) Lingual Gyrus (R) Middle Frontal Gyrus (R) Superior Frontal Gyrus (L/R) Medial Frontal Gyrus (R) Cingulate Gyrus (R)
x
y
z
12 18 36 18 12 15
84 78 21 51 33 24
15 9 36 18 33 36
BA
K
t value
18/19 18/19 9/10 6/8 9 32
369
4.11 3.65 4.61 4.36 3.96 3.90
1 608
Fig. 4. Experiment results caused by viewing 3D images and 2D images in event-related design.
Table 2, viewing 3D images causes significant activations in temporal lobe under the event-related group compared to 2D images, mainly represented in BA19/13/31/37.
3.2.2. Comparison of viewing 3D images with Front/Back depth of field To study the difference of brain activation areas caused by viewing 3D images with front/back depth of field, paired t-test
18
C. Chen et al. / Displays 50 (2017) 14–20
Table 2 The more strongly activated sites produced by viewing 3D images compared to 2D images under event-related design. Activation site (L/R)
MNI coordinate (mm) x
Temporal lobe
Precuneus (L) Precuneus (R)
y
27 33
BA
K
t value
13/19/31/37 13/19/31/37
392 391
4.77 5.05
z 42 42
was conducted on the fMRI data of different depths of 3D image stimulations (topological FDR, P < 0.005, cluster size = 81). Statistical parametric images were displayed on the MNI standard 3D brain template and anatomical indicators were used to mark brain areas with significant differences. Sizes, brain areas, MNI coordinates and related intensity of statistically significant clusters were also recorded. Results on the paired t-test on the fMRI data of watching 3D images with front/back depth of field are shown in Fig. 5, and detailed brain area locations are shown in Table 3. According to the results above, viewing 3D images with front depth of field results in larger activation areas in the right parietal lobe (BA40/2) compared to viewing 3D images with back depth of field. 3.3. Discussion The image depth cognitive process of stereoscopic vision is considered as the analyzing process of dynamic multi-information. Studies have shown that the stereoscopic vision may be produced
12 3
by several regions together, including parietal lobe, occipital lobe and temporal lobe [17]. Jung’s study has shown that visual discomfort caused by stereoscopic image will lead to brain sensory and motor systems get overloaded, which also includes frontal gyrus, temporal gyrus, parietal lobule [8]. Block experiment compared the differences of the activation areas between viewing 3D and 2D images and has indicated that occipital lobe, frontal lobe and limbic lobe are all relevant with the formation of stereovision. In which, occipital lobe includes the extrastriate cortex comprised by BA18 and BA19, frontal lobe includes middle frontal gyrus (BA9/BA10), superior frontal gyrus (BA6/8), medial frontal gyrus (BA9) and limbic lobe includes cingulate gyrus (BA32). As the visual processing region, the extrastriate cortex (BA18/BA19) accepts the primary visual information formed by the primary visual cortex to conduct advanced visual information processing and dispatch to other parts of the brain [18,19]. Middle frontal gyrus (BA9/BA10) mainly takes charge of the cognition; it is associated with high-level executive functions and decision-related processes combined with cingulate gyrus (BA32) [20,21]. BA6
Fig. 5. Experiment results caused by viewing 3D images in front and back depth of field.
Table 3 The more strongly activated sites produced by viewing front depth of field images compared to that of the back one. Activation site (L/R)
Parietal Lobe
MNI coordinate (mm) x
y
Inferior Parietal Lobule (R) Posterior Central Gyrus (R)
39 42
BA
K
t value
40 2
81
9.70 8.26
z 48 27
48 40
C. Chen et al. / Displays 50 (2017) 14–20
and BA8 in frontal lobe form the premotor cortex, providing sensory guidance and planning for movement, especially the frontal eye movement field in BA8, which plays an important role in the control of visual attention and eye movement [22]. Previous studies have also focused on visual fatigue caused by stereoscopic display, stating that the visual cortex in occipital lobe and frontal eye field in frontal lobe are most sensitive to the stimulation caused by stereovision and hence are related to visual fatigue [13,17]. According to the result from the event-related experiment, it is found that brain areas related to stereovision include BA19, which contains regions of the visual areas designated V3, V4, V5 (also known as the middle temporal area, or MT) [11]. Besides BA19, significant activations also occur at BA13 and BA31; both of them are considered to be closely related with human’s emotions, memories and behaviors [23]. Results indicate that middle temporal gyrus (BA37) is also activated during the stimulation of 3D images. BA37 belongs to MT+ of humans, which is involved in the processing of motion information [24]. According to the analysis mentioned above, viewing 3D images will activate advanced visual associated cortex as well as the limbic system. However, a long period of neuron excitation will increase the consumption of energy and oxygen, and thus increasing the burden of the brain [25,26]. Therefore, continuously maintaining the activation status of these areas for a long period will contribute to visual fatigue. From these results, different brain areas will be activated upon viewing 3D images between the block experiment and the eventrelated experiment except BA19. As mentioned above, it is reasonable to have BA19 activated in both groups since it is sensitive to binocular disparity by fMRI experiments [11]. However, different characteristics of each design (average response for block and transient response for event-related) have led to significant variations in activated areas. From the comparison of the activation areas between 3D images with front and back depth of field, it is found that more brain areas will be activated via 3D images with front depth of field, mainly at the right parietal lobe. During the visual cognition, the right hemisphere of the brain mainly takes charge of the complicated shapes and configurations and most studies have shown that the main regulation area of stereovision is located at the right hemisphere of the brain [27]. Meanwhile, research has verified the relationship between frontal lobe and stereoscopic vision. The importance of the frontal lobe (BA40) on cognition includes the movement in depth perception [28], visual flow [29], biological movement [30], spatial cognition [31] etc. Frontal and occipital lobes are associated with depth perception, gaze and the guidance of visual body. Therefore, patients with impaired frontal lobes will suffer from severe stereo damages [32]. When performing visual attention tasks, right frontal lobe has significant advantages compared to left frontal lobe. Patients with impaired right frontal lobes will suffer from severe damage on the cognition of the side space [33]. Based on the analysis and results, the paper explains that viewing 3D images with front depth of field can lead to the activation of right frontal lobe and maintaining the activation status for a long period will contribute to visual fatigue. Therefore, it is suggested that viewing images with front depth of field can result in more severe fatigue than images with back depth of field.
4. Conclusion This paper adopts block and event-related fMRI to conclude that the frontal eye field (BA8) and visual cortex (BA19) are both closely associated with stereovision. It is suggested that stereovision is formed by the integration of the primary visual cortex, followed with the processing of the advanced visual area (BA19). Viewing stereoscopic images will lead to significant activations of occipital
19
lobe, temporal lobe, frontal lobe and limbic lobe of the brain, indicating that the formation of stereovision involves not only the visual cortex, but also the co-function of multiple brain areas, including synergistic fusion flexibility and stamina cognition, etc. This paper preliminarily discusses the brain mechanism of the stereovision formation and provides objective basis to evaluate the visual fatigue caused by stereoscopic videos in the future. Some other experimental conditions, such as crosstalk, resolution, display luminance contrast and ambient illumination, are important factors that lead to visual fatigue. So, in the following work, we will pay attention to these conditions when we explore the mechanism of visual fatigue. Acknowledgement This study was supported by the National Natural Science Foundation of China (Grant No. 61171059). References [1] W.J. Tam, F. Speranza, S. Yano, et al., 3D-TV: visual comfort, IEEE Transact. Broadcast. 57 (2) (2011) 335–346. [2] Jung-Hoon Lee, Jang-Kun Song, Individual variation in 3D visual fatigue caused by stereoscopic images, IEEE Trans. Consum. Electron. 58 (2) (2012) 500–504. [3] M. Lambooij, M. Fortuin, I. Heynderickx, et al., Visual discomfort and visual fatigue of stereoscopic displays: a review, J. Imaging Sci. Technol. 53 (3) (2009), 30201-1-30201-14. [4] S.H. Cho, H.B. Kang, An analysis of visual discomfort caused by watching stereoscopic 3D content in terms of depth, viewing time and display size, J. Imaging Sci. Technol. 59 (2) (2015), 20503-1-20503-16. [5] J.H. Yu, B.H. Lee, D.H. Kim, EOG based eye movement measure of visual fatigue caused by 2D and 3D displays[C]//Biomedical and Health Informatics (BHI), 2012 IEEE-EMBS International Conference on. IEEE, 2012, pp. 305–308. [6] Y.J. Kim, E.C. Lee, EEG based comparative measurement of visual fatigue caused by 2D and 3D Displays, Commun. Comput. Inform. Sci. 174 (2011) 289–292. [7] D.I. Donaldson, Parsing brain activity with fMRI and mixed designs: what kind of a state is neuroimaging, Trends Neurosci. 27 (8) (2004) 442–444. [8] Y.J. Jung, D. Kim, H. Sohn, et al., Towards a physiology-based measure of visual discomfort: brain activity measurement while viewing stereoscopic images with different, screen disparities, J. Display Technol. 11 (2015) 1–11. [9] S. Nakagawa, M. Sugiura, Y. Akitsuki, et al., Compensatory effort parallels midbrain deactivation during mental fatigue: an fMRI study, PloS One 8 (2) (2013) e56606. [10] R.E. Gur, C. McGrath, R.M. Chan, et al., An fMRI study of facial emotion processing in patients with schizophrenia, Am. J. Psychiatry (2014). [11] D. Kim, Y.J. Jung, E. Kim, et al., Human brain response to visual fatigue caused by stereoscopic depth perception[C]//Digital Signal Processing (DSP), 2011 17th International Conference on, IEEE, 2011, pp. 1–5. [12] D. Kim, Y.J. Jung, Y. Han, et al., fMRI analysis of excessive binocular disparity on the human brain, Int. J. Imaging Syst. Technol. 24 (1) (2014) 94–102. [13] C.X. Chen, J. Wang, K. Li, et al., Visual fatigue caused by watching 3DTV: an fMRI study, Biomed. Eng. Online 14 (Suppl 1) (2015) S12. [14] J.A. Meltzer, M. Negishi, R.T. Constable, Biphasic hemodynamic responses influence deactivation and may mask activation in block-design fMRI paradigms, Human Brain Mapp. 29 (4) (2008) 385–399. [15] J.B.C. Marsman, R. Renken, B.M. Velichkovsky, et al., Fixation based eventrelated fmri analysis: using eye fixations as events in functional magnetic resonance imaging to reveal cortical processing during the free exploration of visual images, Human Brain Mapp. 33 (2) (2012) 307–318. [16] C.X. Chen, K. Li, Q.Y. Wu, et al., EEG-based detection and evaluation of fatigue caused by watching 3DTV, Displays 34 (2) (2013) 81–88. [17] B.T. Backus, D.J. Fleet, A.J. Parker, et al., Human cortical activity correlates with stereoscopic depth perception, J. Neurophysiol. 86 (4) (2001) 2054–2068. [18] A.J. Parker, Binocular depth perception and the cerebral cortex, Nat. Rev. Neurosci. 8 (5) (2007) 379–391. [19] G.A. Orban, Higher order visual processing in macaque extrastriate cortex., Physiol. Rev. 88 (1) (2008) 59–89. [20] A. Talati, J. Hirsch, Functional specialization within the medial frontal gyrus for perceptual Go/No-Go decisions based on ‘‘What”, ‘‘When”, and ‘‘Where” related information: an fMRI study, Cogn. Neurosci. J. 17 (2005) 981–993. [21] N. Yeung, J. Ralph, S. Nieuwenhuis, Drink alcohol and dim the lights: the impact of cognitive deficits on medial frontal cortex function, Cogn. Affect. Behav. Neurosci. 7 (4) (2007) 347–355. [22] J.D. Schall, On the role of frontal eye field in guiding attention and saccades, Vision. Res. 44 (12) (2004) 1453–1467. [23] Marco Catani, Flavio Dell’Acqua, Michel Thiebaut De Schotten, A revised limbic system model for memory, emotion and behaviour, Neurosci. Biobehav. Rev. 37 (8) (2013) 1724–1737. [24] R. Goebel, D. Khorram-Sefat, L. Muckli, et al., The constructive nature of vision: direct evidence from functional magnetic resonance imaging studies of
20
[25] [26]
[27]
[28]
C. Chen et al. / Displays 50 (2017) 14–20 apparent motion and motion imagery, Eur. J. Neurosci. 10 (5) (1998) 1563– 1573. H.V. Duinen, R. Renken, N. Maurits, et al., Effects of motor fatigue on human brain activity, an fMRI study, Neuroimage 35 (4) (2007) 1438–1449. J. Lim, W.C. Wu, J. Wang, et al., Imaging brain fatigue from sustained mental workload: an ASL perfusion study of the time-on-task effect, Neuroimage 49 (4) (2010) 3426–3435. L. Kalbfleisch, C. Gillmarten, Left brain vs. right brain: findings on visual spatial capacities and the functional neurology of giftedness, Roeper Rev. 35 (4) (2013) 265–275. M. Ptito, R. Kupers, J. Faubert, et al., Cortical representation of inward and outward radial motion in man, Neuroimage 14 (6) (2001) 1409–1415.
[29] F. Bremmer, J.R. Duhamel, S.B. Hamed, et al., Stages of self-motion processing in primate posterior parietal cortex, Int. Rev. Neurobiol. 44 (2000) 173–198. [30] E. Bonda, M. Petrides, D. Ostry, et al., Specific involvement of human parietal systems and the amygdala in the perception of biological motion, J. Neurosci. 16 (11) (1996) 3737–3744. [31] R. Cabeza, L. Nyberg, Neural bases of learning and memory: functional neuroimaging evidence, Curr. Opin. Neurol. 13 (4) (2000) 415–421. [32] Y. Nishida, O. Hayashi, T. Iwami, et al., Stereopsis-processing regions in the human parieto-occipital cortex, Neuroreport 12 (10) (2001) 2259–2263. [33] S. Agosta, F. Herpich, F. Ferraro, et al., Stimulation of the left parietal lobe improves spatial and temporal attention in right parietal lobe patients: tipping the inter-hemispheric balance with TMS, J. Vision 13 (9) (2013), 287-287.
Contents lists available at ScienceDirect
Displays journal homepage: www.elsevier.com/locate/displa
Using Bold-fMRI to detect cortical areas and visual fatigue related to stereoscopic vision q Chunxiao Chen a,⇑, Jing Wang a, Yupin Liu b, Xin Chen b a b
Department of Biomedical Engineering, Nanjing University of Aeronautics & Astronautics, Nanjing, China Department of Radiology, Guangdong Province Traditional Chinese Medical Hospital, Guangzhou, China
a r t i c l e
i n f o
Article history: Received 8 January 2016 Received in revised form 14 August 2017 Accepted 2 September 2017 Available online 5 September 2017 Keywords: fMRI Stereoscopic vision Block design Event-related design
a b s t r a c t Purpose: The location of cortical areas and visual fatigue related to stereoscopic vision were explored by two types of fMRI designs, namely block stimulation and event-related stimulation. The stimulations consist of 2D/3D images and different depths of the stereoscopic 3D images. Method: 20 normal subjects were randomly divided into the block group and the event-related group. Blood oxygenation level-dependent functional magnetic resonance imaging (Bold-fMRI) was performed in two groups. Functional data was preprocessed and statistically analyzed by SPM8. The result was reported by REST. Result: In the block stimulation group, compared to 2D image stimulation, 3D image stimulation results in more activated brain areas, including frontal lobes, occipital lobes and limbic lobes, especially in the frontal eye field (Brodmann Area 8, BA8) and middle occipital gyrus (BA18/19). In the event-related group, compared to 2D images, viewing 3D images causes significant activations in temporal lobe, mainly represented in BA19/13/31/37. Additionally, 3D image stimulation with the focus set at front depth can lead to the activation of more brain areas compared to the back depth, including inferior parietal lobule and posterior central gyrus. Conclusion: The formation of the stereoscopic vision requires the collaboration of more brain areas, so that viewing stereoscopic videos for a long period may result in visual fatigue; meanwhile, the front depth of field can contribute to more activated brain areas than the back depth of field. As a parameter of stereoscopic images, it is valid to state that the depth of field may affect visual fatigue. Ó 2017 Published by Elsevier B.V.
1. Introduction Recently, 3D technology has become one of the most popular research topics worldwide. Compared to traditional 2D videos, 3D videos are able to bring more realistic perceptions and stronger visual impact. As the 3DTV gradually becomes prevalent, potential health risks caused by watching 3D videos, including severe visual fatigue, nausea and headache, have drawn researchers’ attention [1]. According to the result of the subjective questionnaire, while watching 3D movies, 36% of the subjects experienced severe visual fatigue and 7% had to stop watching due to the visual fatigue [2]. Therefore, studying 3DTV’s influence on people’s health is extremely necessary. Since visual fatigue is the most prominent symptom of long period of watching 3DTV, many studies have been conducted on this q
This paper was recommended for publication by Richard H.Y. So.
⇑ Corresponding author.
E-mail addresses: [email protected] (C. Chen), [email protected] (J. Wang), [email protected] (Y. Liu), [email protected] (X. Chen). http://dx.doi.org/10.1016/j.displa.2017.09.003 0141-9382/Ó 2017 Published by Elsevier B.V.
topic. Lambooij [3] clarified the most pertinent factors contributing to visual fatigue, including temporally changing demand of accommodation-vergence linkage, three-dimensional artifacts and unnatural blur. Cho [4] studied visual discomfort caused by watching stereoscopic 3D content in the term of depth and proved that the visual fatigue is proportional to the depth of field. Yu [5] utilized the eye movement signal to detect the visual fatigue caused by 2D and 3D displays and found that compared with 2D videos, 3D videos would increase the blink frequency and scanning range, which are both proportional to visual fatigue. Kim [6] conducted research on the power spectrum of the electroencephalogram (EEG) and noticed the significant variations of the b wave in the occipital region after watching 3DTV. Functional Magnetic Resonance Imaging (fMRI) is widely used in the research of brain due to its noninvasiveness, high temporal and spatial resolution. There are two major designs for fMRI study—the block design and the event-related design, each with specific advantages and disadvantages. Block visual stimulation requires a long time to stable signals and has relatively stronger activation and larger activated regions. This paradigm suits for
C. Chen et al. / Displays 50 (2017) 14–20
the detection and location of regions of interest (ROI) during particular tasks. However, the block visual stimulation does not account for the transient responses at the beginning and end of task blocks. Event-related visual stimulation will be displayed alternately with short time for each image, which can detect Blood Oxygenation Level Dependent (BOLD) changes caused by instantaneous visual stimulation and will further our understanding of the dynamic nature of the neurophysiology of the human visual cortex. Thus, Event-related visual stimulation has a decrease of signal-tonoise, leading to less activity than block visual stimulation [7]. Jung [8] used fMRI to identify the cortical areas associated with the experience of visual discomfort in the viewing of stereoscopic images. Nakagawa [9] utilized block design fMRI to compare the variations of the brain activation between the first and last quarter during the attention task experiment and significant fatigueinduced deactivation was observed in frontal, temporal, occipital, and parietal cortices, cerebellum and midbrain. Gur [10] explored emotion processing deficits in patients with schizophrenia by using block BOLD fMRI and revealed that patients failed to activate limbic regions during emotion processing tasks. As for the application of block design fMRI in visual research, Dongchan [11] used fMRI to investigate the relationship between visual fatigue and binocular parallax. When the visual stimulation exceeds the comfort depth, neuron activity will be stronger in the frontal eye field (FEF). Additionally, he considered that the excessive binocular disparity stimulus may induce overload to the intraparietal sulcus [12]. Chen [13] revealed that 3D group showed more significant differences in brain activation compared with 2D group in occipital lobe and parietal lobe. However, during this experiment, 3D and 2D image stimulations were carried out alternately without interval and the influence of the hemodynamic response function (HRF) was ignored. Nevertheless, the block design averages the positive and negative responses occurred in a single block [14]. As a result, the urge to extract transient BOLD activity leads to the development of the event-related design. Marsman [15] used eye fixations as events in fMRI study to reveal cortical processing during the free exploration of visual images and found that fixations on different objects in different task contexts resulted in distinct cortical patterns of activation. Visual fatigue caused by watching 3DTV has been evaluated based on the subjective questionnaire in our previous work [16]. This paper incorporated block and event-related fMRI to study the cortex area related stereoscopic vision and its activation intensity under the stimulation with different stereoscopic depth of fields. This paper also discusses the brain functional mechanism that forms the stereoscopic vision, providing a reference to the objective evaluation of the visual fatigue caused by stereoscopic videos.
2. Method 2.1. Subjects A total of 20 healthy dextromanual subjects (10 males, 10 females, with the youngest to be 19 and oldest to be 24, and an average age of 22.3) participated in the study. All subjects have normal vision or corrected vision and normal stereoscopic sense. Eight 3D-movie clips with four levels of depth of field (1/2 in front of and 1/2 behind the focus point) were displayed to ensure that they had normal stereoscopic sensitivity. In the meantime, they do not have medical contraindications such as severe concomitant disease, alcoholism, drug abuse, as well as psychological or intellectual problems which are likely to limit compliance. Subjects lay flat inside the scanner with their heads immobilized, wearing sponge earplugs to avoid audio stimuli during the fMRI scans. All
15
participants have signed the informed consent form. Subjects were divided into two groups (block group and event-related group) randomly by statistical analysis and respectively received block and event-related stimulation. The local ethics committee has approved the study protocol. Each participant has signed an informed consent form and received ¥300 (CNY) in compensation for their participation. 2.2. Experimental device Subjects lay flat inside the scanner and watched the images projected on the optical reflector which fixed in the head coil of MRI. With the help of blue-and-red glasses, which replaced the metal joints with wood material, subjects could obtain the stereo view comfortably. The distance from subjects’ eyes to the reflector was 38 cm. All images had the same 1920 1080 pixel resolution and different depth. Front depth of field 3D images had the average depth of 487 mm and standard deviation of 143 mm, compared to a back depth average of 460 mm and standard deviation of 125 mm. 2.3. Method for the visual stimulation Block visual stimulation and event-related visual stimulation were performed to compare cortical activation between watching 3D and 2D images. 2.4. Block visual stimulation Each subject performed a visual attention task comprised of alternating blocks of two different visual demands (3D images [A] vs. 2D images [B]) and rest [C]. The task sequence was designed by E-prime2.0 and the block design (A-C-B-C) is shown in Fig. 1. The task sequence includes 3D images, 2D images and rest. Block A contains 6 3D images displayed subsequently by red and blue stereoscopy; block B contains 6 2D images with each corresponding to the 3D image; each image will be shown for 2s in the center of the display; block C indicates the rest status, during which subjects will watch the black screen for 12s, with a white cross in the center (‘+’) to minimize eyeball movements. Block A-C-B-C was repeated for 7 times and a 12s rest was arranged prior to the task sequence in order for the subjects to familiarize with the experiment environment. 2.4.1. Event-related visual stimulation The stimulation sequence was designed by E-prime2.0 and the event-related design shown in Fig. 2 was adopted as the visual stimulation. During the design of the visual stimulation, 30 3D and 30 2D images will be displayed alternately with 0.5s for each image. The interval between stimulations is set to be 6s and subject will look at a black screen with a white cross in the middle (‘+’) to minimize the eyeball movement during the rest. Subjects will familiarize with the experiment environment 10s before the test. To study the activation intensity under the stimulation of different stereoscopic depth of fields, front view and back view experiments were conducted separately in two days. Front depth of field and back depth of field 3D images were respectively adopted in each day’s experiment as the stereoscopic stimulation. 2.5. Data collection Functional and anatomical MR images were obtained with GESignaHDx3.0T Magnetic Resonance System in Guangdong Province Traditional Chinese Medical Hospital. The functional images were acquired with Gradient Echo-Echo Planar Imaging (GRE-EPI) sequence for whole brain images, which started slightly prior to
16
C. Chen et al. / Displays 50 (2017) 14–20
Fig. 1. Design and examples of stimuli presented to subjects while block visual task.
Fig. 2. Design and examples of stimuli presented to subjects while Event-related visual task.
the stimulation task until the end of the task. The scanning parameters are: repetition time (TR) = 2000 ms, echo time (TE) = 30 ms, flip angle (FA) = 90°, thickness/gap = 4/0 mm, matrix size of a single slice = 64 64, field of view (FOV) = 240 240 mm and 32 axial slices. Sagittal anatomical images were acquired with 3Dimensional T1-Weighted Gradient Echo Sequence and the scanning parameters are: TR = 7.796 ms, TE = 2.984 ms, FA = 12°, matrix size of a single slice = 256 256, thickness/gap = 2/0 mm, FOV = 256 256 mm and 196 sagittal images.
3. Results and discussion 3.1. Results for block Bold-fMRI Bold-fMRI data analysis was conducted using Statistical Parametric Mapping (SPM8) with post hoc tests of ANOVA method. To study the difference on brain activation areas during watching 3D and 2D images, statistical parametric images were displayed on the Montreal Neurological Institute (MNI) standard 3D brain template, and anatomical indicators were used to mark brain areas with significant differences. Sizes, brain areas, MNI coordinates and related intensity of statistically significant clusters were also
recorded. Fig. 3 shows the SPM results during watching 3D and 2D images from section view and render view; Table 1 displays each brain area’s location, with R and L representing the right hemisphere and left hemisphere respectively, BA referring to Brodmann Area and K meaning the activation cluster size. Larger K value shows that more brain voxels are significantly activated and the color bar represents the relationship between the color and T value in order to indicate activation intensity. Fig. 3 indicates that compared to viewing 2D images, significant differences will occur at occipital lobe, frontal lobe and limbic lobe while viewing 3D images. Occipital lobe includes associated cortex (BA18/BA19), frontal lobe includes middle frontal gyrus (BA9/ BA10), superior frontal gyrus (BA6/8), medial frontal gyrus (BA9) and limbic lobe includes cingulate gyrus (BA32), with BA8 (frontal eye movement area) and BA32 (somatosensory control zone) being the most significant. 3.2. Results for the event-related Bold-fMRI 3.2.1. Comparison of 3D and 2D images The analysis methods are similar to Section 3.1. SPM results between viewing 3D and 2D images are shown in Fig. 4; detailed brain area locations are shown in Table 2. According to Fig. 4 and
17
C. Chen et al. / Displays 50 (2017) 14–20
Fig. 3. Experiment results caused by viewing 3D images and 2D images in block design.
Table 1 The more strongly activated sites produced by viewing 3D images compared to 2D images under block design. Activation site (L/R)
Occipital lobe Frontal lobe
Limbic lobe
MNI coordinate (mm)
Lingual Gyrus (L) Lingual Gyrus (R) Middle Frontal Gyrus (R) Superior Frontal Gyrus (L/R) Medial Frontal Gyrus (R) Cingulate Gyrus (R)
x
y
z
12 18 36 18 12 15
84 78 21 51 33 24
15 9 36 18 33 36
BA
K
t value
18/19 18/19 9/10 6/8 9 32
369
4.11 3.65 4.61 4.36 3.96 3.90
1 608
Fig. 4. Experiment results caused by viewing 3D images and 2D images in event-related design.
Table 2, viewing 3D images causes significant activations in temporal lobe under the event-related group compared to 2D images, mainly represented in BA19/13/31/37.
3.2.2. Comparison of viewing 3D images with Front/Back depth of field To study the difference of brain activation areas caused by viewing 3D images with front/back depth of field, paired t-test
18
C. Chen et al. / Displays 50 (2017) 14–20
Table 2 The more strongly activated sites produced by viewing 3D images compared to 2D images under event-related design. Activation site (L/R)
MNI coordinate (mm) x
Temporal lobe
Precuneus (L) Precuneus (R)
y
27 33
BA
K
t value
13/19/31/37 13/19/31/37
392 391
4.77 5.05
z 42 42
was conducted on the fMRI data of different depths of 3D image stimulations (topological FDR, P < 0.005, cluster size = 81). Statistical parametric images were displayed on the MNI standard 3D brain template and anatomical indicators were used to mark brain areas with significant differences. Sizes, brain areas, MNI coordinates and related intensity of statistically significant clusters were also recorded. Results on the paired t-test on the fMRI data of watching 3D images with front/back depth of field are shown in Fig. 5, and detailed brain area locations are shown in Table 3. According to the results above, viewing 3D images with front depth of field results in larger activation areas in the right parietal lobe (BA40/2) compared to viewing 3D images with back depth of field. 3.3. Discussion The image depth cognitive process of stereoscopic vision is considered as the analyzing process of dynamic multi-information. Studies have shown that the stereoscopic vision may be produced
12 3
by several regions together, including parietal lobe, occipital lobe and temporal lobe [17]. Jung’s study has shown that visual discomfort caused by stereoscopic image will lead to brain sensory and motor systems get overloaded, which also includes frontal gyrus, temporal gyrus, parietal lobule [8]. Block experiment compared the differences of the activation areas between viewing 3D and 2D images and has indicated that occipital lobe, frontal lobe and limbic lobe are all relevant with the formation of stereovision. In which, occipital lobe includes the extrastriate cortex comprised by BA18 and BA19, frontal lobe includes middle frontal gyrus (BA9/BA10), superior frontal gyrus (BA6/8), medial frontal gyrus (BA9) and limbic lobe includes cingulate gyrus (BA32). As the visual processing region, the extrastriate cortex (BA18/BA19) accepts the primary visual information formed by the primary visual cortex to conduct advanced visual information processing and dispatch to other parts of the brain [18,19]. Middle frontal gyrus (BA9/BA10) mainly takes charge of the cognition; it is associated with high-level executive functions and decision-related processes combined with cingulate gyrus (BA32) [20,21]. BA6
Fig. 5. Experiment results caused by viewing 3D images in front and back depth of field.
Table 3 The more strongly activated sites produced by viewing front depth of field images compared to that of the back one. Activation site (L/R)
Parietal Lobe
MNI coordinate (mm) x
y
Inferior Parietal Lobule (R) Posterior Central Gyrus (R)
39 42
BA
K
t value
40 2
81
9.70 8.26
z 48 27
48 40
C. Chen et al. / Displays 50 (2017) 14–20
and BA8 in frontal lobe form the premotor cortex, providing sensory guidance and planning for movement, especially the frontal eye movement field in BA8, which plays an important role in the control of visual attention and eye movement [22]. Previous studies have also focused on visual fatigue caused by stereoscopic display, stating that the visual cortex in occipital lobe and frontal eye field in frontal lobe are most sensitive to the stimulation caused by stereovision and hence are related to visual fatigue [13,17]. According to the result from the event-related experiment, it is found that brain areas related to stereovision include BA19, which contains regions of the visual areas designated V3, V4, V5 (also known as the middle temporal area, or MT) [11]. Besides BA19, significant activations also occur at BA13 and BA31; both of them are considered to be closely related with human’s emotions, memories and behaviors [23]. Results indicate that middle temporal gyrus (BA37) is also activated during the stimulation of 3D images. BA37 belongs to MT+ of humans, which is involved in the processing of motion information [24]. According to the analysis mentioned above, viewing 3D images will activate advanced visual associated cortex as well as the limbic system. However, a long period of neuron excitation will increase the consumption of energy and oxygen, and thus increasing the burden of the brain [25,26]. Therefore, continuously maintaining the activation status of these areas for a long period will contribute to visual fatigue. From these results, different brain areas will be activated upon viewing 3D images between the block experiment and the eventrelated experiment except BA19. As mentioned above, it is reasonable to have BA19 activated in both groups since it is sensitive to binocular disparity by fMRI experiments [11]. However, different characteristics of each design (average response for block and transient response for event-related) have led to significant variations in activated areas. From the comparison of the activation areas between 3D images with front and back depth of field, it is found that more brain areas will be activated via 3D images with front depth of field, mainly at the right parietal lobe. During the visual cognition, the right hemisphere of the brain mainly takes charge of the complicated shapes and configurations and most studies have shown that the main regulation area of stereovision is located at the right hemisphere of the brain [27]. Meanwhile, research has verified the relationship between frontal lobe and stereoscopic vision. The importance of the frontal lobe (BA40) on cognition includes the movement in depth perception [28], visual flow [29], biological movement [30], spatial cognition [31] etc. Frontal and occipital lobes are associated with depth perception, gaze and the guidance of visual body. Therefore, patients with impaired frontal lobes will suffer from severe stereo damages [32]. When performing visual attention tasks, right frontal lobe has significant advantages compared to left frontal lobe. Patients with impaired right frontal lobes will suffer from severe damage on the cognition of the side space [33]. Based on the analysis and results, the paper explains that viewing 3D images with front depth of field can lead to the activation of right frontal lobe and maintaining the activation status for a long period will contribute to visual fatigue. Therefore, it is suggested that viewing images with front depth of field can result in more severe fatigue than images with back depth of field.
4. Conclusion This paper adopts block and event-related fMRI to conclude that the frontal eye field (BA8) and visual cortex (BA19) are both closely associated with stereovision. It is suggested that stereovision is formed by the integration of the primary visual cortex, followed with the processing of the advanced visual area (BA19). Viewing stereoscopic images will lead to significant activations of occipital
19
lobe, temporal lobe, frontal lobe and limbic lobe of the brain, indicating that the formation of stereovision involves not only the visual cortex, but also the co-function of multiple brain areas, including synergistic fusion flexibility and stamina cognition, etc. This paper preliminarily discusses the brain mechanism of the stereovision formation and provides objective basis to evaluate the visual fatigue caused by stereoscopic videos in the future. Some other experimental conditions, such as crosstalk, resolution, display luminance contrast and ambient illumination, are important factors that lead to visual fatigue. So, in the following work, we will pay attention to these conditions when we explore the mechanism of visual fatigue. Acknowledgement This study was supported by the National Natural Science Foundation of China (Grant No. 61171059). References [1] W.J. Tam, F. Speranza, S. Yano, et al., 3D-TV: visual comfort, IEEE Transact. Broadcast. 57 (2) (2011) 335–346. [2] Jung-Hoon Lee, Jang-Kun Song, Individual variation in 3D visual fatigue caused by stereoscopic images, IEEE Trans. Consum. Electron. 58 (2) (2012) 500–504. [3] M. Lambooij, M. Fortuin, I. Heynderickx, et al., Visual discomfort and visual fatigue of stereoscopic displays: a review, J. Imaging Sci. Technol. 53 (3) (2009), 30201-1-30201-14. [4] S.H. Cho, H.B. Kang, An analysis of visual discomfort caused by watching stereoscopic 3D content in terms of depth, viewing time and display size, J. Imaging Sci. Technol. 59 (2) (2015), 20503-1-20503-16. [5] J.H. Yu, B.H. Lee, D.H. Kim, EOG based eye movement measure of visual fatigue caused by 2D and 3D displays[C]//Biomedical and Health Informatics (BHI), 2012 IEEE-EMBS International Conference on. IEEE, 2012, pp. 305–308. [6] Y.J. Kim, E.C. Lee, EEG based comparative measurement of visual fatigue caused by 2D and 3D Displays, Commun. Comput. Inform. Sci. 174 (2011) 289–292. [7] D.I. Donaldson, Parsing brain activity with fMRI and mixed designs: what kind of a state is neuroimaging, Trends Neurosci. 27 (8) (2004) 442–444. [8] Y.J. Jung, D. Kim, H. Sohn, et al., Towards a physiology-based measure of visual discomfort: brain activity measurement while viewing stereoscopic images with different, screen disparities, J. Display Technol. 11 (2015) 1–11. [9] S. Nakagawa, M. Sugiura, Y. Akitsuki, et al., Compensatory effort parallels midbrain deactivation during mental fatigue: an fMRI study, PloS One 8 (2) (2013) e56606. [10] R.E. Gur, C. McGrath, R.M. Chan, et al., An fMRI study of facial emotion processing in patients with schizophrenia, Am. J. Psychiatry (2014). [11] D. Kim, Y.J. Jung, E. Kim, et al., Human brain response to visual fatigue caused by stereoscopic depth perception[C]//Digital Signal Processing (DSP), 2011 17th International Conference on, IEEE, 2011, pp. 1–5. [12] D. Kim, Y.J. Jung, Y. Han, et al., fMRI analysis of excessive binocular disparity on the human brain, Int. J. Imaging Syst. Technol. 24 (1) (2014) 94–102. [13] C.X. Chen, J. Wang, K. Li, et al., Visual fatigue caused by watching 3DTV: an fMRI study, Biomed. Eng. Online 14 (Suppl 1) (2015) S12. [14] J.A. Meltzer, M. Negishi, R.T. Constable, Biphasic hemodynamic responses influence deactivation and may mask activation in block-design fMRI paradigms, Human Brain Mapp. 29 (4) (2008) 385–399. [15] J.B.C. Marsman, R. Renken, B.M. Velichkovsky, et al., Fixation based eventrelated fmri analysis: using eye fixations as events in functional magnetic resonance imaging to reveal cortical processing during the free exploration of visual images, Human Brain Mapp. 33 (2) (2012) 307–318. [16] C.X. Chen, K. Li, Q.Y. Wu, et al., EEG-based detection and evaluation of fatigue caused by watching 3DTV, Displays 34 (2) (2013) 81–88. [17] B.T. Backus, D.J. Fleet, A.J. Parker, et al., Human cortical activity correlates with stereoscopic depth perception, J. Neurophysiol. 86 (4) (2001) 2054–2068. [18] A.J. Parker, Binocular depth perception and the cerebral cortex, Nat. Rev. Neurosci. 8 (5) (2007) 379–391. [19] G.A. Orban, Higher order visual processing in macaque extrastriate cortex., Physiol. Rev. 88 (1) (2008) 59–89. [20] A. Talati, J. Hirsch, Functional specialization within the medial frontal gyrus for perceptual Go/No-Go decisions based on ‘‘What”, ‘‘When”, and ‘‘Where” related information: an fMRI study, Cogn. Neurosci. J. 17 (2005) 981–993. [21] N. Yeung, J. Ralph, S. Nieuwenhuis, Drink alcohol and dim the lights: the impact of cognitive deficits on medial frontal cortex function, Cogn. Affect. Behav. Neurosci. 7 (4) (2007) 347–355. [22] J.D. Schall, On the role of frontal eye field in guiding attention and saccades, Vision. Res. 44 (12) (2004) 1453–1467. [23] Marco Catani, Flavio Dell’Acqua, Michel Thiebaut De Schotten, A revised limbic system model for memory, emotion and behaviour, Neurosci. Biobehav. Rev. 37 (8) (2013) 1724–1737. [24] R. Goebel, D. Khorram-Sefat, L. Muckli, et al., The constructive nature of vision: direct evidence from functional magnetic resonance imaging studies of
20
[25] [26]
[27]
[28]
C. Chen et al. / Displays 50 (2017) 14–20 apparent motion and motion imagery, Eur. J. Neurosci. 10 (5) (1998) 1563– 1573. H.V. Duinen, R. Renken, N. Maurits, et al., Effects of motor fatigue on human brain activity, an fMRI study, Neuroimage 35 (4) (2007) 1438–1449. J. Lim, W.C. Wu, J. Wang, et al., Imaging brain fatigue from sustained mental workload: an ASL perfusion study of the time-on-task effect, Neuroimage 49 (4) (2010) 3426–3435. L. Kalbfleisch, C. Gillmarten, Left brain vs. right brain: findings on visual spatial capacities and the functional neurology of giftedness, Roeper Rev. 35 (4) (2013) 265–275. M. Ptito, R. Kupers, J. Faubert, et al., Cortical representation of inward and outward radial motion in man, Neuroimage 14 (6) (2001) 1409–1415.
[29] F. Bremmer, J.R. Duhamel, S.B. Hamed, et al., Stages of self-motion processing in primate posterior parietal cortex, Int. Rev. Neurobiol. 44 (2000) 173–198. [30] E. Bonda, M. Petrides, D. Ostry, et al., Specific involvement of human parietal systems and the amygdala in the perception of biological motion, J. Neurosci. 16 (11) (1996) 3737–3744. [31] R. Cabeza, L. Nyberg, Neural bases of learning and memory: functional neuroimaging evidence, Curr. Opin. Neurol. 13 (4) (2000) 415–421. [32] Y. Nishida, O. Hayashi, T. Iwami, et al., Stereopsis-processing regions in the human parieto-occipital cortex, Neuroreport 12 (10) (2001) 2259–2263. [33] S. Agosta, F. Herpich, F. Ferraro, et al., Stimulation of the left parietal lobe improves spatial and temporal attention in right parietal lobe patients: tipping the inter-hemispheric balance with TMS, J. Vision 13 (9) (2013), 287-287.