Revisiting mental rotation with stereoscopic disparity: A new spin for a classic paradigm

Brain and Cognition 136 (2019) 103600

Contents lists available at ScienceDirect

Brain and Cognition journal homepage: www.elsevier.com/locate/b&c

Revisiting mental rotation with stereoscopic disparity: A new spin for a classic paradigm

T

⁎

Ford Burlesa,b,c, , James Lua, Edward Slonea,b,e, Filomeno Corteseb,d, Giuseppe Iariaa,b,e, Andrea B. Protznera,b a

Department of Psychology, University of Calgary, Calgary, Alberta, Canada Hotchkiss Brain Institute, Calgary, Alberta, Canada c Rotman Research Institute, Toronto, Ontario, Canada d Seaman Family MR Research Centre, Calgary, Alberta, Canada e Alberta Children's Hospital Research Institute, Calgary, Alberta, Canada b

A R T I C LE I N FO

A B S T R A C T

Keywords: 3D ERP PLS MSE EEG Depth perception

To understand how the presence of stereoscopic disparity influences cognitive and neural processing, we recorded participants’ behavior and scalp electrical activity while they performed a mental rotation task. Participants wore active shutter 3D goggles, allowing us to present stimuli with or without stereoscopic disparity on a trial-by-trial basis. Participants were more accurate and faster when stimuli were presented with stereoscopic disparity. This improvement in performance was accompanied by changes in neural activity recorded from scalp electrodes at parietal and occipital regions; stereoscopic disparity produced earlier P2 peaks, larger N2 amplitudes, and earlier, smaller P300 peak amplitudes. The presence of stereoscopic disparity also produced greater neural entropy at occipital electrode sites, and lower entropy at frontal sites. These findings suggest that the nature of the benefit afforded by stereoscopic disparity occurs at both low-level perceptual processing and higher-level cognitive processing, and results in more accurate and rapid performance.

1. Introduction Through photographs and movies, we often see complex three-dimensional information presented on a two-dimensional medium. This two-dimensional medium retains some depth information, such as perspective and occlusion, while missing other important visual information that would be otherwise present in a real scene, such as binocular disparity. To overcome this limitation, stereoscopic visual media, which present slightly different images to each eye and produce a strong perception of depth, have been adopted since the middle of the 19th century (Brewster, 1856), with a recent resurgence in the entertainment industry in the form of 3D films and head-mounted virtual reality displays. Beyond generating revenue from moviegoers and early adopters, stereoscopic 3D technologies appear to improve behavioral performance on a number of spatial tasks, from teleoperative surgery and bomb disposal, to laboratory-based tasks requiring participants to identify or judge object positions (McIntire, Havig, & Geiselman, 2014). However, the nature of the influence of stereoscopy and 3D technologies on cognitive and neurological processes is poorly understood. One classic experimental task that is well suited to investigating the

⁎

effects of stereoscopy on perceptual and cognitive functions is the Shepard and Metzler (1971) style mental rotation task. In this task, participants view depictions of pairs of simple three-dimensional objects with varying differences in orientation, and must indicate whether each pair consists of two of the same object or if one object is a mirror image of the other. According to traditional models of mental rotation, this task requires participants to generate a mental representation of the objects, mentally rotate one object into congruence with the other, and generate a response (Fig. 1A; Heil, 2002; Ruthruff & Miller, 1995; Stoffels, 1996). Like many traditional models of cognition, there is an emphasis on hierarchical distinctions between perceptual and cognitive systems. The perceptual-level object encoding stage is focused on processing sensory information, and the results of this process would be passed to higher-level cognitive systems, like those concerned with the mental rotation process itself, which would not be expected to be directly influenced by stimulus properties (Corballis, 1988; Gill, O’Boyle, & Hathaway, 1998; Heil, 2002; Leek, Atherton, & Thierry, 2007). From this simple model of mental rotation we can derive some predictions of the effect of stereoscopic disparity on mental rotation performance. The inclusion of stereoscopic disparity clearly affects the initial object

Corresponding author at: Department of Psychology, University of Calgary, Calgary, Alberta, Canada. E-mail address: [email protected] (F. Burles).

https://doi.org/10.1016/j.bandc.2019.103600 Received 29 May 2019; Received in revised form 19 August 2019; Accepted 27 August 2019 0278-2626/ © 2019 Elsevier Inc. All rights reserved.

Brain and Cognition 136 (2019) 103600

F. Burles, et al.

Fig. 1. Panel A depicts a simple but commonly adopted model of the processing steps performed during mental rotation. Panel B depicts an ambiguous Necker Cube alongside the two possible ways to resolve the structure, and panel C depicts a reversible cube tessellation, both examples of ambiguous geometry that are resolved with the presence of depth information. Panel D and E depict the similar ambiguous geometry that is present in typical mental rotation stimuli. The geometry sample in Panel E can be resolved in two ways, as shown in Panel F.

spatial cognition tasks with stereoscopic presentation, and attributed these results to a greater cognitive load associated with the use of stereoscopic technology. Overall, these studies suggest there may be a benefit of stereoscopy in mental rotation tasks, but the nature and neural correlates of this benefit is far from clear (McIntire et al., 2014). It is apparent that these results are incongruent with the traditional hierarchical cognitive model of mental rotation. Particularly, the rotation-axis dependence of the ERP effect of stereoscopy seen in Kozhevnikov and Dhond (2012) work and Price and Lee (2010) interpretation of increased cognitive load associated with stereoscopic presentation indicates that the effects of stereoscopy extend beyond strictly perceptual processes. With this in mind, we set out to characterize the neurological and behavioral effects of adding stereoscopic disparity to mental rotation stimuli. We analyzed behavioral and EEG data from a group of healthy, right-handed university students performing a Shepard and Metzlerstyle mental rotation task in which stimuli were presented with and without stereoscopic disparity and with varying degrees of angular disparity. We first analyzed participants’ mental rotation task behavior and ERPs to ensure the effects of stereoscopic disparity extend beyond purely perceptual processes. If stereoscopic presentation influences higher-level cognitive processes, i.e., the mental rotation process itself, we expected to find the accuracy or reaction benefits of stereoscopy to vary along with the degree of angular disparity present in the stimulus pair. In the context of evoked neurological electrical activity, we first examined changes in the amplitudes and latencies of peaks in ERPs generated by our mental rotation task at parietal and occipital electrode sites. The onset and amplitude modulation of the P300 is thought to be a chronophysiological marker for the magnitude and onset of the cognitive mental rotation process proper (Heil, 2002). This portion of the ERP is delayed if the mental rotation process is delayed (Heil, 2002; Ruthruff & Miller, 1995), and its amplitude is influenced by the angular disparity between stimulus pairs (Milivojevic, Hamm, & Corballis, 2011; Peronnet & Farah, 1989). If stereoscopic disparity affects only low-level perceptual processes, changes would be largely restricted to ERP components preceding the P300, i.e., the P2 or N2 components. These components have been implicated in depth perception (P2;

encoding step, by providing additional depth information, which would aid in resolving ambiguous geometry found in typical mental rotation stimuli (see Fig. 1D–F). This additional information should lead to shorter reaction times, as the processing required to resolve ambiguous local geometry is reduced. After resolving ambiguous geometry, the quality and properties of the mental representation are presumably equivalent between stimuli presented with or without stereoscopic disparity, and therefore no effects of stereoscopic disparity on the mental rotation process itself would be expected. Despite the straightforward predictions outlined above, previous literature investigating the effects of stereoscopic disparity on mental rotation performance is surprisingly inconsistent. Some studies have found generally positive effects of stereoscopic disparity on participants’ mental rotation performance. For instance, Neubauer and colleagues (Neubauer, Bergner, & Schatz, 2010) examined electroencephalographical (EEG) and behavioral data from participants performing mental rotation trials with and without stereoscopic disparity. Participants were slightly faster and more accurate when stimuli were presented with stereoscopic disparity, but Neubauer and colleagues did not detect any changes in event-related alpha desynchronization associated with stereoscopic presentation of their mental rotation stimuli, and did not examine any effects associated with rotation magnitude. In an earlier behavioral experiment, Nagamine and colleagues (Nagamine, Nakayama, & Shimizu, 1992) also had participants perform a mental rotation task with and without stereoscopic disparity. Participants’ reaction times were significantly shorter when stimuli were displayed with stereoscopic disparity (with no investigation of response accuracy), and this effect was constant across varying degrees of angular disparity present in the stimulus pair. In contrast to the generally positive effects noted above, Kozhevnikov and Dhond (2012) compared mental rotation performance with and without stereoscopic disparity, reported marginally lower accuracy in 3D conditions as opposed to 2D conditions, and detected changes to the event-related potential (ERP) measured at parietal scalp electrodes. Their headmounted stereoscopic virtual reality condition produced more negative ERPs at ~270–300 ms post-stimulus under certain rotational axes. Similarly, Price and Lee (2010) found generally slower performance on 2

Brain and Cognition 136 (2019) 103600

F. Burles, et al.

Fig. 2 for sample stimuli). Each object consisted of segments of 2–5 blocks connected perpendicularly and in a non-planar format. Stimulus pairs were generated by selecting 9 ‘starting’ orientations where objects were not grossly self-occluding or ambiguous when presented without stereoscopic disparity. From this starting point, one of the two trial objects was rotated about one of eighteen axes (corresponding to both poles of the three canonical axes, as well as adjacent pairwise couplings thereof) by one of four rotation magnitudes (0°, 60°, 120°, 180°). Matching pairs were either not mirror-flipped, or both flipped about the same canonical plane, whereas for non-matching pairs, only one object was mirror flipped about a canonical plane. Participants were seated approximately 75 cm from the display, and each object occupied M = 24.47, SD = 3.37 (°)2 of screen space centered M = 5.55, SD = 0.81° to the left/right of the fixation cross (estimated from 1466 unique stimuli). We instructed participants to mentally rotate one object into congruence with the other, if necessary, and determine if the objects were the same, or if one was a mirror image of the other. Participants responded with their right hand on a computer mouse, where a left-button press indicated that the objects were the same, and a rightbutton press indicated that they were mirror images of each other. Stimulus pairs remained on screen until participants provided a response. Trials were separated with a randomly jittered inter-stimuli interval between 750 and 1250 ms. Stimuli were generated and displayed using Presentation® software (Version 16.3, neurobs.com), on a 24″ 120 Hz LCD monitor at a resolution of 1920 by 1080 px, and a brightness of approximately 244 Lux. To produce stereoscopic disparity, we used nVidia 3D Vision® (nvidia.com) active shutter glasses synchronized with an infrared tether. Stereoscopic stimuli were centered at monitor-level depth, i.e., the volumetric center of objects had no on-screen disparity, and the close/far portions of the objects had onscreen disparities of up to 3 mm (i.e., ~14 min of arc), intended to represent naturalistic disparity magnitudes roughly equivalent to those expected from real objects of equivalent size, and positioned in the same perceived three dimensional space that the non-stereoscopic stimuli are positioned. The monitor remained in stereoscopic mode for the duration of the experiment, and non-stereoscopic stimuli were presented by applying a previously-generated screenshot of the stimuli as a texture onto a plane at monitor-level depth. Importantly, these features allow for seamless transitions between the presence and absence of stereoscopic disparity by enabling the participant to continuously wear the shutter glasses to view both stereoscopic and non-stereoscopic trials, preventing gross expectation and familiarity effects associated with the use of stereoscopic technologies from contaminating the effect of stereoscopy itself.

Omoto et al., 2010), early object feature processing (P2; Eddy, Schmid, & Holcomb, 2006), and stimulus-driven attention and novelty detection (N2; Folstein & Van Petten, 2008). If these earlier perceptual processes are completed more rapidly, we expected a reduction in the latency of the P300 peak. However, if higher-level cognitive processes are affected, i.e., the mental rotation process itself, we expected to find additional stereoscopic disparity-related changes in the P300 and later portions of the ERP. To foreshadow, we confirmed that the presence of stereoscopic disparity affects the cognitive processes in mental rotation. We therefore set out to characterize how cognitive processes are influenced in this context. Specifically, we wanted to know if stereoscopy enriched or disambiguated the percept, and examined this question by quantifying the regularity of the brain signal using a measure of entropy called multiscale entropy (MSE; Costa, Goldberger, & Peng, 2005). Previous research suggests that entropy increases in brain signals with the amount of information available for a particular stimulus (Heisz, Vakorin, Ross, Levine, & McIntosh, 2014; Heisz, Shedden, & McIntosh, 2012; Mišić, Mills, Taylor, & McIntosh, 2010). Thus, cognitive operations with a greater degree of flexibility produce more stochastic brain responses. Alternately, cognitive operations with fewer degrees of freedom produce more regular brain responses over time. In our experiment, if stereoscopic disparity enriches the observer’s percept, the enriched information would afford a larger variety of neural computations (e.g., by recruiting brain regions processing binocular disparity cells; Freeman, 1999; Goncalves & Welchman, 2017; Parker, 2007) or behavioral strategies (e.g., manual rotation simulation; Price & Lee, 2010). Alternatively, if stereoscopic information reduces perceptual ambiguity (Banks, Read, Allison, & Watt, 2012; Caziot & Backus, 2015), akin to the reduction of ambiguity illustrated by the disambiguation of the reversible cubes illusion (Fig. 1B, C, E), neural operations are simplified because fewer perceptual ‘options’ need to be entertained (Mast & Kosslyn, 2002; Peterson, Kihlstrom, Rose, & Glisky, 1992). To explore these two possibilities, we measured the complexity of the evoked brain activity using multiscale entropy. If the additional depth information disambiguates, and therefore simplifies, the cognitive processes required in mental rotation, we expected the ERP to have lower MSE (Mišić et al., 2010) and behavioral performance to be facilitated, particularly at greater rotational disparities. On the other hand, if stereoscopic disparity creates a richer percept and increases the complexity of the cognitive processes afforded by the stimulus, we expected to detect higher MSE in the ERP (Heisz et al., 2012), and behavioral performance to be slowed, particularly at large rotational disparities. 2. Methods

2.3. Procedure 2.1. Participants Participants were tested across two sessions, one day apart. The first session lasted 30 min, with participants providing informed consent, completing the short-form Edinburgh handedness inventory (Veale, 2014), performing a single random-dot stereogram test of stereopsis which required participants to detect a Δ1 mm on-screen disparity (i.e., a 13 mm vs. 14 mm disparity), and completing 50 random practice trials of the mental rotation task which included trials with and without stereoscopic disparity. The following day, participants performed the mental rotation task for 75 min or 720 trials, whichever occurred first. During this session, continuous EEG data were collected using a BrainVision actiCHamp high impedance system (Brain Products GmbH; Gilching, Germany) from an array of 64 electrodes at standard 10–20 coordinates, at a rate of 500 Hz with the midline, vertex electrode at position Cz acting as a reference. Both sessions were conducted in a double-walled, sound-attenuating, and electromagnetically-shielded chamber. We ensured electrode impedances were below 20 kΩ before beginning EEG recording. No participants reported any events of simulator sickness while performing the study.

We recruited 33 participants for this study; however, five were excluded from all analyses because they were not able to perform the task adequately (i.e., less than 70% accuracy). The remaining 28 participants (14 male, age range: 18–27 yrs., M = 21.5 yrs., SD = 2.66 yrs.) were right-handed (laterality index range: 0.25–1.00, M = 0.83, SD = 0.23) as identified through a 4-item version of the Edinburgh handedness inventory (Veale, 2014). Behavioral analyses were performed on data from these 28 participants. Six additional participants (2 males) were excluded from EEG analyses due to poor EEG data quality, leaving 22 participants for all analyses involving EEG. All participants provided informed consent as approved by the University of Calgary Conjoint Faculties Research Ethics Board. 2.2. Stimuli Designed to mimic Shepard & Metzler’s original stimuli (Shepard & Metzler, 1971), we created ten unique objects with ten cubes each (see 3

Brain and Cognition 136 (2019) 103600

F. Burles, et al.

Fig. 2. Sample stimuli from the mental rotation task.

to 220, 190 to 290, and 250 to 500 ms post-stimulus, respectively. As these parameters are not always able to identify local peaks at every tested electrode in every participant, ten unique datasets were generated via multiple imputation in SPSS to generate values for any missing data. Participants with more than 20% missing data were excluded from the imputation procedure (fully conditional specification, 200 maximum iterations, 10 imputations using linear regression). Data appeared to be missing at random (Little’s MCAR test all ps > 0.99). We performed a series of 17 (channel) by 2 (stereoscopic disparity) by 4 (angular disparity) repeated-measures ANOVAs, with GreenhouseGeisser correction when necessary, to assess the influence of these conditions on P2, N2, and P300 amplitude and latency. Follow-up tests of significant effects were performed with the Bonferroni correction, except for effects of channel, as this effect was of no interest in the current statistical design and experimental context. Mean df, F, and p values from the 10 analysis iterations are reported for imputed datasets. We additionally performed whole-brain, partial least squares (PLS) analysis on our ERP data to examine whether or not our effects were constrained to those expected based on previous literature, or more widespread. For analysis details, see Partial Least Squares Analysis section below.

2.4. Behavioral data analysis The effect of stereoscopic disparity and angular disparity of each stereo pair on accuracy and reaction time means and standard deviations in the mental rotation task was assessed using a 2 (presence or absence of stereoscopic disparity) by 4 (0°, 60°, 120° and 180° angular disparity) repeated-measures ANOVA, with a Greenhouse-Geisser correction when sphericity violations were present. The Bonferroni correction was applied to post-hoc tests to correct for family-wise error. Analyses of reaction time and standard deviation of reaction time were only performed on correct, matching trials. 2.5. EEG acquisition and preprocessing Offline, continuous EEG data for each participant were preprocessed in EEGLAB v12.0.2.3b (Delorme & Makeig, 2004) by first finite impulse response bandpass-filtering between 0.01 and 55 Hz. After visual inspection, channels with excessively noisy data were interpolated prior to the data being re-referenced to the average from all data electrodes. Data were then segmented about stimuli presentation, with the 200 ms preceding used for baseline correction, and the 600 ms following for analysis. Eye motion and other artifacts were removed from the segmented data by performing an independent component analysis separately for each participant’s data, excluding interpolated channels. After removing components with properties not characteristic of neural activity, trials were visually inspected and any remaining trials with excessive noise were removed. Finally, any previously interpolated channels were re-interpolated.

2.7. Brain signal variability Full details of multiscale entropy (MSE) and its relevance for the analyses of signal complexity are provided in a study by Costa et al. (2005). The MSE method calculates entropy as a measure of regularity or predictability of the EEG signal at different timescales, where greater MSE values represent greater entropy. The calculation of MSE involves two steps. First, we resampled the data into six discrete timescales. For each scale, we averaged data points within non-overlapping windows. For example, scale 1 was the raw time series (i.e., 2 ms windows in the context of our 500 Hz sampling rate), scale 2 averaged over two time points (i.e., 4 ms windows), and so on. Second, we calculated sample entropy for each epoch, measuring predictability by evaluating the appearance of repetitive patterns. We calculated MSE for each epoch using the algorithm available at www.physionet.org/physiotools/mse/, with parameter values m (pattern length) = 2 (Small & Tse, 2004) and r (tolerance) = 0.5 (Richman & Moorman, 2000). The length of the time series was 300 data points (corresponding to 600 ms post-stimulus-

2.6. ERP analysis The data were then imported to ERPLAB v3.0.2.1 (Lopez-Calderon & Luck, 2014), and averaged event-related potentials (ERPs) at all parietal and occipital electrodes (17 electrodes in total) were created for each participant from matching trials in which the participant responded correctly. Separate averages were calculated at each electrode for each display condition (with and without binocular disparity) within each rotation magnitude (0°, 60°, 120°, 180°). From these averages, the amplitude and latency of the P2, N2, and P300 components were calculated by selecting a local peak over ± 20 ms from 120 4

Brain and Cognition 136 (2019) 103600

F. Burles, et al.

reaction time benefit associated with stereoscopic depth at 120° disparity (M = 1599 ms) was significantly greater than the benefit at 0° disparity (M = 502 ms; t27 = 2.132, p = .014), whereas the benefits at 60° (M = 930 ms) and 180° (M = 437 ms) were not (t27 = 2.132, p = .042; t27 = 0.182, p = .857, respectively at α < 0.0167). As detailed in Table 1, performance gains in accuracy and reaction time were smaller in individuals who were more accurate, faster, and more consistent, respectively.

onset epochs). To ensure reliable MSE estimation, we included only those timescales for which we had at least 50 samples. Thus, for each participant, electrode- and condition-specific MSE estimates were obtained as a mean across within-epoch entropy measures for scales 1–6 (or 2–12 ms windows). Single trial estimates were averaged across trials to obtain mean MSE for each condition. 2.8. Partial least squares analysis Analysis of MSE and ERP measures was carried out using a hypothesis-driven (i.e., non-rotated) version of partial least squares analysis (PLS; Krishnan, Williams, McIntosh, & Abdi, 2011; Lobaugh, West, & McIntosh, 2001; McIntosh, Kovacevic, & Itier, 2008; McIntosh & Lobaugh, 2004). This is a multivariate technique that allowed us to examine the spatiotemporal distribution of MSE and ERP measures associated with the main effects of rotation magnitude and stereoscopy, and the rotation magnitude by stereoscopy interaction. We first created matrices of mean-centered MSE/ERP averages at each electrode for each display condition (with and without binocular disparity, averaged over rotation), for each rotation magnitude (0°, 60°, 120°, 180°, averaged over display condition), and for all conditions for the interaction. We then conducted inferential analyses on these mean-centered matrices to identify latent variables (LVs). Each LV contained 3 vectors: design saliences, electrode saliences, and a scalar singular value. Design saliences defined contrasts between conditions. Electrode saliences identified the particular pattern of electrodes and time-scales/ERPs that were most related to the condition/group effect expressed in the LV. The scalar singular value indicated the strength of the effect expressed by the LV. We performed statistical assessment in PLS across two levels. First, the overall significance of each LV was assessed with permutation testing (Good, 2000). An LV was considered significant if the observed singular value exceeded the permuted singular value in more than 95% of the permutations (p < 0.05). Second, bootstrap resampling was used to estimate confidence intervals around electrode timescale or amplitude weights in each LV, allowing for an assessment of the relative contribution of particular electrode time-scales or amplitudes (Efron & Tibshirani, 1986). No corrections for multiple comparisons were necessary because the electrode timescale or amplitude weights were calculated in a single mathematical step on the whole brain. For the both MSE and ERP data, we plotted bootstrap ratios (ratio of the individual weights over the estimated standard error) as a proxy for zscores, with a minimum threshold of ± 3.1 corresponding approximately to a 99.9% confidence interval, or p < 0.001.

3.2. EEG analyses To compare neurological activity elicited in correct, matching trials, we examined common components (P2, N2, and P300) of the eventrelated potential at all parietal and occipital electrodes using a univariate analysis of peak amplitudes and latencies (Fig. 4), as well as a PLS analysis of the ERP (Figs. 4 and 5) and MSE (Fig. 6) across all recorded electrodes. 3.3. Univariate ERP results The P2 is a positive-trending potential approximately 150–250 ms after stimulus presentation. The presence of stereoscopic disparity resulted in an earlier P2 peak (mean difference of 7.629 ms, F1,18 = 7.781, p = .012), without significant changes in amplitude (F1,18 = 0.007, p = .948). Changes in angular disparity had no effect on the P2 latency (F2.00,36.14 = 2.345, p = .143) or amplitude (F2.71,48.72 = 1.197, p = .320), and no interaction between stereoscopic disparity and angular disparity was detected (amplitude: F2.29,52.04 = 1.271, p = .295; latency: F2.22,39.97 = 2.100, p = .165). P2 peak amplitude was significantly different across channels (F3.48,62.56 = 16.924, p < .001), however, P2 peak latency was not (F3.27,58.96 = 1.666, p = .213). No interactive effect of channel on P2 latency (ps ≥ 0.303) or amplitude (ps ≥ 0.416) was detected. The presence of stereoscopic disparity did not influence the latency of the subsequent negative peak, i.e., the N2 (F1,20 = 0.261, p = .615), nor did angular disparity (F2.70,53.93 = 1.516, p = .224), nor the interaction between the two (F2.28,45.56 = 1.962, p = .147). However, stereoscopic disparity produced a significantly more negative N2 peak amplitude (mean difference of 0.893 µV, F1,20 = 20.901, p < .001), while angular disparity had no effect (F2.79,55.81 = 0.043, p = .984), and no interaction between the two was detected (F2.66,53.14 = 1.093, p = .361). N2 peak latency (F3.690,73.79 = 4.892, p = .002) and amplitude were significantly different across channels (F3.51,70.13 = 3.486, p = .016). No interactive effects of channel on N2 peak latency or amplitude were detected (ps ≥ 0.187 and 0.145, respectively). The P300 component was greatly influenced by experimental factors, with reduced latency (mean difference of 13.47 ms, F1,21 = 11.385 , p = .003), and decreased amplitude (mean difference of 0.819 µV, F1,21 = 20.377, p < .001) due to the presence of stereoscopic disparity. Concordant with previous research (Schendan & Lucia, 2009), angular disparity produced differences in P300 amplitude (F2.21,46.32 = 7.637, p = .001), but not latency (F2.64,55.39 = 1.010, p = .388). Post-hoc trend analysis revealed a significant linear relationship between angular disparity and P300 amplitude (i.e., the larger the disparity, the greater the amplitude, F1,21 = 31.247, p < .001), but no significant quadratic or cubic trend (ps > 0.145). P300 peak amplitude (F3.68,77.31 = 6.875, p < .001) and latency (F2.92,61.23 = 2.826, p = .047) varied significantly across channels, and the effect of stereoscopic disparity on P300 amplitude was not consistent across all parietal and occipital electrodes (F4.58,96.25 = 3.980, p = .003). No additional interactions were detected in P300 peak latency or amplitude (ps ≥ 0.607 and 0.429, respectively).

3. Results 3.1. Behavioral performance Participants’ behavioral performance, depicted in Fig. 3, revealed that on average, individuals were significantly more accurate (mean difference of 2.25%, t27 = 5.239, p < .001) when responding to stimuli with stereoscopic disparity. For all following analyses, only correct, matching trials were considered, as we could not assume that the magnitude of mental rotation was consistent across participants for incorrect or non-matching trials. For correct, matching trials, participants responded more rapidly to stimuli presented with stereoscopic disparity than to stimuli presented without (mean difference of 867 ms, F1,27 = 23.092, p < .001). As expected, differences in angular disparity within a stimulus pair were associated with changes in reaction time (F1.38,37.33 = 149.267, p < .001). Post-hoc linear trend analyses revealed a significant linear effect of angular disparity on mean reaction time (i.e., the greater the angular disparity, the slower the reaction time; F1,27 = 170.603, p < .001). The effect of stereoscopic depth on reaction time was not consistent across rotation magnitudes (F1.82,49.10 = 3.934, p = .029) with post-hoc testing indicating that the

3.4. PLS ERP results Our PLS analysis examined the main effects of stereoscopic disparity 5

Brain and Cognition 136 (2019) 103600

F. Burles, et al.

Fig. 3. Behavioral effects of stereoscopic disparity on mental rotation performance. (A) Effect of stereoscopic disparity and angular disparity of the stimulus pair on accuracy. (B) Mean reaction time from correct, matching trials only. Error bars represent ± 1 standard error; n = 28. Table 1 *Correct, matching trials only, n = 28. M = mean; SD = standard deviation; Min = minimum value; Max = maximum value; r = Pearson correlation coefficient; p = level of significance. Grand Average

Accuracy (%) Reaction Time (s)*

Benefit of stereo

Performance-to-benefit relationship

M

SD

Min

Max

M

SD

Min

Max

r

p

85.58 3.39

7.14 1.30

70.42 1.75

97.22 6.16

2.25 0.87

2.27 0.95

−3.06 −0.81

6.25 3.64

−0.461 −0.646

0.018 < 0.001

0.49 at the 0°, 60°, 120°, and 180° conditions, respectively). This rotational LV was characterized by increases in MSE in nonzero rotational conditions across frontal, temporal and central electrode sites. There was no significant interaction between stereoscopic disparity and rotation vs. non-rotation conditions (p = .317).

and rotation magnitude, and the interaction between stereoscopic disparity and rotation magnitude. To examine the main effect of stereoscopic disparity, we averaged across rotation magnitudes, and identified a significant LV differentiating between our display conditions (p < .001) commonly expressed as attenuated amplitudes at ~200 ms post-stimulus at posterior and midline frontal electrode sites, and at ~400 ms post-stimulus at posterior and frontal electrode sites (Figs. 4 and 5). To examine the main effect of rotation magnitude, we averaged across the stereoscopic disparity conditions, and again identified a single significant LV (p < .001, 79.39% of crossblock covariance, other LVs ps > 0.094). Detailed in Fig. 5, the significant rotation magnitude LV weights paralleled the form of the behavioral reaction time results, with the 0° condition clearly contrasted from the 120° and 180° conditions, and the 60° condition with an intermediate weight (i.e., weights of −0.84, 0.09, 0.39, 0.37 at the 0°, 60°, 120°, and 180° conditions, respectively). These effects were widespread at frontal, central and parietal electrodes, and were most identifiable beginning at ~300 ms post-stimulus. The stereoscopic disparity by rotation magnitude interaction was not significant (p = .503). Generally, the presence of stereoscopic disparity produced amplitude differences opposing those seen with increases in rotational magnitude, depicted most clearly in Fig. 4A and B.

4. Discussion We used behavioral and ERP measures to characterize the effects of stereoscopic versus traditional presentation of stimuli in the context of a classic, Shepard and Metzler -style mental rotation task. We were particularly interested in understanding the changes in perceptual and cognitive demands produced by stereoscopic disparity, due to the fact that the previous literature has reported both performance improvements and impairments associated with this technology. First, we used our behavioral analyses to confirm the scope of the effects of stereoscopic disparity are not limited to strictly perceptual processes alone. Participants generally responded faster and more accurately to stimuli with stereoscopic disparity. However, the reaction time benefit of stereoscopic disparity was not consistent; participants appeared to benefit the most from stereoscopic disparity in the 120° rotation condition, with notably lesser benefits in the 0° and 180° conditions. The interaction between stereoscopic disparity and rotation magnitude supports our assumption that these effects are not limited to perceptual processes and have an influence on cognitive processes as well. Similarly, our univariate ERP analysis at parietal and occipital electrodes revealed effects that were not restricted to perceptual components alone. Stereoscopic disparity produced an earlier P2 peak, and a more negative N2 peak. The P2 component has been implicated in depth perception and object feature processing (Eddy et al., 2006; Omoto et al., 2010). The posterior N2 component often is linked with attending to relevant stimulus features (Folstein & Van Petten, 2008; Kiss, Van Velzen, & Eimer, 2008). These ERP effects may be representative of

3.5. PLS MSE results Our PLS analysis of the MSE of evoked electrical activity (Fig. 6) identified a clear anterior-posterior difference in the effect of stereoscopic disparity, with the 3D condition exhibiting generally lower MSE at frontal electrode sites, alongside less expansive increases in MSE at occipital electrode sites (LV p = .044). Similar to the ERP PLS analyses, the main effect of rotation magnitude was also significant (p < .001, 81.69% crossblock covariance). This LV differentiated between the 0° and nonzero rotation conditions (i.e., weights of −0.83, 0.27, 0.07, 6

Brain and Cognition 136 (2019) 103600

F. Burles, et al.

Fig. 4. Event-related potentials (ERPs) of mental rotation stimuli evoked at parietal and occipital electrodes by correct, matching trials. The average ERP from all parietal and occipital electrodes (P&O) with respect to stereoscopic disparity is represented in Panel A, and the same average with respect to rotation magnitude is illustrated in panel B. Panel C depicts the ERPs evoked with and without stereoscopic disparity at 0°, 60°, 120°, and 180° rotation magnitudes at electrodes P3 and Oz. ERP PLS highlighting is displayed at a bootstrap ratio of 4 and 3.5 for the stereoscopic disparity and rotation magnitude analyses, respectively; n = 22.

limited to early and unequivocally perception-driven portions of the ERP. Of course, temporally late effects of stereoscopy do not preclude the possibility that these effects are related to perceptual processes, but if these later effects are largely perceptually-driven, it brings the validity of the model depicted in Fig. 1A into question. This commonlyadopted model of mental rotation views the mental rotation process itself as explicitly occurring serially after stimulus evaluation. Thus, once mental rotation begins (at approximately 300 ms post-stimulus presentation, evidenced from the first large peak in Fig. 5C and from previous research; Heil, 2002), it should be the primary cognitive process and presumably a dominant source of EEG activity, yet we continue to see stereoscopy-related effects after the mental rotation process begins. With the aforementioned results indicating that the effects of stereoscopic disparity are not restricted to low-level perceptual processes, we had previously identified two means by which stereoscopic disparity

more robust or temporally-defined visuo-spatial processing of the stimulus pair, and an increased likelihood that participants were rapidly processing relevant properties of the stimuli. We also found differences in the later P300 component, which is often evoked after a stimulus has been evaluated, and is likely indicative of more cognitive, as opposed to perceptual, processes (Pritchard, 1981). Consistently, this component was influenced by angular disparity (larger angular disparity producing more positive amplitudes) but was also influenced by the presence of stereoscopic disparity (with reduced latency and amplitude). Our PLS ERP analysis expanded upon our univariate ERP results, detecting a cluster of earlier effects, beginning at approximately 200 ms post-stimulus presentation and primarily located at occipital and midfrontal electrodes. However, the majority of the stereoscopy-related effects detected by this analysis were in a much later time window, beginning approximately 400 ms post-stimulus presentation, reinforcing our conclusion that the effects of stereoscopic disparity are not 7

Brain and Cognition 136 (2019) 103600

F. Burles, et al.

Fig. 5. Panel A depicts the ERPs evoked at a subset of the analyzed electrodes with significant PLS effects marked above each waveform. For the PLS analysis, the stereoscopic disparity latent variable (blue dots) was plotted at a bootstrap ratio of 4, and the rotation magnitude latent variable (yellow dots) was plotted at a bootstrap ratio of 3.5. Panel B illustrates the scalp scores and corresponding 95% confidence intervals for both latent variables. Panel C displays the temporal distribution of significant PLS ERP effects, normalized such that the area under each curve is equivalent. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

also produces a decrease in the demand upon processes supported by frontal brain regions. Despite the apparent increase in perceptual load, the earlier P300 peak and generally faster response times in the stereoscopic condition suggests that processing is nonetheless completed more quickly. It seems likely that this is the result of additional neural resources, i.e., cells sensitive to binocular disparity (Parker, 2007). The additional stereoscopic depth information can then be processed by these specialized cells and likely removes a significant proportion of the geometrical ambiguity present in sub-figure structure (e.g. Figure D, E, F) that in the non-stereo condition would need to be resolved at the level of the entire figure. As such, the additional low-level load is very rapidly offset by the additional contribution of specialized neuronal architecture, and the early disambiguation provided by these cells seems to have significant positive downstream effects. Overall, the effects of stereoscopy we detected were clearly beneficial, and strongly suggest that the minor increase in perceptual load detected in our MSE analysis is largely offset by what appears to be a more substantial decrease in cognitive load, resulting in clear behavioral performance improvements. This stands in stark contrast to a minority of the extant literature that finds neutral or negative effects of stereoscopic disparity in spatial tasks (e.g., Kozhevnikov & Dhond, 2012; Price & Lee, 2010; for a review, see McIntire et al., 2014). The inconsistent results may be due to the varying manipulations used by

could be influencing the perceptual and cognitive processes in mental rotation. First, it is possible that the additional depth information produces an enriched and more complex stimulus, increasing the load on the perceptual and cognitive processes required to perform mental rotation. Alternatively, we could construe this additional depth information as disambiguating the stimuli, thus reducing the complexity and load on perceptual and cognitive processes associated with mental rotation. While our behavioral and univariate ERP results are not particularly well-suited to differentiating between these two possibilities, the clear behavioral performance improvements associated with stereoscopic disparity appear to be inconsistent with the interpretations of increased cognitive load made by Price and Lee (2010). Similarly, the lower P300 amplitude and latency evoked from stimuli with stereoscopic disparity could indicate these stimuli were less cognitively demanding or were more rapidly processed, as P300 amplitude is related to the amount of attention or effort required during a task (Kok, 2001; Polich, 2007). Our MSE analysis revealed a particularly interesting distribution of effects. Stereoscopic disparity produced limited increases in MSE at the most posterior occipital electrodes, along with a pronounced decrease in MSE at frontal electrodes. These results suggest the additional depth information provided by stereoscopic disparity is increasing the amount of information and perhaps the perceptual load at occipital cortex (as this depth information needs to be processed), but 8

Brain and Cognition 136 (2019) 103600

F. Burles, et al.

Fig. 6. Montage of the PLS MSE results thresholded at a bootstrap ratio of 3.1. Significant effects of stereoscopy are colored in blue, and significant effects of rotation magnitude are colored in yellow. Arrow direction indicates the sign of the effect, with upwards-pointing arrows indicating increases and downwards-pointing arrows indicating decreases in MSE associated with the presence of stereoscopic disparity or rotation magnitude. The error bars on the scalp scores plots represent 95% confidence intervals. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

stereoscopic presentation of mental rotation stimuli in particular would benefit greatly from careful characterization of any changes in eye movement as participants are free to visually explore the stimuli in depth. This is particularly relevant considering that some alternative cognitive models of mental rotation view this task as being performed using more iterative and piecemeal processes, rather than performing mental rotation in a single analog process (Mast & Jäncke, 2007, p. 185). This view is largely supported by observations that participants will shift their gaze multiple times between the objects in a stimulus pair and will shift more often when the objects in a stimulus pair have a greater rotational disparity between them (Just & Carpenter, 1976; Pylyshyn, 2005). Interpreting our results in the context of this model potentially allows for an alternative interpretation of some of our detected neurological effects. Presuming that stereoscopic presentation produces more rapid visual encoding, perhaps every time participants shift their gaze and focus on a new portion of the stimulus they gain a benefit of stereoscopic disparity. In trials with greater rotation magnitudes, in which participants have more time and likely shift their gaze more often (Just & Carpenter, 1976), they exhibit a generally larger benefit of stereoscopy. This trend is realized with exception of the 180° condition, which may be due to participants using a non-rotational strategy on the trials with 180° angular disparity (Shepard & Metzler, 1988), or perhaps participants were searching for and attempting to rule out mirror-symmetry first (as these trials often look similar to 0°

the authors for their stereoscopic conditions. For instance, Kozhevnikov and Dhond (2012) used anaglyph glasses (i.e., red-cyan filtered glasses) as well as a head-mounted display, and Price and Lee (2010) utilized polarized filters (and both found generally negative effects of stereoscopic presentation). Contrasting filtered and head-mounted stimuli presentation with a standard desktop presentation conflates more factors than simply stereoscopic disparity, and confounds effects of stereoscopy with those related to the use and participant expectations of different technologies, e.g., motion-related head-tracking with headmounted virtual reality and loss of contrast and crosstalk with anaglyph or polarized filters. Irrespective of the hardware used to produce stereoscopic disparity, performance detriments could be produced if the depth cues provided with the stimuli are not realistic (e.g., too little or too much stereoscopic disparity), implemented in such a way that makes performing the task awkward (e.g., objects perceived to be significantly closer to the user in 3D space than the display, controls, or fixation point, demanding significant gaze shifts in depth), or simply are not relevant to or even confounding to the task at hand (e.g., including stereo depth in a 2D letter-rotation task). Future studies investigating the benefits of stereoscopic disparity in spatial tasks generally will find these effects greatest when stereoscopic technologies are employed in an ecological manner, such that the depth information does not violate the participant’s expectations, and only when directly relevant to the task at hand. Future studies utilizing 9

Brain and Cognition 136 (2019) 103600

F. Burles, et al.

‘mismatch’ trials) and if unable to do so, assuming that the objects were identical and rotated. Similar non-linearities have been reported in previous studies (Schendan & Lucia, 2009) and in other spatial rotation tasks (Boer, 1991; Roskos-Ewoldsen, McNamara, Shelton, & Carr, 1998). These gaze-related changes could also be partially responsible for some of the later stereoscopy-related ERP effects extending beyond the occipital and parietal cortex, particularly activity near the frontal and supplementary eye fields (Blanke et al., 1999; Clementz, Brahmbhatt, McDowell, Brown, & Sweeney, 2007). However, we would have also expected greater MSE near the electrodes recording from the eye fields in the stereoscopic condition as presumably the participants’ gaze is additionally exploring in depth as well as across the picture plane, yet no such increase was detected. Beyond the possible gaze-related changes, the presence of stereoscopic disparity may have also afforded participants consciously or unconsciously adopting a different strategy. The stimuli with stereoscopic disparity appear tangible and graspable, potentially promoting the use of a simulated manual rotation strategy, as opposed to a more analytic strategy (Gardony, Eddy, Brunyé, & Taylor, 2017). Such a strategy shift is supported by the observed decrease in entropy at frontal electrode sites (that would support analytic strategies), but lacks an expected increase in entropy at parietal and motor regions that would support mental or manual rotation simulation strategies. It is possible that stimuli with stereoscopic disparity are more easily mentally rotated, which would offset the expected load increase with a greater propensity to adopt this strategy. Additionally, our experimental design featured trials with and without stereoscopy mixed throughout the task, as opposed to longer blocks for this manipulation. This less predictable design may have attenuated any explicit strategy adoption, as trials cycle between conditions rapidly enough that it could have produced a switching cost to explicitly adopt two different strategies based on the trial properties. Prospective future studies could perform eye-tracking to test if gaze-related changes can explain some of the effects that we have ascribed to cognitive processes, as well as explicitly examine if stereoscopic disparity and expertise with stereoscopic systems promotes strategy use that may be particularly suited to the presence of the additional depth information.

Research Council of Canada (NSERC) PDF to FB, an NSERC Discovery Grant, Canadian Foundation for Innovation Leaders Opportunity Fund, Alberta Enterprise and Advanced Education Research Capacity Program, and Alberta Alignment Grants to ABP, and an NSERC Discovery Grant to GI. References Backus, B. T., Fleet, D. J., Parker, A. J., & Heeger, D. J. (2001). Human cortical activity correlates with stereoscopic depth perception. Journal of Neurophysiology, 86(4), 2054–2068. Banks, M. S., Read, J. C. A., Allison, R. S., & Watt, S. J. (2012). Stereoscopy and the human visual system. SMPTE Motion Imaging Journal, 121(4), 24–43. https://doi.org/ 10.5594/j18173. Blanke, O., Morand, S., Thut, G., Michel, C. M., Spinelli, L., Landis, T., & Seeck, M. (1999). Visual activity in the human frontal eye field. Neuroreport, 10(5), 925–930. Boer, L. C. (1991). Mental rotation in perspective problems. Acta Psychologica, 76(1), 1–9. https://doi.org/10.1016/0001-6918(91)90050-A. Brewster, D. (1856). The stereoscope; its history, theory and construction, with its application to the fine and useful arts and to education, Etc. John Murray. Caziot, B., & Backus, B. T. (2015). Stereoscopic offset makes objects easier to recognize. PLoS ONE, 10(6), https://doi.org/10.1371/journal.pone.0129101. Clementz, B. A., Brahmbhatt, S. B., McDowell, J. E., Brown, R., & Sweeney, J. A. (2007). When does the brain inform the eyes whether and where to move? An EEG study in humans. Cerebral Cortex, 17(11), 2634–2643. https://doi.org/10.1093/cercor/ bhl171. Corballis, M. C. (1988). Recognition of disoriented shapes. Psychological Review, 95, 115–123. Costa, M., Goldberger, A. L., & Peng, C.-K. (2005). Multiscale entropy analysis of biological signals. Physical Review E, 71(2), https://doi.org/10.1103/PhysRevE.71. 021906. Delorme, A., & Makeig, S. (2004). EEGLAB: An open source toolbox for analysis of singletrial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134(1), 9–21. Eddy, M., Schmid, A., & Holcomb, P. J. (2006). Masked repetition priming and eventrelated brain potentials: A new approach for tracking the time-course of object perception. Psychophysiology, 43(6), 564–568. Efron, B., & Tibshirani, R. (1986). Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy. Statistical Science, 1(1), 54–75. https://doi.org/10.1214/ss/1177013815. Folstein, J. R., & Van Petten, C. (2008). Influence of cognitive control and mismatch on the N2 component of the ERP: A review. Psychophysiology, 45(1), 152–170. Freeman, R. D. (1999). Stereoscopic vision: Which parts of the brain are involved? Current Biology, 9(16), R610–R613. https://doi.org/10.1016/S0960-9822(99)80386-8. Gardony, A. L., Eddy, M. D., Brunyé, T. T., & Taylor, H. A. (2017). Cognitive strategies in the mental rotation task revealed by EEG spectral power. Brain and Cognition, 118, 1–18. https://doi.org/10.1016/j.bandc.2017.07.003. Gill, H. S., O’Boyle, M. W., & Hathaway, J. (1998). Cortical distribution of eeg activity for component processes during mental rotation. Cortex, 34(5), 707–718. https://doi. org/10.1016/S0010-9452(08)70774-3. Goncalves, N. R., & Welchman, A. E. (2017). “What not” detectors help the brain see in depth. Current Biology, 27(10), 1403–1412.e8. https://doi.org/10.1016/j.cub.2017. 03.074. Good, P. (2000). Permutation tests. New York: Springer. Heil, M. (2002). The functional significance of ERP effects during mental rotation. Psychophysiology, 39(5), 535–545. Heisz, J. J., Shedden, J. M., & McIntosh, A. R. (2012). Relating brain signal variability to knowledge representation. NeuroImage, 63(3), 1384–1392. https://doi.org/10.1016/ j.neuroimage.2012.08.018. Heisz, J. J., Vakorin, V., Ross, B., Levine, B., & McIntosh, A. R. (2014). Trade-off between local and distributed information processing associated with remote episodic versus semantic memory. Journal of Cognitive Neuroscience, 26(1), https://doi.org/10.1162/ jocn_a_00466. Just, M. A., & Carpenter, P. A. (1976). Eye fixations and cognitive processes. Cognitive Psychology, 8(4), 441–480. https://doi.org/10.1016/0010-0285(76)90015-3. Kiss, M., Van Velzen, J., & Eimer, M. (2008). The N2pc component and its links to attention shifts and spatially selective visual processing. Psychophysiology, 45(2), 240–249. https://doi.org/10.1111/j.1469-8986.2007.00611.x. Kok, A. (2001). On the utility of P3 amplitude as a measure of processing capacity. Psychophysiology, 38(3), 557–577. Kozhevnikov, M., & Dhond, R. P. (2012). Understanding immersivity: Image generation and transformation processes in 3D immersive environments. Frontiers in Psychology, 3. https://doi.org/10.3389/fpsyg.2012.00284. Krishnan, A., Williams, L. J., McIntosh, A. R., & Abdi, H. (2011). Partial Least Squares (PLS) methods for neuroimaging: A tutorial and review. NeuroImage, 56(2), 455–475. https://doi.org/10.1016/j.neuroimage.2010.07.034. Leek, E. C., Atherton, C. J., & Thierry, G. (2007). Computational mechanisms of object constancy for visual recognition revealed by event-related potentials. Vision Research, 47(5), 706–713. https://doi.org/10.1016/j.visres.2006.10.021. Lobaugh, N. J., West, R., & McIntosh, A. R. (2001). Spatiotemporal analysis of experimental differences in event-related potential data with partial least squares. Psychophysiology, 38(3), 517–530. Lopez-Calderon, J., & Luck, S. J. (2014). ERPLAB: An open-source toolbox for the analysis

5. Conclusion The findings reported in this study provide the first evidence that the presence of stereoscopic disparity in mental rotation stimuli produces concurrent improvements in accuracy and reaction time, as well as significant changes in the neural activity evoked by stimulus presentation. These effects are likely due to facilitated object perception, as participants do not need to rely solely on monoscopic depth cues, such as perspective and occlusion, and can benefit from the neural architecture that processes stereoscopic disparity (Backus, Fleet, Parker, & Heeger, 2001). However, the variable benefit afforded by stereoscopic disparity at different rotation magnitudes indicates the effects are not limited to the initial phase of object encoding alone, and appear to facilitate and simplify the cognitive processes in mental rotation as well. Stereoscopic depth information will particularly benefit individuals in encoding and manipulating spatial information in instances where objects are complex or are camouflaged, or when observer motion is restricted or unnatural; scenarios in which the amount of information afforded by parallax and optic flow is attenuated. A nuanced understanding of the effects stereoscopic disparity can have in simple and more complex tasks will allow us to maximize any potential behavioral performance gains associated with presenting (or withholding) such information in real-world tasks, as well as refine our understanding of visual perception in humans. Acknowledgments This research was supported by a Natural Sciences and Engineering 10

Brain and Cognition 136 (2019) 103600

F. Burles, et al.

Peterson, M. A., Kihlstrom, J. F., Rose, P. M., & Glisky, M. L. (1992). Mental images can be ambiguous: Reconstruals and reference-frame reversals. Memory & Cognition, 20(2), 107–123. https://doi.org/10.3758/BF03197159. Polich, J. (2007). Updating P300: An integrative theory of P3a and P3b. Clinical Neurophysiology, 118(10), 2128–2148. https://doi.org/10.1016/j.clinph.2007.04. 019. Price, A., & Lee, H.-S. (2010). The effect of two-dimensional and stereoscopic presentation on middle school students’ performance of spatial cognition tasks. Journal of Science Education and Technology, 19(1), 90–103. Pritchard, W. S. (1981). Psychophysiology of P300. Psychological Bulletin, 89(3), 506–540. https://doi.org/10.1037/0033-2909.89.3.506. Pylyshyn, Z. W. (2005). Mental imagery: In search of a theory. Behavioral and Brain Sciences, 25, 157–238. Richman, J. S., & Moorman, J. R. (2000). Physiological time-series analysis using approximate entropy and sample entropy. American Journal of Physiology-Heart and Circulatory Physiology, 278(6), H2039–H2049. https://doi.org/10.1152/ajpheart. 2000.278.6.H2039. Roskos-Ewoldsen, B., McNamara, T. P., Shelton, A. L., & Carr, W. (1998). Mental representations of large and small spatial layouts are orientation dependent. Journal of Experimental Psychology. Learning, Memory, and Cognition, 24(1), 215–226. Ruthruff, E., & Miller, J. (1995). Can mental rotation begin before perception finishes? Memory & Cognition, 23(4), 408–424. https://doi.org/10.3758/BF03197243. Schendan, H. E., & Lucia, L. C. (2009). Visual object cognition precedes but also temporally overlaps mental rotation. Brain Research, 1294, 91–105. https://doi.org/10. 1016/j.brainres.2009.07.036. Shepard, R. N., & Metzler, J. (1971). Mental rotation of three-dimensional objects. Science, 171(3972), 701–703. Shepard, S., & Metzler, D. (1988). Mental rotation: Effects of dimensionality of objects and type of task. Journal of Experimental Psychology. Human Perception and Performance, 14(1), 3–11. Small, M., & Tse, C. K. (2004). Optimal embedding parameters: A modelling paradigm. Physica D: Nonlinear Phenomena, 194(3–4), 283–296. https://doi.org/10.1016/j. physd.2004.03.006. Stoffels, E. J. (1996). Inhibition of concurrent processes in letter and orientation discriminations. Acta Psychologica, 91(2), 153–173. https://doi.org/10.1016/00016918(94)00052-2. Veale, J. F. (2014). Edinburgh Handedness Inventory – Short Form: A revised version based on confirmatory factor analysis. Laterality: Asymmetries of Body, Brain and Cognition, 19(2), 164–177. https://doi.org/10.1080/1357650X.2013.783045.

of event-related potentials. Frontiers in Human Neuroscience, 8. https://doi.org/10. 3389/fnhum.2014.00213. Mast, F. W., & Jäncke, L. (Eds.). (2007). Spatial processing in navigation, imagery, and perception. New York: Springer. Mast, F. W., & Kosslyn, S. M. (2002). Visual mental images can be ambiguous: Insights from individual differences in spatial transformation abilities. Cognition, 86(1), 57–70. https://doi.org/10.1016/S0010-0277(02)00137-3. McIntire, J. P., Havig, P. R., & Geiselman, E. E. (2014). Stereoscopic 3D displays and human performance: A comprehensive review. Displays, 35(1), 18–26. https://doi. org/10.1016/j.displa.2013.10.004. McIntosh, A. R., Kovacevic, N., & Itier, R. J. (2008). Increased brain signal variability accompanies lower behavioral variability in development. PLoS Computational Biology, 4(7), e1000106. https://doi.org/10.1371/journal.pcbi.1000106. McIntosh, A. R., & Lobaugh, N. J. (2004). Partial least squares analysis of neuroimaging data: Applications and advances. NeuroImage, 23, S250–S263. https://doi.org/10. 1016/j.neuroimage.2004.07.020. Milivojevic, B., Hamm, J. P., & Corballis, M. C. (2011). About turn: How object orientation affects categorisation and mental rotation. Neuropsychologia, 49(13), 3758–3767. https://doi.org/10.1016/j.neuropsychologia.2011.09.034. Mišić, B., Mills, T., Taylor, M. J., & McIntosh, A. R. (2010). Brain noise is task dependent and region specific. Journal of Neurophysiology, 104(5), 2667–2676. https://doi.org/ 10.1152/jn.00648.2010. Nagamine, K., Nakayama, M., & Shimizu, Y. (1992). Effect of stereo television presentation on “mental rotation”. Electronics and Communications in Japan (Part III: Fundamental Electronic Science), 75(10), 54–65. https://doi.org/10.1002/ecjc. 4430751005. Neubauer, A. C., Bergner, S., & Schatz, M. (2010). Two- vs. three-dimensional presentation of mental rotation tasks: Sex differences and effects of training on performance and brain activation. Intelligence, 38(5), 529–539. https://doi.org/10.1016/j.intell. 2010.06.001. Omoto, S., Kuroiwa, Y., Otsuka, S., Baba, Y., Wang, C., Li, M., ... Suzuki, Y. (2010). P1 and P2 components of human visual evoked potentials are modulated by depth perception of 3-dimensional images. Clinical Neurophysiology, 121(3), 386–391. https://doi. org/10.1016/j.clinph.2009.12.005. Parker, A. J. (2007). Binocular depth perception and the cerebral cortex. Nature Reviews Neuroscience, 8(5), 379–391. https://doi.org/10.1038/nrn2131. Peronnet, F., & Farah, M. J. (1989). Mental rotation: An event-related potential study with a validated mental rotation task. Brain and Cognition, 9(2), 279–288. https://doi. org/10.1016/0278-2626(89)90037-7.

11