A study of cybersickness and sensory conflict theory using a motion-coupled virtual reality system

Journal Pre-proofs A Study of Cybersickness and Sensory Conflict Theory using a Motion-Coupled Virtual Reality System Adrian K.T. Ng, Leith K.Y. Chan, Henry Y.K. Lau PII: DOI: Reference:

S0141-9382(18)30030-1 https://doi.org/10.1016/j.displa.2019.08.004 DISPLA 1922

To appear in:

Displays

Received Date: Revised Date: Accepted Date:

5 April 2018 20 May 2019 26 August 2019

Please cite this article as: A.K.T. Ng, L.K.Y. Chan, H.Y.K. Lau, A Study of Cybersickness and Sensory Conflict Theory using a Motion-Coupled Virtual Reality System, Displays (2019), doi: https://doi.org/10.1016/j.displa. 2019.08.004

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

© 2019 Published by Elsevier B.V.

2

A Study of Cybersickness and Sensory Conflict Theory using a Motion-Coupled Virtual Reality System

3

Adrian K. T. Nga,∗, Leith K. Y. Chana , Henry Y. K. Laua

1

4 5

6 7 8 9 10 11 12 13 14 15 16 17 18 19

a

Department of Industrial and Manufacturing Systems Engineering, The University of Hong Kong, Pokfulam, Hong Kong

Abstract Sensory conflict theory explains that motion sickness in virtual reality (VR) systems can be caused due to the mismatch between visual and vestibular senses. This study examines whether coupling physical motions to visual stimuli in VR could reduce this discomfort. A motion-coupled VR system developed on a motion platform, providing vestibular cues to supplement visual roll from a head-mounted display (HMD), was used. Three conditions were tested: visual rotation only (stationary), visual-physical motion synchronised (synchronous), and vestibular motion with a self-referenced visual environment. Results show that when users are placed under a visual-vestibular synchronised condition, their subjective miserable score of cybersickness was lowered while their comfort level of the overall experience was increased. This indicates that a motion-coupled system, if integrated seamlessly in VR, could mitigate cybersickness symptoms.

21

Keywords: Motion sickness, Visual-vestibular conflict, Multisensory integration, Motion simulation, Virtual reality, Head-mounted display

22

1. Introduction

20

23 24 25 26 27 28 29

Exposure to virtual reality (VR) systems could often lead to nausegenic side-effects known as cybersickness or visually induced motion sickness (VIMS). VIMS is a subset of motion sickness (MS), which includes symptoms such as nausea, stomach awareness, and vertigo. Sensory conflict theory, the most cited and widely accepted MS theory, proposed that the origin of the malady arises from the conflict of afferent signals from visual, vestibular, and somatic sensors, and the incongruence of the signals with an internal model ∗

Corresponding author Emailsubmitted addresses: (Adrian K. T. Ng), [email protected] (Leith6,K. Y. Preprint to [email protected] Displays September 2019 Chan), [email protected] (Henry Y. K. Lau)

30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53

54 55 56 57 58 59 60 61 62 63 64 65 66

in the central nervous system (CNS) [1, 2, 3, 4]. The internal model, or neural store, is assumed to be responsible for making predictions of motion and sensory pattern based on the afferent signals and previous experiences. If the discrepancies between actual and predicted sensory signals vary greatly, MS symptoms will be provoked. Using a VR device sitting or standing in a fixed location, users often get cybersickness because the afferent visual sensations are incoherent with the vestibular sensations. For example, cybertravel experiences provide users with visual moving sensation (afferent visual cues) without any imposed physical motion (no vestibular and somatic cues). Recent years, some advanced VR systems are integrated with motion platforms to provide users with more realistic experiences [5, 6, 7]. These motion-coupled VR systems often use servo motors to generate supplementary physical sensations that complement the prevailing visual experiences. Different from traditional motion simulators, these systems provide additional features, such as stereoscopy and perspective tracking, that improve immersion with visual elements. As motion-coupled VR systems supplement the missing vestibular sensation, these systems may, in theory, reduce the side-effects of cybersickness and provide a more comfortable experience. In fact, the use of physical motion to reduce cybersickness have been evaluated with the use of motion simulator or VR systems without a motion platform, but rarely has it been systematically evaluated with the use of a motion-coupled VR system. Hence, based on the framework of sensory conflict theory, this study compared the effects of multisensory synchronisation on cybersickness and users comfort using a motion-coupled VR system. 1.1. Motion cues in simulators Several driving simulator studies have shown that the use of simulated motion cues leads to reduced simulator sickness [8, 9]. For example, Aykent et al. [8] found that participants experienced less nausea and dizziness on a 6 degree of freedom (DoF) dynamic driving simulator compared to a static one. However, in some cases, similar studies found no effects [10] or increased MS [11, 12] with the use of motion platforms. Dziuda et al. [11] found that physical motion increases oculomotor and disorientation discomfort, and increases the after-effects duration. One possible explanation is that the level of MS is reduced only if the motor-simulated physical motions resemble extremely closely to the actual sensation of the real vehicle driving experience. In some driving simulators, vehicle kinematics are filtered, delayed, and reduced by motion algorithms due to physical limitations of motors [13]. The physical 2

67 68 69

70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88

89 90 91 92 93 94 95 96 97 98

motion generated by the motors mismatches from the actual movements required to represent the vehicle driving experience, hence induces additional conflicts. 1.2. Multisensory interaction In virtual environment (VE), visual-vestibular conflicts are known to induce cybersickness [14, 15]. Active or passive physical body movement that synchronises with visual senses help reduce cybersickness. For instance, Llorach et al. [16] developed a low-accuracy position estimation method using an Oculus Rift DK1 head-mounted display (HMD) with its inertial measurement unit (IMU) that allows navigation by walking. Chen et al. [17] designed a flying navigation technique that uses body (and head) movement to control the visual scene in CAVE-like (cave automatic virtual environment) systems. These techniques with proprioceptive feedback synchronised to the visual cues are found to reduced MS-symptoms. A recent study by Palmisano et al. [18] examined the effects of active physical head movement on vection and cybersickness in an HMD with the use of simple optical flow. The “compensated” condition presented the visual viewpoints ecologically synchronised with the participants’ head yaw motion captured by a tracker, while the “uncompensated” one does not. Their results showed that participants perceived stronger vection in the synchronised condition than in control condition, but there were no significant differences in the severity of cybersickness (see also [19]1 ). 1.3. Self-referenced environment Learnt from our sea travel experience, Tal et al. [20] tested if the content of visual information is essential in reducing MS by providing an Earthreferenced visual artificial horizon in a boat simulator. It was found that providing an artificial visual horizon, a simplified visual representation of the whole synchronised environment, help reduce MS compared to the selfreferenced (control) environment (see also [21, 22]). In static VE, the use of self-referenced frame, restricted field of view (FoV) mask, or a self-referenced virtual body part (a nose) are also found to reduce symptoms of cybersickness [23, 24, 25, 26]. 1

A stationary fixation point was used in the experiment, which may suppress optokinetic nystagmus (OKN) and reduce the severity of cybersickness.

3

111

1.4. Hypothesis In the current study, we create simple motion using a motion platform to create various visual-vestibular synchronisation conditions. A “stationary” condition featured only visual motion; a “synchronous” condition presented a synchronised visual-vestibular experience; a “self-referenced” condition created a fixed self-referenced environment while active physical motions are put in place. It is believed that when the visual and vestibular sensations are in concord with each other, minimum sensory rearrangement would be needed. We hypothesise that the severity of cybersickness should be very much reduced. However, we had no hypotheses regarding the self-referenced environment. A reference frame could either help reduce cybersickness as it does in static VR system or induce more cybersickness as it induces sensory rearrangement.

112

2. Method

99 100 101 102 103 104 105 106 107 108 109 110

113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134

2.1. Stimuli and apparatus Fig. 1 shows the system used in this study. It consists of a 3-DoF motion platform (Motion Systems PS-3TM-350 V3) and an HMD (HTC VIVE). Individuals experienced visual stimuli from the HMD and vestibular stimuli on the motion platform. The participant was seated on a chair with seatbelt centred on top of the platform. A headrest and a travel pillow were used to support and constraint participants head position and orientation. The HMD system included a stereoscopic display headset, a controller for data input, and two base stations for positional tracking (see Table 1). A virtual apartment with realistic furniture was used. The simulated distance from the participant to the surrounding walls were around 2.6 m. Both the virtual and physical movements were rotating in an ipsilateral direction of the participant’s sitting position (i.e., rolling around the x-axis as indicated in Figure 1) at 0.36 Hz at an angle of ±15◦ . The physical rotational axe is located below the participant’s centre of gravity. The actual rotation perceived by the participant is offset (see Table 2). The room-scale tracking system provided by the HMD system was used. Two base stations broadcast infrared tracking signals. Tracking algorithms were used to resolve the objects’ position using the relative relations between photodiodes on top of the headset [27]. The simulated viewpoint is not only manipulated by the visual motion algorithm but is also compensated by the location tracking. The viewpoint tracking system updates the location and 4

+z

+x

+y

Figure 1: Physical setup of the experiment (Left: Stationary, Centre: Synchronous, Right: Self-referenced). The virtual viewpoint from the HMD of that instant is shown on top. Right-hand coordinate system is used.

Table 1: Technical details of the HMD system [28, 29, 30, 31, 32].

Screen Len Resolution Refresh rate Luminance FoVa Weight Tracking-to-display latency a

Stereoscopic OLED screens Fresnel compact lens 1080 × 1200 px per eye (11.4 px/degree) 90 Hz 214 cd/m2 110◦ combined horizontal, 113◦ vertical 550 g 22 ms

Measured at 8 mm eye relief (lens-eye distance).

Table 2: Details of the participant’s experience during the roll motion (Measured around eye height, right-hand coordinate system).

Centre Rollmax

Translation (cm) x y z

Rotation (deg) θ φ ψ

0 8

0 ±9

0 ±28

0 -4

5

0 10

0 ±22

Table 3: Characteristics of the three experimental conditions. Stationary Synchronous Self-referenced

135 136 137

138 139 140 141 142 143 144 145 146 147

148 149 150 151 152 153 154 155 156

Real physical motion Virtual visual motion Resulting sensation

No

.36 Hz ±15◦ roll

.36 Hz ±15◦ roll

.36 Hz ±15◦ roll

Anti-phase with physical motion Physical motion in a self-referenced VE

Sensory rearrangement

Visual in the absence of vestibular

Predicted result

Highest discomfort

In-phase with physical motion Physical motion with ecological virtual visual presentation Visual and vestibular together (minimum to none) Least discomfort

Visual roll

Visual and vestibular together (conflict) Average discomfort

orientation of the virtual stereoscopic camera to prevent unwanted sensory conflicts elicited from participant’s involuntary viewpoint jittering (e.g., vibrational movement due to incomplete head restraint). 2.2. Participants The research study recruited 13 subjects. One participant dropped out of the experiment due to cybersickness discomfort. The final dataset contains the records of 12 participants, aged 18-24 years (Mage = 21.5; SDage = 1.7; 6 males), all of whom completed the procedures. All participants had normal or corrected-to-normal stereopsis ability and visual acuity, presented no obvious signs of oculomotor or neurological pathology, and reported not to have taken any antihistamine in the past 12 hours. All experimental procedures obtained human research ethics approval. Each participant provided written informed consent and received financial compensation. 2.3. Experimental conditions Table 3 summarised the three conditions with variating coupling of visual and physical rotation. The “stationary” condition featured virtual viewport roll, while participants are physically stationary. The “synchronous” condition synchronised the physical roll motion of the motion platform in-phase with an equivalent virtual viewport rotation at the same magnitude and direction. As the spatiotopic location (reference) of the VE remains constant, participants should feel the motions in an ecological fashion as if they were not wearing an HMD (similar to the “compensated” motion in Palmisano 6

157 158 159 160 161

162 163 164 165 166 167 168 169 170 171 172 173 174

175 176 177 178 179 180 181 182 183 184 185 186 187

et al. [18]). Lastly, in “self-referenced” situation, participants experience the same physical rotation as in “synchronous” condition but with a coherent but opposite (anti-phase) virtual viewport roll. The whole virtual room (the self-reference VE) is presented fixed relative to the motion platform and the participant. 2.4. Procedures Each participant was randomly allocated 2 blocks of trials from the 3 conditions. Each block consists of 3 trials, 3.5 minutes each. Between the two blocks, participants removed the HMD for a 5-minute break. After each trial, participants were asked to respond to the simulator sickness questionnaire (SSQ) [33]. In addition, at every 60 seconds during the stimuli experience, participants answered an 11-points misery scale (MISC) and joyfulness scale (JOSC) verbally (see Table 2, 3 of [22]). For MISC, 0 means no problems; any nausea symptoms imply a score of 6 or higher, the experiment would stop at any score higher than 6. For JOSC, 10 means they feel pleasant (0 means bad). The scale is easily repeatable due to the ease of providing a simple number. At the end of the experiment, participants preferential order of the conditions was asked. 2.5. Statistical analysis The data were analysed using generalised linear mixed models (GLMMs). The model can be applied to multiple random effects, catering to random variability [34]. In our case, repeated measures taken at different time. Separated models were created with different dependent variables including SSQ total (T) scores, nausea (N), oculomotor (O), and disorientation (D) subscores, MISC, and JOSC. The independent variables are the three conditions (3), block order (2), trial sequence (3), and minute in the trial (3). Block order is defined by the order of exposure (i.e., before or after a 5-minute break); trial sequence refers to the order of the three trials in each block; minute in the trial refers to the minute elapsed from the start of the trial. An additional model was created to include the SSQ sub-scores to understand the relationship between the sub-scores in our system2 . 2 Noted that similar significances can be achieved if the model is performed by three separate analyses.

7

188

189 190 191 192 193 194 195 196 197 198

199 200 201 202 203 204 205 206 207 208 209 210 211 212

213 214 215 216 217 218 219 220 221

3. Results The SSQ scores were calculated using the standard formula. Table 4 summarised the descriptive statistics. It can be observed that the variances of the ratings and scores are large. The carry-over effect of the second block is huge. The mean differences of MISC between the two blocks for “self-referenced” is 1.31, “synchronous” is .78, and “stationary” is .19. The differences can be visualised in Fig. 2 and Fig. 3. In the initial data screening, it was found that the block order of the experiment created undesirable carryover effects. Participants rated their second block with higher MISC regardless of the conditions. Hence, the block order and participant were included as random effects. 3.1. Subjective ratings Two GLMM were created to evaluate the subjective ratings: MISC and JOSC. The main effects in the analysis included the conditions, trial sequences, and minute in the trial. The model found that condition (F = 134.3, p < .001), trial sequence (F = 26.5, p < .001), and minute in the trial (F = 7.8, p = .001) each have a significant effect on MISC. The following MISC ranking can be observed: “stationary” > “self-referenced” > “synchronous”. MISC increases over time (trial sequence and minute in the trial). See Table 5 for the estimates of fixed effects of each model. The model also found that condition (F = 157.2, p < .001) and trial sequence (F = 7.9, p = .001) each have a significant effect on JOSC, but minute in the trial (F = 1.5, p = .233) does not. The following JOSC ranking can be observed: “synchronous” > “self-referenced” > “stationary”. JOSC reduces over time. 3.2. SSQ GLMM was created to evaluate the SSQ-T score. The main effects in the analysis included conditions and trial sequences, similar to the previous model with random effects included. The model found that condition (F = 16.8, p < .001) and trial sequence (F = 5.8, p = .005) each have a significant effect on the participant’s SSQ-T score. However, the model, in general, is less reliable as the intercept is not significant (F = 2.6, p = .133). The following SSQ score ranking can be observed: “stationary” > “self-referenced” > “synchronous”. The score increases over time (trial sequence).

8

Table 4: Mean and standard deviation of various ratings and scores in three conditions. M Stationary Synchronous Self-referenced

MISC SD

3.57 1.06 1.90

1.69 1.02 1.40

M

JOSC SD

5.21 7.64 6.90

2.95 1.50 1.82

M

SSQ-T SD

48.31 30.23 40.36

M

33.27 43.15 35.26

SSQ-N SD

46.91 21.86 37.37

36.02 37.16 33.63

M

SSQ-O SD

28.43 23.69 28.74

SSQ-D M SD

18.06 28.30 24.69

59.16 37.12 42.92

Table 5: Estimates of fixed effects for MISC, JOSC, and SSQ scores.

β

SE

Z

F

β

SE

MISC 11.58** ***

2.73 Condition Stationary 1.81*** Synchronous −1.08*** Trial sequence 1st trial −1.13*** nd 2 trial −.43** Minute in the trial 1st minute −.61*** nd 2 minute −.24

.66

20.23*** ***

4.14

6.11

1.46

4.14

134.25***

157.18*** ***

−1.89 .21 −9.18 1.76*** .21 8.55

.18 10.15 .18 −6.06 26.45***

7.90*** ***

.16 −7.21 .16 −2.76

.71 .38*

.18 .18

3.97 2.10

.31† .15

.18 .18

1.71 .86

7.80*** .16 −3.92 .16 −1.51

1.47

SSQ-N

SSQ-O,D,T

Intercept

2.64 45.67† 24.80

2.50 51.31* 23.24

1.84

2.21

16.80***

Condition Stationary Synchronous Trial sequence 1st trial 2nd trial Sub-scores Nausea Oculomotor p ≤ .001

F

JOSC

Intercept

***

Z

19.29** 5.79 3.33 −14.17* 5.79 −2.45

31.67***

18.92*** 4.02 4.71 −12.85** 4.02 −3.20 5.77**

***

−16.99 −6.23

9.77*** ***

5.06 −3.36 5.06 −1.23

−15.25 −5.54

3.49 −4.37 3.49 −1.59 15.59***

**

−11.02 3.49 −3.15 *** −19.4 3.49 −5.57 **

p < .01

*

†

p < .05

9

p < .1

45.57 56.83 42.24

Mean MISC Rating

Stationary Synchronous Self-referenced First Block Second Block

1st Trial

2nd Trial Minute in the Trial

3rd Trial

Figure 2: Mean MISC in three conditions by trial sequences, minute in the trial, and block order.

Mean SSQ-T Score

Stationary Synchronous Self-referenced First Block Second Block

Trial

Figure 3: Mean SSQ scores in three conditions by trial sequences and block order.

10

222 223 224 225 226 227 228 229

Additionally, a model was created to include all the sub-categories of the SSQ. The dependent variable is now SSQ score and the sub-categories were included in the analysis. The model, in general, is less reliable as the intercept is not significant (F = 2.5, p = .141). However, the score differences in categories (F = 15.6, p < .001), condition (F = 31.7, p < .001), and trial sequence (F = 9.8, p < .001) each have a significant effect on the participant’s sub-score. The model can provide a good overview of the situation. The following sub-score ranking can be observed D > N > O.

233

3.3. Subjective preference The subjective preferential question showed 58.3 % of participants preferred the “synchronous” condition; 33.3 % preferred the “self-referenced” one; while the remaining 8.3 % preferred the “stationary” one.

234

4. Discussion

230 231 232

235 236 237 238 239

240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255

This study investigates the effect of visual-vestibular interactions on cybersickness under the framework of sensory conflict theory. In particular, a motion-coupled VR system is used. With the help of both the head-tracked HMD and the motion platform, in-phase and out-of-phase visual-vestibular cues were generated to validate several hypotheses. 4.1. Visual-vestibular synchronisation We observed that having a well-synchronised virtual viewport and physical roll motion (“synchronous” condition) leads to a lower subjective rating of cybersickness (MISC), higher user comfort (JOSC), and lower self-rated symptom-based MS score (SSQ) compared to a purely visual roll control condition. Participants preferred this condition more than other ones. Having a cybersickness rating of only 1, participants, in general, does not have specific symptoms of cybersickness. The implementation of motion-coupled VR systems is considered more desirable in providing additional physical motion sensation and reducing MS. However, this result is different from a study carried out using off-vertical axis rotation (OVAR) [35]. Experiencing a physical OVAR spin with the presence of visual cues was significantly more nauseogenic than watching a video playback of the spin on screen. Since it is comparatively easier to resolve a single conflicting cue rather than multiple ones, some subjects may be able to resolve sensory conflict in the video playback condition with 11

256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271

272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291

visual cue being the only conflicting factor. In our experiment, however, participants should experience a higher sense of presence from immersing into a realistic VE. Negative correlation is often found between presence and cybersickness [36]. It would be harder to dissociate oneself from the visual aspect of VE and resolve conflicting cues, even if only one conflicting cue is presented in the control condition of pure visual motion. Our study found that pure visual motion without any vestibular cues led to a much higher cybersickness score and lower user comfort. Very few participants preferred this condition. This is consistent with previous research that simulated viewpoint oscillation affects the severity of simulator sickness [37]. Comparing the SSQ sub-categories ranking order of our experiment with other previous data, we observed that our conditions have a similar characteristic to cybersickness, but not simulator sickness or other kinds of MS [38, 39]. This confirmed our underlying assumption that our motioncoupled VR system induces cybersickness, since the majority of effects are driven by visual cues. 4.2. Effect of self-reference environment A self-referenced environment was created by generating a virtual viewport rotation out-of-phase to the physical rotation to evaluate the effect of stabled vision cues during physical rotation on cybersickness. Results show that the self-referenced condition counteracts a certain amount of cybersickness. The average subjective cybersickness rating is 2, meaning that participants, in general, only experienced very slight amount of discomfort. Although it is not as effective as a visual-vestibular synchronised condition, a self-referenced environment produces similar cybersickness reduction effect as a self-referenced environment (or object) in a VR experience without having a motion platform [25, 26]. It is possible that the cognitive system contributes to a higher ranking to the visual stimuli input compared to the vestibular one during conflicts. The earth-fixed synchronous condition was found to induce less cybersickness discomfort than the self-reference condition, which is consistent with previous simulator experiments that use visual earth-fixed horizon to reduce simulator sickness [21, 22]. Our study further found that self-referenced environment can help mitigate a certain degree of MS effect. This technique can be considered on interactive VR systems designed for moving vehicles since it is harder to synchronise real-time vehicle motion.

12

303

4.3. Limitations This study had a number of limitations. First, multiple trials were conducted over short durations that are performed continuously for a participant. The short exposure time is similar to most entrainment setups, in particuar, motion-coupled VR applications (e.g., VR games in shopping malls and VR rides in theme parks) [7]. Typically, cybersickness symptoms will develop within the first few minutes of VR exposure [40]. Second, the break time between the two blocks of different conditions is relatively short. Carry-over effects such as sensitisation and habituation were found to be significant, regardless of the condition. This is addressed by randomisation using GLMM. In retrospect, we realised that short break time is insufficient for recovery even though the duration for exposure is short.

304

5. Conclusion

292 293 294 295 296 297 298 299 300 301 302

309

This study suggests that a motion-coupled VR system with synchronised visual and vestibular cues is the most preferred condition, as it significantly lowers the severity of cybersickness. We propose that a perfectly synchronised condition is ideal for a motion-coupled VR system and can decrease the severity of MS by reducing sensory conflict.

310

Acknowledgements

305 306 307 308

311 312 313 314 315

This work is partly supported by the Hong Kong PhD Fellowship under the UGC of the HKSAR government. The authors would like to thank Jeffrey A. Saunders from the Department of Psychology, HKU for his advice and Raymond Law of the Department of Industrial and Manufacturing Systems Engineering, HKU for the system setup.

316 317

Declarations of interest: none

318

Appendix A. Supplementary material

319 320

321 322

Research data associated with this article can be found online, at https: //doi.org/10.17632/mrkmjj2f4m.1. [1] J. T. Reason, J. J. Brand, Motion Sickness, Academic Press, London, 1975. 13

323 324 325 326

327 328 329

330 331

332 333 334 335

336 337 338

339 340 341 342

343 344 345 346 347

348 349 350 351

352 353

[2] C. M. Oman, A heuristic mathematical model for the dynamics of sensory conflict and motion sickness hearing in classical musicians, Acta Otolaryngol. 94 (Suppl 392) (1982) 4–44, doi:10.3109/00016488209108197. [3] W. Bles, J. E. Bos, B. de Graaf, E. Groen, A. H. Wertheim, Motion sickness: only one provocative conflict?, Brain Res. Bull. 47 (5) (1998) 481–487, doi:10.1016/s0361-9230(98)00115-4. [4] J. E. Bos, W. Bles, E. L. Groen, A theory on visually induced motion sickness, Displays 29 (2) (2008) 47–57, doi:10.1016/j.displa.2007.09.002. [5] J. Chen, Y. Fang, H. Y. K. Lau, A novel design of robotic air bridge training system, in: 2016 IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO), IEEE, Shanghai, 2016, pp. 67–72, doi:10.1109/arso.2016.7736258. [6] S. Song, J. Yang, Configurable component framework supporting motion platform-based VR simulators, J. Mech. Sci. Technol. 31 (6) (2017) 2985–2996, doi:10.1007/s12206-017-0542-1. [7] Happy Finish, The slide thrilling VR experience from the view from the Shard, https://www.happyfinish.com/work/the-slide-thrillingvr-experience-for-the-view-from-the-shard, 2017, (accessed 29 August 2017). [8] B. Aykent, F. Merienne, C. Guillet, D. Paillot, A. Kemeny, Motion sickness evaluation and comparison for a static driving simulator and a dynamic driving simulator, Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering 228 (7) (2014) 818–829, doi:10.1177/0954407013516101. [9] R. Curry, B. Artz, L. Cathey, P. Grant, J. Greenberg, Kennedy SSQ results: fixed vs. motion-based FORD simulators, in: Proceedings of Driving Simulation Conference (Conference Simulation de Conduite) (DSC 2002), Driving Simulation Association, Paris, 2002, pp. 289–300. [10] B. Keshavarz, R. Ramkhalawansingh, B. Haycock, S. Shahab, J. Campos, Comparing simulator sickness in younger and older

14

354 355 356

357 358 359 360

361 362 363 364

365 366 367 368

369 370 371 372

373 374 375 376 377

378 379 380 381

382 383 384 385

adults during simulated driving under different multisensory conditions, Transp. Res. Part F Traffic Psychol. Behav. 54 (2018) 47–62, doi:10.1016/j.trf.2018.01.007. [11] L. Dziuda, M. P. Biernacki, P. M. Baran, O. E. Truszczy´ nski, The effects of simulated fog and motion on simulator sickness in a driving simulator and the duration of after-effects, Appl. Ergon. 45 (3) (2014) 406–412, doi:10.1016/j.apergo.2013.05.003. [12] M. Kl¨ uver, C. Herrigel, S. Preuß, H.-P. Sch¨oner, H. Hecht, Comparing the incidence of simulator sickness in five different driving simulators, in: Proceedings of Driving Simulation Conference Europe (DSC 2015), Driving Simulation Association, Tuebingen, 2015, pp. 87–94. [13] S. Casas, C. Portal´es, M. Fern´andez, To Move or Not to Move?, in: K. C. C. Yang (Ed.), Advances in Multimedia and Interactive Technologies, IGI Global, 2019, pp. 124–144, doi:10.4018/978-1-5225-59122.ch006. [14] H. Akiduki, S. Nishiike, H. Watanabe, K. Matsuoka, T. Kubo, N. Takeda, Visual-vestibular conflict induced by virtual reality in humans, Neurosci. Lett. 340 (3) (2003) 197–200, doi:10.1016/s03043940(03)00098-3. [15] S. Nishiike, S. Okazaki, H. Watanabe, H. Akizuki, T. Imai, A. Uno, T. Kitahara, A. Horii, N. Takeda, H. Inohara, The effect of visualvestibulosomatosensory conflict induced by virtual reality on postural stability in humans, J. Med. Invest. 60 (3.4) (2013) 236–239, doi:10.2152/jmi.60.236. [16] G. Llorach, A. Evans, J. Blat, Simulator sickness and presence using HMDs, in: Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology, ACM Press, Edinburgh, 2014, pp. 137–140, doi:10.1145/2671015.2671120. [17] W. Chen, A. Plancoulaine, N. Frey, D. Touraine, J. Nelson, P. Bourdot, 6DoF navigation in virtual worlds, in: Proceedings of the 19th ACM Symposium on Virtual Reality Software and Technology, ACM Press, Singapore, 2013, pp. 111–114, doi:10.1145/2503713.2503754.

15

386 387 388

389 390 391

392 393 394

395 396 397 398 399

400 401 402

403 404 405 406 407

408 409 410 411 412

413 414 415 416 417

[18] S. Palmisano, R. Mursic, J. Kim, Vection and cybersickness generated by head-and-display motion in the Oculus Rift, Displays 46 (2017) 1–8, doi:10.1016/j.displa.2016.11.001. [19] B. Arcioni, S. Palmisano, D. Apthorp, J. Kim, Postural stability predicts the likelihood of cybersickness in active HMD-based virtual reality, Displays (In Press) (2018) 1–9, doi:10.1016/j.displa.2018.07.001. [20] D. Tal, A. Gonen, G. Wiener, R. Bar, A. Gil, Z. Nachum, A. Shupak, Artificial horizon effects on motion sickness and performance, Otol. Neurotol. 33 (5) (2012) 878–885, doi:10.1097/mao.0b013e318255ddab. [21] J. E. Bos, S. N. MacKinnon, A. Patterson, Motion sickness symptoms in a ship motion simulator: effects of inside, outside, and no view, Aviat. Space Environ. Med. 76 (12) (2005) 1111–1118, https://www.ingentaconnect.com/content/asma/ asem/2005/00000076/00000012/art00003. [22] P. J. Feenstra, J. E. Bos, R. N. H. W. van Gent, A visual display enhancing comfort by counteracting airsickness, Displays 32 (4) (2011) 194–200, doi:10.1016/j.displa.2010.11.002. [23] Z. Cao, J. Jerald, R. Kopper, Visually-induced motion sickness reduction via static and dynamic rest frames, in: K. Kiyokawa, F. Steinicke, B. Thomas, G. Welch (Eds.), 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), IEEE, Reutlingen, 2018, pp. 102–112, doi:10.1109/VR.2018.8446210. [24] A. S. Fernandes, S. K. Feiner, Combating VR sickness through subtle dynamic field-of-view modification, in: B. H. Thomas, R. Lindeman, M. Marchal (Eds.), 2016 IEEE Symposium on 3D User Interfaces (3DUI), IEEE, Greenville, SC, 2016, pp. 201–210, doi:10.1109/3dui.2016.7460053. [25] T. Nguyen-Vo, B. E. Riecke, W. Stuerzlinger, Simulated reference frame: a cost-effective solution to improve spatial orientation in VR, in: K. Kiyokawa, F. Steinicke, B. Thomas, G. Welch (Eds.), 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), IEEE, Reutlingen, 2018, pp. 415–422, doi:10.1109/VR.2018.8446383.

16

418 419 420 421 422

423 424 425 426

427 428 429 430

431 432 433 434

435 436

437 438 439 440

441 442

443 444 445 446

447 448 449 450

[26] D. M. Whittinghill, B. Ziegler, T. Case, B. Moore, Nasum virtualis: a simple technique for reducing simulator sickness, in: Games Developers Conference (GDC), UBM Tech Game Network, San Francisco, 2015, https://www.purdue.edu/newsroom/releases/2015/ Q1/virtual-nose-may-reduce-simulator-sickness-in-video-games.html. [27] A. K. T. Ng, L. K. Y. Chan, H. Y. K. Lau, A low-cost lighthousebased virtual reality head tracking system, in: 2017 International Conference on 3D Immersion (IC3D), IEEE, Brussels, 2017, pp. 1–5, doi:10.1109/ic3d.2017.8251910. [28] D. C. Niehorster, L. Li, M. Lappe, The accuracy and precision of position and orientation tracking in the HTC Vive virtual reality system for scientific research, i-Perception 8 (3) (2017) 1–23, doi:10.1177/2041669517708205. [29] Eagleshadow, Vive Pro vs Vive screen brightness and color accuracy measurement results, https://www.reddit.com/r/Vive/comments/ 8c4luc/vive pro vs vive screen brightness and color/, 2018, (accessed 10 July 2018). [30] O. Kreylos, The display resolution of head-mounted displays, Revisited, http://doc-ok.org/?p=1694, 2018, (accessed 20 June 2018). [31] A. Borrego, J. Latorre, M. Alcaiz, R. Llorens, Comparison of Oculus Rift and HTC Vive: feasibility for virtual reality-based exploration, navigation, exergaming, and rehabilitation, Games Health J. 7 (3) (2018) 151–156, doi:10.1089/g4h.2017.0114. [32] O. Kreylos, Optical properties of current VR HMDs, http://docok.org/?p=1414, 2016, (accessed 4 May 2018). [33] R. S. Kennedy, N. E. Lane, K. S. Berbaum, M. G. Lilienthal, Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness, Int. J. Aviat. Psychol. 3 (3) (1993) 203–220, doi:10.1207/s15327108ijap0303 3. [34] D. Hedeker, Generalized linear mixed models, in: B. Everitt, D. C. Howell (Eds.), Encyclopedia of Statistics in Behavioral Science, vol. 2, Wiley Online Library, Chichester, U.K., 2005, pp. 729–738, doi:10.1002/0470013192.bsa251. 17

451 452 453 454

455 456 457 458

459 460 461

462 463 464

465 466 467 468

469 470 471 472 473

[35] M. M. C. Bijveld, A. M. Bronstein, J. F. Golding, M. A. Gresty, Nauseogenicity of off-vertical axis rotation vs. equivalent visual motion, Aviat. Space Environ. Med. 79 (7) (2008) 661–665, doi:10.3357/asem.2241.2008. [36] B. Keshavarz, H. Hecht, Visually induced motion sickness and presence in videogames: the role of sound, in: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 56, SAGE Publications, 2012, pp. 1763–1767, doi:10.1177/1071181312561354. [37] S. Palmisano, F. Bonato, A. Bubka, J. Folder, Vertical display oscillation effects on forward vection and simulator sickness, Aviat. Space Environ. Med. 78 (10) (2007) 951–956, doi:10.3357/asem.2079.2007. [38] L. Rebenitsch, C. Owen, Review on cybersickness in applications and visual displays, Virtual Reality 20 (2) (2016) 101–125, doi:10.1007/s10055016-0285-9. [39] K. M. Stanney, K. S. Hale, I. Nahmens, R. S. Kennedy, What to expect from immersive virtual environment exposure: influences of gender, body mass index, and past experience, Hum. Factors 45 (3) (2003) 504–520, doi:10.1518/hfes.45.3.504.27254. [40] S. Davis, K. Nesbitt, E. Nalivaiko, Comparing the onset of cybersickness using the Oculus Rift and two virtual roller coasters, in: Y. Pisan, K. Nesbitt, K. Blackmore (Eds.), 11th Australasian Conference on Interactive Entertainment (IE 2015), vol. 167, ACS, Sydney, 2015, pp. 3–14, http://crpit.com/confpapers/CRPITV167Davis.pdf.

18

Highlights -

The effect of visual-vestibular interaction on cybersickness was investigated. An HMD VR system with motion platform was used to generate accurate synchronization. Visual-vestibular synchronised motion was found to be the most enjoyable condition. Adding self-referenced visual element to physical motion reduces sickness severity. This experiment’s SSQ symptoms profile is similar to that of cybersickness.