Manipulating subjective realism and its impact on presence: Preliminary results on feasibility and neuroanatomical correlates

Interacting with Computers 24 (2012) 227–236

Contents lists available at SciVerse ScienceDirect

Interacting with Computers journal homepage: www.elsevier.com/locate/intcom

Manipulating subjective realism and its impact on presence: Preliminary results on feasibility and neuroanatomical correlates q Stéphane Bouchard a,⇑, Stéphanie Dumoulin a, Jeanne Talbot b, André-Anne Ledoux c, Jennifer Phillips d, Johana Monthuy-Blanc a,e, Geneviève Labonté-Chartrand a, Geneviève Robillard a, Matteo Cantamesse f, Patrice Renaud a a

Cyberpsychology Lab of UQO, Université du Québec en Outaouais, C.P. 1250, Succursale Hull, Gatineau, Québec, Canada J8X 3X7 Royal Ottawa Mental Health Center, Department of Psychiatry, Ottawa, Canada School of Psychology, Université d’Ottawa, Ottawa, Canada d University of Ottawa Institute of Mental Health Research, Ottawa, Canada e Université du Québec à Trois-Rivières, Department of Education, Trois-Rivières, Canada f Università Cattolica del Sacro Cuore, Centro Studi e Ricerche di Psicologia della Comunic, Milano, Italy b c

a r t i c l e

i n f o

Article history: Available online 15 May 2012 Keywords: Feeling of presence Virtual reality Subjective realism fMRI Parahippocampus

a b s t r a c t The feeling of presence has been shown to be an important concept in several clinical applications of virtual reality. Among the factors influencing presence, realism factors have been examined extensively from the angle of objective realism. Objective realism has been manipulated by altering numerous technological characteristics such as pictorial quality, texture and shading, or by adding more sensory information (i.e., smell, touch). Much less studied is the subjective (or perceived) realism, the focus of the two pilot studies reported in this article. In Study 1, subjective realism was manipulated in order to assess the impact on the feeling of presence. Method: Presence was measured in 31 adults after two immersions in virtual reality. Participants were immersed in a neutral/irrelevant virtual environment and subsequently subjected to the experimental manipulation. Participants in the experimental condition were falsely led to believe that they were immersed live in real time in a ‘‘real’’ room with a ‘‘real’’ mouse in a cage. In the control condition, participants believed they were immersed in a replica of the nearby room. All participants were actually immersed in the exact same virtual environment. Results: A manipulation check revealed that 80% of the participants believed in the deception. A 2 Times by 2 Conditions repeated measure ANOVA revealed that leading people to believe they were seeing a real environment digitized live in virtual reality increased their feeling of presence compared to the control condition. In Study 2, the same experimental design was used but with simultaneous functional magnetic resonance imaging (fMRI) in order to assess brain areas potentially related to the feeling of presence. fMRI data from five participants were subjected to a within subject fixed effect analysis to verify differences between the experimental immersion (higher presence) and the control immersion (lower presence). Results revealed a statistically significant difference in left and right parahippocampus areas. Conclusion: Results are discussed according to layers of presence and consciousness and the meaning given to experiences occurring in virtual reality. Some suggestions are formulated to target core presence and extended presence. Ó 2012 British Informatics Society Limited. All rights reserved.

1. Introduction Virtual reality (VR) has been described as ‘‘an application that lets users navigate and interact with a three-dimensional, computer-generated (and computer-maintained) environment in real time’’ (Pratt et al., 1995, p. 17). Virtual environments (VEs) can

q

This paper has been recommended for acceptance by Dr. D. Murray.

⇑ Corresponding author. Address: Laboratory of Cyberpsychology, Université du Québec en Outaouais, PO Box 1250, Station Hull, Gatineau, Québec, Canada J8X 3X7. Tel.: +1 819 595 3099; fax: +1 819 595 2250. E-mail address: [email protected] (S. Bouchard).

elicit strong emotional reactions that can be used in psychotherapy. Advocates of virtual reality interventions in the treatment of anxiety disorders contend that VE can elicit stronger reactions than imaginal exposure and, as a result, are more likely to activate the underlying neural fear processing network necessary for habituation to occur (see Foa and Kozak, 1986; Rothbaum et al., 1996). Presence is considered an important factor involved in the mechanism of exposure therapy using virtual reality (Wiederhold & Wiederhod, 2005). The sense of presence is often defined as the subjective impression of being there in the VE (Sadowski and Stanney, 2002), or as the illusion of being unaware of the medium used to create the immersion (the so called illusion of

0953-5438/$ - see front matter Ó 2012 British Informatics Society Limited. All rights reserved. http://dx.doi.org/10.1016/j.intcom.2012.04.011

228

S. Bouchard et al. / Interacting with Computers 24 (2012) 227–236

non-mediation, Lombard and Ditton, 1997). Researchers in the field of VR psychotherapeutic applications continue to debate several issues related to the sense of presence. Firstly, it is unclear whether presence is a moderator or a mediator of treatment outcome. Secondly, it is not known whether the relationship between presence and outcome is linear or even whether presence is associated with better treatment outcome in general or if it is associated only with the potential of virtual stimuli to induce anxiety – which subsequently is associated with the development of new mental representations and hence treatment outcome (Bouchard et al., 2007a). The sense of presence has been a challenge to define. The impression of being there provides a widely accepted description of the concept, but it does not identify the factors influencing presence, nor does it describe the exact nature of the experience. According to Witmer and Singer (1998), the feeling of presence experimented in a VE varies as a function of the characteristics of the VE (sound, graphic, etc.) and individual differences (anxiety, beliefs, etc.). More specifically, four major categories of interacting factors that can influence the feeling of presence have been studied: control factors, sensory factors, distraction factors, realism factors (Witmer and Singer, 1998). Most studies have focused on the first three factors (see Bouchard et al., 2010; Youngblut, 2007 for reviews). Studies have shown that presence in a VE is significantly increased when the user has more control, the senses are better stimulated, there is less distraction while immersed and there is increased realism. However, the realism in most studies has been examined essentially from the specific angle of objective realism (Bouchard et al., 2010; Youngblut, 2007). Realism (Witmer and Singer, 1998) encompasses a few elements. The first main element is that scene realism increases as a function of VE scene realism (as governed by scene content, texture, resolution, light sources, field of view, dimensionality, etc.). Scene content does not require an exact replica of the physical reality, but refers to the connectedness and continuity with objective reality of the stimuli being experienced. Consistency of information with the physical reality constitutes the second element associate with objective realism. The more consistent the information conveyed by a VE is with real-world experience, the more presence should be experienced in the VE (Held and Durlach, 1992). A third important element is the meaningfulness of the experience. Meaningfulness is often related to many other factors, such as motivation to learn or perform, task saliency, and previous experience. Objective realism has been studied by manipulating a variety of objective technological characteristics of the immersion, such as the field of view, stereoscopy, sound spatialization, and tactile augmentation (Hendrix and Barfield, 1996; Hoffman et al., 1998; Ijsselsteijn et al., 2001; Lin et al., 2002). It is possible to differentiate objective realism, such as realism of geometric forms and lighting of the VE from subjective realism (Slater et al., 2009b). If objective realism is based on technological characteristic such as visual characteristics, subjective realism addresses characteristic such as the beliefs that the VE is true. It is surprising to notice the paucity of empirical research on the role of factors such as subjective realism. As early as 1997, Lombard and Ditton reviewed the available literature and described six interrelated and distinct conceptualizations of presence, and made an important distinction between objective and perceived realism. According to them, one must differentiate between what most people commonly think about realism, which is how stimuli in VR are similar to those in objective reality, and subjective realism, which is how stimuli in VR match with what is expected by the user. Expectation is central for this subjective component of presence, more than photorealism. Presence occurs when VR equipment and the medium becomes unnoticed by the user and produce the perceptual illusion of non-mediation (Lombard and Ditton, 1997). The illusion of

non-mediation implies that the user immersed in VR: (a) forgets there are two sides to the mediated experience, the ‘‘real’’ and the virtual; (b) is subjected to stimuli that are perceptually and psychologically immersive; (c) feels that objects and entities are real. Feeling present is a perceptual illusion that ‘‘is a property of a person’’ (Lombard and Ditton, 1997, p. 9). Baumgartner et al. (2008) have examined realism and presence with neuroimaging techniques and other studies on this topic are under way (Clement et al., 2011). In their study, Baumgartner et al. (2008) manipulated several factors associated with the objective properties of the VE. The high presence condition consisted of a spectacular ascending, descending and looping roller coaster ride, with phases inducing anticipation and thrill, fast rotations in all three dimensions, accelerations and braking, sounds of the cart being pulled up and driving matching with spatial cues, etc. The low presence condition referred to a horizontal straight ride at varying velocities on sinuous tracks. The comparison of data acquired in the two conditions revealed the activation of many brain areas that might be associated with presence. They observed widespread activation in the brain areas associated with egocentric spatial processing, visual analyse and recognition, sensory-motor processes, acoustic processing and emotional processing. Unfortunately, the differences observed in these regions may be explained by differences in the stimuli presented and by the emotions induced by a spectacular and thrilling roller coaster ride. They correlated changes in the brain areas and changes in presence and concluded that presence was modulated by the dorsolateral prefrontal cortex and this mechanism was different for adult and children. They (Baumgartner et al., 2008; Jancke, Cheetham & Baumgartner, 2009) interpreted this finding as strategy for regulating presence by constraining the egocentric processing of the roller coaster stimulus and preparing for actions in the VE. The aim of this article it to explore the impact of subjective realism on presence by conducting two preliminary studies with the objectives of showing that it is possible to influence presence by manipulating only subjective realism (Study 1) and exploring which areas of the brain may be influenced by this manipulation (Study 2). 2. Study 1 2.1. Method 2.1.1. Sample After receiving authorization from the ethics review board from the University of Québec in Outaouais, a total of 31 adults – 26 women and 5 men – aged between 19 and 62 years old (M = 32.26; SD = 14.75) participated after being recruited from advertisements in the media (newspaper, poster, etc.). 2.1.2. Procedure The flowchart describing the experimental procedure (see Fig. 1) illustrates the six steps of the study. First, participants completed a general assessment that consist of consent forms and pre-immersion questionnaires measuring predispositions to feel present and baseline side effects). Step 2 was a first immersion in a neutral/irrelevant virtual environment conducted to allow participants learning how to navigate in a virtual environment, experiencing what presence is and providing a reference point to assess the impact of the experimental immersion. This immersion was followed by an assessment of presence (Step 3). From this step, participants were randomly assigned to either the Control condition (Ctrl) or the Experimental condition (Exp). In the Ctrl condition, participants were physically in Room A and watched a 2-min video recorded in Room B showing a research assistant

S. Bouchard et al. / Interacting with Computers 24 (2012) 227–236

Step 1 General assessment

Step 2 Neutral/irrelevant virtual environment

Step 3 Assessment of presence (pre-immersion)

Step 4 Control condition (Ctrl) or Experimental condition (Exp)

Step 5 Experimental immersion

Step 6 Assessment of presence (post-immersion) Fig. 1. Flow diagram of six procedural steps.

looking at a mouse inside a cage (see Fig. 2). After the video, participants in the Ctrl condition were told that they will explore a copy of room B that was entirely created for the purpose of the study. In the Exp condition, participants were in Room A and discussed, via a videoconference system, with the research assistant located in Room B about the mouse in a cage near the assistant. After the 2-min live discussion in videoconference, the experimenter turned off the television monitor, connected and launched the mock ‘‘live immersion system’’ (see Fig. 3) and told the participants that they will now explore Room B in VR while the images from room B and the mouse were digitalized live in real-time. Note that Step 4 is based on a verbal deception and the VE explored by the participants was the same in both conditions. Step 5 consisted of the 5-min experimental immersion. The VE depicted a replica of room

229

B with a mouse moving in a cage. Post-immersion (Step 6), participant completed again three measures (presence, side effects, manipulation check) and were debriefed. 2.1.3. Equipment 2.1.3.1. Mock live immersion. In order to manipulate subjective realism, participants were lead to believe that they were connected in real time with a remote location that was digitized live. This was done by having participants discussed verbally with a research assistant by videoconference. The experimental room (Room A) and Room B were linked at 384 kb/s by two Tandberg 2500 videoconference systems (codec, camera, and NT3 communication port). Images from the videoconference system were displayed on a 20-inch television monitor in full-screen. The mock set-up also included four computers (see Fig. 3) that were turned on at the end of the discussion in videoconference while the experimenter was saying it would ‘‘provide a live digitalized video feed from the videoconference unit directly in our most powerful computer system so you will see the real room with the real mouse, but in VR, as if the room was projected in your goggles’’. Lights were blinking on a CISCO switch, sounds were coming from the computers and cables labeled ‘‘live video feed’’ were plugged into the VR system while the participant was witnessing the scene. The immersion in VR was performed with a PC computer (IBM Pentium IV, ATI 128 MB), a Cy-Visor head mounted display (HMD) and an Intersense Intertrax2 motion tracker (Bouchard et al., 2008). The virtual environment replicated Room B, a small room with no furniture and a mouse in a cage. The mouse was animated (sleeping, walking, drinking and running in a wheel) during the immersion. 2.1.4. Measures 2.1.4.1. Feeling of presence. A single-item self-report was used to assess presence on a scale of 0 (not at all) to100 (totally) scale: ‘‘To what extent do you feel present in the virtual environment right now?’’ All participants were told before the immersion that ‘‘Presence is defined as the subjective impression of really being there in the virtual environment’’. Single-item measures have already been used in presence research (e.g., Bouchard et al., 2008; Freeman et al., 1999; Slater and Usoh, 1993) and have been found to be reliable and not obstructive (Bouchard et al., 2005). 2.1.4.2. Predisposition to feel present. The Immersive Tendencies Questionnaire (ITQ; Witmer and Singer, 1994) was used to measure the extent to which a person succeeds in tuning out external distractions in order to become involved in common activities. The tendency to become involved or immersed should theoretically be related to the likelihood of experiencing presence in virtual environments. The 18 items of the questionnaire were scored according to four sub-scales: focus (state of mental alertness, ability to concentrate or enjoyable activities, abilities to block out distractions),

Fig. 2. Picture of the live mouse (left) and screenshot of the virtual mouse (right).

230

S. Bouchard et al. / Interacting with Computers 24 (2012) 227–236

Fig. 3. Illustration of the mock set-up.

involvement (subject’s propensity to get involved passively in some activity, such reading books, watching TV or viewing movies), emotion (intensity of lived emotions during or after viewing movies) and game (frequency to play video games). The participants were asked to indicate how frequently they were characterized by each of the items on a seven-point scale ranging from ‘‘never’’ (1) to ‘‘often’’ (7). This scale was used to ensure that participants in both experimental conditions do not differ in terms of likelihood of feeling present. 2.1.4.3. Side effects. The Simulator Sickness Questionnaire (SSQ; Kennedy et al., 1993) was used to assess the intensity of VRinduced symptoms (i.e. cybersickness) before and after the immersion. Instead of following Kennedy et al.’s (1993) suggested scoring procedure, only the sum of the raw scores from the 16 items of the questionnaire are reported (Bouchard et al., 2007b, 2009). This scale was used to: (a) document if the experiment induced any side effects, and (b) ensure that potential side effects would not bias the results. 2.1.4.4. Manipulation check. After the last immersion, all participants were asked if they consider the mouse as realistic, and only those in the experimental condition were asked if they really believed they were in Room B. 2.2. Results Preliminary analyses revealed no significant differences between the conditions in terms of gender [v2ð1Þ ¼ 1:92, p = .17], age [t(29) = .17, p = .87], immersive tendencies [t(29) = .85, ns] and side effects [t(29) = 1.73, p = .4]. A manipulation check was performed and analyzed with a v2 test. The mouse observed during the immersion in VR was considered realistic by 81% of the participants in the experimental condition and by 80% of the participants in the control condition, which was not significantly different [v2ð1Þ ¼ :008, p = .93]. The deception was considered effective since 80% of the participants in the Exp condition believed that Room B was really digitized live and sent to the head mounted display through the live immersion/videoconference system. After confirming that the distribution of the data did not depart from the normal distribution, the impact of subjective realism on the feeling of presence was analyzed with a two way repeated measures ANOVA. Participants in the Ctrl condition reported a mean presence score of 64.06 (SD = 22.6) after the neutral/irrelevant immersion, and a mean of 58.13 (SD = 29.21) after the immersion in the VR environment described as a copy of Room B. Those in

the Exp condition reported a mean presence score of 53.67 (SD = 26.42) after the neutral/irrelevant immersion, and a mean of 61.0 (SD = 29.47) after the immersion in the VR environment depicted as a live projection from Room B. The results from the two way repeated measures ANOVA revealed no significant main effects of conditions [F(1,29) = .17, p = .69] and repeated immersions [F(1,29) = .05, p = .82] on the feeling of presence and, as hypothesized, a significant interaction effect between subjective realism and repeated immersions on the feeling of presence [F(1,29) = 4.73, p < .05, partial eta-squared effect size of .14]. Note that removing from the study participants who did not believed in the experimental manipulation did not change the pattern of results, with the interaction effect remaining significant [F(1,26) = 6.82, p < .05]. 3. Study 2 3.1. Method 3.1.1. Sample Recruitment was independent from Study 1. Five adults (three females, two males, mean age of 33.4, SD = 11.24) were recruited after an initial interview showed: (a) they were not suffering from a mental disorder, as assessed with the Structured Clinical Interview for DSM-IV (First et al., 2002); (b) they were right handed, according to the Edinburgh Handedness Inventory (Oldfield, 1971), (c) had no known medical condition, and (d) scored on average or more on the Immersive Tendencies Questionnaire (Witmer & Singer, 1998). All participants provided a written informed consent and were fully debriefed after the experiment. For ethical reasons as well as exclusion of participants with brain abnormalities, a scan was performed when participants were in the scanner, with a low angle recalled echo (FLARE) sequence acquired in the axial plane to produce images suitable for review by a radiologist to rule out clinically significant abnormalities, hydrocephalus or intra-cranial masses. 3.1.2. Procedure At their arrival in the fMRI clinic, participants were prepared for the scan. A deception strategy based on manipulating the narrative context provided for the immersion combined with a mock live immersion was used, based on the findings of Study 1. The deception took place in a staff room adjacent to the scanner room, where participants were told that during their brain scan they would at times see a live video-feed from this room (Higher Presence Condition; HCP) or a good 3D copy of the room (Lower Presence Condition; LPC), although only one virtual environment was

S. Bouchard et al. / Interacting with Computers 24 (2012) 227–236

developed and presented. The environment was a replica of the staff room adjacent to the scanner (see Fig. 4). The exact narrative was: ‘‘We want to know which areas of the brain are involved in the experience of virtual reality by using a high tech device that will take images of your brain in action. For the very first time, we were able to create a system that allows projecting, in real time, images taken from this adjacent staff room and project it directly, live, in the fMRI scanner. You will therefore be able to see this room, as you are just doing now, but seeing through the eyes of virtual reality. Our cameras will record images in real time as they move in the room, send them to our computers and our virtual reality software; they will recreate the virtual reality and send it directly into the scanner. The cameras will move along a predetermined path so you would not have anything to do but let yourself be immersed in the virtual environment and looking around. We also built a copy of this staff room. After having the unique chance of being immersed in the real staff room, we will also show you a copy of the staff room. The experience will be repeated twice, so you will again see the real room and the copy of the room’’. When the experience began in the scanner, the instructions were projected in the HMD ‘‘In this unique experience, you will visit the real staff room, then a copy. This will be repeated twice. Let go, observe these virtual environments and we will talk about it after the scan’’. In the HPC immersion, the following message was displayed before starting the 60-s immersion: ‘‘Live staff room. Let go of yourself and take the time to observe the virtual environment’’ and in the LPC immersion the message stated ‘‘Copy of staff room. Let go of yourself and take the time to observe the virtual environment’’. Both immersions were repeated once. 3.1.3. Equipment 3.1.3.1. Neuroimaging. The scanner was a 1.5 T Siemens Magnetom Symphony System scanner (Siemens AG, Erlangen, Germany). Diagnostic (FLARE), structural (T1), and functional (BOLD) scans were acquired during the scanning session. A preliminary rapid sagittal T1-weighted scan was used as a localizer to verify participant head position and image quality. A three-dimensional gradient echo acquisition was used to collect 160 contiguous, 1 mm T1 weighted structural images in the sagittal plane for corregistration with the Echoplanar images (EPIs). Structural images were acquired using the modified International Consortium for Brain Mapping (ICBM) T1 Protocol with the following parameters: repetition time (TR) = 22 ms; echo time (TE) = 9.2 ms; bandwidth = 70 Hz/Px; field of view = 256 mm; flip angle = 30°. Blood oxygen level dependent (BOLD) signals were obtained using 32 contiguous, 4 mm axial slices, positioned parallel to the hippocampus and covering the entire brain (64 by 64 matrix; TR = 3000 ms; delay in TR = 219 ms; TE = 50 ms, field of view = 256 mm, flip angle = 90°). The virtual environments were displayed using a stereoscopic MRI-compatible HMD system (Silent VisionTM Model SV-7021 Fibre Optic Visual System with In Control Software, Avotec, Inc.) and the participants were wearing noise cancellation headphones.

231

3.1.3.2. Mock live immersion. As in Study 1, subjective realism was manipulated by leading participants to believe that they were connected in real time with a remote location that was digitized live. Fig. 5 shows the physical set-up and the cameras mounted on a wheeled kart. In order to show participants the system was working, two cameras were functional and displaying live images of the participants and the experimenter, one by wireless transmission to an Alienware gaming laptop positioned at the entrance of both rooms and one to the TV monitor connecting to the cameras and the computers on the kart.

3.1.4. Measures 3.1.4.1. Selection questionnaires. The Structured Clinical Interview for DSM-IV, Non-Patient Edition (First et al., 2002) was used to exclude participants who suffer from a psychiatric disorder, such as schizophrenia and claustrophobia. The Edinburgh Handedness Inventory (Oldfield, 1971) is a traditional measured used to determine participant’s handedness and ‘‘dominant’’ hemisphere. The medical screening confirmed the participant’s ability to undergo fMRI scanning when a trained nursed assessed past injuries, surgeries and metal implants that may warrant exclusion from the study. Given the costs associated with a neuroimaging experiment, its exploratory nature and small sample size, it was decided that participants would be excluded if they scored below 60 on the Immersive Tendencies Questionnaire (Witmer and Singer, 1998), a measure of user’s disposition to feel immersed in a virtual environment. No participant was excluded based on this criterion. Their scores ranged from 60 to 98, which suggest a strong tendency to feel immersed. A one-item measure of presence rated with a visual slider was included in the research design but cannot be used due to technical difficulties.

3.1.4.2. Analysis of fMRI results. DICOM images were converted to NIFTI format using Statistical Parametric Mapping (SPM5) conversion script (Wellcome Department of Imaging Neuroscience, London, UK). MRI image preprocessing and analysis was conducted with SMP5. Preprocessing of functional images involved the alignment of the first image of the functional series for each subject relative to the anterior and posterior commisures, realignment of all subsequent images to the first image, normalization of the resulting images to the standardized MNI template, and spatial filtering of the images with a Gaussian filter equal to twice the size of EPI voxels (FWHM = 8 mm3). The time series were high-pass filtered (minimum cutoff frequency of 1/128 Hz) to remove low-frequency artifacts. To determine the effect of the task as a group, a within subject, fixed effect analysis was conducted for the statistical analysis of the functional images. The first level group-specific design matrix contained the following regressors for each participants: (1) three regressors encoding the average BOLD responses at each of the three states (Instruction, HPC and LPC immersion tasks); and (2) a nuisance partition containing six regressors that encoded the

Fig. 4. Picture of the staff room (left) and two screenshots of the virtual environment, from the same point of view of the picture (center) and from a broader perspective (right).

232

S. Bouchard et al. / Interacting with Computers 24 (2012) 227–236

Fig. 5. Picture of the mock-up set-up and device created to give the impression the virtual environment was scanned live. The operation of the system was supervised by a laptop positioned at the at the entrance of the staff and the scanner rooms and displaying live images from the room (first left), both rooms were adjacent (second left), the live recording device consisted of seven cameras mounted on a kart equipped with wheels, several running computers and a TV monitor (third left) and the participants could see live on the monitor their image captured during the visit of the staff room (right).

movement displacement as estimated from the affine part of the image realignment procedure. The design matrix was estimated and for the purpose of this study, the contrast of interest was the difference between HPC > LPC for the group. Areas of significant brain activation were localized using the Wake Forest University PickAtlas version 2.3 (Tzourio-Mazoyer et al., 2002) and confirmed through manual inspection. 3.2. Results The fixed effect analysis (average of group) was used to determine areas of significant activation in HPC versus LPC immersions. Statistical images were assessed for voxel-wise significance using an uncorrected threshold of p < 0.001 and significant differences were found in parahippocampus areas. As seen in Fig. 6 a, the left parahippocampus and in Fig. 6b the right parahippocampus were activated when participants were lead to believe the virtual environment was real (HPC compared to LPC immersion). 3.3. Discussion The objectives of these combined studies were to test if subjective realism actually influences presence and if this strategy could be used in a neuroimaging experiment. In Study 1, when misleading people to believe that a virtual environment was real, their impression of being there in the virtual environment significantly increased. This finding supports Lombard and Ditton’s (1997) contention that user’s expectations play a role in the subjective component of presence. We concur that photorealism and immersive

properties of the system strongly contribute to the feeling of presence (Bouchard et al., 2010), as demonstrated empirically by other researchers (e.g., Hendrix and Barfield, 1996; Hoffman et al., 1998; Ijsselsteijn et al., 2001; Lin et al., 2002; Witmer and Singer, 1998; Youngblut, 2007). But Study 1 suggests that the meaning given to the experience may also be important. Despite the small sample size in Study 2, the fMRI results were significant and specific to one anatomical area. The manipulation activated the parahippocampal cortex on both sides of the brain. For several years, the parahippocampal cortex was dubbed as the ‘‘place area’’ to describe the growing body of knowledge suggesting that it was not related to recognition and retrieval of information related to places, but to the current perception of places (Epstein et al., 2007). Epstein et al. (2007) revealed that activity in this area does not depend on memory retrieval, and Bar et al. (2008) demonstrated that this area of the brain actually mediates the representation and processing of contextual associations, providing meaning for scenes and places. Our fMRI data did not reveal a specific activation of dorsolateral prefrontal cortex as suggested by Baumgartner et al. (2008). However, the two methodologies are quite different since we manipulated interpretations of the stimuli instead of the stimuli themselves. We speculate that the reflexive interpretation of the experience felt in the VE describes a different dimension of presence, most likely involving the parahippocampus gyrus after multisensory integration had led to the explicit feeling of actually being in the virtual environment and not in the physical reality. Nonetheless, Study 2 should be considered as a pilot study and must be replicated with a larger sample, and more controls, including a manipulation check.

Fig. 6. The attached images show the voxels of significant activation in the parahippocampus when participants were lead to believe the virtual environment was real (HPC > LPC). (a) Represents the activity in the left parahippocampus (MNI coordinates: 28, 16, 24; t-value of 3.23, p < 0.001 unc., cluster size = 5 voxels) and (b) demonstrates the activity in the right parahippocampus (MNI coordinates: 28, 4, 36; t-value of 3.66, p < 0.001 unc., cluster size = 10 voxels). Statistical parametric maps were thresholded at an uncorrected p value of p < 0.001.

S. Bouchard et al. / Interacting with Computers 24 (2012) 227–236

If subjective realism relates to presence, how does it fit with our general understanding of presence? This question is even more important given the topic of this journal’s special issue because our verbal manipulation did not require change in interaction between the user and the VE. The brief answer is based on a conception of presence based on essentially two factors located on a continuum, from a basic but strong perceptual illusion to the meaning given to the experience felt in VE. Subjective realism probably contributes directly to extended presence, while objective realism and the user’s interactions with the synthetic environment probably contribute more directly to core presence. The elaborate answer requires describing processes akin to the three layers of presence described by Riva et al. (2004), Riva (2009), and Waterworth et al. (2010). These three layers, proto presence, core presence and extended presence, where proposed to relate to Damasio’s (1999, 2003) three neurological layers of consciousness and the self, which are proto consciousness, core consciousness and extended consciousness. As Blanke et al. (2004) and Makin et al. (2008) reported, consciousness is a mechanism that ascertain that what is seen and felt by the organism are both synchronized and consistent with representations of what is and should be occurring in a given situation. Discrepancies from expected internal mental representations stimulate this internal monitoring process (Beaumeister and Masicampo, 2010; Damasio, 1999, 2003; Panksepp and Northoff, 2009) and we propose that a nuanced understanding of consciousness can explain users’ comments immersed in VR such as ‘‘wow, I really have the impression of being there’’. If a user immersed in VR has the impression of falling from a cliff, this should activate processes involved in consciousness because it is both relevant for the organism and egodystonic. The following paragraphs will provide a brief description of how the three layers of consciousness relate to presence and how it may integrate some of our results. The presentation will however be more oriented toward VR as essentially being a strong perceptual illusion combined with a sense of meaning, as opposed to focusing on the differentiation between internal and external events (Riva et al., 2004; Waterworth et al., 2010). According to Damasio (1999, 2004), the proto self maps the physical state of the organism and the corresponding proto presence is the most basic form of presence. At one end of the continuum, proto presence operates automatically at a basic level of perception. For example, proto presence describes a user’s startle reaction when hearing a sound, when subjected to vection illusions or when looking at traditional visual illusions such as Kanizsa triangle or the Duck–Rabbit illusion (e.g., Goldstein, 2007) while immersed in a VE. With classical visual illusions such as those mentioned above, proto presence describes the user’s perception of self-motion and that the triangle or the duck is really there on the visual plane. Some might consider the occurrence of basic reflexes or primary perceptions as objective signs of presence, although many would be hesitant and consider these responses as too primary. Actually, proto presence does not inform us much about the user and its experience in the VE, but it is the starting point of the presence continuum. If a user does not react to such basic stimuli, it is most likely there is no presence at all. The two experiments reported in the current paper manipulated presence at a higher level of interaction and consciousness than proto presence, as do typical experimental manipulations of the objective realism of the VE. According to Damasio (1999, 2003), core consciousness occurs along with the emergence of the feeling of knowing how the organism is being affected by the processing of stimuli, the state of the organism and the interactions between the two. Multisensory integration represents a second-order treatment of information coming from the senses that contributes to bodily self-consciousness. An immersion in VR is an excellent example of multisensory

233

integration. In VR, many senses are stimulated, from simple immersions using visual and audio stimuli to complex interactions with avatars combined with haptic feedback and olfactory cues in a CAVE. Since proprioceptive information provides feedback on the user’s postural state in the VE, as soon as motion is tracked in VR there is some basic interaction with the VE. Head rotation or body sway will call for a reaction from the computer and modifications in the VE. Even simple vergence, in the case of stereoscopic displays, leads to converging information from the visual and proprioceptive senses. Empirical data support the notion that multisensory integration can lead to powerful and credible illusions. Research on multisensory integration is blooming and several findings relate directly to presence, the illusion of non-mediation and the feeling of being there. As recent work from Blanke, Ehrsson and Slater shows, it is possible to create multisensory illusions that influence the user’s representation of his own body and its location. It is feasible to make people believe they are located somewhere else in the room than they are physically (Lenggenhager et al., 2007), they are in someone else body (Petkova and Ehrsson, 2008) or their body is the size of a doll or a giant (van der Hoort et al., 2011). Complex interactions with computer generated environments are quite powerful to create illusions such as body transfer (Slater et al., 2010), but even minimal interaction found in basic immersions in VR may be sufficient, due to the integration of visual information with proprioceptive sensations from motor activity (Sanchez-Vives et al., 2010), and this impact is much stronger using a first-person than a third-person reference frame (Petkovas et al., 2011). To create an illusion as strong as VR can provide, the integration of the stimuli must be credible and consistent with realistic state of the organism (Held and Durlach, 1992; Slater et al., 2009a). For example, asynchronous stimulation of senses hinders or prevents illusions based on multisensory integration (e.g., Sanchez-Vives et al., 2010; Slater et al., 2010; van der Hoort et al., 2011). Actually, locating one’s body in the physical space is so much dependent on automatic and implicit integration of multisensory information that it can even be manipulated in monkeys (Graziano, 1999) and researchers have reported insects immersed in synthetic environment that behaved as if the stimuli were real (e.g., Mouritsen and Frost, 2002; Sempo et al., 2006; Takalo et al., 2012). Extending Lombard and Ditton’s (1997) concept of illusion of non-mediation, it is possible to propose that the phenomenon that many describe as presence is actually just an illusion. At least, this is true for core presence. An immersion in VR is essentially the organism being subjected to a complex apparatus that enables strong perceptual illusions based on synchronous multisensory integration. Core presence is experienced because of the integration of multisensory information leading the organism to interpret and react implicitly, without thinking, to stimuli as if they were not mediated. Moving around to avoid a collision with a virtual wall is an objective example of such implicit illusion of non-mediation based on the integration of multiple sensory inputs that are consistent with internal representations. The illusion is possible because of the way the brain integrates perceptions. Stimulating more senses, improving the sense of agency and allowing more complex interactions with computer generated stimuli are all strategies that will increase core presence and be felt by the user at the level of core consciousness. All these operations occur at an automatic and implicit level (Beaumeister and Masicampo, 2010; Damasio, 2003; Morin, 2006; Schubert, 2009; Panksepp and Northoff, 2009; Waterworth et al., 2010). As mentioned by Biocca (1997), the experience of presence did not emerge with the arrival of VR. Riva (2009) and Waterworth et al. (2010) argued that presence can be experienced even when the experience is not mediated by technologies. Integration of perception is part of the normal functioning of core consciousness, is

234

S. Bouchard et al. / Interacting with Computers 24 (2012) 227–236

essential for the organism self-regulation, and can be altered in many non-mediated ways, from ingestion of various substances to dreams. But thanks to its multisensory nature, VR is quite an effective tool to manipulate representations of what is actually occurring to the organism. For example, in a classical experiment Hendrix and Barfield (1996) showed that simply adding head tracking during an immersion increases the feeling of presence. Increasing the number of stimuli that meaningfully inform the organism about its internal state, the state of the environment and how they interact together will strengthen the perceptual illusion of non-mediation. Research on objective realism during immersions thus corresponds to manipulating what is afforded to the senses and influences directly core presence. Also, ego-centric reference frame (first-person point of view) favors a sense of agency (Balconi, 2010; Gallager, 2000), which in turn is important for presence (Riva, 2009; Waterworth et al., 2010). Attention is also a contributing factor (Damasio, 1999; Waterworth et al., 2010), as is interaction (Slater et al., 2009a,b). However, core presence and core consciousness do not describe the meaning given to the experience. Meaning and a genuine sense of self-awareness are a third and more reflexive process (Damasio, 1999; Morin, 2006), associated with extended consciousness and extended presence. In Study 1, the experimental manipulation targeted only the meaning of the experience, without modifying interactions or the stimuli presented and integrated by the users’ sense. As much as proto presence is based on automatic perception of simple stimuli and core presence comes from implicit multisensory integration, the explicit and reflexive meaning given to the experience corresponds to the concepts of extended consciousness and extended presence. Extended consciousness provides an elaborated sense of self through the monitoring of the state of the organism in relation with its environment in the context of past and anticipated events. If the illusion created by multisensory integration is effective, the user immersed in VR will become aware that he or she was there in the VE, and also that he or she is the one causing the actions in the VE. Extended presence is associated with explicit judgments, as well as a sense of self-being and interacting in the VE. This sense of autobiographical self includes awareness of the location where the events occurred and were experienced, as it can be felt after an immersion in a VE or a dream, as well as other meanings including those relevant for social regulation (e.g., ‘‘my behavior in the VE was not appropriate’’), planning behaviors (e.g., ‘‘I must not do that next time’’), values (e.g., ‘‘I like the experience’’), judgments (e.g., ‘‘it was beautiful’’), etc. Discrepancies between physical and virtual realities will call for attention and meaning, and what some call breaks in presence (Garau et al., 2008). Discrepancies may come from minor details revealing weaknesses in the realism of the stimuli or the interactions, from intrusive thoughts about being in a mediated experiment, or from awareness of the physical reality. Providing meaning about ‘‘real or fake’’ – about whether the experience is mediated or not – occurs after the operation of basic perception (proto presence) and implicit internal representations of the interactions between the stimuli and the senses (core presence), through a third-order and self-reflexive process influenced from multiple inputs. Narrative manipulating the context of the experience, and hence subjective realism, is one of these inputs that provides meaning and would contribute to extended presence. When the user becomes aware that his or her basic senses have been fooled into believing the VE was true, the surprise and ‘‘wow reaction’’ are examples of cognitive dissonance at the explicit and reflexive level of extended consciousness. Similar reflexive phenomena occur during immersions in augmented reality, where the synthetic stimuli are integrated by the user as if they were real, but without the illusion of spatial location. In augmented reality, the spatial location displayed is the physical reality, but the user may describe the virtual

stimuli as ‘‘really there’’ and therefore show signs of core presence. In light of the concepts of core and extended presence, what are the practical implications of the data reported in this article? There are consequences for the assessment of presence and for considerations when developing powerful mediated experiences. Behavioral indices of presence could measure well core presence, but would not be as efficient in measuring extended presence as subjective measures (Schubert, 2009). However, self-report measures could provide crude information about the users’ automatic responses at the sensorimotor level such as basic perceptions (proto presence) and at the level of multisensory integration based on plausible interactions with the VE (core presence), and would contribute significantly to differentiate what is occurring at the level of explicit significance of the events to the user (extended presence). Assessing reflexive interpretation of what happens during the immersion should be broader than focusing only on spatial location and include other judgments about what was felt as ‘‘real’’ in the VR. Replicating the experimental manipulation used in Study 1 and Study 2 may provide information about extended presence but will miss core presence. Probably as much as manipulating emotions associated with the immersion would (Bouchard et al., 2008; Bouchard and Labonté-Chartrand, 2011; Riva et al., 2007). Replications of Study 1 should therefore carefully measure the continuum of presence instead of only extended presence. The brain imaging data observed in Study 2 is consistent with the meaning given to the place visited and to extended presence. But again, further experimental manipulations should include blocks where core presence is manipulated. Brain imaging studies, such as Baumgartner et al., (2008) or Clemente et al. (2011) should take into account the different layers of consciousness when interpreting brain areas activated in VR because some are associated with basic perceptions, other with more complex multisensory integration and with meanings. Baumgartner et al. (2008) manipulation may have elicited core presence, although the two experimental conditions used in their study differed in many aspects, but it is unclear to what extent it elicited extended consciousness. The bottom-up processes on the presence and consciousness continuums are most likely bidirectional. Extended presence may exert some influence on the more automatic and implicit processes, but only indirectly and to a lesser degree than raw perceptions and multisensory integration. Our findings suggest to developers of virtual environments that to obtain powerful results at the level of core presence, it is more fruitful to improve the perceptual illusion through multisensory manipulations and probably pay less attention to reflexive and meaning-related processes. On the other hand, developers aiming at more explicit judgments about suspended disbeliefs, impressions of being transported in another location, or judgments about the realism of the experience may want to invest in subtle details that influence the meaning of the experience, including contextual narratives, emotions and mood state, user’s apprehensions and expectations, etc. In providing narratives, it is interesting to leave room for the user to fill in missing details as it facilitates appropriation of the experience. Therapists facing challenges to induce presence in a VE may want examine the level where the problem occurs in the patient’s presence continuum. If it is at the level of core presence, the immersion may need to be augmented with more plausible stimuli (e.g., more realistic objects or better HMD), with better and richer interactions (e.g., better tracking), or by stimulating more senses (e.g., adding olfaction). If the challenges occur at the level of extended presence, the therapist may target patient’s internal representations of the immersive experience, the narrative contextualizing the immersion, expectations or negative beliefs that would hinder presence, openness to new experiences, etc.

S. Bouchard et al. / Interacting with Computers 24 (2012) 227–236

Clearly, both studies need to be replicated with more elaborated measures of presence. In the case of Study 1, a sample including more males would facilitate generalization of the results to both genders and in Study 2 a larger sample would be required. But the results are consistent with what is experienced by therapists using VR to treat mental disorders (Bouchard et al., 2007a; Wiederhold and Wiederhold, 2005). Patients suffering from anxiety disorders who keep telling themselves during an immersion that the stimuli are fake and they are playing a game are more difficult to treat with VR. Telling a patient immersed in VR to ‘‘click on the mouse button to walk in the virtual room’’ is more detrimental to extended presence than saying ‘‘walk in the room’’. Providing a narrative to a patient suffering from a specific phobia of flying suggesting she is ‘‘in an airplane departing from Ottawa for her vacations in Europe’’ and that ‘‘the pilot decided to go ahead with that airplane despite the repairs done recently on the engine’’ enhances extended presence much more than reminding the patient that the entire experience is virtual. It may indirectly influence core presence by facilitating how information from multiple senses is integrated and interpreted by the core self. The same can be said about attributing different meaning and intentions to virtual humans used in the treatment of social phobia. Saying that the ‘‘first avatar on the right is a really important person’’ or ‘‘is your boss’’ has a strengthening impact on extended presence. The perceive realism of virtual humans may also impact how people will emotionally react when interacting with the avatars of a person they know, either in a virtual environment or on social networks, no matter how photorealistic the avatar is. From a slightly different stand point, using deception to manipulate extended presence without modifying the stimuli may be used in further fMRI studies to differentiate self-location and meaning given to experiences, and therefore improve our understanding of the role of the paraphippocampus gyrus. It will help exploring with brain imaging techniques the areas of the brain potentially associated with core presence versus extended presence. For example, an fMRI experiment contrasting immersions in virtual environments with or without stereoscopy would implicitly activate different areas related to vision and depth perception. But if the effects specific to each sense can be documented and then extracted from the general impact of multisensory integration, it may be possible to assess brain areas associated with core presence and extended presence. To conclude, further research should experimentally manipulate stimuli, interactions, and meanings to address (a) first-order reactions automatically and implicitly experienced in a virtual environment due to basic automatic perceptual processes, (b) more complex second-order reactions that are still implicit but based on the integration of sensory information, and (c) third-order self-reflexive explicit processes about the significance to the self of what is occurring during the immersion in a VE. These preliminary findings suggest that further elucidating factors that affect the subjective experience occurring in virtual reality may not only lead to more effective psychotherapeutic applications but that VR methodological paradigms paired with functional imaging technology may offer a unique platform from which to study indeterminate and long debated concepts such as consciousness and presence.

Acknowledgements The authors want to thank Hugh Baras for his input on the idea of manipulating subjective realism during a discussion at the ABCT conference in Reno. This project was supported in part by grants from the Canada Research Chairs program, the Canadian Foundation for Innovation and NSERC awarded to the first author. Portions

235

of Study 2 have been described in the proceedings of the Presence conference in 2009.

References Balconi, M., 2010. Neuropsychology of the Sense of Agency: From Consciousness to Action. Springer-Verlag Italia, Milan. Bar, M., Aminoff, E., Schacter, D.L., 2008. Scenes unseen: the parahippocampal cortex intrinsically subserves contextual associations, not scenes or places per se. Journal of Neuroscience 28, 8539–8544. Baumgartner, T., Speck, D., Wettstein, D., Masnari, O., Beeli, G., Jäncke, L., 2008. Feeling present in arousing virtual reality worlds: prefrontal brain regions differentially orchestrate presence experience in adults and children. Frontiers in Human Neuroscience 2, Article 8. Beaumeister, R.F., Masicampo, E.J., 2010. Conscious thought is for facilitating social and cultural interactions: how mental simulations serve the animal–culture interface. Psychological Review 117 (9), 945–971. Biocca, F., 1997. The cyborg’s dilemma: progressive embodiment in virtual environments. Journal of Computer-Mediated Communication 3 (2). Blanke, O., Landis, T., Spinelli, L., Seeck, M., 2004. Out-of-body experience and autoscopy of neurological origin. Brain 127, 243–258. Bouchard, S., Labonté-Chartrand, G., 2011. Emotions and the emotional valence afforded by the virtual environments. In: Kim, J.-J. (Ed.), Virtual Reality. InTech, Croatia, pp. 501–514 (Chapter 24). Bouchard, S., Robillard, G., St-Jacques, J., Dumoulin, S., Patry, M.J., Renaud, P., 2005. Reliability and validity of a single-item measure of presence in VR. In: IEEE International Workshop on Haptic Audio Visual Environments and their Applications, Ottawa, October. Bouchard, S., Côté, S., Richard, D.S., 2007a. Virtual reality applications of exposure. In: Richard, D.S., Lauterbach, D. (Eds.), Handbook of Exposure. Academic Press, New York, pp. 347–388 (Chapter 16). Bouchard, S., Robillard, Renaud, P., 2007b. Revising the factor structure of the simulator sickness questionnaire. Annual Review of CyberTherapy and Telemedicine 5, 117–122. Bouchard, S., St-Jacques, J., Robillard, G., Renaud, P., 2008. Anxiety increases the sense of presence in virtual reality. Presence: Teleoperators and Virtual Environments 4, 376–391. Bouchard, S., St-Jacques, J., Renaud, P., Wiederhold, B.K., 2009. Side effects of immersions in virtual reality for anxious people. Journal of Cybertherapy and Rehabilitation 2, 127–137. Bouchard, S., Guitard, T., Bernier, F., Robillard, G., 2010. Virtual reality and the training of military personnel to cope with acute stressors. In: Brahanam, S., Jain, L.C. (Eds.), Advances Intellectual Paradigms in Healthcare. Springer-Verlag, Inc., New York, NY, pp. 109–128. Clemente, M., Rodríguez, A., Rey, B., Rodríguez, A., Baños, R.M., Botella, C., Alcaniz, M., Ávila, C., 2011. Analyzing the level of presence while navigating in a virtual environment during an fMRI scan. Lecture Notes in Computer Science, LNCS 6949 (4), 475–478. Damasio, A., 1999. The Feeling of What Happens: Body and Emotion in the Making of Consciousness. Harcourt Inc., Orlando. Damasio, A., 2003. Looking for Spinoza: Joy, Sorrow and the Feeling Brain. Harcourt Inc., Orlando. Epstein, R.A., Parker, W.E., Feiler, A., 2007. Where am I now? Distinct roles for parahippocampal and retrosplenial cortices in place recognition. Journal of Neurosicence 27 (23), 6141–6149. First, C.D., Spitzer, R.L., Gibson, M., Williams, J.B.W., 2002. Structured Clinical Interview for DSM-IV-TR Axis I Disorders, Research Version, Non-patient Edition (SCID-I/NP). Biometrics Research, New York State Psychiatric Institute, New York. Foa, E.B., Kozak, M.J., 1986. Emotional processing of fear: exposure to corrective information. Psychological Bulletin 99, 20–35. Freeman, J., Avons, S.E., Pearson, D.E., Ijsselsteijn, W.A., 1999. Effects of sensory information and prior experience on direct subjective ratings of presence. Presence: Teleoperators and Virtual Environments 8, 1–13. Gallager, S., 2000. Philosophical conceptions of the self: implications for cognitive science. Trends in Cognitive Sciences 4, 14–21. Garau, M., Friedman, D., Ritter Widenfeld, H., Antley, A., Brogni, A., 2008. Temporal and spatial variations in presence: qualitative analysis of interviews from an experiment on breaks in presence. Presence 17 (3), 293–309. Goldstein, E.B., 2007. Sensation and Perception. Wadsworth, Belment, CA. Graziano, M.S.A., 1999. Where is my arm? The relative role of vision and proprioception in the neuronal representation of limb position. Proceedings of National Academy of Science United States of America 96, 10418–10421. Held, R., Durlach, N., 1992. Telepresence. Presence: Teleoperators and Virtual Environments 1, 109–112. Hendrix, C., Barfield, W., 1996. Presence within virtual environments as a function of visual display parameters. Presence: Teleoperators and Virtual Environments 5, 274–289. Hoffman, H.G., Hollander, A., Schroder, K., Rousseau, S., Furness, T.A., 1998. Physically touching and tasting virtual objects enhances the realism of virtual experiences. Virtual Reality: Research, Development and Application 3, 226– 234. Ijsselsteijn, W.A., de Ridder, H., Freeman, J., Avons, S.E., Bouwhuis, D., 2001. Effects of stereoscopic presentation, image motion, and screen size on subjective and

236

S. Bouchard et al. / Interacting with Computers 24 (2012) 227–236

objective corroborative measures of presence. Presence: Teleoperators and Virtual Environments 10, 298–311. Jäncke, L., Cheetham, M., Baumgartner, T., 2009. Virtual reality and the role of the prefrontal cortex in adults and children. Frontiers in Human Neuroscience 3 (1), 52–59. Kennedy, R.S., Lane, N.E., Berbaum, K.S., Lilienthal, M.G., 1993. Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness. International Journal of Aviation Psychology 3, 203–220. Lenggenhager, B., Tadi, T., Metzinger, T., Blanke, O., 2007. Video ergo sum: manipulating bodily self-consciousness. Science 317 (5841), 1096–1099. Lin, J.J.-W., Duh, H.B., Albi-Rached, H., Parker, D.E., Furness, T.A., 2002. Effects of field of view on presence, enjoyment, memory and simulator sickness in a virtual environment. Proceedings of IEEE Virtual Reality 2002, 164–171. Lombard, M., Ditton, T., 1997. At the heart of it all: the concept of presence. Journal of Computer-Mediated Communication 3, 1–37. Makin, T.R., Hilmes, N.P., Ehrsson, H.E., 2008. On the other head: dummy hands and personal space. Behavioural Brain and Research 191, 1–10. Morin, A., 2006. Levels of consciousness and self-awareness: a comparison and integration of various neurocognitive views. Consciousness and Cognition 15, 358–371. Mouritsen, H., Frost, B.J., 2002. Virtual migration in tethered flying monarch butterflies reveals their orientation mechanisms. PNAS 99 (15). Oldfield, R.C., 1971. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9, 97–113. Panksepp, J., Northoff, G., 2009. The trans-species core SELF: the emergence of active cultural and neuro-ecological agents through self-related processing within subcortical–cortical midline networks. Consciousness and Cognition 18, 193–215. Petkova, V.I., Ehrsson, H.H., 2008. If I were you: perceptual illusion of body swapping. PLoS ONE 3 (12), e3832. Petkovas, V.I., Khoshnevis, M., Ehrsson, H.H., 2011. The perspective matters! Multisensory integration in egocentric reference frames determines full-body ownership. Frontiers in Psychology 2 (35). Pratt, D.R., Zyda, M., Kelleher, K., 1995. Virtual reality: in the mind of the beholder. IEEE Computer 28, 17–19. Riva, G., 2009. Is Presence a Technology Issue? Some Insights from Cognitive Sciences, vol. 13(3). Springer-Verlag London Limited. Riva, G., Waterworth, J.A., Waterworth, E.L., 2004. The layers of presence: a biocultural approach to understanding presence in natural and mediated environments. CyberPsychology & Behavior 7 (4), 402–416. Riva, G., Mantovani, F., Capideville, C.S., Preziosa, A., Morganti, F., Villani, D., Gaggioli, A., Alcaniz, M., 2007. Affective interactions using virtual reality: the link between presence and emotions. CyberPsychology & Behavior 10 (1), 45– 56. Rothbaum, B.A., Hodges, L.F., Watson, B.A., Kessler, G.D., Opdyke, D., 1996. Virtual reality exposure therapy in the treatment of fear of flying: a case report. Behavior Research and Therapy 34, 477–481.

Sadowski, W., Stanney, K., 2002. Presence in virtual environments. In: Stanney, K.M. (Ed.), Handbook of Virtual Environments: Design, Implementation, and Applications. Lawrence Elbaum Associates, Publishers, Mahwah, NJ, pp. 791– 806. Sanchez-Vives, M.V., Spanlang, B., Frisoli, A., Bergamasco, M., Slater, M., 2010. Virtual hand illusion induced by visuomotor correlations. PLoS ONE 5 (4), e10381. Schubert, T.W., 2009. A new conception of spatial presence: once again, with feeling. Communication Theory 19, 161–187. Sempo, G., Depickère, S., Amé, J.-M., Detrain, C., Halloy, J., Deneubourg, J.-L., 2006. Intergration of an autonomous artificial agent in an insect society: experimental validation. Lectures Notes in Computer Science, 4095 LNAI, 703–712. Slater, M., Usoh, M., 1993. Representation systems, perceptual position, and presence in immersive virtual environment. Presence: Teleoperators and Virtual Environments 2, 221–233. Slater, M., Lotto, B., Arnold, M.M., Sanchez-Vives, M.V., 2009a. How we experience immersive virtual environments: the concept of presence and its measurement. Anuario de Psicologia 40 (2), 193–210. Slater, M., Khanna, P., Mortensen, Jesper., Yu, I., 2009b. Visual realism enhances realistic response in an immersive virtual environment. Computer Graphics & Applications 29, 76–84. Slater, M., Spanlang, B., Sanchez-Vives, M.V., Blanke, O., 2010. First person experience of body transfer in virtual reality. PLoS ONE 5 (5), e10564. Takalo, J., Piironen, A., Honkanen, A., Lempea, M., Aikio, M., Tuukkanen, T., Vahasoyrinki, M., 2012. A fast and flexible panoramic virtual reality system for behavioural and electrophysiological experiments. Scientifi Reports 2, Article 324 (published on-line). Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., Mazoyer, B., Joliot, M., 2002. Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage 15 (1), 273–289. Van der Hoort, B., Guterstam, A., Ehrsson, H.H., 2011. Being Barbie: the size of one’s own body determines the perceived size of the world. PLoS ONE 6 (5), e20195. Waterworth, J.A., Waterworth, E.L., Mantovani, F., Riva, G., 2010. On feeling (the) present. Journal of Consciousness Studies 17 (1–2), 167–188. Wiederhold, B.K., Wiederhold, M.D., 2005. Virtual Reality Therapy for Anxiety Disorders. Advances in Evaluation and Treatment. American Psychological Association Press, Washington, DC. Witmer, B.G., Singer, M.J., 1994. Measuring Immersion in Virtual Environments. ARI Technical Report 1014. Alexandria, VA: US Army Research Institute for the Behavioral and Social Sciences. Witmer, B.G., Singer, M.J., 1998. Measuring presence in virtual environments: a presence questionnaire. Presence 7, 225–240. Youngblut, C., 2007. What a Decade of Experiments Reveals about Factors that Influence the Sense of Presence: Latest Findings. Institute for Defense Analyses, VA, Virginia.