Predicting presence: Constructing the Tendency toward Presence Inventory

ARTICLE IN PRESS

Int. J. Human-Computer Studies 67 (2009) 62–78 www.elsevier.com/locate/ijhcs

Predicting presence: Constructing the Tendency toward Presence Inventory Carol A. Thornson, Brian F. Goldiez, Huy Le Institute for Simulation and Training, University of Central Florida, 3100 Technology Parkway, Orlando, FL 32826, USA Received 18 December 2007; received in revised form 23 August 2008; accepted 26 August 2008 Communicated by S. Greenberg Available online 29 August 2008

Abstract We used a rational-empirical approach to construct the Tendency toward Presence Inventory (TPI), constructing scales to measure the individual difference human factors hypothesized to predict a person’s tendency to experience the cognitive state of presence. The initial pool of 105 items was administered to 499 undergraduate psychology students at a university in the southeastern United States in order to empirically validate the underlying factor structure associated with this tendency. Six factors were derived, resembling the original conceptual model. Construct validity and reliability evidence are presented. Future empirical work is needed to explore the criterion and predictive validities of the factors constituting this inventory. r 2008 Elsevier Ltd. All rights reserved. Keywords: Presence; Telepresence; Virtual reality; Augmented reality; HCI; Test validation

1. Introduction For humans, a sense of presence is a phenomenon enabling people to interact with, and feel connected to, the world outside their physical bodies. It is defined as a person’s subjective sensation of being there in a scene depicted by a medium, usually virtual in nature (Barfield et al., 1995). Our definition of presence is adopted from the one in use by the International Society of Presence Research, where ‘‘presence (a shortened version of the term ‘‘telepresence’’) is a psychological state or subjective perception in which even though part or all of an individual’s current experience is generated by and/or filtered through human-made technology, part or all of the individual’s perception fails to accurately acknowledge the role of the technology in the experience’’ (ISPR, 2000). Thus, presence is a psychological phenomenon that occurs in the human mind and not in the specific technology. The technology is merely a means used to arrive at this state of mind. In fact, awareness of the technology actually hinders Corresponding author. Tel.: +1 407 430 2402; fax: +1 407 977 2954.

E-mail address: [email protected] (C.A. Thornson). 1071-5819/$ - see front matter r 2008 Elsevier Ltd. All rights reserved. doi:10.1016/j.ijhcs.2008.08.006

the experience of presence because presence involves the perceptual illusion of non-mediation, whereby the techno logy becomes invisible and is unnoticed by the user (Lombard and Ditton, 1997). Experiencing presence in a virtual environment (VE) is analogous to viewing a mountain scene through a clear, clean window—one does not notice the window (i.e., the technology) itself. In fact, if one’s focus were on the medium (e.g., the window), it would take away from the experience and lessen his/her sense of ‘‘presence.’’ In this way, presence is best served when the medium is transparent. When the technology (e.g., window) obstructs the view beyond the window (i.e., the virtual or augmented experience), the sense of presence is reduced. Of course, it goes without saying that the technology used to design and deliver the virtual or augmented experience is of utmost importance; however, it is not the technology itself that engages the user but what goes on inside the user’s mind. Therefore, our position is that presence is ‘‘a property of an individual and varies across people and time; it is not a property of a technology or one of the technologies commonly referred to as a medium, although technologies or media with specific constellations

ARTICLE IN PRESS C.A. Thornson et al. / Int. J. Human-Computer Studies 67 (2009) 62–78

of characteristics are likely to evoke a similar set of presence responses across individuals and across time (e.g., an IMAX 3D presentation typically produces greater presence in viewers than a small television presentation)’’ (ISPR, 2000). Consequently, high-fidelity environments— those that look, sound, and react in the same way as environments in the physical world—are more likely to evoke a higher degree of presence. Yet, it is our contention that it is still the user’s perception that the VE matches the physical world that defines presence rather than any actual (real-world) correspondence between the two (Heeter, 2003). Therefore, in this way, the user’s sense of presence functions as a crucial facilitator in achieving the goals of virtual and augmented environments (van Schaik et al., 2004), which include how people might use robotic entities as extensions of the self. Presence research, therefore, has become a central issue in the engineering, design, and evaluation of interactive technology such as that found in augmented reality (AR) and telerobotics. This is why entire communities of engineers, designers, psychologists, and other researchers are collaborating to understand, measure, and engineer presence (Biocca, 1996) and its impact on various aspects of human performance. Because presence is a multidimensional concept involving complex psychological processes, the search for the important antecedents to evoking a sense of presence is a recurrent theme in the literature related to VEs, AR, and human–computer interaction (HCI). Such research has both theoretical and practical significance. With regard to practical significance, it is widely agreed that information systems can and should be designed to accommodate individual differences in human factors so that they may be better suited to the unique needs of the individual user. With advances in VE, and the more recent extension to AR, developing a deeper understanding of the user’s psychological make-up and cognitive demands when interacting with information systems is warranted. This is especially true in AR because virtual objects must be perceived as a contiguous feature of the environment. Thus, when interface designers better understand the benefits and difficulties a user might encounter when communicating through such an interface, the designer will be better able to adapt and tailor the interface to the individual, aiding the designers of virtual and augmented environments by targeting the key antecedents that function to evoke a sense of presence. 1.1. Our purpose Theoretical significance involves understanding the antecedents, or the predictors, of presence in order to better predict the environments that are likely to evoke such a state in different individuals. Only by taking this crucial first step in understanding individual differences in the tendency to experience presence can we hope to gain a handle on the multidimensional nature of this construct.

63

Although the literature is replete with theories proposing various environmental or human factors that may or may not evoke a sense of presence in the user (e.g., Heeter, 1992; Slater et al., 1994; Witmer and Singer, 1998; Lessiter et al., 2001), the diversity of the constructs proposed, as well as the lack of an integrative framework, have limited the development of a multidimensional inventory with strong psychometric and conceptual underpinnings that examines the antecedents to presence. Therefore, the purpose of this study is to propose and develop an inventory of human individual difference factors that (a) captures the multidimensional nature of the construct of presence by predicting important higher order constructs that are antecedent to, and predictive of, a user’s likelihood to experience a sense of presence; (b) paves the way for future presence researchers to test the criterion validity of this inventory against an empirically derived post-test or objective measures of the criterion, namely presence itself, within an environment designed to evoke such a state; and (c) establishes the foundation for the construct validation process of the resulting inventory. 2. Current theories of presence Again, our focus is to examine the hypothesized predictors, or antecedents, to presence, and not the higher order factors that make up the construct of presence itself. However, adding to the confusion is the fact that this relatively new field approaches the conceptualization of presence from a variety of avenues, some more focused on the technology than others. Some theories emphasize the psychological and multidimensional nature of presence itself, such as that put forth by Heeter (1992). She proposed a model that includes three dimensions of presence: personal presence, social presence, and environmental presence. According to this researcher, personal presence is a measure of the extent to which a participant feels that he or she is actually inside the virtual (or remote) environment. It is the ‘‘sense of being there’’ and might be expressed by a user as, ‘‘It seemed as if I was someplace else!’’ (ISPR, 2000). She theorizes that there are individual differences in how and when presence is experienced because different people may pay attention to different stimuli, depending on past experience, current mood, and the nature of the stimuli (Heeter, 2003). According to this theory, presence also includes a representation of the self in the social world. Social presence refers to the extent to which the user experiences other beings (living or synthetic) in the virtual world, which beings to appear to react to the participant. It is often expressed as, ‘‘It seemed like we were really interacting!’’ (ISPR, 2000). Finally, in this theory, environmental presence represents the extent to which the environment itself appears to know that the user is there and reacts to him/her (Heeter, 1992). This theory has been an influential one in the field and is important to us in that it emphasizes the psychological nature of the factors related to presence. Although we do not seek to uncover

ARTICLE IN PRESS 64

C.A. Thornson et al. / Int. J. Human-Computer Studies 67 (2009) 62–78

the dimensions of presence itself, we hypothesize that the antecedents (i.e., individual difference human factors) that determine one’s tendency to experience the future state of presence are also psychological and multidimensional and it is this factor structure we are endeavoring to uncover. 2.1. Antecedents to presence Other researchers have looked into the antecedents that function individually or interact to evoke one’s sense of presence. One such theory put forth by Lessiter et al. (2001) involves interacting components that influence presence, and includes characteristics of the media or technology form (e.g., image size and quality, viewing distance, use of motion and color, audio volume and fidelity, obtrusiveness of technology, etc.); characteristics of the media or technology content (e.g., social realism, quality of writing, quality of acting, physical appearance of actors, fame of actors, and the nature of the task or activity); and characteristics of the user (e.g., willingness to suspend disbelief, knowledge of and prior experience with the technology, age, and gender). It is the latter type of antecedents, those that are associated with the user, that are of interest to us. As such, we leave the technology content and form to the expertise of the designers and engineers. Another influential theory regarding the predictors (e.g., antecedents) of presence was put forth by Witmer and Singer (1998), who define presence as a normal awareness phenomenon that requires directed attention and is based on the interaction between sensory stimulation, environmental factors that encourage involvement and enable immersion, and internal tendencies to become involved. Again, our focus is on the latter—the individual’s internal tendencies to experience the future state of presence. There are different definitions of immersion as well, and many researchers conceive of this concept as some technological aspect of sensory coupling with a VE (e.g., Bowman and Raja, 2004, among others). Though this view is certainly a valid one, our own definition of immersion is adopted from the International Society of Presence Research, which states that immersion occurs ‘‘when part or all of a person’s perception fails to accurately acknowledge the role of technology that makes it appear that s/he is in a physical location and environment different from her/his actual location and environment in the physical world.’’ Thus, we again are focusing on predicting the antecedents to the user’s subjective experience rather than on the specific technologies used. 3. Presence measurement There are two general approaches to measuring the degree of presence a user experiences: subjective and objective measures. Of course, our interest is in measuring not presence itself, but the predictors of presence (i.e., the individual difference factors that predict the tendency

toward presence). Because these are individual difference factors that are not situation-specific (but are specific to the individual user), they must be assessed via subjective measurement. We briefly review the advantages and disadvantages of objective and subjective measures, in general, and the ways that presence researchers have used each in assessing our eventual criterion of interest, presence itself. 3.1. Objective measures Objective or behavioral measures are carried out by observing the automatic responses of participants (Freeman et al., 2000). Experimenters observe any physiological and automatic, unplanned responses to stimuli. These measures are typically administered during, rather than before or after, the virtual experience. These automatic (and autonomic) responses might include measures of heart rate, skin conductance, blood pressure, muscle tension, respiration rate, ocular response, posture, and so on. The main advantage of objective measures, in general, is that they are free of self-report and recollection biases; however, there is probably not any way to measure such tendencies toward presence as an objective measure, though we certainly advocate the advancements being made in the field with using objective measures to assess presence itself (e.g., Slater et al., 2006). A disadvantage of objective measures is the lack of correlation with the degree of presence experienced subjectively by the user (Prothero et al., 1995); thus, there is difficulty in establishing criterion-related validity. According to these researchers who define presence as a subjective psychological state, it cannot be fully captured by objective measures alone (Sheridan, 1992). Because there needs to be a strong relationship between the measured responses and presence, objective measures may be appropriate only in certain contexts, such as under stress (Meehan et al., 2002), making it unlikely that a single objective measure of presence can be developed for us across all situations, though a new avenue using objective measures to assess breaks in presence seems promising (Slater et al., 2006). 3.2. Subjective measures Subjective measures are items contained in a questionnaire or inventory that inquire as to a user’s subjective experience. They may be conducted through pretest (measuring the tendency toward presence) and/or post-test (the user’s recollections of having experienced presence), though post-test measures engender more of the disadvantages of subjective measures, as outlined below (Slater et al., 1994; Witmer and Singer, 1998; Lessiter et al., 2001). Disadvantages of subjective measures, in general, include reliability problems, interpretation problems, lack of introspection, and demand characteristics (i.e., the wording of the items leads participants to predict the type of

ARTICLE IN PRESS C.A. Thornson et al. / Int. J. Human-Computer Studies 67 (2009) 62–78

responses expected of them). Reliability problems refer to unstable and inconsistent responses across participants, time, and study settings (see Freeman et al., 2000). Misinterpretation by participants occurs if items are too difficult for participants to understand or if the items explicitly refer to an unfamiliar concept (i.e., presence) without explaining what it means. Lack of introspection as well as faulty memory may also influence responses in unpredictable ways; thus, the responses may not be accurate reflections of the true experience (e.g., Freeman et al., 1999; Slater, 1999, 2004; Usoh et al., 2000). Nevertheless, most of the problems with subjective measures can be addressed and remedied with more careful and systematic design of the inventories as well as the prudent use of exploratory and confirmatory statistical analyses. The main advantages of subjective measures are that they are generally easy to administer and interpret and if psychometrically valid and reliable, are worthwhile measures to use. However, a universal inventory has yet to be developed and validated, resulting in few researchers using the same sets of items, making comparisons across studies extremely difficult, if not impossible. Therefore, a general call has gone out in the field for a pretest inventory which would be valid and reliable across different participant groups, experimental conditions, stimuli, and settings (e.g., Witmer and Singer, 1998; Lombard et al., 2000; Lessiter et al., 2001; Schubert et al., 2001). It is the goal of this research to try and answer this call. 3.3. Subjective pretest measures currently in use The ITC-Sense of Presence questionnaire (Lessiter et al., 2001) is a worthwhile attempt to develop a cross-media questionnaire and is based upon the previously discussed theory of Lessiter and colleagues. Other presence questionnaires (PQs) are usually created by researchers for measuring presence in the specific VEs in which they design or work; therefore, making comparisons across studies impossible. However, the focus on the technology necessitates this type of design. The reliability of such questionnaires is mostly unknown (see Youngblut and Perrin, 2002). Even though a vast number of PQs exist, many researchers continue to develop their own questionnaire to suit their own needs as no single questionnaire has yet been shown to be widely accepted or valid. Another reason we chose not to focus on the technology used, is stated eloquently by Slater (1999), who wrote, ‘‘For two such subjects one reports a high degree of ability to control events, and another reports a very low ability. Of course the system is the same in both cases. What has determined the difference in response is nothing at all to do with the immersive system, but is due to differences in the individuals, their experience, psychological make-up, dexterity, and so on. They each report a different immersive response’’ (Slater, 1999, p. 6). This is the very strength of the cross-media measure we are trying to develop, to differentiate individual differences that predict

65

presence from those that are associated with the specifics of the technology (media form, content, or task). The Witmer–Singer PQ (Witmer and Singer, 1998) has been widely used and serves as the basis for many of the questionnaires in use today. In keeping with their theory (that presence is based on the interaction between sensory stimulation, environmental factors that encourage involvement and enable immersion, and internal tendencies to become involved), the authors constructed an immersive tendencies questionnaire (ITQ), a pretest subjective instrument. The ITQ measures the tendency toward the psychological states of involvement and immersion in a wide variety of stimuli and experiences. This was the inspiration for our own Tendency toward Presence Inventory (TPI); however, unlike the ITQ, our focus is solely on the individual difference predictors of the tendency to experience presence and not on the environmental factors so much. Witmer and Singer followed this pretest with a post-test PQ. The results of four experiments showed that individual tendencies as measured by the ITQ predicted presence as measured by the PQ. However, according to Youngblut and Perrin (2002), there is no clear evidence thus far that the PQ is a valid measure of presence. That is, though it correlated with the ITQ, we do not know if it is really measuring presence itself. For example, Slater (1999), in his response to the PQ, also had some problems with the confounding of measures. Namely, because the ITQ was used to measure individual differences to assess immersive tendencies in predicting presence, and because the authors used the same individual differences (i.e., subjective responses) to assess various aspects of the system in the PQ, post-hoc, it seems that they confounded the predictor variable with the criterion variable. Slater (1999) does conclude, however, that he would use the ITQ, ‘‘since this stands alone as an attempt to measure important psychological characteristics of individuals.’’ Likewise, our TPI, like the ITQ, is also a pretest measure. However, by focusing on the individual difference predictors themselves, we attempt to amalgamate the best that each of the preceding pretest measures has to offer. Because of this focus, it is our hope that the TPI will have broader application across users, media, tasks, and technology, as further elucidated below. 4. Proposed inventory design In order to develop an inventory that is psychometrically valid and reliable—that measures what it purports to measure with a high degree of reliability and consistency— we adhered to the five steps of questionnaire design: conceptualization, construction, tryout, item analysis, and revision (Cohen and Swerdlik, 2002). Starting with the conceptualization phase, we reviewed the literature on the existing inventories and questionnaires that have been designed to measure presence, and such a review indicated that these measures could be improved upon in terms of

ARTICLE IN PRESS 66

C.A. Thornson et al. / Int. J. Human-Computer Studies 67 (2009) 62–78

psychometric soundness and construct-related validity. This served as the stimulus for developing our new measure (Cohen and Swerdlik, 2002, p. 189). Because the tendency to experience presence is most likely comprised of various individual difference factors, in order to tap as many possible manifestations of different content areas that may be relevant to this tendency, many more questions than needed were initially pilot-tested. Therefore, as large an item pool of questions as possible was used to ensure adequate content coverage. Moreover, according to Lessiter et al. (2001), questions should not make reference to specific media systems and properties; therefore, we did not include terms such as ‘‘virtual reality,’’ ‘‘VEs’’ ‘‘robotics’’ or ‘‘presence,’’ as these terms may have different, vague, or nebulous meanings for different participants, depending upon their individual backgrounds and experiences. We managed social desirability bias by instructions to avoid spending too long on any one question and by emphasizing that the first response is usually the best response. The results of the literature review on possible antecedents follow. 5. Proposed antecedents in the literature One area of individual difference factors, somewhat neglected in much of the research on presence is the area of personality traits. Because presence is a cognitive state (as well as having affective and behavioral components), it makes theoretical sense that personality variables influence the tendency toward such a state, as well as the motivation to focus upon or attend to the media or task at hand. Thus, individual differences in personality could prove promising avenues to explore. Indeed, some aspects of personality have been shown to be related, individually or in combination with other aspects, to influence the tendency toward presence. These factors are often examined in conjunction with cognitive factors, as when Sas and O’Hare, in 2003, studied absorption, creative imagination, empathy, and cognitive style in their experiments on presence. They found positive correlations with only two of the antecedents, however, namely the two personality traits—creative imagination and empathy—both of which were positively correlated with one’s tendency to experience presence. In addition to personality, several other individual differences have been hypothesized to evoke presence. For example, Stanney and Salvendy (1998) identified level of experience, human cognitive abilities, the ability to manipulate objects in a virtual world, personality characteristics, and age as possible significant influences in VE experiences from previous HCI studies. Thus, based upon a review of the literature, we also examined each of these antecedents, as well as others, for consideration in our initial item pool. The proposed factors that we hypothesized to be related to the tendency to experience presence are outlined and described below, as well as presented graphically in our model (see Fig. 1).

Psychomotor Involvement Cognitive Involvement (Passive)

Cognitive Involvement (Active) Spatial Ability

Presence

Ability to Construct Mental Models Visual Style Openness to Experience Introversion Empathy Fig. 1. Hypothesized tendency toward presence model.

5.1. Age Bangay and Preston (1998) conducted a study using 355 participants and found that subjects between the ages of 35 and 45 tended to have lower scores in presence than participants between the ages of 10 and 20. Schuemie et al. (2005) studied height situations in a VE with 41 participants and found no correlations between presence and absorption, gender, computer experience or the level of acrophobia; however, they did find a positive correlation between presence and age. It seems that the more complex a system is designed, the more influential the effects of age, especially if information from different sensory channels is to be integrated. Thus, we will examine age as one of our demographic variables in order to assess its correlation with the derived item scales (factors). 5.2. Level of experience The level of computer experience indicates whether a user is an expert or a novice and has been hypothesized to predict presence. However, it seems that experience may not exert a direct influence on one’s tendency toward presence, but the amount of experience a user has may be related to the user’s ability to perform and the manner in which the user understands and organizes task information. Thus, it seems logical that experience may relate

ARTICLE IN PRESS C.A. Thornson et al. / Int. J. Human-Computer Studies 67 (2009) 62–78

specifically to how users mentally represent a VE over time. This ability is believed to affect the perception of the navigational complexity and the benchmark performance of users (Egan, 1988; Dix et al., 1998). Another reason that experience is logically thought to be related to the tendency to experience presence is because having prior experience with something exerts an influence upon one’s expectations (Heeter, 2003). Knowing what to expect, it is theorized that users ‘‘experience a greater sense of presence in an environment if they are able to anticipate or predict what will happen next y’’ (Held and Durlach, 1992, p. 229). However, despite these predictions, there is very little empirical evidence suggesting any direct links. Some studies (e.g., Youngblut and Perrin, 2002; Jurnet et al., 2005) have found no link at all between computer experience and presence. Another issue to consider is whether computer experience includes task experience as well. In fact, Youngblut and Perrin (2002) examined over 80 presence-related studies in the literature and found no relationship, or a negative relationship, between game playing experience and presence, and between VE experience and presence. Thus, we did not include any direct experience items in our initial item pool. 5.3. Psychomotor ability Related to spatial orientation (another hypothesized predictor), but not extensively studied in the presence literature, is psychomotor ability. There is a clear link between enhanced spatial abilities and psychomotor task performance, especially in the surgical literature (e.g., Kyllonen and Chaiken, 2003; Keehner et al., 2004). Anecdotally, it seems that individuals who tend to engage in a variety of sports, martial arts, and/or who enjoy working with their hands tend to be higher in spatial orientation and spatial ability as well. Thus, for exploratory purposes, we included items related to psychomotor skills and abilities in our initial instrument; however, these items were eventually dropped from the final instrument following analysis of the data. 5.4. Cognitive involvement (passive) We define passive cognitive involvement as a cognitive state in which the person is fully engaged in what s/he is doing, characterized by a feeling of energized focus and full involvement in the activity. In 1999, Ban˜os et al. found that absorption (defined as the tendency to become involved or immersed in everyday events or the tendency to totally immerse oneself with the attentional objects) increased presence in VEs. In this way, presence has been theorized as relating to Csikszentmihalyi’s theory of flow (Trevino and Webster, 1992; Webster et al., 1993). Such an experience entails a ‘‘merging of action and awareness,’’ during which a person loses self-consciousness and a sense of time, focusing on the present, while ‘‘blocking out the

67

past and the future’’ (Csikszentmihalyi, 1982). Flow theory describes the experience of those who participate in activities they find rewarding, activities where abilities and skills match the challenges encountered. Stress results when challenges (demands) exceed one’s abilities and skills (resources); however, if one’s resources exceed the demands (challenges), boredom results. Both boredom and stress undermine, rather than enhance a sense of absorption; therefore, both are theorized to influence presence. As such, we included several items that ask respondents’ about their general tendency to experience this type of passive involvement across different kinds of activities. 5.5. Cognitive involvement (active) Active involvement is the tendency to become involved and immersed in tasks that involve decision-making and judgment, where the user intentionally (actively) tunes out distractions (internal and external) in order to focus attention on the experience at hand. Interesting, according to Ijsselsteijn (2005), interactivity appeared to be a more important factor than immersion in VEs. This is shown when interactive yet unrealistic displays are able to engender substantial levels of presence. Thus, ‘‘being there’’ has been considered by some as related to the ability to ‘‘do there.’’ Unlike flow and passive involvement, the user is actively engaged with the task (e.g., videogame, activity). Such selective attention involves actively tuning out and ignoring internal distractions (personal problems) as well as external distractions (the environment) from diverting one’s attentional resources away from the experience (Barfield and Weghorst, 1993; Draper et al., 1998). Several researchers have studied the tendency to become actively involved and its relation to presence. It has been hypothesized that those individuals who are motivated to actively engage in the VE will experience a greater sense of presence (Witmer and Singer, 1998) due to the allocation of more attentional resources from working memory to the experience itself. In fact, the ITQ includes several items designed to measure this tendency to become involved or absorbed and several of the items in our inventory were inspired by these. We hypothesize that the user must be actively willing to focus (Slater and Usoh, 1993) and to become absorbed (Lombard and Ditton, 1997) as prerequisites for experiencing presence. 5.6. Spatial orientation Spatial orientation is related to spatial ability, and has also been studied as it relates to evoking a sense of presence in the virtual world. Spatial orientation refers to our natural ability to maintain our body orientation and/or posture in relation to the surrounding environment (physical space) at rest and during motion. Research suggests that individuals who score low on spatial memory tests generally have longer mean execution times and more

ARTICLE IN PRESS 68

C.A. Thornson et al. / Int. J. Human-Computer Studies 67 (2009) 62–78

first-try errors, particularly system navigation issues. In HCI, differences in spatial orientation lead to certain users performing more efficiently than others at information search and information retrieval. This performance difference does not mean that users with lower spatial orientation cannot find information, but that they tend to be slower at finding it. The difficulties experienced by low spatial memory have been regarded as being particularly related to system navigation issues during an information search, but such difficulties disappear when a hierarchical structure is presented as a completely explicit 2D visual hierarchy with no hidden structure layers (Stanney and Salvendy, 1995). In fact, some researchers have suggested that spatial orientation is more relevant than either computer or VE experience alone in predicting the ability to construct mental models, which we also hypothesize to be predictive of one’s tendency to experience presence (Jurnet et al., 2005; Schuemie et al., 2005).

In Slater and Usoh’s (1993) study, the researchers used the therapeutic technique known as Neuro Linguistic Programming (NLP) to characterize the user’s psychological representational and perceptual systems. Their findings suggest that the degree of visual dominance was positively correlated with a sense of presence; users who preferred the auditory representational system experienced less presence. In a similar study, Slater et al. (1994) exposed 24 subjects to a VE and confirmed that users who preferred the visual or the kinesthetic representational system (in this case, if a virtual body was included) experienced higher levels of presence. In their 1995 study using eight participants, Slater et al. found that including dynamic shadows in a VE only resulted in a higher degree of presence in those individuals who preferred the visual representational system. Thus, we also included items assessing if individuals were visual, as opposed to auditory, learners. However, these items were eventually dropped following statistical analyses of the data.

5.7. Ability to construct mental models 5.9. Openness to experience Another perceptually related ability is the ability to construct accurate mental models. A mental model is an internal scale-model representation of an external reality. This ability has been shown to predict certain types of task as well as team performance (Smith-Jentsch et al., 2001). As described above, it seems that computer experience matters mostly when it predicts the ability to construct an accurate mental representation of the self as spatially surrounded by the VE (i.e., literally, a sense of ‘‘being there’’). Being able to construct mental models of the self has been studied as ‘‘embodied cognition,’’ a mental representation of bodily actions as possible actions inside a virtual world. When movements of one’s own body (or body parts) in the VE are represented mentally as possible actions, presence emerges (Schubert et al., 2001). The user is able to project the self more easily into the VE and to interact with other objects and beings (Regenbrecht et al., 1998). Loomis (1992) refers to this same ability as ‘‘distal attribution.’’ This is compared to the phenomenological concept of how a tool or a vehicle can become an extension of one’s own body when it is used often enough, even though it is physically not a part of the body, (Draper et al., 1998). Thus, we felt it important to include items related to this important cognitive ability as it relates to the tendency toward presence in VEs. 5.8. Visual learning style According to the spatial-functional model of Schubert et al. (2001), having a visual learning style is predictive of experiencing a sense of presence. Specifically, some suggest that those individuals whose preferred and habitual approach to organizing and representing information as visual may be more likely to experience a higher degree of presence than verbal or auditory learners (Riding, 1998).

Openness to Experience describes a dimension of personality that distinguishes imaginative, creative people from down-to-earth, conventional people. As elucidated above, the literature suggests that creative imagination, a main facet of openness, is related to presence. For instance, Sas and O’Hare (2003) found that creative imagination was positively related to presence and theorized that individuals must utilize their imaginations in the suspension of disbelief often required in VE or AR. Thus, being open to new experiences might be related to one’s ability and willingness to suspend disbelief and imagine themselves as part of whatever virtual or augmented world in which designers of such environments wish to place them. 5.10. Introversion People who are introverted are energized and excited when they are involved with the ideas, images, memories, and reactions that are a part of their inner world. Introverts often prefer solitary activities or spending time with one or two others with whom they feel an affinity. They also tend to have a calming effect on those around them. With their orientation toward the inner world, introverts truly like the idea of something, often better than the something itself; ideas are almost solid things for them. Thus, introverts are considered to be more reflective in nature than their more extraverted peers (Myers, 1980). For the foregoing reasons, it has been hypothesized that Introverts may be better able to suppress distracting sensory information than extraverts due to their reflective natures as well as their tendency to have narrower ranges of attention (Althaus et al., 2004). It seems they may be better able to suppress conflicting sensory inputs such as the stimuli of the hardware or the stimuli of the real world as well as to construct mental models of the virtual space.

ARTICLE IN PRESS C.A. Thornson et al. / Int. J. Human-Computer Studies 67 (2009) 62–78

The suppression of conflicting stimuli and the allocation of attention to the virtual stimuli are both believed to be related to one’s tendency to experience a sense of presence (Witmer and Singer, 1998; Schubert et al., 2001). Research has in fact shown that those individuals who score higher in Introversion will tend to experience higher levels of presence (Sas et al., 2004; Jurnet et al., 2005). 5.11. Empathy Empathy involves the ability to adopt another’s psychological point of view and to experience the emotions observed in the experience of others. In a study by Sas and O’Hare (2003), it was found that participants higher in empathy, namely, those who were Feeling types (as measured through the Myers–Briggs Type Indicator (MBTI)) were then able to experience a higher degree of presence. One year later, Sas et al. (2004) confirmed the 2003 study that users scoring higher in Feeling, as measured through the MBTI, had a greater tendency to experience a deeper level of presence. Therefore, the MBTI has been used to predict which individuals would be likely to have the capacity to feel for other characters in the VE and thus, will tend to experience a greater sense of presence than others. As such, we also included items adapted from the MBTI to assess Empathy in our inventory as well. 6. Research methodology We followed the construct validation approach to inventory development to develop the scales of interest, as advocated by psychometricians (e.g., Nunnally and Bernstein, 1994; Clark and Watson, 1995), we adhered to the protocol regarding the five steps of questionnaire design (Cohen and Swerdlik, 2002). Accordingly, following our review of the literature, we derived a conceptual model of the factors we believe are related to an individual’s tendency to experience presence (see Fig. 1). The scale development process began by using this conceptual model as a foundation. Specifically, we first generated items representing each of the proposed factors in the conceptual model. After discussion and revision by the first two study authors, the 105 items were administered to a sample of college students (N ¼ 306). We carried out an exploratory factor analysis (EFA) to empirically examine the factors underlying these items. Items were screened on the basis of their rotated factor matrices. Next, confirmatory factor analysis (CFA) was implemented on data from a new sample (N ¼ 193) to (a) confirm the factor structure and (b) reselect the items. The process is outlined in further detail below. 6.1. Item generation procedure An initial item pool was constructed, based upon a review of the literature as well as the adaptation and

69

incorporation of existing validated scales used to measure several of the personality constructs in our conceptual model (e.g., Introversion, Openness, Empathy). For each of the other constructs, the first author, an advanced doctoral student in industrial and organizational psycho logy, with a background in psychometrics, wrote the items. After generating items independently, meetings were held with the second author, whose expertise lay in modeling and simulation, in order to discuss which items were to be retained and/or revised. This was done to ensure content coverage as well as to ensure that all items were phrased carefully, simply, and unambiguously, as recommended by psychometricians (Rust and Golombok, 1989). This procedure yielded an initial item pool of 105 items assessing the nine different hypothesized factors related to one’s tendency to experience a sense of presence. We then randomized the order of the 105 items, representing the nine hypothesized constructs, for administration purposes. 6.2. Inventory design A five-point Likert scale was used throughout the inventory, facilitating both scoring and the respondents’ ability to complete the entire inventory in a timely manner. Several items were reverse-coded in order to discourage random responding as well as to enhance validity. Participants rated the extent to which they completely disagreed (1) to completely agreed (5) with each statement. With regard to the problem with Likert scales rendering ordinal response data rather than interval data, and therefore not being amenable to advanced statistical analysis (Slater and Garau, 2007), we followed the current thinking in psychometrics, namely that ‘‘anchors’’ (descriptors underneath the scale numbers) should only be placed on the ends of the scale and not underneath each number (1–5). This way, any differences in the ratings are more likely to be due to ‘‘true’’ differences inferred by the respondents rather than due to unique individual differences in the interpretation of the descriptors, as different people have different interpretations for words such as ‘‘almost’’ or ‘‘somewhat’’ or ‘‘very’’ in their own minds. Social desirability bias was managed with instructions to avoid spending too long on any one question and by emphasizing that the first response is usually the best response. 6.3. Procedure Having created a large pool of items, it was necessary to administer the questionnaire to as large a pool of participants as possible. After applying for approval through the University of Central Florida’s Internal Review Board, the inventory was administered online via the university’s Research Participation website, Sona Systems.

ARTICLE IN PRESS 70

C.A. Thornson et al. / Int. J. Human-Computer Studies 67 (2009) 62–78

6.4. Participants Participants were undergraduate students at the University of Central Florida in Orlando, who were enrolled in an undergraduate psychology class. In addition, the students had to be registered on the Psychology Department’s online Research Experience website (Sona Systems). All participants were age 18 and over, and volunteered to participate for extra course credit as determined by their instructors. Data were collected online through Sona Systems, where anonymity was maintained as each participant is identified only through a randomly generated user ID code. Informed consent was provided. Extra credit was awarded for completion of the inventory, which took participants approximately 15–20 min to complete. The total number of completed inventories was 499. Participants were mostly female (71.9%) and mostly 18–20 years of age (81.6%), but ranged in age from 18 to over 50 years in age. 7. Analysis 7.1. Step 1: exploratory factor analysis Factor Analysis is a statistical technique used to (1) estimate factors or latent variables, or (2) reduce the dimensionality of a large number of variables to a fewer number of factors. We employed EFA in order to uncover the underlying structure of our data. These analyses were carried out on approximately two thirds of the total sample (N ¼ 306). We used SPSS Version 12.0 and specified principal-axis factoring (PAF) as the extraction method (cf. Gorsuch, 1983; Nunnally and Bernstein, 1994). Principle components analysis (PCA) and PAF differ in several ways; thus, the decision on which one to use should be guided by the analyst’s purposes. PCA is a method of factoring a correlation matrix directly, without estimating communalities and as such, it takes into account all sources of variance including common, specific, and error variance. This analysis is known to overestimate reliability; thus, it is mostly used for measuring things such as height, distance, etc. Physical scientists use this approach more often. PAF, on the other hand, is a method of factor analysis where successive factors are extracted to explain the most variation in a set of variables but PAF only takes into account common sources of variance. That is, PAF works with the reduced correlation matrix, after subtracting out the unique variance of each input. It does not assume perfectly correlated variables and uses covariances rather than correlation coefficients. This analysis is known to underestimate reliabilities and is more commonly used by social psychologists. Therefore, as it is more appropriate to use PCA in a physical science situation where variables such as height are perfectly correlated with one another, we used PAF, which

is often preferred to detect structure. We chose the number of factors to be retained on the basis of several criteria, including examination of the resulting scree plot, parallel analysis, and factor interpretability (Gorsuch, 1983). The factors were then rotated using Varimax rotation. The two most common types of orthogonal rotation are Quartimax and Varimax and again the purpose must be considered. For instance, if a general factor was expected, then Quartimax would provide a much better solution because Varimax has the tendency to minimize solutions towards a general factor. Varimax, however, simplifies the columns (factors), while Quartimax simplifies the rows (variables). Therefore, when one is more interested in defining and differentiating the factors than in the variables, as in this case, Varimax should be used. Although the Quartimax solution is analytically simpler than the Varimax solution, Varimax seems to give a clearer separation of the factors. Varimax is the most popular orthogonal rotation choice because of its ability to reapportion variance in a way that minimizes the complexity of factors so that they become relatively equal in importance and the factor pattern obtained is more invariant than that obtained by Quartimax (Gorsuch, 1983; Nunnally and Bernstein, 1994). 7.2. Step 2: confirmatory factor analysis CFA is a type of factor analysis performed to confirm a hypothesized factor structure. Thus, our next step was to confirm the factors determined by the exploratory analysis. CFA was carried out on the remaining one third of the total sample (N ¼ 193). We specified the measurement model on the basis of the pattern of item–latent factor relationships found in the exploratory step. Specifically, for each item, the path from its respective latent factor (i.e., regression weight for the factor or path coefficient) was allowed to be freely estimated while the paths from other factors were constrained to be zero. We examined the extent to which the model fit the data by using the combination of several fit indexes (i.e., the goodness of fit index [GFI], the root mean square error of approximation [RMSEA], and the standardized root mean square residual [SRMR]; Hu and Bentler, 1999).We then reselected the items on the basis of the magnitudes of their path coefficients using LISREL (version 8.30) (Jo¨reskog and So¨rbom, 1999). 7.3. Step 3: analyses to determine scale properties For these analyses, we used the entire sample (N ¼ 499) to estimate the internal consistency reliability (i.e., Cronbach’s coefficient a) of scores on the resulting scales for the six factors determined in the previous steps. We also estimated the intercorrelations among the factors, as well as their correlations with the demographic variables (i.e., age, gender).

ARTICLE IN PRESS C.A. Thornson et al. / Int. J. Human-Computer Studies 67 (2009) 62–78

8. Results 8.1. Exploratory factor analysis results EFA (PAF) was carried out. On the basis of the resulting scree plot (Cattell, 1966), we initially extracted 11 factors which converged in 18 iterations. The factors were then rotated using different rotation methods to thoroughly examine the factor solutions, including different oblique [i.e., promax and oblimin with different parameters] and orthogonal [varimax and quartimax] methods, all of which yielded essentially the same results. On item examination, however, we decided to eliminate several items and retain only six (6) factors because of the uninterpretability of the items belonging to Factor IV (not very meaningful), Factor X (only 2 items loaded), and Factor XI (difficult to interpret). Additionally, the meanings of Factors V and VIII seemed to group together not because of any underlying meaningful construct, but due to the way the items were worded. After deleting these poorer items, as well as those items with low factor loadings (which can make the interpretability of the factors difficult), we reran the EFA specifying a six-factor solution using a varimax orthogonal rotation for the reasons elucidated above. The six factors appeared interpretable and accounted for 42.37% of the total variance (please see Table 1). 8.2. Results of CFA CFA was carried out by the first and third authors to examine the model by specifying six latent factors with 42 items as indicators. Data for this analysis were the remaining 193 responses. The items were specified to be indicators of the factors determined in the exploratory analysis. We used the maximum likelihood estimation method, with sample size specified as the geometric mean of sample sizes of all correlation pairs (n ¼ 193; cf. Viswesvaran and Ones, 1995). Once we examined the various fit indices outlined above, we confirmed the factor structure determined in the exploratory analysis. We then proceeded to reselect the items on the basis of the magnitudes of the regression weights for their assigned factors. Following this step, we eliminated one item from the final inventory as it failed to reach significance (Factor II, Item 10; see Appendix A). The resulting model showed a good fit (chi-square ¼ 3468.38, df ¼ 820, p ¼ .05, GFI ¼ .77, RMSEA ¼ .053; SRMR ¼ .079), confirming the factor structure determined in the exploratory analysis. Thus, 41 items were selected to represent the final six factors (see Appendix A). 8.3. Examination of scale and factor properties Data for this analysis included all 499 responses. Table 2 features the final factors, their definitions, sample items used to assess each of the factors, as well as the number of items belonging to each factor, and descriptive statistics.

71

We estimated the internal consistency reliability (i.e., Cronbach’s coefficient a) of scores on the resulting scales for the six factors (see Table 2), as well as the factor correlations with demographic variables (i.e., age, gender; Table 4), and their convergent–discriminant pattern (Table 2), with factors correlating more strongly with factors conceptually related to one another than with other factors. 8.3.1. Evidence of construct (convergent–discriminant) validity Convergent and discriminant validities were examined (see Table 3). Evidence of convergent validity is shown when factors that are conceptually related are more highly correlated than factors that are theoretically unrelated to one another. Thus, we would expect that Cognitive Involvement (active) and Cognitive Involvement (passive) scales would be more highly correlated with one another than they would be with conceptually dissimilar factors. This was confirmed (see Table 3) in that the correlation between Cognitive Involvement (active) and Cognitive Involvement (passive) was .29 (po.01). The correlation between Spatial Orientation and the Ability to Construct Mental Models (which we would expect to be related) was .27 (po.01). Discriminant validity is evidenced when the correlation between conceptually distinct constructs is lower than the correlations between similar constructs. This was found when constructs such as Cognitive Involvement (active) and Spatial Orientation were uncorrelated (.06, n.s.), evidence of discriminant validity as there is no reason to expect these two different constructs to be related. Likewise, Spatial Orientation was not related to Empathy (.06, n.s.), attesting to the scale’s ability to discriminate between factors that are conceptually and theoretically unrelated (see Table 3). 8.3.2. Internal consistency of scales The internal consistency reliability (i.e., Cronbach’s coefficient a) estimates ranged from .61 for Factor V, the Ability to Construct Mental Models, to .91 for Factor I, Cognitive Involvement (active) (see Table 2), providing evidence that the items comprising each scale or factor are measuring one dimension, a prerequisite to establishing content and construct validity. Even the lower internal consistencies (e.g., Factor V) are still adequate, given the sensitivity of alpha (internal consistency) to the number of items in a scale, as these scales are comprised of only 4–5 items. 8.3.3. Relationships of factors with demographic variables Interestingly, all factors but one were significantly correlated with gender (see Table 4). Specifically, these correlations indicate that females tended to report higher levels of passive Cognitive Involvement and Empathy, while males tend to report higher levels of active Cognitive Involvement, Spatial Orientation, and the Ability to Construct Mental Models. The differences in Spatial

ARTICLE IN PRESS 72

C.A. Thornson et al. / Int. J. Human-Computer Studies 67 (2009) 62–78

Table 1 Rotated factor matrix: results of exploratory factor analysis Factor 1 When I’m involved with the characters in a videogame, I momentarily ‘‘forget’’ that the characters aren’t actually real people. I sometimes catch myself wondering about what will happen to the characters in a videogame. Even though I’m in one location, while playing a videogame it can feel as though my mind has been transported to another location entirely. When involved with the fictional characters in a videogame, I’m able to feel what they’re feeling (anger, sadness, grief, etc.). When I play a videogame, it’s easy for me to imagine myself as an integral part of the action. When involved in a videogame, it seems as if the real world around me disappears. I often find myself physically reacting to something that occurs inside the game as if it were real. When I’m playing a videogame, I don’t seem to realize how quickly time is going by. I know I have a good sense of direction. I rarely get lost. When driving to a new location, even if given explicit and accurate directions, I usually make at least one mistake.—R When I’m inside a building, I can point outside to another location I know—and be absolutely accurate. After having driven somewhere once, I can find it again pretty easily. Sometimes, I just seem to instinctively know which direction is north. I need to consult printed directions and/or a map several times before going somewhere for the first time.—R When someone gives me directions to a location, I can picture the map showing the route of how to get there in my mind. When someone shows a new technique to me, I find I need little to no practice before I can do it on my own. I lack good hand-eye coordination.—R People often describe me as being ‘‘quiet.’’ I’m often quiet when I’m around strangers. People describe me as ‘‘the life of the party.’’—R I usually talk to a variety of people at parties rather than sticking to a few people I know well.—R I’m uncomfortable meeting new people. People describe me as calm or reserved. I don’t mind being the center of attention.—R I prefer to keep to myself mostly, away from the scrutiny of others. When involved with fictional characters in a TV show, movie, or book, I’m able to feel what they are feeling (anger, sadness, grief, etc.) When involved in a TV show, movie, or good book, it seems as if the world around me disappears. When I’m watching something I enjoy, or reading a good book, I don’t seem to realize how quickly time is going by. When choosing a book to read (other than a textbook), I will choose fiction (science fiction, fantasy, mystery, etc.) over non-fiction (history, biographies, etc.). After I’m finished watching a TV show or movie, or have read a good book, I might think about the characters and wonder what’s going to happen to them now. I often played make-believe or role-playing games (house, war, etc.) as a child. I do not enjoy spending time imagining possibilities.—R Most people use the words warm, compassionate, and sympathetic to describe me. Taking other people’s points of view into account is a top priority for me. Making sure that everyone gets along in my circle of friends is one of my priorities. I think I’m too tenderhearted and quick to forgive insults. I enjoy one or more hobbies related to making things (e.g., carpentry, arts, crafts, etc.) As a child, I loved to pull things apart to see if I could reconstruct them. When I was little, I spent hours building sophisticated designs with construction toys (blocks, Lego sets, etc.) or other materials. I can usually draw a pretty accurate (to scale) representation of the rooms of a house or building that I know well. If I’m trying to locate an office in an unfamiliar area of town, I prefer that someone draws me a map.

2

3

4

5

6

.811 .795 .792 .761 .723 .678 .626 .621 .854 .762 .673 .640 .613 .577 .491

.307

.475 .423 .313 .783 .712 .691 .629 .625 .599 .590 .450 .603 .430

.551 .504 .477 .363 .358 .357 .653 .633 .499 .431 .521 .516 .510 .333

Extraction method: principal axis factoring; rotation method: varimax with Kaiser normalization. Rotation converged in 7 iterations.

.425 .390

ARTICLE IN PRESS C.A. Thornson et al. / Int. J. Human-Computer Studies 67 (2009) 62–78

73

Table 2 Factor definitions and scale properties Factor name, definition, and sample item

Number of items

a

M

SD

Cognitive Involvement (active): the tendency to become involved and immersed in tasks that involve decision making and judgment. Example item: ‘‘I often find myself physically reacting to something that occurs inside the game as if it were real.’’ Spatial Orientation: refers to the natural ability to maintain body orientation and/or posture in relation to the surrounding environment (physical space) at rest and during motion. Example item: ‘‘Sometimes, I just seem to instinctively know which direction is north.’’ Introversion: tendency to prefer solitary activities or spending time with one or two others with whom they feel an affinity. Example item: ‘‘I prefer to keep to myself mostly, away from the scrutiny of others.’’ Cognitive Involvement (passive): a cognitive state in which the person is fully engaged in what s/he is doing, characterized by a feeling of energized focus and full involvement in the activity. Example item: ‘‘When I’m watching something I enjoy, or reading a good book, I don’t seem to realize how quickly time is going by.’’ Ability to Construct Mental Models: a mental model is an internal scale-model representation of an external reality. Example item: ‘‘I can usually draw a pretty accurate (to scale) representation of the rooms of a house or building that I know well.’’ Empathy: the ability to adopt another’s psychological point of view and to experience the emotions observed in the experience of others. Example item: ‘‘Taking other people’s points of view into account is a top priority for me.’’

8

.91

2.37

.94

9

.84

3.06

.80

8

.84

2.90

.80

7

.72

3.71

.67

5

.61

3.23

.77

4

.63

3.33

.72

Note: Means and standard deviations are the averages of the items belonging to each scale, with a potential range of 1–5.

Table 3 Intercorrelations among the final factors Factor name

1

2

3

1. 2. 3. 4. 5. 6.

– .06 .15 .26 .23 .10

– .15 .16 .27 .05

– .11 – .08 .10 – .11 .27 .13 –

Cognitive Involvement (active) Spatial Orientation Introversion Cognitive Involvement (passive) Ability to Construct Mental Models Empathy

4

5

6

Note: n ¼ 499. Correlation is significant at the .01 level (2-tailed). Correlation is significant at the .05 level (2-tailed).

Table 4 Correlations of the factors with demographic variables Factor

Age rangea

Genderb

Cognitive Involvement (active) Spatial Orientation Introversion Cognitive Involvement (passive) Ability to Construct Mental Models Empathy

.04 .06 .01 .01 .03 .03

.31 .21 .05 .20 .19 .11

Note: n ¼ 499. Correlation is significant at the .01 level (2-tailed). Correlation is significant at the .05 level (2-tailed). a (18–20) ¼ 1, (21–25) ¼ 2, (26–30) ¼ 3, (31–40) ¼ 4, (41–50) ¼ 5, (50+) ¼ 6. b Male ¼ 0; female ¼ 1.

Orientation (and the related Ability to Construct Mental Models) make sense because spatial ability is one of the few domains where clear sex differences in cognition appear consistently in the literature. Interestingly, it has also been found that spatial ability correlates with verbal ability in

females but not in males, suggesting that women may use different strategies for spatial visualization tasks than males use (Alfonso, 1998). While males report higher levels of Introversion than females, this relationship did not reach the level of significance for this sample. It should be noted, however, that the proportion of males and females in the current sample does not reflect that in the general population. Therefore, on the one hand, this gender skew may have attenuated the observed correlations, which means our results underestimate the correlations, so that the true correlations may be even higher in the population than in our sample. On the other hand, there might be some factors that are correlated with both female participants and the characteristics under consideration, so the correlations found here might not be replicated in the population. Accordingly, the findings related to gender must be treated as tentative, pending future replication. As to age differences, despite some of the prior research, age was not significantly correlated with any of the factors (see Table 4). However, this may be due to the restricted range of age in our young sample and as such, further replication with more diverse and heterogeneous populations is warranted. 9. Discussion As noted, the lack of a consistent and reliable inventory with which to predict the tendency toward presence across different media and tasks has hindered research in the field by making comparisons across different studies— that use different modalities—difficult, if not impossible. Therefore, in an effort to determine the individual difference/human factors predictors of presence across different types of tasks and media, we took on the goal to

ARTICLE IN PRESS 74

C.A. Thornson et al. / Int. J. Human-Computer Studies 67 (2009) 62–78

design a psychometrically valid inventory with which to measure these predictors. Pending further validation, this inventory was designed to be administered as a pretest to assess the tendency toward presence across different participant groups, experimental conditions, stimuli, and settings. Forty-one items out of an initial pool of 105 were selected to represent the 6 factors found to predict a tendency toward presence, out of the original 9 factors (latent constructs) that were proposed. These factors are (in order of the magnitude of the factor loadings): (1) Cognitive Involvement (active); (2) Spatial Orientation; (3) Introversion; (4) Cognitive Involvement (passive); (5) Ability to Construct Mental Models; and (6) Empathy (see Appendix A). These factors are defined as follows. Cognitive Involvement (active) is the tendency to become involved and immersed in tasks that involve decisionmaking and judgment. An example item from our inventory is: ‘‘I often find myself physically reacting to something that occurs inside the game as if it were real.’’ Spatial Orientation refers to the natural ability to maintain body orientation and/or posture in relation to the surrounding environment (physical space) at rest and during motion. An example item is: ‘‘Sometimes, I just seem to instinctively know which direction is north.’’ Introversion is the tendency to prefer solitary activities or spending time with one or two others with whom they feel an affinity. An example item is: ‘‘I prefer to keep to myself mostly, away from the scrutiny of others.’’ Cognitive Involvement (passive) is a cognitive state in which the person is fully immersed in what s/he is doing, characterized by a feeling of energized focus and full involvement in the activity. An example item is: ‘‘When I’m watching something I enjoy, or reading a good book, I don’t seem to realize how quickly time is going by.’’ The Ability to Construct Mental Models is the ability to construct an internal scale-model representation of an external reality. An example item is: ‘‘I can usually draw a pretty accurate (to scale) representation of the rooms of a house or building that I know well.’’ Finally, Empathy is the ability to adopt another’s psychological point of view and to experience the emotions observed in the experience of others. An example item is: ‘‘Taking other people’s points of view into account is a top priority for me.’’ Construct validity was demonstrated with the resultant convergent–discriminant validity pattern, with factors that are conceptually and theoretically related to one another correlating more highly than with factors that are dissimilar. For instance, Cognitive Involvement (active) and Cognitive Involvement (passive) are more highly correlated with one another than with other factors (see Table 3). Discriminant validity evidence was also demonstrated, in that factors that should not be related theoretically to one another are uncorrelated with one another. Correlations with important demographic variables, namely Age and Gender, were also examined. These

variables have been hypothesized to interact with presence and its antecedents, but have not consistently and empirically been shown to do so. Perhaps our tentative finding that different factors (e.g., scales) are differently and uniquely related to gender (male vs. female) will help to shed some light on this issue. It is interesting to find that males reported higher levels of active Cognitive Involvement, Spatial Orientation, and the Ability to Construct Mental Models, while females report higher levels of passive Cognitive Involvement and Empathy. Thus, based on these results, if they are replicated in future research, this would mean that if a virtual or augmented task requires the user to be actively engaged, to manipulate objects in 2D or 3D space, and to be able to construct an accurate representation of the self in the VE (all factors more highly correlated with being male in this study), then males may tend to be more engaged and present in such an environment than females would be. On the other hand, if the VE involves a very complex storyline and requires that one become emotionally invested and involved with the characters and plot, then we might expect females, in general, to experience a higher degree of presence. Thus, it is our hope that the TPI might be used to provide further guidance to designers and researchers engaged in various interface options at some point in the future. 10. Conclusions 10.1. Summary The main objective of this research was to propose and develop an inventory of individual human difference factors that (a) captures the multidimensional nature of the construct of presence by tapping important individual difference constructs that might predict one’s tendency toward experiencing a sense of presence (see Fig. 2); (b) paves the way for future presence researchers to test the criterion and predictive validity of this inventory against an

Spatial Orientation Empathy

Cognitive Involvement (Passive)

Tendency Toward Presence

Cognitive Involvement (Active)

Introversion Ability to Construct Mental Models

Fig. 2. Empirically derived tendency toward presence model.

ARTICLE IN PRESS C.A. Thornson et al. / Int. J. Human-Computer Studies 67 (2009) 62–78

empirically derived post-test measure; and (c) establishes the foundation for the construct validation process of the resulting inventory. Based upon the foregoing discussion, we believe that this overall objective was met, though we acknowledge this is only a first step in the validation process, albeit an important one. We constructed our inventory with the same goal in mind that Witmer and Singer (1998) had for their ITQ, that is, to predict which individual difference factors predict the ability to experience a sense of presence. We started by basing our inventory upon those factors hypothesized or empirically shown in the literature to predict the tendency of an individual to experience a higher degree of presence. However, there is a critical difference between our instrument and the ITQ. Our goal was to offer a crossmedia inventory for widespread use no matter the task or media involved; therefore, instead of focusing, as the ITQ does, on how the user responds to specific types of technology, our inventory examines the factors related to the user himself or herself, in general, as an individual, and this person’s tendency to experience presence across tasks and media. This is the strength of the cross-media generality we hope we have achieved with our inventory. 10.2. Limitations A limitation in this study, inherent in any study that uses a student population, should be noted. Generalizability is an issue that applies to most research studies using undergraduate student samples. Furthermore, as the participants were from the psychology department of a university, the sample was largely female. As addressed above, this gender skew may have caused our results to be either more or less likely to generalize to the general population. The restricted age range of this sample, as in all studies conducted in undergraduate university settings, may also limit generalizability. However, despite the restricted age range, there were no significant relationships found between any of the factors and age, and the sample size was quite large (N ¼ 499). Nevertheless, it is suggested that future researchers determine the extent to which the findings presented in this paper can be expanded to include other persons, settings, and times (Cook and Campbell, 1979). Another limitation, that also points the way for future researchers, is that we did not attempt to assess criterionrelated validity in this study. We accomplished our goal— to design an internally (construct) valid instrument that accurately measures the individual differences expected to relate to one’s tendency to experience a sense of presence. 10.3. Future directions This study was not designed to measure the criterion itself, that is, the degree of presence experienced by the user in a virtual or augmented environment. Our purpose was to

75

examine the factor structure of a group of hypothesized higher-order factors theorized to predict an individual’s tendency to experience presence (e.g., the criterion). Whether the factors assessed by the TPI will in actuality predict the degree of presence experienced by the user remains to be seen. As such, our study was only a first step, albeit an important one, in the process of validating the TPI. Future empirical work is needed to explore the criterion and predictive validities of the factors constituting this inventory. Therefore, it is our hope that future researchers will administer our inventory before participants engage in a virtual or augmented environment (e.g., as a pretest). Thus, they can then correlate the factor scores on the pretest data with the degree of presence actually experienced by the user in the virtual or augmented environment. Such data on the user’s actual degree of presence will, of course, be assessed via researchers’ objective measures of presence, or validated subjective post-test measures, or ideally, both types of measures for cross-validation purposes. It would be interesting to see if certain individual difference factors predict the tendency toward presence across media or if some predictors are more important for certain types of media (e.g., VR, AR, etc.) than others. By providing designers with valid pretest data on the human factors related to one’s tendency to experience presence in various systems using different modalities, our goal of helping designers utilize these data to design various interface options in virtual reality, AR, or telerobotics systems will thus be realized. Acknowledgments This work was supported in part by the US Army Research Laboratory under Cooperative Agreement W911NF-06-2-0041. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the ARL or the US government. We would like to thank Dustin Chertoff for his advice on earlier versions of the TPI. Appendix A. Tendency toward Presence Inventory (TPI) Rated on five-point scale (1 ¼ completely disagree to 5 ¼ completely agree) A.1. FACTOR I—Cognitive Involvement (Active—games): 1. When I’m involved with the characters in a videogame, I momentarily ‘‘forget’’ that the characters aren’t actually real people. (.814) 2. I sometimes catch myself wondering about what will happen to the characters in a videogame. (.795) 3. Even though I’m in one location, while playing a videogame it can feel as though my mind has been transported to another location entirely. (.789)

ARTICLE IN PRESS 76

C.A. Thornson et al. / Int. J. Human-Computer Studies 67 (2009) 62–78

4. When involved with the fictional characters in a videogame, I’m able to feel what they’re feeling (anger, sadness, grief, etc.) (.762) 5. When I play a videogame, it’s easy for me to imagine myself as an integral part of the action. (.720) 6. When involved in a videogame, it seems as if the real world around me disappears. (.676) 7. I often find myself physically reacting to something that occurs inside the game as if it were real. (.623) 8. When I’m playing a videogame, I don’t seem to realize how quickly time is going by. (.620) A.2. FACTOR II—Spatial Orientation 1. I know I have a good sense of direction. (.854) 2. I rarely get lost. (.763) 3. When driving to a new location, even if given explicit and accurate directions, I usually make at least one mistake. (.675) 4. When I’m inside a building, I can point outside to another location I know—and be absolutely accurate. (.639) 5. After having driven somewhere once, I can find it again pretty easily. (.614) 6. Sometimes, I just seem to instinctively know which direction is north. (.578) 7. I need to consult printed directions and/or a map several times before going somewhere for the first time. (.486) 8. When someone gives me directions to a location, I can picture the map showing the route of how to get there in my mind. (.477) 9. When someone shows a new technique to me, I find I need little to no practice before I can do it on my own. (.425) 10. I lack good hand–eye coordination. (.311)1 A.3. FACTOR III—Introversion 1. 2. 3. 4. 5. 6. 7. 8.

People often describe me as being ‘‘quiet.’’ (.781) I’m often quiet when I’m around strangers. (.714) People describe me as ‘‘the life of the party.’’ (.693) I usually talk to a variety of people at parties rather than sticking to a few people I know well. (.631) I’m uncomfortable meeting new people. (.623) People describe me as calm or reserved. (.603) I don’t mind being the center of attention. (.584) I prefer to keep to myself mostly, away from the scrutiny of others. (.447)

A.4. FACTOR IV—Passive Cog. Involvement/Absorption 1. When involved with fictional characters in a TV show, movie, or book, I’m able to feel what they are feeling (anger, sadness, grief, etc.) (.623) 1

This item was eliminated following Confirmatory Factor Analysis.

2. When involved in a TV show, movie, or good book, it seems as if the world around me disappears. (.564)—also cross-loads onto Factor I, active involvement, at .422 3. When I’m watching something I enjoy, or reading a good book, I don’t seem to realize how quickly time is going by. (.510) 4. When choosing a book to read (other than a textbook), I will choose fiction (science fiction, fantasy, mystery, etc.) over non-fiction (history, biographies, etc.). (.454) 5. After I’m finished watching a TV show or movie, or have read a good book, I might think about the characters and wonder what’s going to happen to them now. (.393) 6. I do not enjoy spending time imagining possibilities. (.390) 7. I often played make-believe or role-playing games (house, war, etc.) as a child. (.362) A.5. FACTOR V—Ability to Construct Mental Models 1. As a child, I loved to pull things apart to see if I could reconstruct them. (.552) 2. When I was little, I spent hours building sophisticated designs with construction toys (blocks, Lego sets, etc.) or other materials. (.551) 3. I enjoy one or more hobbies related to making things (e.g., carpentry, arts, crafts, etc.) (.494) 4. I can usually draw a pretty accurate (to scale) representation of the rooms of a house or building that I know well. (.401) 5. If I’m trying to locate an office in an unfamiliar area of town, I prefer that someone draws me a map. (.342) A.6. FACTOR VI—Empathy 1. Most people use the words warm, compassionate, and sympathetic to describe me. (.653) 2. Taking other people’s points of view into account is a top priority for me. (.627) 3. Making sure that everyone gets along in my circle of friends is one of my priorities. (.496) 4. I think I’m too tenderhearted and quick to forgive insults. (.432) References Alfonso, D.L., 1998. The effects of individual differences in spatial visualization ability on dual-task performance. Dissertation (HTML), retrieved on 2007-09-14. Althaus, P., Ishiguro, H., Kanda, K., Miyashita, T., Christensen, H.I., 2004. Navigation for human-robot interaction tasks. In: Proceedings of the 2004 IEEE International Conference on Robotics and Automation, pp. 1894–1900. Bangay, S., Preston, L., 1998. An investigation into factors influencing immersion in interactive virtual environments. In: Riva, G., Wiederhold, B.K., Molinari, E. (Eds.), Virtual Environments in Clinical Psychology and Neuroscience. Ios Press, Amsterdam. Ban˜os, R.M., Botella, C., Perpin˜a, C., 1999. Virtual reality and psychopathology. CyberPsychology & Behavior 2, 283–292.

ARTICLE IN PRESS C.A. Thornson et al. / Int. J. Human-Computer Studies 67 (2009) 62–78 Barfield, W., Weghorst, G., 1993. The sense of presence within virtual environments: a conceptual framework. In: Proceedings of the Fifth International Conference of Human–Computer Interaction, pp. 699–704. Barfield, W., Zeltzer, D., Sheridan, T.B., Slater, M., 1995. Presence and performance within virtual environments. In: Barfield, W., Furness, III, T.A. (Eds.), Virtual Environments and Advanced Interface Design. Oxford University Press, Oxford, pp. 473–513. Biocca, F., 1996. Intelligence augmentation: the vision inside virtual reality. In: Gorayska, B., Mey, J.L. (Eds.), Cognitive Technology: in Search of a Humane Interface. Elsevier, Amsterdam, pp. 59–73. Bowman, D., Raja, D., 2004. A method for quantifying the benefits of immersion using the CAVE. Presence-Connect, 4. Retrieved August 11, 2008 from /http://www.presence-connect.comS. Cattell, R.B., 1966. The scree test for the number of factors. Multivariate Behavioral Research 1, 245–276. Clark, L.A., Watson, D., 1995. Constructing validity: basic issues in objective scale development. Psychological Assessment 7, 309–319. Cohen, R.J., Swerdlik, M.E., 2002. Psychological Testing and Assessment: An Introduction to Tests and Measurement, fifth ed. McGraw-Hill, New York. Cook, T.D., Campbell, D.T., 1979. Quasi Experimentation: Design and Analytical Issues for Field Settings. Rand McNally, Chicago. Csikszentmihalyi, M., 1982. Beyond Boredom and Anxiety. Jossey-Bass, San Francisco. Dix, A., Finlay, J., Abowd, G., Beale, R., 1998. Human–Computer Interaction, second ed. Prentice Hall, Englewood Cliffs. Draper, J.V., Kaber, D.B., Usher, J.M., 1998. Telepresence. Human Factors 40, 354–375. Egan, D.E., 1988. Individual differences in human–computer interaction. In: Helander, M.E. (Ed.), Handbook of Human–Computer Interaction. Elsevier Science, New York. Freeman, J., Avons, S.E., Pearson, D.E., IJsselsteijn, W.A., 1999. Effects of sensory information and prior experience on direct subjective ratings of presence. Presence: Teleoperators and Virtual Environments 8, 1–13. Freeman, J., Avons, S.E., Meddis, R., Pearson, D.E., Ijsselsteijn, W., 2000. Using behavioural realism to estimate presence: a study of the utility of postural responses to motion-stimuli. Presence: Teleoperators and Virtual Environments 9, 149–164. Gorsuch, R.L., 1983. Factor Analysis. Lawrence Erlbaum Associates, Hillsdale, NJ. Heeter, C., 1992. Being there: the subjective experience of presence. Presence: Teleoperators and Virtual Environments 1, 262–271. Heeter, C., 2003. Reflections on real presence by a virtual person. Presence 12, 335–345. Held, R., Durlach, N., 1992. Telepresence. Presence: Teleoperators and Virtual Environments 1, 109–112. Hu, L., Bentler, P.M., 1999. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling 6, 1–55. Ijsselsteijn, W.A., 2005. Towards a neuropsychological basis of presence. Annual Review of CyberTherapy and Telemedicine: A Decade of VR 3, 25–30. International Society for Presence Research, 2000. The concept of presence: explication statement. Retrieved August 7, 2008 from /http://ispr.info/S. Jo¨reskog, K.G., So¨rbom, D., 1999. LISREL 8.30 and PRELIS 2.30. Scientific Software International, Chicago. Jurnet, I.A., Beciu, C.C., Maldonado, J.G., 2005. Individual differences in the sense of presence. In: Paper presented at the PRESENCE 2005: The 8th Annual International Workshop on Presence, London. Keehner, M.M., Tendick, F., Meng, M.V., Anwar, H.P., Hegarty, M., Stoller, M.L., Duh, Q.Y., 2004. Spatial ability, experience, and skill in laparoscopic surgery. American Journal of Surgery 188, 71–75. Kyllonen, P.C., Chaiken, S., 2003. Dynamic spatial ability and psychomotor performance. International Journal of Testing 3, 233–249.

77

Lessiter, J., Freeman, J., Keogh, E., Davidoff, J., 2001. A cross-media presence questionnaire: the ITC-sense of presence inventory. Presence: Teleoperators and Virtual Environments 10, 282–297. Lombard, M., Ditton, T.B., 1997. At the heart of it all: the concept of telepresence. Journal of Computer-Mediated Communication 3. Lombard, M., Ditton, T.B., Crane, D., Davis, B., Gil-Egui, G., Horvath, K., Rossman, J., Park, S., 2000. Measuring presence: a literature-based approach to the development of a standardized paper-and-pencil instrument. In: Presented at the Third International Workshop on Presence, Delft, The Netherlands. Loomis, J.B., 1992. The evolution of a more rigorous approach to benefit transfer: benefit function transfer. Water Resources 28, 701–705. Meehan, M.B., Insko, M.C., Whitton, F.P., Brooks Jr., 2002. Physiological measures of presence in stressful virtual environments. ACM Transactions on Graphics 21, 645–652 (Proceedings of the ACM SIGGRAPH 2002, San Antonio, TX). Myers, I.B., 1980. Gifts Differing: Understanding Personality Type. Davies-Black Publishing, Mountain View, CA. Nunnally, J.C., Bernstein, I.H., 1994. Psychometric Theory, third ed. McGraw-Hill, New York. Prothero, J.D., Hoffman, H.G., Parker, D.E., Furness III, T.A., Wells, M.J., 1995. Foreground/background manipulations affect presence. In: Proceedings of the Human Factors and Ergonomics Society 39th Annual Meeting. Human Factors Society, Santa Monica, CA. Regenbrecht, H., Schubert, T., Friedmann, F., 1998. Measuring the sense of presence and its relations to fear of heights in virtual environments. International Journal of Human–Computer Interaction 10, 233–250. Riding, R.J., 1998. Cognitive Styles Analysis—Research Applications. Learning and Training Technology, Birmingham, UK. Rust, J., Golombok, S., 1989. Modern Psychometrics: The Science of Psychological Assessment. Taylor & Frances/Routledge, Florence, KY. Sas, C., O’Hare, G.M.P., 2003. Impact of cognitive style upon sense of presence. In: Proceedings of the 10th International Conference on Human Computer Interaction, HCII 2003. 22–27 August 2003, Crete, Greece, Erlbaum, London, pp. 696–700. Sas, C., O’Hare, G.M.P., Reilly, R., 2004. Presence and task performance: an approach in the light of cognitive style. Cognition, Technology and Work 6, 53–56. Schubert, T., Friedmann, F., Regenbrecht, H., 2001. The experience of presence: factor analytic insights. Presence: Teleoperators and Virtual Environments 10, 266–281. Schuemie, M.J., Abel, B., van der Mast, C.A.P.G., Krijn, M., Emmelkamp, P.M.G., 2005. The effect of locomotion technique on presence, fear and usability in a virtual environment. In: Proceedings of Euromedia 2005 Conference, Toulouse, France, pp. 129–135. Sheridan, T.B., 1992. Musings on telepresence and virtual presence. Presence: Teleoperators and Virtual Environments 1, 120–126. Slater, M., 1999. Measuring presence: a response to the Witmer and Singer presence questionnaire. Presence: Teleoperators and Virtual Environments 8, 560–565. Slater, M., 2004. How colorful was your day? Why questionnaires cannot assess presence in virtual environments. Presence: Teleoperators and Virtual Environments 13, 484–493. Slater, M., Garau, M., 2007. The use of questionnaire data in presence studies: do not seriously Likert. Presence: Teleoperators and Virtual Environments 16, 447–456. Slater, M., Usoh, M., 1993. Representational systems, perceptual position, and presence in virtual environments. Presence: Teleoperators & Virtual Environments 2, 221–233. Slater, M., Usoh, M., Steed, A., 1994. Depth of presence in virtual environments. Presence: Teleoperators and Virtual Environments 3, 130–144. Slater, M., Usoh, M., Chrysanthou, Y., 1995. The influence of dynamic shadows on presence in immersive virtual environments. In: Second Eurographics Workshop on Virtual Environments, Monte Carlo. Slater, M., Guger, C., Edlinger, G., Leeb, R., Pfurtscheller, G., Antley, A., Garau, M., Brogni, A., Friedman, D., 2006. Analysis of physiological

ARTICLE IN PRESS 78

C.A. Thornson et al. / Int. J. Human-Computer Studies 67 (2009) 62–78

responses to a social situation in an immersive virtual environment. Presence: Teleoperators & Virtual Environments 15, 553–569. Smith-Jentsch, K.A., Campbell, G.E., Milanovich, D.M., Reynolds, A.M., 2001. Measuring teamwork mental models to support training needs assessment, development, and evaluation: two empirical studies. Journal of Organizational Behavior 22, 179–194. Stanney, K.M., Salvendy, G., 1995. Information visualization: assisting low spatial individuals with information access tasks through the use of visual mediators. Ergonomics 38, 1184–1198. Stanney, K.M., Salvendy, G., 1998. Aftereffects and sense of presence in virtual environments: formulation of a research and development agenda. International Journal of Human–Computer Interaction 10, 135–187. Trevino, L.K., Webster, J., 1992. Flow in computer-mediated communication. Communication Research 19, 539–573.

Usoh, M., Catena, E., Arman, M., Slater, M., 2000. Presence questionnaires in reality. Presence: Teleoperators and Virtual Environments 9, 497–503. van Schaik, P., Turnbull, T., van Wersch, A., Drummond, S., 2004. Presence within a mixed reality environment. CyberPsychology & Behavior 7, 540–552. Viswesvaran, C., Ones, D.S., 1995. Theory testing: combining psychometric meta-analysis and structural equations modeling. Personnel Psychology 48, 865–885. Webster, J., Trevino, L.K., Ryan, L., 1993. The dimensionality and correlates of flow in human–computer interactions. Computers in Human Behavior 9, 411–426. Witmer, B.G., Singer, M.J., 1998. Measuring presence in virtual environments: a presence questionnaire. Presence 7, 225–240. Youngblut, C., Perrin, B.M., 2002. Investigating the relationship between presence and performance in virtual environments. In: Paper presented at IMAGE 2002 Conference, Scottsdale, AZ.