Self-evaluative intrusive thoughts impede successful searching on the Internet

Self-evaluative intrusive thoughts impede successful searching on the Internet

Computers in Human Behavior 20 (2004) 85–101 www.elsevier.com/locate/comphumbeh Self-evaluative intrusive thoughts impede successful searching on the...

190KB Sizes 0 Downloads 32 Views

Computers in Human Behavior 20 (2004) 85–101 www.elsevier.com/locate/comphumbeh

Self-evaluative intrusive thoughts impede successful searching on the Internet§ Penny L. Yeea,*, Ingrid Hsieh-Yeeb, Gregory R. Piercea, Rebekah Gromea, Lindsey Schantza a Department of Psychology, Hamilton College, 198 College Hill Road, Clinton, NY 13323, USA School of Library and Information Science, Catholic University of America, Washington, DC 20064, USA

b

Abstract This study examined the association between self-evaluative intrusive thoughts and performance in an Internet search task. Participants performed an information search on the Internet, completed the Cognitive Interference Questionnaire [CIQ; Journal of Consulting & Clinical Psychology, 46 (1978), 102], and then responded to a self-assessment questionnaire on their search performance. Participants reported fewer self-evaluative than other task-related intrusive thoughts; however, higher levels of self-evaluative intrusions were predictive of poorer search performance. Participants who experienced more self-evaluative intrusive thoughts were also less satisfied with their search performance. Other task-related thoughts were unrelated to measures of search performance or participant satisfaction with their searches. These results are discussed in the context of information processing and self-regulatory models of cognitive interference and performance and provide one explanation for how concerns with self-evaluation can undermine performance in Internet search tasks. # 2003 Elsevier Ltd. All rights reserved. Keywords: Cognitive interference; Intrusive thoughts; Information search behavior; Internet use; Information processing load; Self-evaluative thoughts

A primary Internet activity of most Web users is searching for information. Research suggests that 57% of Internet users search on the Web every day with almost a third searching more than once a day (Statistical Research, 2000). In §

Portions of these data were presented at the Thirty-eighth Annual Meeting of the Psychonomic Society, Philadelphia, PA. * Corresponding author. Fax: +1-315-859-4807. E-mail address: [email protected] (P. L. Yee). 0747-5632/03/$ - see front matter # 2003 Elsevier Ltd. All rights reserved. doi:10.1016/S0747-5632(03)00042-6

86

P.L. Yee et al. / Computers in Human Behavior 20 (2004) 85–101

absolute time, Internet users have reported spending an average of 1.5 h each week searching for information on the Web. Consistent with this pattern, a separate survey reported that Internet users ranked searching as the most important activity, rating it at 9.1 on a 10-point scale (Search Engine Watch, 2002). Given the pervasiveness of Internet searching behavior and the challenges of using computers to find information (Bloom, 1990; Kellogg & Richards, 1995) researchers have recently sought to better understand factors that affect search performance. From an information retrieval perspective, the search for information can be understood within a framework that assumes three components (Marchionini, 1995). Two components, system content and system capabilities, emphasize features of the search environment such as the organization and representation of information in the system (system content) or a system’s search features and algorithms for searching (search capabilities). While these factors certainly play a role in search performance, we are more interested in the third component that emphasizes the role that user characteristics play in the effectiveness of information searches. The user component refers to factors that the user brings to the search task, such as dispositional traits like anxiety, cognitive abilities, or previous experience. A review of the literature on Web search behavior by Hsieh-Yee (2000) summarizes the many approaches that researchers have adopted to investigate Web search behavior. Studies have varied in their emphasis on each of the three components proposed by Marchionini (1995), but common findings based on descriptive and empirical studies suggest that search performance on the Web decreases as the cognitive demands of the task increase. For instance, Khan and Locatis (1998a, 1998b) examined how the presentation of information in Web pages influenced search efficiency and accuracy. In their research they varied the number of links per display and the presentation format of the links. Links were presented in a list format or embedded in text. Participants exhibited better search performance when displays contained fewer links and when links were presented in a list format. Both of these conditions reflect situations that reduce the information processing demands for users. Other researchers have also noted that the differential search preferences of novice and expert Web users indicate that novice searchers prefer strategies that require less cognitive load (Carlson & Kacmar, 1999; Marchionini & Shneiderman, 1993). Findings such as these underscore the point that characteristics of the task can influence search performance by altering the demands placed on cognitive processes. Previous research has also demonstrated that characteristics of the user can influence search performance. Feelings of self-efficacy, anxiety, and cognitive abilities have all been recognized as important individual differences variables that influence performance on computer tasks (Bloom & Hautalouma, 1990; Durndell & Haag, 2002; Kellogg & Richards, 1995; Presno, 1998; Rozell & Gardner, 2000; Torkzadeh & Van Dyke, 2002). A central goal of this study was to investigate a component of anxiety—the experience of intrusive thoughts—that may impede information search behavior on the Internet. Intrusive thoughts have often been considered the cognitive component of anxiety (Sarason, 1975; Wine, 1971, 1982), and as such have also been referred to as cognitive interference (Sarason & Stoops, 1978). Consistent with

P.L. Yee et al. / Computers in Human Behavior 20 (2004) 85–101

87

this assertion, Sarason (1984) observed positive associations between anxiety and intrusive thinking. In his study high test anxious individuals experienced more intrusive thoughts than low test-anxious individuals. Studies such as this have lead to the conclusion that intrusive thoughts are the means by which anxiety exerts its influence on performance. Intrusive thoughts are believed to compete for attentional resources while a person is performing a task; individuals either focus their attention in an inward direction on the worrisome cognitions or outwardly on the task at hand (Sarason & Stoops, 1978). Thus, just as characteristics of a search task can influence search performance by varying demands on information processing, user characteristics such as the experience of intrusive thoughts may affect Internet search performance by similarly altering information processing demands.

1. Cognitive interference and performance The behavioral consequences of cognitive interference have been observed with various intellective tasks such as anagram, memory, and problem solving tasks (Blankstein, Toner, & Flatt, 1989; Comunion, 1993; Ellis, Moore, Varner, Ottaway, & Becker, 1997; Hammermaster, 1989; Kurosawa & Harackiewicz, 1995; Pierce, Ptacek, Yee, Hanson, & Boudreau, under review). The typical finding is that the experience of intrusive thoughts is linked to poorer task performance. For instance, Hammermaster (1989) examined the association between performance levels on the Wisconsin Cart Sorting Test (WCST), cognitive interference, and test anxiety. She observed significant differences in the performance levels of individuals who experienced high levels of cognitive interference and those who experienced lower levels; individuals who experienced high levels of cognitive interference performed more poorly. High test-anxious individuals were also more likely to experience significantly more cognitive interference. Blankstein et al. (1989) examined the relationship between cognitive interference and performance on an anagram task, and found a similar association. Those individuals who experienced more cognitive interference correctly solved fewer anagram problems. In yet another study Comunion (1993) observed that children who scored high on a measure of cognitive interference had the lowest school performance as measured by grades. We predict that the effects of cognitive interference will extend to search performance on the Internet. Theoretical models accounting for the impact of cognitive interference on task performance share in common an emphasis on the role that attention plays in the link between intrusive thoughts and performance (Martin & Gill, 1991; Smith, 1986, 1996). In particular, these models focus on the limited mental processing capacity that is available at any given time for executing a task (Baddeley, 1986). The basic premise proposes that intrusive thoughts compete with on-task processing for limited capacity mental resources and thereby reduce efficiency and overall performance (Sarason, 1975; Wine, 1971, 1982). Recent models have proposed several specific means by which intrusive thoughts might undermine performance across a range of domains. Because individuals have

88

P.L. Yee et al. / Computers in Human Behavior 20 (2004) 85–101

a limited capacity for keeping information active in working memory (Baddeley, 1986), cognitive interference may compete with task-relevant information for storage in working memory. In addition, intrusive thoughts may reduce overall processing efficiency within working memory by competing for central processing resources that may be required for judgments and decisions necessary to successfully perform the primary task at hand (Eysenck, 1992; Eysenck & Calvo, 1992; Mikulincer, Kedem, & Zilkha-Segal, 1989). In the context of Internet searches, intrusive thoughts may displace a key term from working memory, reduce the searcher’s ability to assess system response, interfere with the selection of search moves or undermine the searcher’s ability to evaluate the search results and determine the logical next steps. These mechanisms of the general information processing model do not distinguish between the types of intrusive thoughts that an individual may experience. Thus, in the processes described by these mechanisms, one task-irrelevant intrusion is likely to be as debilitating as any other non-task relevant cognition; the critical factor here is the proportion of working memory or central processing resources that are taken up by cognitive interference as opposed to on-task information. 1.1. Self-regulatory models of cognitive interference Other theoretical frameworks have been proposed that distinguish between the effects of specific categories of intrusive thoughts. These theoretical analyses suggest that self-evaluative task-related intrusions, particularly those that are negatively valenced, e.g. ‘‘I’m no good at doing this!’’ (Deffenbacher, 1980; Hembree, 1988), are especially detrimental to the successful completion of a task (see, e.g. Carver, 1996; Wine, 1971, 1982). These models are particularly appealing because of their potential relevance to research on the association between self-efficacy and computer performance that has been the focus of many studies recently (see, e.g. Durndell & Haag, 2002; Presno, 1998; Torkzaden & Van Dyke, 2002). According to Carver (1996), when pursuing a task, an individual may occasionally assess his or her performance by considering the likely outcome of his or her current behavior (i.e. ‘‘Am I getting closer to finding the information I am looking for?’’). Confidence that current behavior is likely to generate a successful outcome leads the person to continue with the currently implemented search strategy. Worry about the efficacy of current behavior may lead the individual to switch strategies to successfully complete the search. However, if these worrisome thoughts about the outcome increase, they consume additional attentional resources needed for the task at hand (Paulman & Kennelly, 1984; Tobias, 1985). In order to regulate levels of performance anxiety, individuals experiencing worrisome thoughts must expend attentional resources in order to suppress these cognitions (Kanfer & Ackerman, 1996). In addition, negative self-evaluative cognitions may lead the individual to reduce effort because he or she anticipates that successful task performance cannot be achieved. Thus, while modest levels of worry may enhance levels of performance because they may lead the individual to switch to a more effective strategy, preoccupation with worrisome cognitions are likely to yield a net loss in overall performance

P.L. Yee et al. / Computers in Human Behavior 20 (2004) 85–101

89

efficiency. In short, while positively oriented thoughts about performance might be associated with higher levels of performance by reinforcing current problem solving strategies, recurrent negatively oriented self-evaluative thoughts may lead to decrements in performance. These two theoretical approaches to cognitive interference and performance, information processing and self-regulation models, share in common an emphasis on the negative impact that intrusive thoughts can exert on performance. However, a strictly information-processing model of cognitive interference predicts that cognitive intrusions, regardless of content, will undermine performance to the degree that they consume attentional resources needed for the task at hand. Thus, according to this model we would expect that the experience of self-evaluative intrusive thoughts and other task-related cognitive interference will exhibit similar relationships to search performance outcomes because each class of cognitive intrusions would reduce the efficiency of information-processing mechanisms in equivalent ways. In contrast, self-regulatory models further distinguish between the content of intrusive thoughts and, therefore, predict that self-evaluative intrusive thoughts, particularly worrisome thoughts, will exert a stronger negative influence on search performance than would other task-related thoughts because, in addition to competing for attentional resources, negative self-evaluative thoughts may engender other responses that may further undermine performance (e.g. by initiating disengagement from or reducing effort towards the task). Consistent with the predictions of self-regulatory models, Pierce et al. (under review) observed that the experience of high levels of self-evaluative thoughts were significantly and negatively associated with athletic and academic performance, while other task-related thoughts were not. Division III college football and crosscountry runners completed a measure of cognitive interference, the Cognitive Interference Questionnaire (CIQ; Sarason & Stoops, 1978), following matches during their competitive season. Pierce et al. (under review) found that self-evaluative intrusions, but not other task-related cognitive interference, predicted their performance. Reports of self-evaluative cognitive interference, and not other task-related thoughts were also predictive of performance on course examinations. Similarly, Blankstein et al. (1989) reported that negative self-referent thoughts were predictive of performance in solving anagrams; experiencing more negative self-referent thoughts was predictive of poorer performance. In this study, we were particularly interested in task-related intrusive thoughts that involve self-evaluation, as opposed to task-related intrusions that are relatively free from personal evaluation (‘‘I wonder why this has to get done today?’’). We investigated the unique contributions that self-evaluative and other task-related intrusive thoughts each made to search performance on the Internet.

2. This study The goal of this study was to examine the association between cognitive interference and search performance on the Internet. In particular, we were interested in

90

P.L. Yee et al. / Computers in Human Behavior 20 (2004) 85–101

exploring the impact of self-evaluative intrusive thoughts. In order to assess cognitive interference, we employed a revised version of the CIQ (Sarason, Sarason, Keefe, Hayes, & Shearin, 1986; Sarason & Stoops, 1978). The CIQ is a widely used self-report measure that assesses the frequency of occurrence of task-related and task-irrelevant thoughts as well as general mind-wandering. However, Pierce et al. (under review) proposed a revised scoring method of the original CIQ to focus on the nature of task-related intrusive thoughts and to better reflect the distinction between task-related intrusive thoughts that are self-evaluative (e.g. ‘‘I thought about how I should work more carefully’’ and ‘‘I thought about how poorly I was doing) and those that are non-self-evaluative (‘‘I thought about how much time I had left’’ and ‘‘I thought about the purpose of the experiment’’). In their work, the self-evaluative items reliably predicted performance while the non-self-evaluative items did not. In this study we relied on the revised scales to assess self-evaluative and other task-related thoughts. We examined Internet search behavior for an open as opposed to a closed search task as a measure of performance. A closed search addresses narrow search questions that seek out a specific piece of information, for example, finding the weather forecast for a particular city. An open search addresses broader search questions whose answers are less specific, for example compiling biographical information about Nobel Peace Prize recipients (Catledge & Pitkow, 1995). We relied on an open search in order to elicit a broader range of search behaviors in which users must explore and browse the available database to locate relevant information. Thus, as a search task we asked participants to locate five separate Web pages that describe programs that parents or educators could use to protect children from undesirable materials and people on the Internet. We relied on three primary measures of search performance. The first was a count of the number of Web sites (i.e. programs) located; we refer to this variable as the number of hits. The second variable, negative critical incidents, referred to the number of times a user backtracked or was forced to start again due to a negative result, for instance, if a search term failed to produce relevant results, or if an explored link did not progress the search in the desired direction. In this way, a negative critical incident is analogous to an error in other classes of tasks. The last variable examined was search time, the time required to complete the search for five Web pages. While search time can be influenced by fluctuating traffic on the Internet, we believe the measure is indicative of search performance because all searches were conducted between 2 and 5 p.m. during the week and assume that the level of web traffic was comparable for all participants. We predicted that individuals who experienced more self-evaluative intrusive thoughts would have fewer hits, more negative critical incidents, and take longer to complete the search than individuals who experienced fewer self-evaluative thoughts. Post-search satisfaction scores were also obtained as a way of measuring participants’ self-evaluation of their search performance. We predicted that individuals with lower levels of satisfaction in their performance would also report experiencing more self-evaluative intrusive thoughts.

P.L. Yee et al. / Computers in Human Behavior 20 (2004) 85–101

91

3. Method 3.1. Participants Participants were 62 undergraduates between the ages of 18 and 22 (31 males and 31 females) from a selective liberal arts college. Volunteers were screened and recruited for participation if they had a minimum total of 5 h of previous experience with the Internet. They were paid $8.00 for their participation. The testing session, which included time for the participants to complete the relevant self-report measures, the computer task, and for the experimenter to explain the tasks to the participants, lasted a total of approximately 40 min. Due to computer complications and missing data, only complete data on 54 participants was available. Later we report the data from the remaining participants (26 males, 28 females). 3.2. Procedure Participants were given 15 min to complete a search task on the Internet. During the search task the image on the computer monitor was recorded to videotape for later transcribing of search behavior. During this phase, a research assistant remained in the room to make written transcriptions of a participant’s search behavior, to deal with mechanical problems with the computer, and to judge the participants’ answers to the given task. In the final phase of the session participants reported the level of intrusive thinking they experienced during the testing session, and they responded to a survey that allowed them to self-assess their performance. 3.3. Apparatus Participants performed an information search on a Macintosh IIci computer equipped with a 14 inch color monitor and the Web browser NetScape Navigator 3.0. A television and video recorder was linked to the computer with a TeleVyse Pro scan converter that enabled the computer session to be recorded directly to videotape. 3.4. Measures 3.4.1. Search task Participants were presented with the following search task to complete: Parents and educators are concerned about protecting kids from undesirable materials and people on the Internet (often referred to as the information superhighway), and several computer programs have been created to assist parents and educators in this effort. Please find five of these programs. This search problem was selected because it required searchers to explore the Internet for information on a topic that is easily understood by most people, does

92

P.L. Yee et al. / Computers in Human Behavior 20 (2004) 85–101

not require knowledge of a particular discipline, and is well represented by Web resources. We experimented with several search problems and searched the Web to determine how well they were represented. In screening the search problem we easily found more than 50 sites with information on the chosen search topic and concluded that this topic would be of low to medium difficulty for Internet searchers. Search behavior and performance was examined with three variables that were believed to reflect the effectiveness of the search process. The first dimension, number of hits, represented the number of correctly identified Web pages. Web sites were considered hits if they described a computer program that would block access to designated Internet sites. The second dimension, negative critical incidents (NCIs), reflected the number of times that the participants’ search path led to an outcome in which the participant needed to backtrack or start a new search strategy in order to move the search forward. The third variable examined was search time and was scored as the total amount of time needed to find five hits that were relevant to the search problem. A maximum time of 15 min was allotted for participants to complete the search. 3.4.2. CIQ The degree to which participants’ experienced intrusive thoughts during the search task was measured with the CIQ (Sarason et al., 1986; Sarason & Stoops, 1978). The CIQ was designed to be administered following performance on a task, and assesses the frequency with which people experienced various types of thoughts during that task. The CIQ consists of 22 items. On the first 21 items participants rate the frequency of occurrence of particular types of thoughts on a scale of 1–5: (1) never, (2) once, (3) a few times, (4) often, and (5) very often. On the first 10 items, participants report how often they experienced a variety of task-related thoughts while working on the task; both self-evaluative and non-self evaluative thoughts were listed. On the next 11 items participants rate how often they experienced intrusive thoughts whose contents do not refer to the task (e.g. ‘‘I thought about personal worries’’ and ‘‘I thought about something that made me feel guilty’’). The final item provides a global rating on a seven-point scale of the degree of mind-wandering experienced while working on the task. Results from numerous studies suggest that the CIQ has desirable psychometric properties and has good construct validity (e.g. Blankstein et al., 1989; Pierce et al., 1998; Sarason et al., 1986; Sarason & Stoops, 1978). However, following Pierce et al. (under review) our investigation relies on responses to just the first 10 task-related items. Pierce et al. (under review) developed two new scales from the task-related items of the CIQ that were used in this study. These two new scales (presented in Table 1) were constructed to provide indices of the frequency with which participants experienced self-evaluative intrusive thoughts (i.e. those that dealt with thoughts about the individual’s level of performance) and other task-related intrusive thoughts (i.e. those that, while focused on the task, did not include an evaluation of the individual’s own performance). In this study, internal consistency estimates (Cronbach’s a) for the CIQ self-evaluative and other task-related thoughts scales were acceptable (a=0.82 and 0.63, respectively). The internal consistency estimate

P.L. Yee et al. / Computers in Human Behavior 20 (2004) 85–101

93

Table 1 Items assigned to the CIQ self-evaluative and other task-related scales CIQ self-evaluative task-related items I thought about how poorly I was doing I thought about how I should work more carefully I thought about my level of ability I thought about how I would feel if I were told how I performed I thought about how often I got confused I thought about what the experimenter would think of me CIQ other task-related items I thought about how much time I had left I thought about how others have done on this task I thought about the difficulty of the experiment I thought about the purpose of the experiment

for the other task-related thoughts scale was lower than some of the estimates obtained for this scale in previous research (Pierce et al., under review). None of the items from the task-unrelated or mind-wandering scales were used in this study. 3.4.3. Self-reported post-search performance A self-report measure of post-search appraisals was developed for this study to assess participants’ self-evaluations of search performance. These questions asked participants (1) if they were satisfied with their search, (2) if they would change the way they searched if given the opportunity, and (3) if they had previous knowledge of the search topic. These questions were answered in a yes/no format. For purposes of analyses ‘no’ responses were coded as ‘0’ and ‘yes’ responses as ‘1’.

4. Results 4.1. Descriptive statistics 4.1.1. Search behavior and intrusive thoughts The mean and standard deviation were computed for each of the search behaviors and each of the CIQ Scales; the results are reported for each gender and for the overall sample in Table 2. Overall, the mean search time was 13.22 min (S.D.=3.05). In that time participants located an average of 2.40 (S.D.=2.29) of the requested five critical Web pages. The average number of negative incidents during the search was 5.20 (S.D.=2.64). In order to examine possible gender differences in search performance independent groups t-tests were performed on each of the search variables. Significant gender differences were observed for the total number of hits, t(52)=3.44, P < 0.01. Males found significantly more Web pages that were relevant to the search question. There were no significant differences in the number of NCIs or in the amount time males and females took to perform the search task.

94

P.L. Yee et al. / Computers in Human Behavior 20 (2004) 85–101

Table 2 Mean scoresa by gender and overall for search behaviors, CIQ self-evaluative, other task-related intrusive thoughts and self-reported post-search assessment Males (N=26)

Females (N=28)

Overall (N=54)

3.42 (2.21) 4.92 (2.58) 12.59 (3.38)

1.46 (1.97) 5.46 (2.71) 13.81 (2.64)

2.40 (2.29) 5.20 (2.64) 13.22 (3.05)

Intrusive thoughts Self-evaluative task-related Other task-related

2.29 (0.77) 2.59 (0.84)

2.57 (0.92) 2.63 (0.89)

2.43 (0.85) 2.61 (0.86)

Post-search assessment Satisfied with search Change search Previous topic knowledge

0.85 (0.37) 0.31 (0.47) 0.31 (0.47)

0.39 (0.50) 0.68 (0.48) 0.08 (0.26)

0.61 (0.49) 0.50 (0.50) 0.19 (0.39)

Search behaviorsb Hits NCIs Search time

a

Standard deviations in parentheses. Hits refers to the number of Web pages found that were relevant to the search question; NCIs refers to the number of negative critical incidents; Search time refers to minutes spent on task. b

In order to examine possible differences in the levels of self-evaluative compared with other task-related thoughts, as well as possible gender differences in the experience of intrusive thoughts, we conducted a 2 (Type of Task-related thought: Selfevaluative vs. Other)2 (Gender: Male vs. Female) repeated measures ANOVA. A significant main effect for type of intrusive thought was observed, F(1, 52)=4.04, P < 0.05, Mean Square Error (MSE)=0.22; participants reported experiencing fewer self-evaluative (M=2.43, S.D.=0.85) than other (M=2.61, S.D.=0.86) task-related intrusive thoughts (Z2=0.072). There was no significant main effect for gender, F(1, 52)=0.55, P=0.50, MSE=1.26, and there was no significant interaction between type of intrusive thought and gender, F(1, 52)=1.75, P=0.20, MSE=0.22. 4.1.2. Post-search assessment Responses on the post-search questionnaire were evaluated to assess participants’ satisfaction with their search experience. In our analyses ‘no’ responses were coded as ‘0’ and ‘yes’ responses as ‘1’. The mean and standard deviation for each of the three items are reported in Table 2. These results indicate that the overall sample was moderately satisfied with their search, and expressed some desire to do things differently if given the opportunity. In addition, the participants appeared to have a modest degree of familiarity with the search topic before participating in the study. These responses were also examined for sex differences with independent groups t-tests; significant differences between males and females were observed for each of the items (all Ps < 0.05), indicating that males were more satisfied with their search and more likely to have had previous knowledge about the search topic. Females were more likely to express a desire to change their search in some way if offered an opportunity to perform the search again.

P.L. Yee et al. / Computers in Human Behavior 20 (2004) 85–101

95

4.2. Associations among measures Before investigating associations between intrusive thinking and search performance variables, we examined the correlations among the Cognitive Interference Scales, as well as correlations among the performance variables. The CIQ Scales were strongly correlated with each other r(52)=0.70, P < 0.01. This association is not surprising considering that the items forming the self-evaluative and the other task-related intrusive thoughts scales used in this study constituted a single scale of task-related intrusive thoughts in the original CIQ instrument reported by Sarason and Stoops (1978). It should be noted that, given the internal consistency estimates of the subscales in this study and the Spearman–Brown formula, the observed correlation between these subscales is nearly 1.0 when corrected for unreliability of measurement. Because psychometric assessment of the original CIQ indicated moderate associations among these items (Sarason et al., 1986; Sarason & Stoops, 1978), and in previous research these subscales showed greater empirical distinctiveness, and the distinction between these two classes of intrusive thoughts is of theoretical interest, we elected to retain the two subscales. Measures of search performance involving the number of hits, NCIs, and search time were significantly correlated with each other with correlations ranging from 0.50 to 0.60 (P < 0.01). These correlations indicated that a higher number of hits was negatively associated with search time and the number of NCIs [r(52)= 0.60 and r(52)= 0.50, respectively, P < 0.01]. Thus, finding a greater number of correct Web sites was associated with shorter search times and fewer thwarted actions during the search. Interestingly, search time was also positively correlated with NCIs [r(52)=0.59, P < 0.01], indicating that searches took longer when participants ran into more dead-ends. 4.3. Intrusive thoughts, search performance, and search satisfaction We computed correlation coefficients to assess the degree to which participants’ self-evaluative and other intrusive thoughts scores predicted their search performance. Table 3 presents these correlations. Generally, participants who reported a high level of self-evaluative thoughts performed more poorly than did other participants; individuals with high levels of self-evaluative intrusive thoughts had fewer hits, more NCIs, and had longer search times than did individuals who experienced lower levels of self-evaluative intrusive thoughts. In contrast, participants’ reports of other task-related intrusive thoughts were unrelated to search performance. Consistent with this finding are the associations observed with post-search assessment measures of satisfaction. Table 3 presents correlations between CIQ Scales and post-search responses. Individuals who experienced higher levels of self-evaluative intrusive thoughts were more likely to indicate a desire to change their search strategy if given the opportunity [r(52)=0.40, P < 0.01]. A trend was also observed suggesting that individuals with higher levels of self-evaluative cognitive interference were less satisfied with their search performance [r(52)= 0.23, P=0.09]. Measures of other task-related intrusive thoughts were unrelated to post-search satisfaction.

96

P.L. Yee et al. / Computers in Human Behavior 20 (2004) 85–101

Table 3 Correlations between the CIQ Scales and search behaviors and self-reported post-search assessment (N=54) CIQ Scales Self-evaluative

Other task-related

a

Search behaviors Hits NCIs Search time

0.39** 0.29* 0.36**

0.17 0.15 0.20

Post-search assessment Satisfied with search Change search Previous topic knowledge

0.23 0.40** 0.20

0.10 0.23 0.02

a Hits refers to the number of Web pages found that were relevant to the search question; NCIs refers to the number of negative critical incidents; Search time refers to time spent on task. * P <0.05. ** P <0.01.

4.4. Regressions To evaluate whether self-evaluative thoughts were more uniquely related than other task-related thoughts to search performance, we computed regression equations with each search performance measure serving as a dependent variable. For each regression equation, participants’ self-evaluative and other task-related intrusive thoughts scores were entered simultaneously to evaluate the unique contribution of each category of intrusive thoughts to the prediction of search performance. We observed a consistent pattern of findings across all three regression equations. Self-evaluative and other task-related scales jointly predicted between 17% of the variance in the number of hits (P < 0.05), 14% of the variance in search time (P < 0.05), and 9% of the variance in NCIs (P=0.10). Self-evaluative intrusive thoughts significantly and uniquely predicted 14.1% of the variance in the number of hits (P < 0.01), 10.0% of the variance in the search time (P < 0.05), and 6% of the variance in NCIs (P=0.07). Participants’ ratings of other task-related intrusive thoughts failed to predict unique variance in search performance in any of the regression equations. These results support our hypothesis that self-evaluative intrusive thoughts are more strongly related than other task-related thoughts to search performance. The regression analyses reported earlier were repeated using participants’ gender as a covariate to evaluate the possible role of gender differences in the observed associations; the inclusion of the gender variable did not alter any of the reported associations between self-evaluative intrusive thoughts and search performance.

P.L. Yee et al. / Computers in Human Behavior 20 (2004) 85–101

97

5. Discussion We conducted a study to examine the impact of intrusive thoughts on search behavior on the Internet. While we expected the experience of intrusive thoughts to impede performance, we were particularly interested in the unique contributions that self-evaluative and other task-related thoughts each made to search performance. Consistent with our predictions was the fact that self-evaluative intrusive thoughts were predictive of objective and subjective self-reports assessments of performance; other task-related thoughts were unassociated with performance measures. Consistent with previous research reported by Pierce et al. (under review), our participants reported experiencing self-evaluative intrusive thoughts significantly less frequently on the average than other task-related thoughts. The fact that experiencing higher levels of self-evaluative thoughts predicted poorer search performance paired with the observation that participants reported experiencing self-evaluative thoughts less often on the average than other task-related thoughts, emphasizes the powerful role that concerns about task performance can have on search behavior. We explore the broader implications of this finding later. One of the many challenges in investigating Internet search performance is measuring the behavior on-line as it unfolds. Our study presents one viable approach for tracking and measuring Internet search performance. The detrimental impact of self-evaluative intrusive thoughts was observed in three objective measures of search behavior. These measures reflected the effectiveness of a user’s search in different ways; we expected efficient search strategies to result in a greater number of hits, shorter search times, and fewer negative critical incidents. Consistent with previous research on the association between cognitive interference and task performance, participants who experienced high levels of self-evaluative intrusive thoughts located fewer critical Web pages, took more time for their search, and experienced more negative critical incidents. The fact that we also observed associations between the experience of self-evaluative thoughts and subjective self-assessments of performance in our post-search survey provides convergent evidence that self-evaluation is an important factor in the association between cognitive interference and search performance. As Carver and Scheier (1981) proposed, the act of self-assessment can be beneficial, facilitating performance when it signals a person to continue with a successful approach to a problem or to abort an unproductive strategy. However, individuals engaging in high levels of self-assessment as they performed the search task, may also begin to question their own abilities in performing the task successfully at all. One consequence is that individuals may reduce their efforts or completely disengage from the search task. Our observation that individuals who experienced high levels of self-evaluative intrusive thinking were less satisfied with their search, were more likely to want to change something about their search strategy if offered an opportunity to do so, and performed more poorly in the search task is certainly consistent with Carver and Scheier’s (1981) predictions. Individuals who experienced more self-evaluative thoughts were less happy with their performance. Consistent with these observations other research supports the idea that an underlying factor in the association

98

P.L. Yee et al. / Computers in Human Behavior 20 (2004) 85–101

between self-evaluative thoughts and performance is self-efficacy. Participants who perceive themselves as not making progress may begin to doubt their ability to perform the task. In a study of Internet anxiety Presno (1998) suggested that concerns about low self-efficacy played a central role in the various forms of Internet anxiety that she identified. She reported that participants frequently blamed themselves for not being capable of getting to desired sites on the Internet, or not finding the desired information. Interestingly, self-evaluative thoughts predicted performance while other taskrelated thoughts did not despite the finding that participants reported experiencing other task-related thoughts more frequently on the average. This combination of findings underscores the importance of the content rather than the sheer frequency of occurrence of intrusive thinking. Such a view supports the self-regulatory models of cognitive interference and performance (Carver & Scheier, 1981; Kanfer & Ackerman, 1996). According to these models, both cognitive and emotional processes require monitoring; and regulation of these processes demands resources. Self-regulation of the cognitive component may evoke a switch in cognitive strategies or an increase in overall effort, the latter of which is enacted by increasing the resources dedicated to those processes. Self-regulation of the emotional component involves monitoring arousal levels and requires keeping negative emotions and anxieties suppressed. These processes demand attentional resources as well. Thus, the self-evaluative thoughts that an individual experiences not only present competition for limited attentional resources, but also engage mechanisms to regulate the performance anxiety that is triggered by self-assessments. Results reported in this paper and those obtained by other investigators suggest that the link between intrusive thoughts—especially self-evaluative cognitions—and performance, may be complex. While our results allude to the detrimental effects of a preoccupation with negative self-evaluation, they should not be taken as evidence that the process of self-assessment is itself a poor strategy. As indicated earlier, selfassessment is useful in gauging one’s progress towards a goal. However, preoccupation with self-evaluation, or self-evaluative cognitions that are unduly negative, may undermine performance. It is therefore the case that programs designed to encourage self-reflection in the acquisition of computer skills may need to consider more fully the multifaceted nature of self-reflective thought and its role in performance. This study replicates and extends the results observed by Pierce et al. (under review) to a more an applied setting. Pierce et al. (under review) examined performance in academic (examination performance) and athletic task domains. In our study, we examined performance in an Internet search task—an activity that statistics suggest is rapidly becoming a daily activity and is becoming increasingly difficult to avoid because the advantages garnered by those who choose to search the Internet will make it indispensable (Kellogg & Richards, 1995). As this becomes a reality, research efforts to better understand search performance will become increasingly valuable, and these efforts will need to take into consideration the abilities and limitations of the user. This study is not without limitations. As we noted earlier, the other task-related intrusive thoughts subscale had only modest reliability; and, importantly, the relia-

P.L. Yee et al. / Computers in Human Behavior 20 (2004) 85–101

99

bility of this subscale was lower than the reliability found for the self-evaluative intrusive thoughts subscale. It is possible, therefore, that findings from the correlational and regression analyses reported in this paper are, at least in part, due to differences in the reliabilities of these two subscales. This is because unreliability reduces the magnitude of observed associations when compared with population values (Cohen & Cohen, 1983). In addition, the substantial correlation between the two subscales raises the problem of multicollinearity among the predictor variables in the regression analyses (Cohen & Cohen, 1983). The effect of multicollinearity among predictor variables is to reduce the observed unique associations between the predictor variables and the dependent variable. In this case, the implication of this observation is that the percentage of variance accounted for by each of the two classes of intrusive thoughts may be an underestimate of their true association in the population (Cohen & Cohen, 1983). The fact that the present findings parallel those obtained in three previous studies using the same scales, in which higher internal consistency estimates were obtained for the other task-related intrusive thoughts subscale (Pierce et al., under review), strengthen our conclusion that the pattern of observed associations reflects actual differences in the potential role of these two classes of intrusive thoughts. This study represents one of the first attempts to examine associations between affective cognitive processes and actual Internet search performance. While researchers in the information science field have examined the role of affective states in Internet search behavior (Nahl, 1998; Wang, Hawk, & Tenopir, 2000), none has investigated possible mechanisms underlying associations between anxiety and search performance. The results of our study showed that self-evaluative intrusive thoughts were closely related to search strategy, search results, and searcher assessment of performance. These findings suggest that self-evaluative intrusive thoughts may be an important factor mediating the relationship between performance and anxiety. Should this be the case, interventions focused on decreasing the level of intrusive thoughts may be successful in creating a generation of more effective Internet and computer users. References Baddeley, A. D. (1986). Working memory. Oxford: Clarendon Press. Blankstein, K. R., Toner, B. B., & Flett, G. L. (1989). Test anxiety and the contents of consciousness: Thought-listing and endorsement measures. Journal of Research in Personality, 23, 269–286. Bloom, A. J. (1990). Anxiety management training as a strategy for enhancing computer user performance. Computers in Human Behavior, 6, 337–349. Bloom, A. J., & Hautaluoma, J. E. (1990). Anxiety management training as a strategy for enhancing computer user performance. Computers in Human Behavior, 6, 337–349. Carlson, J. R., & Kacmar, C. J. (1999). Increasing link marker effectiveness for WWW and other hypermedia interfaces: an examination of end-user preferences. Journal of the American Society for Information Science, 50, 386–398. Carver, C. S. (1996). Cognitive interference and the structure of behavior. In I. G. Sarason, G. R. Pierce, & B. R. Sarason (Eds.), Cognitive interference: theories, methods, and findings (pp. 25–45). Mahwah, NJ: Erlbaum.

100

P.L. Yee et al. / Computers in Human Behavior 20 (2004) 85–101

Carver, C. S., & Scheier, M. F. (1981). Attention and self-regulation: a control-theory approach to human behavior. New York: Springer-Verlag. Catledge, L., & Pitkow, J. (1995). Characterizing browsing strategies in the World-Wide Web. Computer networks & ISDN Systems, 27, 1065–1073. Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation research for the behavioral sciences. Hillsdale, NJ: Erlbaum. Comunian, A. L. (1993). Anxiety, cognitive interference, and school performance of Italian children. Psychological Reports, 73, 747–754. Deffenbacher, J. L. (1980). Worry and emotionality in test anxiety. In I. G. Sarason (Ed.), Test anxiety: theory, research, and applications (pp. 111–128). Hillsdale, NJ: Erlbaum. Durndell, A., & Haag, Z. (2002). Computer self-efficacy, computer anxiety, attitudes toward the Internet and reported experience with the Internet, by gender, in an East European sample. Computers in Human Behavior, 18, 521–535. Ellis, H. C., Moore, B. A., Varner, L. J., Ottaway, S. A., & Becker, A. S. (1997). Depressed mood, task organization, cognitive interference, and memory: irrelevant thoughts predict recall performance. Journal of Social Behavior & Personality, 12, 453–470. Eysenck, M. W. (1992). Anxiety: the cognitive experience. Hove, UK: Erlbaum. Eysenck, M. W., & Calvo, M. G. (1992). Anxiety and performance: the processing efficiency theory. Cognition & Emotion, 6, 409–434. Hammermaster, C. S. (1989). Levels of performance and cognitive interference in test anxious subjects. The Alberta Journal of Educational Research, 35, 164–170. Hembree, R. (1988). Correlates, causes, effects, and treatment of test anxiety. Review of Educational Research, 58, 47–77. Hsieh-Yee, I. (2000). Research on Web search behavior. Library & Information Science Research, 23, 1–19. Kanfer, R., & Ackerman, P. L. (1996). A self-regulatory skills perspective to reducing cognitive interference. In I. G. Sarason, G. R. Pierce, & B. R. Sarason (Eds.), Cognitive interference: theories, methods, and findings (pp. 153–171). Mahwah, NJ: Erlbaum. Kellogg, W. A., & Richards, J. T. (1995). The human factors of information on the Internet. In J. Nielson (Ed.), Advances in human–computer interaction (Vol. 5) (pp. 1–36). Norwood, NJ: Ablex Publishing. Khan, K., & Locatis, C. (1998a). Searching through cyberspace: the effects of link display and link density on information retrieval from hypertext on the World Wide Web. Journal of the American Society for Information Science, 49, 176–182. Khan, K., & Locatis, C. (1998b). Searching through cyberspace: the effects of link cues and correspondence on information retrieval from hypertext on the World Wide Web. Journal of the American Society for Information Science, 49, 1248–1253. Kurosawa, K., & Harackiewicz, J. M. (1995). Test anxiety, self-awareness, and cognitive interference: a process analysis. Journal of Personality, 63, 931–951. Marchionini, G. (1995). Information seeking in electronic environments. Cambridge, MA: Cambridge University Press. Marchionini, G., & Shneiderman, B. (1993). Finding facts versus browsing knowledge in hypertext systems. In B. Shneiderman (Ed.), Sparks of innovation in human–computer interaction (pp. 103–122). Norwood, NJ: Ablex Publishing. Martin, J. J., & Gill, D. L. (1991). The relationships among competitive orientation, sport confidence, self-efficacy, anxiety, and performance. Journal of Sport & Exercise Psychology, 13, 149–159. Mikulincer, M., Kedem, P., & Zilkha-Segal, H. (1989). Learned helplessness, reactance, and cue utilization. Journal of Research in Personality, 23, 235–247. Nahl, D. (1998). Ethnography of novices’ first use of Web search engines: affective control in cognitive processing. Internet Reference Services Quarterly, 3, 51–72. Paulman, R. G., & Kennelly, K. J. (1984). Test anxiety and ineffective test taking: different names, same construct? Journal of Educational Psychology, 76, 279–288. Pierce, G. R., Ptacek, J. T., Taylor, B., Yee, P. L., Henderson, C. A., Lauventi, H. J., & Bourdeau, C. M. (1998). The role of dispositional and situational factors in cognitive interference. Journal of Personality & Social Psychology, 75, 1016–1031.

P.L. Yee et al. / Computers in Human Behavior 20 (2004) 85–101

101

Pierce, G. R., Ptacek, J. T., Yee, P. L., Hanson, M. D., & Bourdeau, C. M. Self-evaluative and other taskrelated intrusive thoughts: their temporal stability, cross-situational consistency, and differential role in task performance (under review). Presno, C. (1998). Taking the byte out of Internet anxiety: instructional techniques that reduce computer/ Internet anxiety in the classroom. Journal of Educational Computing Research, 18, 147–161. Rozell, E. J., & Gardner, W. L. (2000). Cognitive, motivation, and affective processes associated with computer-related performance: a path analysis. Computers in Human Behavior, 16, 199–222. Sarason, I. G. (1975). Anxiety and self-preoccupation. In I. G. Sarason, & C. D. Spielberger (Eds.), Stress and anxiety (Vol. 2) (pp. 27–44). Washington, DC: Hemisphere. Sarason, I. G. (1984). Stress, anxiety, and cognitive interference: reactions to tests. Journal of Personality & Social Psychology, 46, 929–938. Sarason, I. G., Sarason, B. R., Keefe, D. E., Hayes, B. E., & Shearin, E. N. (1986). Cognitive interference: Situational determinants and traitlike characteristics. Journal of Personality & Social Psychology, 51, 215–226. Sarason, I. G., & Stoops, R. (1978). Test anxiety and the passage of time. Journal of Consulting & Clinical Psychology, 46, 102–109. Search Engine Watch. (2002). The Search Engine index. Retrieved 17 July from http://www.searchenginewatch. com/reports/seindex.html. Smith, R. E. (1986). A component analysis of athletic stress. In M. R. Weiss, & D. Gould (Eds.), Sport for children and youths (pp. 107–111). Champaign, IL: Human Kinetics. Smith, R. E. (1996). Performance anxiety, cognitive interference, and concentration enhancement strategies in sports. In I. G. Sarason, G. R. Pierce, & B. R. Sarason (Eds.), Cognitive interference: theories, methods, and findings (pp. 261–283). Mahwah, NJ: Erlbaum. Statistical Research. (2000). New study shows internet users are loyal to Web ‘‘niches’’. Retrieved 16 July 16 from http://www.sriresearch.com/press/pr20000217.htm. Tobias, S. (1985). Test anxiety: interference, defective skills, and cognitive capacity. Educational Psychologist, 20, 135–142. Torkzadeh, G., & Van Dyke, T. P. (2002). Effects of training on Internet self-efficacy and computer user attitudes. Computers in Human Behavior, 18, 279–494. Wang, P., Hawk, W. B., & Tenopir, C. (2000). Users’ interaction with World Wide Web resources: an exploratory study using a holistic approach. Information Processing & Management, 36, 229–251. Wine, J. D. (1971). Test anxiety and direction of attention. Psychological Bulletin, 76, 92–104. Wine, J. D. (1982). Evaluation anxiety: a cognitive-attentional construct. In H. W. Krohne, & L. Laux (Eds.), Achievement, stress, and anxiety. Washington, DC: Hemisphere.