Evaluating the consistency of immediate aesthetic perceptions of web pages

ARTICLE IN PRESS

Int. J. Human-Computer Studies 64 (2006) 1071–1083 www.elsevier.com/locate/ijhcs

Evaluating the consistency of immediate aesthetic perceptions of web pages Noam Tractinskya,, Avivit Cokhavia, Moti Kirschenbauma, Tal Sharfib a

Department of Information Systems Engineering, Ben-Gurion University of the Negev, Beer Sheva, Israel Department of Industrial Engineering and Management, Ben-Gurion University of the Negev, Beer Sheva, Israel

b

Received 23 August 2005; received in revised form 5 June 2006; accepted 13 June 2006 Communicated by P. Zhang Available online 23 August 2006

Abstract Two experiments were designed to replicate and extend [Lindgaard et al.’s, 2006. Attention web designers: you have 50 ms to make a good first impression! Behaviour and Information Technology 25(2), 115–126] findings that users can form immediate aesthetic impression of web pages, and that these impressions are highly stable. Using explicit (subjective evaluations) and implicit (response latency) measures, the experiments demonstrated that, averaged over users, immediate aesthetic impressions of web pages are remarkably consistent. In Experiment 1, 40 participants evaluated 50 web pages in two phases. The average attractiveness ratings of web pages after a very short exposure of 500 ms were highly correlated with average attractiveness ratings after an exposure of 10 s. Extreme attractiveness evaluations (both positive and negative) were faster than moderate evaluations, landing convergent evidence to the hypothesis of immediate impression. The findings also suggest considerable individual differences in evaluations and in the consistency of those evaluations. In Experiment 2, 24 of the 50 web pages from Experiment 1 were evaluated again for their attractiveness after 500 ms exposure. Subsequently, users evaluated the design of the web pages on the dimensions of classical and expressive aesthetics. The results showed high correlation between attractiveness ratings from Experiments 1 and 2. In addition, it appears that low attractiveness is associated mainly with very low ratings of expressive aesthetics. Overall, the results provide direct evidence in support of the premise that aesthetic impressions of web pages are formed quickly. Indirectly, these results also suggest that visual aesthetics plays an important role in users’ evaluations of the IT artifact and in their attitudes toward interactive systems. r 2006 Elsevier Ltd. All rights reserved. Keywords: Web-page design; Aesthetic perceptions; Attractiveness; Classical aesthetics; Expressive aesthetics; Response latency; Response time; First impression

1. Introduction First impressions colour subsequent search for information and sway judgement and choice processes. One of the most notable sources of first impressions is visual appearance. In a seminal paper, Dion et al. (1972), demonstrated that a person’s physical appearance influences other aspects of the social interaction. Subsequent studies showed how prevalent and powerful this phenomenon is. For example, beautiful people earn more on the marketplace (Hamermesh and Biddle, 1994), and better Corresponding author. Tel.: +972 7 6472226; fax: +972 7 6472958.

E-mail address: [email protected] (N. Tractinsky). 1071-5819/$ - see front matter r 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.ijhcs.2006.06.009

looking university instructors receive higher teaching evaluations (Hamermesh and Parker, 2005). Clearly, effects of visual appearance are not limited to perceptions of humans. We are affected by the aesthetics of nature and of architecture (e.g. Nasar, 1988a; Porteous, 1996) as well as the beauty of everyday objects and artifacts (Postrel, 2002; Coates, 2003; Norman, 2004a). There is also growing evidence suggesting that evaluations of interactive systems are influenced in general by the systems’ visual appearance (Tractinsky et al., 2000) and by the appearance of web pages in particular (Karvonen, 2000; Schenkman and Jonsson, 2000; Zhang and von Dran, 2000; Zhang et al., 2001; van der Heijden, 2003; Kim et al., 2003). However, it is not clear whether this influence stems from immediate,

ARTICLE IN PRESS 1072

N. Tractinsky et al. / Int. J. Human-Computer Studies 64 (2006) 1071–1083

first impression or from a more elaborated examination of the web site. In fact some even doubt that such influence even exists (Hassenzahl, 2004). Thus, the causal relationship between aesthetic perceptions and other perceived attributes of the IT artifact cannot be taken for granted. One of the reasons for the major role of aesthetics in everyday life may stem from its immediate effect on our senses and, consequently on our judgement. However, there are two prerequisites to this possible explanation. Firstly, it should be demonstrated that aesthetic impressions could be made quickly. Secondly, the validity of this argument may diminish if those impressions were short-lived and transient. In this case, initial aesthetic impressions will be swept aside by new ones, and more elaborated aesthetic evaluations may change the overall evaluations. Thus, it is important to demonstrate empirically that first aesthetic impressions are not only quick, but that they are also lasting and consistent.1 Such demonstration lies at the heart of this study. Our major objective is to demonstrate the immediacy and the consistency of aesthetic impressions. In a series of experiments, Lindgaard et al. (2006) provided evidence in support of both prerequisites in the context of users’ evaluations of web pages. In the central experiment conducted by Lindgaard et al. (2006, Study 2), participants watched images of 50 web pages, each for a brief exposure of 500 ms. This exposure time was intended to be long enough to form a first impression, yet not sufficiently long to evaluate other features of the web site, such as its semantic content. After each page was shown the participants rated its visual attractiveness2 by using a continuous rating scale, ranging from 0 (for very unattractive web pages) to 100 (for very attractive web pages). Then, each participant viewed the 50 pages for a second time in a newly randomized order. The correlation between the mean evaluation of the visual attractiveness of web pages in the first phase and the mean evaluations in the second phase was 0.97, indicating that, when aggregating individual evaluations, even very short exposure resulted in remarkably consistent aesthetic evaluation. In a subsequent experiment (Study 3 in Lindgaard et al., 2006), exposure time was limited to 50 ms in an attempt to demonstrate that evaluating web pages may follow the pattern of ‘‘mere exposure effect’’ (Zajonc, 1980; Bornstein, 1992). The aggregated individual evaluations under the 50 ms condition were still very consistent, but intra-user consistency was considerably lower than in the 0.5 s condition. Lindgaard et al. speculate that this difference may have stemmed from users’ being able to take in more page content during the 500 ms condition than in the 50 ms condition. 1 We do not suggest that initial aesthetic impressions never change upon further reflections or experience. Our claim is that, in general, those impressions last considerable time. 2 In this work, we use the term ‘‘attractiveness’’ following its use by Lindgaard et al. (2006). In essence, we use this term interchangeably with ‘‘visually pleasing’’ or ‘‘(visual) aesthetics perceptions’’ or e.g. Lavie and Tractinsky (2004).

This study was designed to replicate, extend, and augment the novel and important findings of Lindgaard et al.’s study. In the experiments reported herein we have concentrated on the results of the 0.5 s condition and not on the mere exposure effect experiment, although our findings may shed some light on this issue as well. Replication and extension research, especially by independent researchers (i.e. not the original researchers reporting new findings) serves an important function in the advancement of science (Ehrenberg, 1990; Hubbard and Armstrong, 1994) by helping to evaluate the validity and the generalizability of previous studies. We elaborate on the elements of replication and extension in Section 2. Sections 3 and 4 present two experiments that were conducted to accomplish the study’s objectives. 2. Study objectives As mentioned above, the major objective of this study is to demonstrate the immediacy and the consistency of aesthetic impressions. This will be done in large part by replicating and extending the findings of Lindgaard et al. (2006). This process includes four sub-goals: 1. Generalization: Ehrenberg (1990) argues that in order to face the realities of real-world data ‘‘we must always be looking to see whether there is a generalizable result that holds across many different sets of data’’ (p. 196). Thus, the study’s most fundamental objective is to generalize Lindgaard et al.’s (2006) findings by using a new set of data that is based on different web pages, a different rating scale and with participants from a different culture. The specific details of the characteristics of our data sets are presented in the sections that describe the two experiments. Beyond generalization, the study has three additional goals: 2. Convergence: Here, we attempt to validate the results using converging evidence from an implicit measure (response latency) in addition to the use of an explicit measure (subjective rating). 3. Comparison of two types of evaluation consistency: The first type refers to the consistency by which evaluations of web pages are averaged over users in the two rounds of evaluations. However, it is also of interest to study a second type of consistency, one that gauges the degree to which individuals are internally consistent in evaluating aesthetic stimuli (e.g. Hassenzahl, 2004) and in particular stimuli that were presented for a very short duration. 4. Association of design dimensions and attractiveness of web pages: Here, we examine the relations between the attractiveness of web pages and two perceived dimensions of web-page design—classical aesthetics and expressive aesthetics (Lavie and Tractinsky, 2004). We elaborate on the last three goals below.

ARTICLE IN PRESS N. Tractinsky et al. / Int. J. Human-Computer Studies 64 (2006) 1071–1083

2.1. Response latency Lindgaard et al. (2006, Study 2) used a single-item subjective measure to study whether very short exposure can elicit aesthetic response. Naturally, the results obtained by such measures can be susceptible to measurement bias and error. In this study we use, in addition to Lindgaard et al.’s measure, an objective measure, response latency. Convergence of the results obtained from these two different measures will substantially increase the findings’ validity. Response latency is the length of time taken by a respondent to answer a question. As a measure, response latency has several general advantages: it is unobtrusive and is very easy to collect over computerized systems. In addition, there are three specific reasons for our interest in this measure. 1. Our major thesis is that certain aesthetic responses are immediate and precede elaborated cognitive processes. Obviously, if the process of interest involves chronometric considerations, the appropriate measure of choice should be one that measures durations. Indeed, research on the immediacy of affective responses, on the spontaneity of judgement and on the automaticity of information processing has made considerable use of response latency data (e.g. Herr and Page, 2004). For example, by measuring response latencies, Duckworth et al. (2002) have shown that evaluative response to both novel and familiar stimuli can be immediate, unintentional, and appropriate (in the sense that the response matches the stimulus message). 2. Research suggests that response latency can be used as a measure of strength of preferences between alternatives (e.g. MacLachlan et al., 1979; Tyebjee, 1979; Aaker et al., 1980). Response latency is also indicative of implicit attitudes towards objects (e.g. Greenwald et al., 1998) and of the strength of attitudes (e.g. Bassili, 1996). Research on consumer behaviour suggests that the time needed to choose between two brands is inversely proportional to the ‘psychological distance’, which separates the two brands. This is because if the brands appear to be more similar, the choice ‘‘will be more difficult, and hence take longer, than if one brand clearly dominates the others’’ (Tyebjee, 1979, p. 96). MacLachlan et al. (1979) suggest that ‘‘the faster an answer is given, the stronger the respondent’s conviction’’ (p. 573). This effect has been demonstrated regardless of whether the questions were phrased in positive or negative terms (Shipley et al., 1946). The effect was also demonstrated regardless of whether people replied to factual items or expressed subjective preferences. Thus, in MacLachlan et al. (1979), participants were asked about the makers of certain cereal brands. Average response time for correct answers was 1.5 s faster than for wrong answers. Dashiell (1937) showed subjects pairs of colours and asked what colour was preferred. The stronger a colour was preferred, the faster it was chosen. In addition, it was found that attitudes expressed in shorter response latencies predict greater resistance to persuasion (Stocke´, 2003) and that behaviour-

1073

al intentions (e.g. voting) expressed in shorter latencies were better predictors of actual behaviour (Bassili, 1996). Finally, conflicting thoughts or feelings take longer to express than coherent ones (Bassili, 1996). Most studies employing response latency measures to infer preferences have used binary choice tasks in which the psychological distance between the two stimuli was manipulated and its effects on response latencies observed. Because of the large space required to present web pages for users’ evaluations, our study is limited to registering latencies from judgements of stimuli (i.e. a web page) that are presented one-by-one. Relatively little research has been done using response latencies in this mode. However, it is reasonable to propose that the more extreme evaluations of web-page attractiveness will be associated with shorter response latencies. This proposition is based on two lines of reasoning. Firstly, Ostrom and Gannon (1996) found that extreme ratings on an evaluation scale are easier to generate than are ratings at the middle of the scale. Hence, we believe that generating extreme ratings should be shorter than nonextreme ratings. Secondly, based on the studies surveyed in this sub-section, users’ conviction will be stronger when they judge pages that are considerably more or considerably less attractive than average pages. Presumably, very attractive or very unattractive web pages represent a coherent object towards which feelings are manifested quickly. Web pages that are not decidedly attractive or unattractive are probably not perceived coherently from an aesthetic viewpoint and therefore their evaluations will be slower due to the inconsistent feelings towards them. Indeed, Pham et al. (2001) found that extreme ratings of pictures—whether positive or negative—were associated with lower latencies than were more moderate ratings. Bassili (1996) reports similar pattern of the relations between response latency and extremity of opinion. In the context of aesthetic evaluations of furniture, Ritterfeld (2002) found that response latency was shorter for prototypical stimuli (i.e. stimuli that already conform to a preference schema) than for polyvalent stimuli (i.e. stimuli that involve greater level of preference uncertainty). In addition, Ritterfeld (2002) predicts that latencies of very negative evaluations will be shorter than latencies of very positive evaluations, because negative evidence looms larger during the decision-making process. For example, in the context of HCI, users may look for additional evidence before deciding that a web page is perfect, whereas a few salient negative features may suffice to conclude that a web page is totally unattractive. 3. Finally, Aaker et al. (1980) suggest that measuring response latency can be helpful in assessing the construct validity of preference measurements. The idea is that a ‘‘true test of construct validity requires ‘maximally different methods’ to determine convergent and discriminant validity’’ (p. 237). As a very different measurement method relative to explicit ratings of stimuli, response latency can serve as such a method. Thus, the use of the

ARTICLE IN PRESS 1074

N. Tractinsky et al. / Int. J. Human-Computer Studies 64 (2006) 1071–1083

two methods (rating and response latency) can serve as a test for convergence across different measures of the same ‘‘thing’’ (Cook and Campbell, 1979), given that, as mentioned above, we expect response latency to covary with rating extremity. 2.2. Evaluation consistency As mentioned above, in order to support the premise that aesthetic perceptions of the IT artifact (in our case, web pages) influence subsequent judgements of the artifact, it is necessary to demonstrate that these perceptions remain stable (i.e. consistent) over time. The consistency of webpage aesthetics can be assessed in two different ways: (1) Users’ evaluations of web pages can be averaged for each page. The association between averages of repeated evaluations can then be indicative of the degree to which the study’s sample (and by generalization the target population) maintains a consistent rating of each page relative to other pages. (2) The evaluations of the web pages are compared to repeated evaluations of the same pages within individual users. This method gauges the degree to which individuals are internally consistent in evaluating aesthetic stimuli (e.g. Hassenzahl, 2004). According to Monk (2004), for HCI designers—who design for populations rather than for individual users—the interesting question is whether, on average, products are rated consistently higher or lower relative to other products. However, it is also of interest to study the second type of consistency, that is, to assess the stability of aesthetic perceptions within individuals. Lindgaard et al. (2006) found that for very short stimuli exposure (0.5 s) sample-averaged attractiveness estimations were considerably more consistent than estimations of individual users. This difference may be due to the fact that the error associated with single estimations is reduced when independent estimations (i.e. from different, independent raters) are averaged. Lindgaard et al. (2006) examined intra-individual consistency based on correlations of each individual’s ratings with the average rating of the entire sample. We assess consistency in this study by correlating participants’ own ratings of the web pages on two occasions—the first after a 0.5 s exposure and the second after a 10 s exposure. 2.3. Aesthetic dimensions of web-page design We are also interested in analysing how design affects attractiveness. In other words, we were looking for design factors that are associated with attractive or unattractive web pages. The design of web pages involve many technical, low-level aspects that may also interact with each other, yielding a virtually infinite number of components to consider when analysing the site’s design. It is therefore essential to consider higher-level, more abstract concepts of web-site design. The HCI literature has yet to generate considerable research of such design

concepts. First steps in this direction have been taken by Kim and his associates (Kim et al., 2003; Park et al., 2004) who identified design factors that relate to web-page objects, backgrounds and the relationships between the objects and the backgrounds. Kim et al.’s (2003) work, however, does not allow for an overall score for any of the higher-level design factors, while Park et al.’s (2004) work identified 13 design factors, which may indicate insufficient level of abstraction for the purpose of this study. Lindgaard et al. (2006) have also attempted to associate several design factors with overall assessment of web-page attractiveness. However, it seems that their design factors were chosen ad hoc and the results failed to distinguish between the contributions of each factor to overall attractiveness evaluations. In another line of research, Lavie and Tractinsky (2004) proposed that people identify two high-level aestheticrelated dimensions of web pages. The dimensions— classical aesthetics and expressive aesthetics—represent users’ evaluation of the design. Classical aesthetics refers to the orderliness and clarity of the design. Expressive aesthetics refers to the originality, creativity and the richness of the design. While the two dimensions are based on users’ subjective evaluations of web pages, they seem to represent general and consensusal notions of aesthetics. For example, the two dimensions, which emerged in the context of web pages, closely resemble the aesthetic dimensions that emerged in works in other fields, e.g. environmental aesthetics (Kaplan, 1988; Nasar, 1988b) and design (Coates, 2003). The dimensions proposed by Lavie and Tractinsky (2004) resemble aesthetic notions such as concinnity and novelty (Coates, 2003). Classical aesthetic represent order and familiarity, whereas expressive aesthetics represent novelty—two visual aspects of the environment that induce pleasure. Indeed, Lavie and Tractinsky found that both dimensions are positively correlated with pleasing interaction, but that classical aesthetics is also strongly correlated with perceived usability. However, the above-mentioned correlates of the two aesthetics dimensions represent constructs that appear to stem from reflection, whereas attractiveness represents a more visceral response. Thus, it is difficult to predict which of the aesthetic dimensions is more strongly correlated with the attractiveness of web pages. Based on the similar correlations with the pleasure construct, as reported by Lavie and Tractinsky (2004), we expect that both dimensions will be correlated similarly with attractiveness. To summarize this section, we propose to study the immediacy and the consistency of aesthetic impressions. The study will be based on replicating the findings of Lindgaard et al. (2006); on extending that research with an implicit, independent measure; and on examining what high-level design dimensions correlate with immediate attractiveness evaluations. The study is comprised of two experiments. Experiment 1 was designed to answer the first three goals stated at the beginning of this section.

ARTICLE IN PRESS N. Tractinsky et al. / Int. J. Human-Computer Studies 64 (2006) 1071–1083

1075

Experiment 2 was designed to answer the fourth goal as well as to provide additional evidence for the first goal.

350

3. Experiment 1

250

70

200

60

3.1.2. Stimuli Fifty web pages (as in Lindgaard et al.’s studies 2 and 3) were evaluated in this study. The 50 web pages were arbitrarily selected for this experiment based on two criteria: (1) They did not belong to well-known web sites (to reduce the possible influence of familiarity on evaluations). (2) The stimulus set had to cover a wide range of attractiveness in order to be ecologically representative. Hence, we selected 25 pages that we considered to be relatively attractive and 25 pages that we considered to be relatively unattractive.3 The web pages came from a variety of domains (e.g. web developers, entertainment, art and design, retail, and personal web pages). Screen shots of the web pages were captured at a resolution of 1024
768 pixels in 24-bit true colour, but were compressed to JPG format with a resolution of 800
600 before being presented in the experiment. A subset of 24 web pages used in this experiment is presented in the Appendix.

80

50 150

40

100

30 20

50

Cumulative Percent

3.1.1. Sample Forty undergraduate business students (25 female, 15 male), enrolled in decision making and organizational behaviour classes volunteered to participate in the study for course credit. They were 19–28 years old (average ¼ 23.7). Prior to the experiment, the students did not participate in any class related to web-page design.

90

300

Frequency

3.1. Method

100

10

0

0 1

2

3 4 5 7 6 Attractiveness Ratings

Frequency

8

9

10

Cumulative Percent

Fig. 1. Frequency distribution and cumulative percent distribution of 2000 attractiveness ratings in Phase 1 (1 ¼ very unattractive, 10 ¼ very attractive).

A ‘‘Very Unattractive’’ and a ‘‘Very Attractive’’ verbal anchors were placed below the ‘‘1’’ button and the ‘‘10’’ button, respectively. There were no instructions or time limits regarding speed of rating.4 Before the experimental stimuli, a block of 10 trial images was administered to get the participants acquainted with the rating method and the short display times. Next, the 50 web-page images were presented in a random order; participants rated each image in turn, and pressed a ‘‘Continue’’ button when they were ready to proceed to the next image. After that phase, an instructions page informed the participants that they are done with the first phase and that the second phase is about to begin. The stimuli and the procedure in Phase 2 were identical to those of Phase 1, with the exception that images were presented for 10 s rather than for 0.5 s. The order of presentation of web-page images was again randomized for each participant.

3.1.3. Procedure Participants were briefed about the study’s general purpose and were given written instructions regarding the experimental task. In addition, each experimental phase was preceded by online instructions. The participants interacted with a computer system that included a P4 1.7 MHz processor and a 19 in display. A C# program in .NET environment was built to control the procedure, to present images of web sites, to control the display time of the images and to collect user data, including ratings of the web-site images and response latencies. The study consisted of two main phases. In Phase 1, each web-page image was displayed for 500 ms, after which the rating scale was displayed on the screen and the participants were asked to rate the attractiveness of the page that they just saw. The rating scale was represented by 10 radio buttons arranged in order from left to right.

Overall, 2000 attractiveness ratings and response latencies were collected in Phase 1 (40 participants
50 web pages). As Fig. 1 shows, the distribution of attractiveness evaluations suggests a quasi-normal distribution where more evaluations concentrate at the middle of the scale and fewer pages are evaluated as extremely attractive or unattractive. Also, extreme positive ratings are even rarer than extreme negative ratings. While the stimuli for this experiment were selected arbitrarily, there appears to be a reasonable distribution of attractiveness ratings, which limits concerns that a skewed sample of stimuli might limit

3 The rationale behind the selection was not to select extremely attractive and extremely unattractive pages. Rather, we wanted to cover the range of attractiveness that one would encounter on the Web, including intermediate levels of attractiveness. As can be seen in Section 3.2, this objective was met.

4 Practice differs regarding the use of speed instructions between the preference measurement paradigm (no such instructions) and the implicit attitude measurement paradigm. In both cases, though, the results are quite consistent regarding the relation between response latency and attitude or preference strength.

3.2. Results

ARTICLE IN PRESS N. Tractinsky et al. / Int. J. Human-Computer Studies 64 (2006) 1071–1083

1076

the generalizability of the study’s results (e.g. Norman, 2004b).

9

9 8

Phase 2

7 6 5 4 3 2 2

3

4

6 5 Phase 1

7

8

9

Fig. 2. Average raw rating for each web page in Phases 1 and 2 (each dot reflects the mean rating over 40 participants of each of 50 web pages).

Average Ratings in Phase 2

8

3.2.1. Attractiveness evaluations of web pages Evaluations were somewhat more favourable in Phase 2 (mean ¼ 5.29, SD ¼ 1.31) compared to Phase 1 (mean ¼ 5.06, SD ¼ 1.25). This difference is significant according to a paired-sample t-test (t(49) ¼ 3.23, p ¼ 0.002). In both cases, the average ratings were just slightly below the middle of the rating scale, indicating that the set of web pages chosen for this study was quite balanced in terms of the pages’ attractiveness. I n Phase 1, the average attractiveness of the web pages ranged from 2.5 for the least attractive page to 7.98 for the most attractive page. In Phase 2 the attractiveness evaluations ranged from 3.3 to 8.3. For each participant, the ratings of the web pages in each phase were transformed into z-scores to control for individual rating tendencies. The means of the raw scores and of the z-scores for each web page were calculated separately for each of the two experimental phases. The correlation of the mean z-scores of the 50 pages between the two phases was 0.92, which was practically identical to the correlation between the mean raw ratings of visual appeal for each web page on both phases (r ¼ 0.92). The relation between the raw ratings in both phases is depicted in Fig. 2. The high-explained variance (r2 ¼ 0.85) indicates that even with minimal exposure (i.e. only 0.5 s), the evaluation of web-page attractiveness (averaged over participants) was very consistent. These findings replicate the results of Study 2 of Lindgaard et al. (2006) despite the use of different web pages, a different sample from a different culture, an extended exposure in the second phase of the experiment and a slightly modified methodology (e.g. recording participants’ ratings using 10 radio buttons rather than a slider indicating a range of 0–100).

7 6 5 4 3 2 2

3

4 5 6 7 Average Ratings in Phase 1

8

9

Fig. 3. Average ratings by participants in Phases 1 and 2 (each dot reflects the mean rating over 50 web pages by one of the study’s 40 participants).

3.2.2. Within-participants analyses The mean evaluations within participants (over 50 web pages) ranged from 3.20 to 6.66 in Phase 1 and from 2.78 to 6.94 in Phase 2. The results are depicted in Fig. 3, which shows a noticeable variance in participants’ average ratings, together with a slight tendency to rate the pages as more attractive in the second phase relative to the first phase. The correlations of web-page evaluations within participants reflect the level of each participant’s consistency in rating the attractiveness of the 50 web pages in both phases of the study. The within-subject correlations ranged from 0.09 to 0.90, with an average correlation of 0.55 and a median of 0.60. These results indicate large variation in individual consistency. 3.2.3. Relation between rating extremity and response latency The columns in Fig. 4 depict the mean and the median of the 2000 response latencies as a function of each attractiveness rating obtained in Phase 1. It can be seen that both the mean and the median latencies of very attractive or very unattractive web pages are shorter than latencies of ratings that were placed at the middle of the scale. To test the relation between extremity of attractiveness rating and reaction times, an ANOVA was performed with ratings as random factors and the transformed latencies as a dependent variable.5 Ratings were treated as random factors with 5 levels based on their distance from the scale’s mid-points. For example, ratings of 5 and 6—which are the scale’s mid-point—belong to Category 0, whereas ratings of 1 and 10—which are the most extreme ratings—belong to category 4. There was a significant effect of extremity of 5 A logarithmic transformation is often used to reduce the skewness of response latency data.

ARTICLE IN PRESS N. Tractinsky et al. / Int. J. Human-Computer Studies 64 (2006) 1071–1083

1077

4. Experiment 2 Mean Median

Latency (sec.)

2.5

2

1.5

1 1

2

3

4

5

6

7

8

9

10

Attractiveness Evaluation Fig. 4. Evaluation latencies as a function of attractiveness ratings.

Table 1 Pairwise (Tukey’s HSD) post-hoc comparisons between transformed (ln) latencies of web-page ratings Rating categorya

Against categorya 3

4 (ratings of 1 and10) 3 (ratings of 2 and 9) 2 (ratings of 3 and 8) 1 (ratings of 4 and 7) 0 (ratings of 5 and 6)

ns —

2

1

0





ns —

ns ns —

   

—

 po0.05. a Categories 0–4 represent the extremity of evaluations of web-page attractiveness (0 ¼ middle of the scale; 4 ¼ ends of scale).

the rating on response latency (F(4, 1995) ¼ 10.815, po0.001). The weighted linear trend was significant (F ¼ 38.502, po0.001), while the term of deviation from linearity was insignificant (F ¼ 1.586, p ¼ 0.191). Table 1 displays the post-hoc comparisons between response latencies of the five pooled rating categories. The comparisons support the premise that the most extreme evaluations (Category 4) were significantly faster than other evaluations and that the least extreme evaluations (Category 0) were the slowest. Finally, we tested the difference in latencies between extremely positive ratings (ratings of 9 and 10) and extremely negative ratings (1 and 2). We collapsed the ratings of 9 and 10 into one category of extreme positive evaluations (for a total of 148 evaluations) and ratings of 1 and 2 into a category of extreme negative evaluations (317 evaluations). As expected, latencies of positive evaluations (mean ¼ 0.66, SD ¼ 0.37) were significantly longer than latencies of negative evaluations (mean ¼ 0.58, SD ¼ 0.40) [F(1, 463) ¼ 7.786, p ¼ 0.029].

Experiment 1 established that, on average, web pages that are perceived as attractive after a very short exposure are also perceived as attractive after longer exposure. An obvious question, thus, is what design characteristics distinguish between the more and the less attractive web pages. One may look for low level, very specific design factors (e.g. the use of a certain colour combination or the placement of a certain object in a certain place). Such research is still in its infancy (e.g. Park et al., 2005) and solid findings were not available when this study was conducted. Alternatively, one can look for higher level, more general characteristics. In this study, we used two dimensions that represent the aesthetics of web-page design (Lavie and Tractinsky, 2004). These dimensions describe high-level attributes of web-page design as perceived by the users. They are very comparable to dimensions that describe the aesthetics of other environments (Kaplan, 1988; Nasar, 1988b). The dimensions—classical and expressive—are distinct (albeit correlated) aspects of webpage aesthetics. Classical aesthetics refers to the orderliness, or clarity of the design. Expressive aesthetics refers to the creativity and the richness of the design. Thus, in this experiment we examined whether the attractiveness of web pages is related to their design as captures by the classical and expressive aesthetics dimensions. In addition, the experiment was conducted such that a comparison can be made with Experiment 1 regarding the relative attractiveness of web pages. High correlations between attractiveness ratings in different samples would increase the generalizability of the study’s findings. 4.1. Method 4.1.1. Procedure The procedure used for this experiment was similar to Experiment 2 with two modifications. In the second phase, instead of once again rating the pages’ attractiveness (as in Experiment 1) the participants rated the pages on 6 items measuring expressive and classical aesthetics. The six items were shortened versions of the original scales developed by Lavie and Tractinsky (2004). We chose to use the short version to reduce the effects of repetitiveness and fatigue on users’ ratings. In both phases the web pages were presented in a random order. Because the evaluations of classical and expressive aesthetics require more reflection than a single attractiveness rating, the exposure time of web pages in the second phase was not limited (as opposed to a 10 s limit in Experiment 1). 4.1.2. Stimuli Twenty-four web pages were selected from the 50 pages used in Experiment 1. The pages were selected based on their attractiveness ratings in that experiment. We selected the eight most attractive pages, the eight least attractive

ARTICLE IN PRESS 1078

N. Tractinsky et al. / Int. J. Human-Computer Studies 64 (2006) 1071–1083

pages, and eight pages that were rated in the middle of the pack. The 24 web pages are presented in the Appendix.

9 8

4.1.4. Measures In addition to an attractiveness measure obtained in Phase 1, users’ evaluations of the web pages in Phase 2 were captured by 10-point rating scales. The rating scales asked participants to mark their level of agreement with statements regarding the page’s design. The statements included three expressive aesthetics statements (sophisticated, creative and fascinating) and three classical aesthetics statements (clean, pleasant and aesthetic). Cronbach’s a reliabilities of these scales were computed separately for each page. The reliabilities ranged from 0.70 to 0.92, with an average of 0.84 for both the classical and the expressive scales over all 24 pages. 4.2. Results The average attractiveness ratings after 0.5 s exposure are presented below the thumbnail of each page in the Appendix. The results of the attractiveness ratings were compared to the results of the attractiveness ratings of the same subset of pages gathered after the same exposure time in Experiment 1. Fig. 5 plots the average attractiveness ratings of the 24 pages in Experiments 1 and 2. Overall, ratings were somewhat higher in this experiment (mean ¼ 5.63, SD ¼ 1.35) compared to the average ratings of the same 24 pages in Experiment 1 (mean ¼ 5.15, SD ¼ 1. 56). Still, the average attractiveness evaluations of the web pages in the two samples were highly correlated (r2 ¼ 0.84). The mean attractiveness ratings of the 3 groups of web pages, after a .5 s exposure, were in accordance with expectations (See Table 2). The attractiveness ratings of the top-eight pages were higher than that of the middle group, and the attractiveness of the bottom group was the lowest. A repeated measures ANOVA with Attractiveness Group as the dependent variable was highly significant (F(1.70, 88.63) ¼ 250.993, po0.001, degrees of freedom are Greenhouse–Geisser corrected). Tests of within-subjects contrasts between all groups were highly significant as well (po0.001). Figure 6 presents three scores (attractiveness, classical and expressive aesthetics) for each web page, averaged over 53 participants. The web pages are sorted in this figure from left to right in a descending order of attractiveness ratings (denoted by triangles). We split the data points in Fig. 6 to two categories (rather than use the original three categories) because such visualization supports better insights than if we had preserved the original three-group

7 Experiment 2

4.1.3. Sample Fifty-three undergraduate engineering students (who did not participate in Experiment 1) participated in this experiment for class credit. The sample included 37 male and 16 female students. The average age was 24.9 years.

6 5 4 3 2 2

3

4

5 6 Experiment 1

7

8

9

Fig. 5. Average attractiveness ratings of 24 web pages after 0.5 s exposure in Experiments 1 and 2. Table 2 Mean ratings of attractiveness (in the first phase, after 0.5 s exposure), classical aesthetics and expressive aesthetics (in the second phase, unlimited exposure time) of the top-8, middle-8 and bottom-8 web pages Pages

Attractiveness

Classical aesthetics

Expressive aesthetics

Top-8 Middle-8 Bottom-8

7.03 (0.88) 5.68 (0.87) 4.19 (0.96)

7.28 (0.97) 5.77 (0.93) 4.80 (1.14)

7.12 (1.01) 4.93 (0.80) 2.88 (0.85)

Grand mean

5.63 (0.73)

5.95 (0.79)

4.98 (0.70)

classification. (An analysis of 3 groups provides the same results for the most attractive and for the least attractive groups, with inconclusive results for the intermediate group). As can be seen clearly in Fig. 6, the three measures are correlated: in general, the more attractive pages scored higher on both aesthetic dimensions (r ¼ 0.86 with classical aesthetics and r ¼ 0.95 with expressive aesthetics). These results demonstrate the very strong relationships between the very brief attractiveness ratings and the more elaborated assessments of classical and expressive aesthetics. Still, the most salient pattern observed in this figure suggests that the least attractive pages (the right-hand side of Fig. 6) are markedly low on expressive aesthetics (denoted by rectangles). For example, the page with the lowest expressive aesthetics score in the top half of attractive pages was still 1.75 points above the page with the highest expressive aesthetics score in the bottom half of attractive pages. The differences in classical aesthetics (denoted by diamonds) between the two groups were similar, though less pronounced. To assess the relations between attractiveness and the two aesthetics dimensions we conducted two types of

ARTICLE IN PRESS N. Tractinsky et al. / Int. J. Human-Computer Studies 64 (2006) 1071–1083

10 9 8

Rating

7 6 5 4 3 2 1 Most Attractive

Least Attractive Web Pages

Attractiveness

Classical

Expressive

Fig. 6. Average ratings (N ¼ 53) of attractiveness (in the first phase, after 0.5 s exposure), classical aesthetics and expressive aesthetics for each of 24 web pages, sorted by attractiveness rating. Vertical dashed bar separates the top 12 and bottom 12 pages according to attractiveness level. Horizontal dashed bar indicates the middle of the rating scales.

1079

were obtained in Phase 1, and the two aesthetic dimensions, which were obtained in Phase 2. A linear regression analysis was conducted for the entire sample of 24 web pages and separately for the group of 12 most attractive pages and for the group of 12 least attractive pages. Attractiveness ratings were regressed on classical aesthetics and expressive aesthetics scores. For the entire sample of web pages, only classical aesthetics contributed significantly to the equation (standardized regression coefficient ¼ 0.356, p ¼ 0.014). Considering only this predictor, the model’s adjusted R2 was 0.20. Within the group of the most attractive pages (the left-hand side in Fig. 6), only expressive aesthetics contributing significantly (b ¼ 0.336, p ¼ 0.03). Considering only this predictor, the model’s adjusted R2 was 0.18. Within the group of the least attractive pages (the right-hand side in Fig. 6), classical aesthetics was the only significant contributor (b ¼ 0.426, p ¼ 0.003), with adjusted R2 ¼ 0.25. The results indicate that overall, and among the least attractive pages, classical aesthetics contributed more to explaining variations in immediate attractiveness ratings. Expressive aesthetics explained only the attractiveness variations within the group of the more attractive pages. 5. Discussion

analyses.6 The first analysis was designed to test whether the evaluations of classical and expressive aesthetics obtained in the second phase of this experiment were associated with the three pre-defined attractiveness categories (8 web pages in each of the high, medium, and low levels). We used a repeated measures multivariate analysis of variance (MANOVA), with Attractiveness Group as a three-level within-subject factor and expressive and classical aesthetics as the dependent variables. The multivariate within-subjects test was highly significant (Wilks’ Lambda ¼ 0.088, F(4,206) ¼ 122.056, po0.001). The univariate tests were highly significant for both classical (F(1.57, 81.83) ¼ 134.47, po0.001, Partial Z2 ¼ 0.721) and expressive aesthetics (F(1.69, 87.80) ¼ 510.84, po0.001, Partial Z2 ¼ 0.908) (Greenhouse–Geisser corrected degrees of freedom in both tests). For both dependent variables, within-subjects contrasts between the three groups of web pages were significant (po0.001). The differences were more pronounced on the expressive aesthetics dimension (Partial Z2 ¼ 0.88 and 0.84 between the top group and the middle group and between the middle group and the bottom group, respectively) than on the classical aesthetics dimensions (Partial Z2 ¼ 0.77 and 0.45 for the same contrasts, respectively). A second, complementary, analysis was designed to test the association between the attractiveness ratings, which 6

The analyses were conducted to test associations, not causality; hence, the use of independent or dependent variables in these analyses do not imply causes or effects.

The major objective of this study was to demonstrate that aesthetics impressions of a web site are formed after a very brief exposure to the site and that that such impressions are not transient. Experiment 1’s findings demonstrate that users are able to form immediate and consistent evaluation of the attractiveness of web pages. These evaluations were very consistent across web pages. That is, the degree to which web pages were regarded, on average, as attractive after a very short exposure remained stable given a considerably longer exposure, lending support to the proposition that the relative attractiveness of web pages is determined quickly (Lindgaard et al., 2006). These results are in line with Zhang and Li’s (2004b) findings regarding the immediate and continuous effect of the system’s affective quality on users’ cognition and usage patterns. Past studies that have demonstrated the immediacy of certain affective reactions (e.g. Duckworth et al., 2002) have mostly been based on simple stimuli. This study and the studies cited above indicate that such reactions also occur for visual aspects of computer software, and especially of web pages, which is considerably more complex. Further support for the immediate aesthetic evaluation proposition is provided by the response latency measure. One of the study’s major objectives was to find convergence between the explicit and the implicit measures of attractiveness. Such convergence was demonstrated for both attractive and unattractive evaluations, in line with similar findings from different contexts (e.g. Bassili, 1996; Ostrom and Gannon, 1996; Pham et al., 2001; Ritterfeld, 2002). In addition to the specific contribution of this measure to the research context, this study demonstrates the potential of

ARTICLE IN PRESS 1080

N. Tractinsky et al. / Int. J. Human-Computer Studies 64 (2006) 1071–1083

response-latency as an easy-to-collect, unobtrusive measure of preferences and attitudes in HCI research. There are very few studies about individual differences in reactions to aesthetics of IT products (see Hassenzahl, 2004). This research area is important if we assume that users’ aesthetic preferences play a role in their evaluation of web pages and in their subsequent interactions with web sites. Thus, it is interesting to note the differences in the participants’ average ratings of the set of 50 web pages. Whereas some participants rated the entire set of web pages as fairly unattractive (lower than 4 on a 1–10 scale), other participants rated it much higher (close to 7 on that scale). It is also important to note that while the relative attractiveness of web pages (i.e. designed objects) remained stable between the two phases of Experiment 1 when averaged over evaluators (i.e. users), the evaluations within individual users were less consistent. There are several possible reasons for this finding: It may be that there are individual differences in people’s ability to consistently rate the attractiveness of objects, especially given such short exposures as in the first phase. In addition, people may differ in their ability to distinguish nuances in design. People who are better at that may have detected additional information during the longer exposure times in Phase 2. This may have led to lower consistency between the two phases for these participants. Finally, the differences may simply reflect the statistical property of sample means to be more consistent than single observations. Thus, the results highlight the two sides of aesthetic evaluations. On the one hand, evaluations over a sample of users can provide a reliable and consistent measure of the general attractiveness of web pages. At the same time, individual users may differ in terms of their tastes and evaluations of web pages. Hence, despite the benefits of designing for the average user, there is still room and need for tailoring the visual design of web pages to various users’ tastes. The proliferation of software skins reflects the demand for such aesthetic personalization of IT applications (Tractinsky and Zmiri, 2006). Some promising work has already been done on the question of what design characteristics affect evaluations of web pages (Kim and Moon, 1998; Kim et al., 2003; Park et al., 2005). In Experiment 2 we examined the association of immediate first impressions with two aesthetic dimensions of web pages, classical aesthetics and expressive aesthetics (see Fig. 6). In general, the results indicate that positive immediate impressions were associated with high levels of both aesthetic dimensions. However, unattractive pages were associated mainly with very low levels of expressive aesthetics. Perhaps the lack of expressiveness in those web pages left a dull design that users could not consider as attractive. In addition, the more attractive web pages were associated to various degrees with the two aesthetic dimensions. That is, the design dimensions measured in this study do not offer ‘‘golden rules’’ for designing attractive web pages. These results reflect the findings that the relationships between various design

dimensions and perceptions of web sites were not stable (Chen et al., 2002). Rather, they depend on the type of web sites evaluated and the population from which users are sampled. Thus, while some general conclusions can be drawn—e.g. that less attractive pages are characterized by very low levels of expressive aesthetics, or that highly attractive pages reflect high levels of both aesthetics dimensions—there are still many contingencies involved in the creation of first impressions. There is ample room for future research to elaborate on this issue. For example, cultural and individual differences, as well as the web domain (Zhang et al., 2001) may be important determinants of how users perceive the attractiveness of web sites. While the findings of this study relate to a presumably minor issue of web site design, the implications may be far reaching. Recent studies have argued that positive affect improves decision making, trust and social interactions (e.g. Isen (2001). Other studies found that decisions consist of a mix of conscious and nonconscious processes (Bargh, 2004) and that the degree to which nonconscious processes influence choice processes ‘‘is much greater than most choice researchers believe’’ (Fitzsimons et al., 2002). Studies have also shown that one’s affective states are related to aesthetics of one’s environment, be it in the working place (Rafaeli and Vilnai-Yavetz, 2004), the home or the neighbourhood (Nasar, 1988b), the store (Russell and Pratt, 1980), or the web site (Kim et al., 2003; Zhang and Li, 2004a). In line with Lindgaard et al. (2006), this study supports a possible explanation for various studies that found influence of aesthetics on attitudes towards the IT artifact (e.g. Schenkman and Jonsson, 2000; Tractinsky et al., 2000; Lindgaard and Dudek, 2003; van der Heijden, 2003). Since aesthetic information is evaluated immediately, it is largely responsible for the users’ first impressions. Subsequently, new information tends to be processed in a way that is biased towards those first impressions (Fitzsimons et al., 2002). And because the aesthetic impressions are quite stable, there is probably a need for significant counter evidence on other system attributes (e.g. usability, reliability, functionality) to alter the users’ first impressions. The findings do not imply that first aesthetic impressions are solely responsible for users’ attitudes towards web pages, just as attitudes towards other humans are not determined by aesthetic first impressions alone (Eagly et al., 1991). Many factors can potentially moderate the relations between first aesthetic impressions of an IT artifact and the attitudinal or behavioural consequences of the interaction (Tractinsky, 2004). Still, as the adage goes, there is no second chance to make a first impression. 6. Summary The study fulfilled the four objectives set forth at the beginning of this paper. First, we replicated Lindgaard et al.’s (2006) findings using different web sites, different users and a slightly different rating scale. Second, we

ARTICLE IN PRESS N. Tractinsky et al. / Int. J. Human-Computer Studies 64 (2006) 1071–1083

provided converging evidence from an independent, implicit measure (response latency) to the premise that users can consistently judge the attractiveness of web pages even after very brief exposure. Third, we showed that while average ratings of web-page attractiveness are highly consistent, there is considerable variance in the degree to which individual users are consistent in their evaluations. Fourth, we found some associations between immediate attractiveness ratings and more elaborated evaluations of two aesthetic dimensions of web pages—classical and

1081

expressive. Overall, the findings suggest that visual aesthetics plays an important role in users’ evaluations of web pages and of interactive systems in general.

Appendix Thumbnails of 24 web pages used in Experiment 2, sorted by average attractiveness ratings (shown below each web page).

ARTICLE IN PRESS 1082

N. Tractinsky et al. / Int. J. Human-Computer Studies 64 (2006) 1071–1083

References Aaker, D.A., Bagozzi, R.P., Carman, J.M., MacLachlan, J.M., 1980. On using response latency to measure preference. Journal of Marketing Research 17, 237–244. Bargh, J.A., 2004. Bypassing the will: towards demystifying the nonconscious control of social behavior. In: Hassin, R.R., Uleman, J.S., Bargh, J.A. (Eds.), The New Unconscious. Oxford University Press, New York.

Bassili, J.N., 1996. The how and why of response latency measurement in telephone surveys. In: Schwarz, N., Sudman, S. (Eds.), Answering Questions. Jossey-Bass Publishers, San Francisco. Bornstein, R.F., 1992. Subliminal mere exposure effects. In: Bornstein, R.F., Pittman, T.S. (Eds.), Perception Without Awareness: Cognitive, Clinical, and Social Perspectives. The Guilford Press, New York, pp. 191–210. Chen, Q., Clifford, S.J., Wells, W.D., 2002. Attitude toward the Site II: new information. Journal of Advertising Research, 33–45.

ARTICLE IN PRESS N. Tractinsky et al. / Int. J. Human-Computer Studies 64 (2006) 1071–1083 Coates, D., 2003. Watches Tell More than Time. McGraw-Hill, New York. Cook, T.D., Campbell, D.T., 1979. Quasi-Experimentation. Houghton Miffin Co., Boston. Dashiell, J.F., 1937. Affective value distances as determinants of esthetic judgment-times. American Journal of Psychology 50, 57–67. Dion, K., Berscheid, E., Walster, E., 1972. What is beautiful is good. Journal of Personality and Social Psychology 24 (3), 285–290. Duckworth, K.L., Bargh, J.A., Garcia, M., Chaiken, S., 2002. The automatic evaluation of novel stimuli. Psychological Science 13 (6), 513–519. Eagly, A.H., Ashmore, R.D., Makhijani, M.G., Longo, L.C., 1991. What is beautiful is good, buty: a metaanalytic review of research on the physical attractiveness stereotype. Psychological Bulletin 110 (1), 109–128. Ehrenberg, A.S.C., 1990. A hope for the future of statistics: MSOD. American Statistician 44, 195–196. Fitzsimons, G.J., Hutchinson, J.W., Williams, P., Alba, J.W., Chartrand, T.L., Huber, J., Kardes, F.R., Menon, G., Raghubir, P., Russo, J.E., Shiv, B., Tavassoli, N.T., 2002. Non-conscious influences on consumer choice. Marketing Letters 13 (3), 269–279. Greenwald, A.G., McGhee, D.E., Schwartz, J.L.K., 1998. Measuring individual differences in implicit cognition: the implicit association test. Journal of Personality and Social Psychology 74, 1464–1480. Hamermesh, D., Biddle, J., 1994. Beauty and the labor market. American Economic Review 84 (December), 1174–1194. Hamermesh, D., Parker, A.M., 2005. Beauty in the classroom: professors’ pulchritude and putative pedagogical productivity. Economics of Education Review 24 (4), 369–376. Hassenzahl, M., 2004. The interplay of beauty, goodness and usability in interactive products. Human–Computer Interaction 19 (4), 319–349. van der Heijden, H., 2003. Factors influencing the usage of websites: the case of a generic portal in the Netherlands. Information and Management 40, 541–549. Herr, P.M., Page, C.M., 2004. Asymmetric association of liking and disliking judgments: so what’s not to like? Journal of Consumer Research 30, 588–601. Hubbard, R., Armstrong, J.S., 1994. Replications and extensions in marketing—rarely published but quite contrary. International Journal of Research in Marketing 11, 233–248. Isen, A.M., 2001. An influence of positive affect on decision making in complex situations: theoretical issues with practical implications. Journal of Consumer Psychology 11 (2), 75–85. Kaplan, S., 1988. Perception and landscape: conceptions and misconceptions. In: Nasar, J.L. (Ed.), Environmental Aesthetics: Theory, Research, and Applications. Cambridge University Press, Cambridge. Karvonen, K., 2000. The beauty of simplicity. In: Proceedings of the ACM Conference on Universal Usability (CUU 2000), November 16–17, 2000, Washington DC, USA. Kim, J., Moon, J.Y., 1998. Designing towards emotional usability in customer interfaces—trustworthiness of cyber-banking system interfaces. Interacting with Computers 10 (1), 1–29. Kim, J., Lee, J., Choi, D., 2003. Designing emotionally evocative homepages: an empirical study of the quantitative relations between design factors and emotional dimensions. International Journal of Human–Computer Studies 59 (6), 899–940. Lavie, T., Tractinsky, N., 2004. Assessing dimensions of perceived visual aesthetics of Web sites. International Journal of Human–Computer Studies 60, 269–298. Lindgaard, G., Dudek, C., 2003. What is this evasive beast we call user satisfaction? Interacting with Computers 15, 429–452. Lindgaard, G., Fernandes, G.J., Dudek, C., Brownet, J., 2006. Attention web designers: you have 50 ms to make a good first impression!. Behaviour and Information Technology 25 (2), 115–126. MacLachlan, J., Czepiel, J., LaBarbera, P., 1979. Implementation of response latency measures. Journal of Marketing Research 16, 573–577.

1083

Monk, 2004. The product as a fixed effect fallacy. Human–Computer Interaction 19 (4), 371–375. Nasar, J.L. (Ed.), 1988a. Environmental Aesthetics: Theory, Research, and Applications. Cambridge University Press, Cambridge. Nasar, J.L., 1988b. Perception and evaluation of residential street scenes. In: Nasar, J.L. (Ed.), Environmental Aesthetics: Theory, Research, and Applications. Cambridge University Press, Cambridge. Norman, D.A., 2004a. Emotional Design: Why We Love (or Hate) Everyday Things. Basic Books, New York. Norman, D.A., 2004b. Introduction to the Special Issue on Beauty, Goodness and Usability. Human–Computer Interaction 19 (4), 311–318. Ostrom, Gannon, 1996. Exemplar generation: assessing how respondents give meaning to rating scales. In: Schwarz, N., Sudman, S. (Eds.), Answering Questions. Jossey-Bass Publishers, San Francisco. Park, S., Choi, D., Kim, J., 2004. Critical factors for the aesthetic fidelity of web pages: empirical studies with professional web designers and users. Interacting with Computers 16, 351–376. Park, S., Choi, D., Kim, J., 2005. Visualizing E-Brand Personality: exploratory studies on visual attributes and E-Brand Personalities in Korea. International Journal of Human–Computer Interactions 19 (1), 7–34. Pham, M.T., Cohen, J.B., Pracejus, J.W., Hughes, G.D., 2001. Affect monitoring and the primacy of feelings in judgment. Journal of Consumer Research 28, 167–188. Porteous, J.D., 1996. Environmental Aesthetics: Ideas, Politics and Planning. Routledge, London. Postrel, V., 2002. The Substance of Style. Harper Collins, New York. Rafaeli, A., Vilnai-Yavetz, I., 2004. Emotion as a connection of physical artifacts and organizations. Organization Science 15 (6), 671–686. Ritterfeld, U., 2002. Social heuristics in interior design preferences. Journal of Environmental Psychology 22, 369–386. Russell, J.A., Pratt, G., 1980. A description of the affective quality attributed to environments. Journal of Personality and Social Psychology 38 (2), 311–322. Schenkman, B.N., Jonsson, F.U., 2000. Aesthetics and preferences of web pages. Behavior and Information Technology 19 (5), 367–377. Shipley, W.C., Norris, E.D., Roberts, M.L., 1946. The effect of changed polarity of set on decision time of affective judgments. Journal of Experimental Psychology 1, 237–243. Stocke´, V., 2003. Measuring Information Accessibility and Predicting Response-Effects: The Validity of Response-Certainties and ResponseLatencies, Sonderforschungsbereich 504, 03-33. University of Mannheim. Tractinsky, N., 2004. Towards the Study of Aesthetics in Information Technology. In: Proceedings of the 25th Annual International Conference on Information Systems (ICIS). Washington, DC, December 12–15, pp. 771–780. Tractinsky, N., Zmiri, D., 2006. Exploring attributes of skins as potential antecedents of emotion in HCI. In: Fishwick, P. (Ed.), Aesthetic Computing. MIT Press, Cambridge, MA, pp. 405–422. Tractinsky, N., Shoval-Katz, A., Ikar, D., 2000. What is beautiful is usable. Interacting with Computers 13, 127–145. Tyebjee, T.T., 1979. Response latency: a new measure for scaling brand preference. Journal of Marketing Research 16, 96–101. Zajonc, R.B., 1980. Feeling and thinking: preferences need no inferences. American Psychologist 35 (2), 151–175. Zhang, P., Li, N., 2004a. The importance of affective quality. Communications of the Association for Computing Machinery 48 (9), 105–108. Zhang, P., Li, N., 2004b. Love at first sight or sustained effect? the role of perceived affective quality on users’ cognitive reactions to IT. In: Proceedings of the International Conference on Information Systems (ICIS), Washington, DC. Zhang, P., von Dran, G.M., 2000. Satisfiers and dissatisfiers: a two-factor model for website design and evaluation. Journal of the American Society for Information Science 51 (14), 1253–1268. Zhang, P., von Dran, G., Blake, P., Pipithsuksunt, V., 2001. Important design features in different website domains: an empirical study of user perceptions. e-Service Journal 1 (1).