- Email: [email protected]
The role of feedback in the assessment of news
Infi~rmation Processing & Management. Vol.33, No. 5, pp. 583-594, 1997
© 1997ElsevierScienceLtd All rightsreserved,Printedin GreatBritain 0306-4573/97$17+0.00
Pergamon Pll: S0306-4573(97)00018-6
THE ROLE OF FEEDBACK
IN THE ASSESSMENT
OF NEWS
JOHN E. NEWHAGEN College of Journalism, Universityof Maryland, College Park, College Park, MD 20742, USA Abstract--The 25-year search for a compelling online news delivery system has been fraught with frustration, This study examines the problem by focusing on how mass media audiences and online users differ in their assessment of news. It employed a two-wave national survey (N=!335) to study the perception of interactivity in mass media and computer networks, and its relationship to the assessment of news. The first wave looked at a national probability sample, while the second wave targeted viewers of NBC Nightly News who responded to the show via e-mail. NBC respondents rated mass media to be less interactive, while they rated computer communication more interactive than the national sample. The NBC group also rated mass media news less important and of lower quality than did the national sample. Interactivity ratings did not predict mass media credibility assessments for either group. However, respondents who defined interactivity as cybernetic feedback or who contacted NBC by e-mail rated computer communication to be more credible than those who did not. Caution is suggested in attempting to design online news filters solely to replicate the editorial news selection process employed by mass media. Based on the evidence that users assess news differently in an interactive media system than in a mass media system, more design emphasis on feedback is encouraged. © 1997 Elsevier Science Ltd
I. THE ROLE OF FEEDBACK IN THE ASSESSMENT OF NEWS
The potential for computer networks, such as the Intemet, to play a role in the delivery of news has captured the imagination of professionals and researchers alike. Yet, the implementation of that idea remains elusive, fraught with both conceptual ambiguity, and practical intransigence. Despite the intuitive appeal of using computers to deliver news, 25 years of experiments and pilot projects have yet to develop into a viable system. Naysayers to the current wave of interest recall Knight Ridder's failed video text experiment in the 1970s (for details, see Emery & Emery, 1988). Even as large newspapers move into an online environment, they do so hesitantly, and none expect to realize a profit any time soon. One source of trouble in the transition of news from print to electronic text may relate to theoretical ambiguity about the nature of the relationship between content and user. This is especially apparent in understanding the importance of feedback in user assessment of content in an online environment. This study examines data from a two-wave national survey (N= 1335) to consider the role of user feedback in news assessment. The survey included questions focusing on the perception and use of computers and mass media as sources for news. The first wave looked at a national random sample, while the second wave targeted viewers of NBC Nightly News who responded to the show via e-mail. Respondents of the NBC subsample are unique in that they spontaneously employed the Internet as a dynamic feedback channel to a traditional mass media news source. First, the concept of news as a special category of survival relevant information will be suggested as an alternative to the simple assumption of traditional mass media operational definitions. This opens the possibility of looking at differences in user assessment of information importance, quality, and credibility between mass media and online platforms. Next, the idea of feedback will be considered as the dimension distinguishing truly interactive computer-based information systems from mass media systems. 583
584
John E. Newhagen
1.1. The problem with news Asking a seasoned editor to define just what news is will frequently lead to an embarrassed silence. Difficulty explaining an over-learned skill is not unusual (see Sweller, 1988), and journalists are no exception. Rather than applying some explicit set of proscriptive rules, editors make decisions based on intuitional judgments guided by a century-old set of professional canons. This lack of a clear notion of what distinguishes news as a special class of information leads to the substitution of operational definitions: news is what appears in newspapers. The advent of television did not really change this formula: broadcast news is information that could be in a newspaper, only with sound and moving pictures. The problem then contaminates research to the degree that news is frequently conceptualized solely from an operational standpoint. For instance, Allen's benchmark discussion (Allen, 1990) of user model-based filters in an online news delivery system does exactly that, simply defining news to be the contents of newspapers or wire services. The limits of this approach appear in the inability to predict users preference beyond coarse-grained production-oriented categories that, not coincidentally, correspond to the sections of a newspaper. An early attempt in communication research to model mass media use can be seen in the 'uses and gratifications' approach (see Blumler & Katz, 1974). For instance, readers might look at the front section of a newspaper for different reasons than they do the sports section. However, to the degree that the uses and gratifications perspective also is based on standard productionoriented categories, it failed to describe how information in those categories might be theoretically unique. For instance, a report concerning legislation about anti-trust waivers for professional baseball could as easily appear in a newspaper's main section as it could in its sports section. The challenge to building an online news delivery system is even more difficult because it has to deal with the dynamic nature of both news and the user's information needs in an environment that is, by its nature, much more volatile than mass media. Newhagen & Levy (1997) define news as survival relevant information useful in adapting to change. This information-processing perspective models a user making survival-contingent decisions in a hostile environment based on limited information in real time (see Geiger & Newhagen, 1993). This theory helps ground concepts frequently studied in the assessment of news, including importance, quality, and credibility. Each of these dimensions will be discussed in the context of news as survival contingent information. 1.1.1. Information importance. Importance bears most directly on the novelty and proximity of news. Novelty, especially when it is proximate, activates emotion-based action states such as fear, and leads to information gathering used in making survival-contingent decisions (Frijda, 1988). Negative images of human death and suffering in television news, for instance, compel viewer attention, and affect memory (see Newhagen & Reeves, 1992). Thus, novel information, especially when it has negative implications for the user's survival, will be deemed important. 1.1.2. Information quality. The assumption is that high quality information is better that low quality information when making survival-contingent decisions. However, a system compelled to make decisions under real-time stress might be willing to de-emphasize quality for the sake of efficiency. Gilbert (1991), for instance, proposes that in order to save time, information is first processed as if it were true, with veracity judgment postponed until later stages of cognition. He asserts this is an adaptive response to time pressure based on the assumption that the senses do not lie. However, this assumption becomes problematic when the information is technologically mediated, where the epitaph shifts from 'seeing is believing' to 'the camera does not lie'. Thus, while journalists set information quality as a high priority, users might not, especially when it is complex. 1.1.3. Credibility. The concept of credibility has dominated mass media research into the assessment of news. The prominence of credibility is, perhaps, a reflection of its importance to the canons of professional journalism, which hinge on the perception that reporting is unbiased and neutral. Empirical research into mass media credibility has focused on how message characteristics
Feedback and news
585
evoke behavioral or cognitive changes in the message receiver. For example, the physical appearance of a political candidate on television news has been shown to affect viewer perceptions of credibility (Lanzetta et al., 1985). Frequently the other two concepts mentioned here, importance and quality, are assumed to be collinear with credibility: that is, a highly credible message has both importance and quality. However, factor analysis searching for social psychological dimensions underlying credibility have been difficult to interpret. 1 While both theoretical and methodological reasons have been suggested for this lack of coherence, another reason might have to do with the very architecture of mass media systems themselves. Mass media are one-way delivery systems, designed to generate a practically infinite number of replica messages to very large audiences. But the cost of this efficiency to the user is reflected in the absence of real-time user feedback and distorts the natural information assessment process. For instance, viewers of political TV spots, finding themselves unable to query candidates about their positions on issues, may evaluate credibility in terms of physical appearance (Garramone, 1984). On the other hand, because of their architecture, computer networks are inherently interactive and open a feedback channel for users. This two-way capacity, then, opens the possibility that information consumers might evaluate news in an interactive environment much differently than they do when the message stream is restricted to a one-way flow. This is reflected in the way use is conceptualized; viewers passively watch television, while computer users actively search the Net. Feedback, then, may provide a clue to understanding how online users assess news differently from members of a mass media audience, and what difference that might make to constructing user models.
1.2. Conceptualizing feedback in a news delivery system Much of the content of face-to-face communication can be thought of as news. This is important because interpersonal exchanges are generally thought to be the most highly interactive form of human communication. Prior to the invention of movable type, information about current events, called tidings, frequently was communicated in face-to-face exchanges (Emery & Emery, 1988). However, by their nature, mass media news delivery systems are not interactive, and preclude the possibility of feedback (Newhagen et al., 1995) 2. Thus, it may be a mistake to attempt to map mass media user models onto an interactive computer network environment. For instance, Allen (1990) suggests gauging the performance of an adaptive user model for news selection by applying a 'Turing control', that is, comparing it with a journalist engaged in similar task (p. 516). However, a critical assumption underlying the use of a Turing control is that the mass media system represents a functional standard for an interactive communication model. In simpler times the journalist might have sampled user feedback through informal daily interactions with members of the community. However, as urban populations have increased and society has become more diverse and complex, this technique has reached its limit. Indeed, journalism is increasingly criticized for its detachment from its audience. Thus, the real promise of an online news delivery system may not have to do with its efficiency as an automated filter, but with its capacity to return feedback to the communication experience. Having said that, the next challenge is to address the conceptual ambiguity of the term interactivity itself: does it have to do with the responsiveness of the system? Its mutability? The intuitive quality of its interface? As a link between human agents? qn a literature review of credibility studies, Newhagen and Nass (1989) found factor analysis yielded anywhere from 2 to 12 dimensions. "Federal Communications Commission policy on broadcast license renewal accepts the notion that viewer mail constitutes a viable audience feedback system. That is, the policy promotes the idea that viewers will send mail to stations with critical comments about what they saw. Someoneat the station, then, would read the mail and use the information in subsequent programming decisions. However, the feasibility of such a system is limited even if broadcasters are given the benefit of the doubt concerning their good intentions to receive and respond to their audience. Traditional mail, after all, requires effort to write, send, and read and is not particularly timely in the context of the fast-paced world of broadcasting. While most newspapers also print a page of letters to the editor, neither does this practice represent a timely or functional feedback circuit in any practical sense.
586
John E. Newhagen
In the context of human communication, interactivity might best be conceptualized in terms of cybernetic feedback. While Wiener's work in cybernetic feedback had to do with engineering problems associated with servo-synchro mechanisms, the metaphor was soon applied to interactive social systems (Dechert, 1966). In either case cybernetic theory defines interactivity in terms of the feedback necessary to the maintenance of a self-regulating system (Rosenblueth et al., 1943). For instance, voting can be seen as feedback in a democracy, a self-regulating political system. From this perspective journalists working for newspapers or television do not employ adaptive user models, which incorporate feedback, when they make news judgments (see Newhagen & Levy, 1997). The other side of that coin leaves mass media users outside the news creation process, forcing them either to forgo message assessment, or employ criteria external to journalistic methodology. Hall (1980) details the problem message receivers have decoding meaning from news in a mass media system. On the other hand, a media system with a viable feedback channel might allow users to assess more actively the news they receive. This goes beyond a normative comment about which system is better; it suggests assessment processes for the two may be qualitatively different. That prospect should be of substantial interest to designers, to the degree that it will affect the very nature of the user models such systems embody.
1.3. A crude beginning: Using the Internet as a feedback channel for mass media One way to look at computers in a news delivery system, at least during the early stages of their development, is in terms of their role as a feedback channel for traditional mass media. This application can be seen in the explosion of television news programs promoting World Wide Web Intemet sites. Some of these pilot projects have become quite sophisticated. For instance, the weather forecaster on a Washington DC station publishes a Web page that enables viewers to see both text and images not shown on the broadcast, and allows viewers to respond directly to the on-camera personalities via e-mail. Web resources include real-time radar images as storms pass through the area and detailed lists of school and office closings during snow storms. The station activity promotes remote weather stations at local schools that can be accessed from the Web site, further giving it at the perception of interactive community involvement. Examining such a mass media-lnternet hybrid system has value in the development of a stand-alone computer-based news delivery system for two reasons: • First, anecdotal evidence suggests such a system compels use, while news filters on standalone computer-based systems do not. For example newspapers appear unable to spark much interest, not to mention turn a profit, with their online products. Most online newspaper systems seemed to be used primarily as archival clipping services, rather than as a primary source for breaking news. • Second, the hybrid system embodies feedback between human agents, while news filters do not. This second point harks back to the importance of the link between news assessment and feedback. In a mass media system, the user is essentially passive in the news selection process, making only coarse decisions within a rigid, predetermined framework. Attempts to build a computer-based user model seem motivated by a philosophy of more or less replicating that experience, only increasing its efficiency through automation. A key assumption of this approach is that automation will reduce user effort, and that such a reduction is desirable. However, if the value added by a computer-based system has to do with its interactivity, it may by its very nature require increased user effort. This may be especially apparent in the shift in news assessment from the news provider to the user. This focuses on the special problem of the dynamic nature of news, and of the user's need for it. This suggests modeling an online communication technology would best be done in the context of a client-server paradigm, and not as a database problem. This study looks at a group of NBC Nightly News viewers who responded to an Internet e-mail address shown during a week-long series of programs dealing with new technology. What NBC
Feedbackand news
587
appeared to be doing in this program was to patch together its highly effective message delivery system with the Internet, a cost-effective feedback channel) The NBC experience under study here was, from an engineering standpoint, clumsy at best. But it does offer the opportunity to take a closer look at a new breed of news consumers who are wired into the Internet and actively using it.
2. METHOD
2.1. Sample selection A mail survey was executed in two waves, the first to a sample of NBC Nightly News viewers who responded to the show via e-mail, and the second to a random probability sample administered nationally. The first wave sample was drawn from a universe of e-mail messages written in response to the 'Almost 2001' series broadcast by NBC Nightly News. The series began during the last week of December 1993, and the actual messages were sent to NBC from 31 December to 14 February 1994. NBC's archive contained 3459 messages. However, examination of the message set revealed that some were not written in response to the series, and others were duplicates, bringing the actual universe of audience response messages to about 3200. Of that universe, 1000 messages were selected randomly. The authors of those messages were invited via e-mail to participate in a survey by returning their postal address. Of the 289 who agreed to take part, 206 completed and returned the survey, for a response rate of 71%. The second part of the sample was a national mail survey. Executed during September 1994, it generated a total of 1129 valid, completed surveys. The sample was randomly drawn from a database with a 30-million-household mailout developed by Donnelly Marketing, Inc. The sampling universe represents an address list of 17 million households generated by a rolling master database updated on a monthly basis. A total of 3000 addresses were randomly selected by zip code and state, and with 1129 completed questionnaires returned, the mailing had an overall response rate of over 30%.
2.2. Instrument construction The questionnaire included a section measuring respondents' assessment of the importance, quality, and credibility of information from newspapers, television news, and in computer discussion groups. While information in computer discussion groups is not news in the strict journalistic sense, it does provide a point of contrast. It also provides an interesting comparison with mass media in that it represents user-generated information about the environment not available in traditional media systems. Respondents were asked to make ratings using a thermometer scale, where 0 indicated the lowest rating and 100 indicated the highest rating. Respondents also rated their ability to understand and use computers on a 7-point Likert-like scale. They also were asked about their age, education, profession, and income.
3. RESULTS
3.1. Group differences There were no marked demographic differences between the two sample groups. For instance, there w e r e no statistically significant differences b e t w e e n the N B C group and the national 3When asked about the project Vice President for NBC Nightly News, William Wbeatley, told a group of journalism educators, "We're on the on-ramp of the Information Superhighway---the engine's running-but we don't really know where we are going yet" (comments given to The Freedom Forum Media Studies Center Technology Studies Seminar, New York, 25 October 1994). Since then NBC and Microsoft Corp. have entered into a joint venture, MSNBC, in an attempt to create the very kind of hybrid system described here. Results on the success of that system are still not available.
588
John E. Newhagen
sample for income or profession. The two groups did differ according to their education, t(1324)=5.56, P<0.001. The NBC group was better educated (M=3.09) than the national sample (M=2.66). 4
3.2. Efficacy and computer use Indexes were created to test differences between the NBC group and the national sample on their use of computers to communicate, and their sense of self efficacy in using computers. First, items related to computer use were combined into indexes, using factor loadings for questions conceming access to the Internet and to commercial online services. Variables were standardized and an index was created by combining scores from the two variables. Similarly, factor loadings for questions concerning how well respondents felt they understood how computers work, and how sure of themselves they felt when using computers to communicate, were combined. Figure 1 shows that the NBC group scored higher (M=0.26) than the national sample ( M = - 0.05) on the computer usage index, t(1229)=9.89, P<0.001. Similarly, the NBC group scored higher (M=0.41) than the national sample ( M = - 0.07) on the computer efficacy index, t(1312) = 13.95, P < 0.001. A total of 25.4% of the national sample reported having access to the Internet. However, Internet access is frequently reported to be just below 20% nationally (Barkow, 1996). The difference might be accounted for by the fact that the question looked at both home and work site access. 5
0.5
[] 0.4
National
[]NeC
0.3-
Standardized Index Score
0.2
0.1
0.0.
-0.1 Internet Access
Computer Efficacy
Fig. I. Intemet access and computer efficacyindexes and sample group. 42= Some college, 3 =college degree. ~Amongthose who said they did have Intemet access, the two groups also differed, X2(3)=525.36, P<0.001. Of those reporting access to the lnternet at home only, the national sample scored higher (59.4%) than the NBC group (40.6%). Similarly, of those reporting Internet access at work only, the national sample (82.5%) topped the NBC group (17.5%). However. for those reporting Internet access at both work and home, the NBC group scored higher (68.5%) than the national sample (31.5%).
Feedback and news
589
3.3. Information assessment, interactivity definition, and sample group Respondent interactivity definition and sample group are used as independent variables, while assessment of information across media systems is used as a dependent variable. Rafaeli's definition (Rafaeli, 1988) of interactivity in computer-mediated communication was employed to code an open-ended question asking respondents to comment on what they thought the word meant. He categorizes interactivity as either one-way, reactive, or truly interactive. One-way communication offers the message receiver no opportunity for a response. The content of a message in reactive, or two-way, communication is affected by the content of the previous message only. Truly interactive communication is cybernetic feedback in the sense that message content depends on the substance of multiple messages in a dynamic communication stream. Coding the open-ended response as either two-way or feedback proved effective, because most respondents explicitly mention one or the other. 3.3.1. Media system interactivity. Analysis of variance shows that respondents in the national sample rated newspapers to be more interactive (M=2.16) than the NBC group (M=1.53), regardless of respondent definition of interactivity, F(1,783)=21.52, P<0.001. 6 Similarly, respondents in the national sample rated television news to be more interactive (M=2.3) than the NBC group (M= 1.61), regardless of respondent definition of interactivity, F(1,783)=20.13, P<0.001. However, the NBC group rated e-mail to be more interactive (M=5.22) than the national sample (M = 4.96), regardless of their definition of interactivity, F( 1,730) = 6.10, P < 0.006. As a point of reference, results for the interactivity of face-to-face communication were very similar to e-mail, with the NBC group (M=6.82) scoring above the national sample (M=6.58), F(1,783)=6.10, P<0.01. Results are summarized in Fig. 2.
7.
I~ NationalSample IINBC
Interactivity 4 Rating 3,
Face-to-Face
E-mall
/ i/ Television N e w s
Newspapers
Medium
Fig. 2. Media interactivityratings and samplegroup. 6Interactivitywas measured on a 7-point Likert-likescale, where 1= not interactive,and 7 = very interactive.
590
John E. Newhagen
3.3.2. Information importance. Respondents in the national sample rated the importance of information in newspapers to be higher (M=71.5) than the NBC group (M=62.2), regardless of respondent definition of interactivity, F(1,785) = 13.3, P < 0.001.7 Analysis of variance shows an interaction between group and interactivity definition for the importance of television news, F(I, 1,776) = 3.04, P<0.01. The NBC group defining interactivity as two-way communication rated the importance of television news at about the same level (M=64.19) as the national sample (M=66.5). The NBC group, which defined interactivity as feedback, rated television news importance lower (M=57.4) than the national sample (M=67.57). The NBC group also rated importance of information in computer discussion groups higher (M=53) than the national sample (M=39.9), regardless of their definition of interactivity, F(1,692)=22.99, P<0.03. Results are summarized in Fig. 3. 3.3.3. Information quality. Analysis of variance shows an interaction approaching statistical significance between sample group and interactivity definition for the quality of newspaper news, F(I, 1,778) = 2.78, P <0.09. The NBC group defining interactivity as two-way communication rated the quality of television news at about the same level (M= 61.8) as the national sample (M=60). However, NBC group respondents who defined interactivity as feedback rated newspaper news quality lower (M=56.8) while the rating for the national sample increased (M=68.4). Analysis also shows an interaction that approaches statistical significance between sample group and interactivity definition for the quality of television news, F(1,1,778)=3.26, P<0.07. Members of the NBC group defining interactivity as two-way communication rated the quality of television news at about the same level (M= 63) as the national sample (M= 63.89). However, the NBC group who defined interactivity as feedback rated television news quality lower (M=56.12) than the national sample (M=64.3). Finally, the NBC group rated the quality of information in computer discussion groups to be higher (M=63.05) than the national sample (M=52.45), regardless of their definition of interactivity, F(1,693)= 3.26, P< 0.03. Results are summarized in Fig. 4. 3.3.4. Information credibility. There were no statistically significant differences in credibility ratings for either newspaper or television news. However, credibility ratings for information in computer discussion groups differed for the national sample and the NBC group, F(1,575)=4.95, P<0.02. There also was a main effect for interactivity definition, F(1,575)= 13.48, P < 0.001. For respondents who defined interactivity as two-way communication, the NBC group was higher (M=54.5) than the national sample (M=44.4). Credibility ratings rose for both groups when respondents defined interactivity as feedback, with the NBC group higher (M=56.8) than the national sample (M=48.6). Results are summarized in Fig. 5.
4. DISCUSSION This study set out to see if people's assessment of news varies according to their perception of a media system's interactivity. First, respondents were classified according to the way they defined interactivity, as either two-way communication or as cybernetic feedback. The study was further enhanced by the opportunity to look at a group that actually used the Internet as a feedback channel to a mass media news source. This enabled categorization of respondent interactivity according to both their own definition of the concept and by their behavior. First, the NBC group consistently rated mass media lower and computer communication higher on both information importance and quality than the national sample. In some tests that difference was heightened by differences in the respondents' definition of interactivity. Those in the NBC group who defined interactivity as feedback rated mass media much more critically, and computer communication more positively, than other respondents. 7Respondents rated information importance, quality, usefulness, and credibility on lO0-degree thermometer scales, where O=very low, and lO0=very high.
Feedback and news
591
Neither measure of interactivity made a difference in how respondents rated the credibility of mass media. This give credence to the proposition a mass media system that excludes interactive feedback limits or distorts the natural information assessment processes. However, the NBC group did rate computer discussion groups to be more credible than the national sample, and ratings increased for members of both groups who defined interactivity as feedback. Thus, when a media system includes a feedback channel, interactivity becomes a factor in the credibility assessment equation. While drawing causal inference from cross-sectional surveys is problematic, these trends do lead to a number of interesting conclusions: first, people seem to conceptualize two levels of interactivity in media, and they employ those levels in their quality and importance assessment of news. Not surprisingly, those who defined interactivity more powerfully, that is as feedback,
Newspapers
73" 71 69
"'0"
67-
Two-Way Feedback
65' 63 61 59
I
National Sample
M~C 60"
Computer Discussion Groups
I
50
J
Two-Way
40
Feedback
Television News 30
|
National Sample
70
I
NBC
56,
666462-
58 56
54
I
J
National Sample
NBC
Fig. 3. Information importance, interactivity definition, and sample group.
592
John E. Newhagen
rated computer communication higher and mass media lower than those who did not. This trend was especially heightened when the respondent had actually communicated with a mass media news source via the Internet. However, user interactivity only predicted credibility assessments for computer information, and not for mass media. It might be that credibility and feedback are associated in ways previous research into the topic have not revealed. Newhagen and Nass (1989) point out that news consumers sometimes rate the credibility of the news source or the medium rather than actual news content. The lack of association between credibility and interactivity for mass media found here serves to underscore the ambiguity of such assessments made in the absence of feedback. We are only beginning to recognize the computer as a communication technology. Its early use by scientific, engineering, and business communities led to its conceptualization as a calculating machine. When the task was defined in terms of language, the metaphor for the computer became that of information processor. For instance, the computer as typewriter is a
Newspapers
62 61
6O 68
56
Two-Way
ss £
F~ck
54
!
Computer Discussion
National Sample
Groups
70
65
55 ¸
50,
Television News
70
45 68
|
I
l~k.l¢mal Sample
NBC
68 6462-
C•
0
80$856,
--'El-"
F~I~N:k
54 52
|
National Sample
NBC
Fig. 4. Information quality, interactivity definition, and sample group.
Feedback and news
593
60
55 n" q)
E
50
o
a) I-
45
-,,,040
Two-Way F ~
!
National Sample
P~C
Fig. 5, Information credibility, interactivity definition, and sample group.
'word processor'. Attempts to construct news filters are really built on this informationprocessing metaphor. These engines might best be thought of as preprocessing filters, pre-selecting information for users in the same way newspaper editors do (see Beniger & Nass, 1986 for a discussion of the importance of pre-processing in a complex information environment). However, news has proved to be a tough nut to crack and filters have been able to only operate at the level of the sections of a daily newspaper. This study suggests progress might be made if designers conceptualize the computer as an interaction machine rather than as a calculating machine, that is, as an agent in communication. This position is based on a theory of news assessment where user needs and the nature of the information itself are both extremely fluid, sometimes changing from moment to moment. Given the dynamic nature of this relationship, it ought not to be surprising that user models employing past performance to predict behavior come up short. Conceptualizing the computer as a communication device opens the door to its use as a feedback channel between journalists and news consumers. This study demonstrates that the existence of such a channel does make a difference in the way people assess news. The implicit goal of building computer filters for news is to make use 'easier' for the consumer. However, shifting from a calculating machine metaphor to a communication metaphor might mean a shift in the goal of the system itself, from 'easier' to 'more useful'. In the first case, 'easier' is conceptualized in terms of data reduction, or preprocessing. However, mass media systems have already proven themselves to be extremely effective at this task, and improving on them might be problematic. Accessing news in an online environment might actually be inherently more effortful due to the complexity of both the technology and the news itself. Notice the shift that takes place when we move from a discussion of the television news consumer as a 'viewer', that is, as a passive recipient, to a computer 'user', an active agent. Working from that premise, the true value of the computer as a news delivery agent may be as a control apparatus, communicating information about a constantly changing environment to the user, while providing a channel back from the user to the news source.
REFERENCES Allen, R. B. (1990). User models: Theory, method, and practice. International Journal of Man-Machine Studies, 32, 511-543. Barkow, T. (1996). Raw data. Wired, 4(10), 82 Beniger, J., & Nass, C. (1986). Preprocessing: The neglected component in sociocybernetic models. In R. E Geyer, & J. van der Zauwen (Eds) Sociocybernetic paradoxes. Beverly Hills, CA: Sage Publications. Blumler, J., & Katz, E. (1974). The uses of mass communications. Current Perspectives on gratifications research. Beverly Hills, CA: Sage Publications. Dechert, C. R. (1966). The development of cybernetics. In Charles R. Dechert (Ed.) The social impact of cybernetics (pp. 11-37). Notre Dame, IN: University of Notre Dame Press.
594
John E. Newhagen
Emery, E., & Emery, M. (1988). The press and America: An interpretive history of the mass media (6th edn). Englewood Cliffs, NJ: Prentice-Hall. Frijda, N. (1988). The laws of emotion. American Psychologist, 43(5), 349-358. Garramone, G. (1984). Candidate image formation. In L. L. Kaid., D. Nimmo, and K. R. Sanders (Eds), New perspectives on political advertising (pp. 235-247). Carbondale, IL: Southern Illinois University Press. Geiger, S., & Newhagen, J. E. (1993). Revealing the black box: An information processing approach to understanding media effects. Journal of Communication, 43(4), 42-50. Gilbert, D. (1991). How mental systems believe. American Psychologist, 46(2), 107-119. Hall, S. (1980). Encoding/decoding. In S. Hall (Ed.), Culture, media, language: Working papers in cultural studies 1972-79 (pp. 128-138). London: Hutchinson. Lanzetta, J., Sullivan, D., Masters, R., & McHugo, G. (1985). Emotional and cognitive responses to televised images of political leaders. In S. S. Krans, and R. Perloff (Eds), Mass media and political thought: An information processing approach. Beverly Hills, CA: Sage Publishing. Newhagen, J. E., & Levy, M. R. (1997). The future of journalism in a distributed communication architecture. In D. L. Borden and K. Harvey (Eds), The electronic grapevine: Rumor, reputation and reporting in the new online environment (in press). Hillsdale, NJ: Lawrence Erlbaum Associates. Newhagen, J. E., & Nass, C. (1989). Differential criteria for evaluating credibility of newspapers and TV news. Journalism Quarterly, 66(2), 277-284. Newhagen, J. E., & Reeves, B. (1992). This evenings bad news: Effects of compelling negative television news images on memory. Journal of Communication, 42(2), 25-41. Newhagen, J. E., Cordes, J. W., & Levy, M. R. (1995). [email protected]: Audience mail and the perception of inter'activity. Journal of Communication, 45(2), 168-179. Rafaeli, S. (1988). Interactivity: From new media to communication. In R. P. Hawkins, J. M. Wiemann, and S. Pingree (Eds), Advancing communication science: Merging mass and interpersonal processes (pp. 110-134). Newbury Park, CA: Sage Publications. Rosenblueth, A., Wiener, R., & Bigelow, J. (1943). Behavior, purpose and teleology. Philosophy of Science, 10, 18-24. Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257-285.
© 1997ElsevierScienceLtd All rightsreserved,Printedin GreatBritain 0306-4573/97$17+0.00
Pergamon Pll: S0306-4573(97)00018-6
THE ROLE OF FEEDBACK
IN THE ASSESSMENT
OF NEWS
JOHN E. NEWHAGEN College of Journalism, Universityof Maryland, College Park, College Park, MD 20742, USA Abstract--The 25-year search for a compelling online news delivery system has been fraught with frustration, This study examines the problem by focusing on how mass media audiences and online users differ in their assessment of news. It employed a two-wave national survey (N=!335) to study the perception of interactivity in mass media and computer networks, and its relationship to the assessment of news. The first wave looked at a national probability sample, while the second wave targeted viewers of NBC Nightly News who responded to the show via e-mail. NBC respondents rated mass media to be less interactive, while they rated computer communication more interactive than the national sample. The NBC group also rated mass media news less important and of lower quality than did the national sample. Interactivity ratings did not predict mass media credibility assessments for either group. However, respondents who defined interactivity as cybernetic feedback or who contacted NBC by e-mail rated computer communication to be more credible than those who did not. Caution is suggested in attempting to design online news filters solely to replicate the editorial news selection process employed by mass media. Based on the evidence that users assess news differently in an interactive media system than in a mass media system, more design emphasis on feedback is encouraged. © 1997 Elsevier Science Ltd
I. THE ROLE OF FEEDBACK IN THE ASSESSMENT OF NEWS
The potential for computer networks, such as the Intemet, to play a role in the delivery of news has captured the imagination of professionals and researchers alike. Yet, the implementation of that idea remains elusive, fraught with both conceptual ambiguity, and practical intransigence. Despite the intuitive appeal of using computers to deliver news, 25 years of experiments and pilot projects have yet to develop into a viable system. Naysayers to the current wave of interest recall Knight Ridder's failed video text experiment in the 1970s (for details, see Emery & Emery, 1988). Even as large newspapers move into an online environment, they do so hesitantly, and none expect to realize a profit any time soon. One source of trouble in the transition of news from print to electronic text may relate to theoretical ambiguity about the nature of the relationship between content and user. This is especially apparent in understanding the importance of feedback in user assessment of content in an online environment. This study examines data from a two-wave national survey (N= 1335) to consider the role of user feedback in news assessment. The survey included questions focusing on the perception and use of computers and mass media as sources for news. The first wave looked at a national random sample, while the second wave targeted viewers of NBC Nightly News who responded to the show via e-mail. Respondents of the NBC subsample are unique in that they spontaneously employed the Internet as a dynamic feedback channel to a traditional mass media news source. First, the concept of news as a special category of survival relevant information will be suggested as an alternative to the simple assumption of traditional mass media operational definitions. This opens the possibility of looking at differences in user assessment of information importance, quality, and credibility between mass media and online platforms. Next, the idea of feedback will be considered as the dimension distinguishing truly interactive computer-based information systems from mass media systems. 583
584
John E. Newhagen
1.1. The problem with news Asking a seasoned editor to define just what news is will frequently lead to an embarrassed silence. Difficulty explaining an over-learned skill is not unusual (see Sweller, 1988), and journalists are no exception. Rather than applying some explicit set of proscriptive rules, editors make decisions based on intuitional judgments guided by a century-old set of professional canons. This lack of a clear notion of what distinguishes news as a special class of information leads to the substitution of operational definitions: news is what appears in newspapers. The advent of television did not really change this formula: broadcast news is information that could be in a newspaper, only with sound and moving pictures. The problem then contaminates research to the degree that news is frequently conceptualized solely from an operational standpoint. For instance, Allen's benchmark discussion (Allen, 1990) of user model-based filters in an online news delivery system does exactly that, simply defining news to be the contents of newspapers or wire services. The limits of this approach appear in the inability to predict users preference beyond coarse-grained production-oriented categories that, not coincidentally, correspond to the sections of a newspaper. An early attempt in communication research to model mass media use can be seen in the 'uses and gratifications' approach (see Blumler & Katz, 1974). For instance, readers might look at the front section of a newspaper for different reasons than they do the sports section. However, to the degree that the uses and gratifications perspective also is based on standard productionoriented categories, it failed to describe how information in those categories might be theoretically unique. For instance, a report concerning legislation about anti-trust waivers for professional baseball could as easily appear in a newspaper's main section as it could in its sports section. The challenge to building an online news delivery system is even more difficult because it has to deal with the dynamic nature of both news and the user's information needs in an environment that is, by its nature, much more volatile than mass media. Newhagen & Levy (1997) define news as survival relevant information useful in adapting to change. This information-processing perspective models a user making survival-contingent decisions in a hostile environment based on limited information in real time (see Geiger & Newhagen, 1993). This theory helps ground concepts frequently studied in the assessment of news, including importance, quality, and credibility. Each of these dimensions will be discussed in the context of news as survival contingent information. 1.1.1. Information importance. Importance bears most directly on the novelty and proximity of news. Novelty, especially when it is proximate, activates emotion-based action states such as fear, and leads to information gathering used in making survival-contingent decisions (Frijda, 1988). Negative images of human death and suffering in television news, for instance, compel viewer attention, and affect memory (see Newhagen & Reeves, 1992). Thus, novel information, especially when it has negative implications for the user's survival, will be deemed important. 1.1.2. Information quality. The assumption is that high quality information is better that low quality information when making survival-contingent decisions. However, a system compelled to make decisions under real-time stress might be willing to de-emphasize quality for the sake of efficiency. Gilbert (1991), for instance, proposes that in order to save time, information is first processed as if it were true, with veracity judgment postponed until later stages of cognition. He asserts this is an adaptive response to time pressure based on the assumption that the senses do not lie. However, this assumption becomes problematic when the information is technologically mediated, where the epitaph shifts from 'seeing is believing' to 'the camera does not lie'. Thus, while journalists set information quality as a high priority, users might not, especially when it is complex. 1.1.3. Credibility. The concept of credibility has dominated mass media research into the assessment of news. The prominence of credibility is, perhaps, a reflection of its importance to the canons of professional journalism, which hinge on the perception that reporting is unbiased and neutral. Empirical research into mass media credibility has focused on how message characteristics
Feedback and news
585
evoke behavioral or cognitive changes in the message receiver. For example, the physical appearance of a political candidate on television news has been shown to affect viewer perceptions of credibility (Lanzetta et al., 1985). Frequently the other two concepts mentioned here, importance and quality, are assumed to be collinear with credibility: that is, a highly credible message has both importance and quality. However, factor analysis searching for social psychological dimensions underlying credibility have been difficult to interpret. 1 While both theoretical and methodological reasons have been suggested for this lack of coherence, another reason might have to do with the very architecture of mass media systems themselves. Mass media are one-way delivery systems, designed to generate a practically infinite number of replica messages to very large audiences. But the cost of this efficiency to the user is reflected in the absence of real-time user feedback and distorts the natural information assessment process. For instance, viewers of political TV spots, finding themselves unable to query candidates about their positions on issues, may evaluate credibility in terms of physical appearance (Garramone, 1984). On the other hand, because of their architecture, computer networks are inherently interactive and open a feedback channel for users. This two-way capacity, then, opens the possibility that information consumers might evaluate news in an interactive environment much differently than they do when the message stream is restricted to a one-way flow. This is reflected in the way use is conceptualized; viewers passively watch television, while computer users actively search the Net. Feedback, then, may provide a clue to understanding how online users assess news differently from members of a mass media audience, and what difference that might make to constructing user models.
1.2. Conceptualizing feedback in a news delivery system Much of the content of face-to-face communication can be thought of as news. This is important because interpersonal exchanges are generally thought to be the most highly interactive form of human communication. Prior to the invention of movable type, information about current events, called tidings, frequently was communicated in face-to-face exchanges (Emery & Emery, 1988). However, by their nature, mass media news delivery systems are not interactive, and preclude the possibility of feedback (Newhagen et al., 1995) 2. Thus, it may be a mistake to attempt to map mass media user models onto an interactive computer network environment. For instance, Allen (1990) suggests gauging the performance of an adaptive user model for news selection by applying a 'Turing control', that is, comparing it with a journalist engaged in similar task (p. 516). However, a critical assumption underlying the use of a Turing control is that the mass media system represents a functional standard for an interactive communication model. In simpler times the journalist might have sampled user feedback through informal daily interactions with members of the community. However, as urban populations have increased and society has become more diverse and complex, this technique has reached its limit. Indeed, journalism is increasingly criticized for its detachment from its audience. Thus, the real promise of an online news delivery system may not have to do with its efficiency as an automated filter, but with its capacity to return feedback to the communication experience. Having said that, the next challenge is to address the conceptual ambiguity of the term interactivity itself: does it have to do with the responsiveness of the system? Its mutability? The intuitive quality of its interface? As a link between human agents? qn a literature review of credibility studies, Newhagen and Nass (1989) found factor analysis yielded anywhere from 2 to 12 dimensions. "Federal Communications Commission policy on broadcast license renewal accepts the notion that viewer mail constitutes a viable audience feedback system. That is, the policy promotes the idea that viewers will send mail to stations with critical comments about what they saw. Someoneat the station, then, would read the mail and use the information in subsequent programming decisions. However, the feasibility of such a system is limited even if broadcasters are given the benefit of the doubt concerning their good intentions to receive and respond to their audience. Traditional mail, after all, requires effort to write, send, and read and is not particularly timely in the context of the fast-paced world of broadcasting. While most newspapers also print a page of letters to the editor, neither does this practice represent a timely or functional feedback circuit in any practical sense.
586
John E. Newhagen
In the context of human communication, interactivity might best be conceptualized in terms of cybernetic feedback. While Wiener's work in cybernetic feedback had to do with engineering problems associated with servo-synchro mechanisms, the metaphor was soon applied to interactive social systems (Dechert, 1966). In either case cybernetic theory defines interactivity in terms of the feedback necessary to the maintenance of a self-regulating system (Rosenblueth et al., 1943). For instance, voting can be seen as feedback in a democracy, a self-regulating political system. From this perspective journalists working for newspapers or television do not employ adaptive user models, which incorporate feedback, when they make news judgments (see Newhagen & Levy, 1997). The other side of that coin leaves mass media users outside the news creation process, forcing them either to forgo message assessment, or employ criteria external to journalistic methodology. Hall (1980) details the problem message receivers have decoding meaning from news in a mass media system. On the other hand, a media system with a viable feedback channel might allow users to assess more actively the news they receive. This goes beyond a normative comment about which system is better; it suggests assessment processes for the two may be qualitatively different. That prospect should be of substantial interest to designers, to the degree that it will affect the very nature of the user models such systems embody.
1.3. A crude beginning: Using the Internet as a feedback channel for mass media One way to look at computers in a news delivery system, at least during the early stages of their development, is in terms of their role as a feedback channel for traditional mass media. This application can be seen in the explosion of television news programs promoting World Wide Web Intemet sites. Some of these pilot projects have become quite sophisticated. For instance, the weather forecaster on a Washington DC station publishes a Web page that enables viewers to see both text and images not shown on the broadcast, and allows viewers to respond directly to the on-camera personalities via e-mail. Web resources include real-time radar images as storms pass through the area and detailed lists of school and office closings during snow storms. The station activity promotes remote weather stations at local schools that can be accessed from the Web site, further giving it at the perception of interactive community involvement. Examining such a mass media-lnternet hybrid system has value in the development of a stand-alone computer-based news delivery system for two reasons: • First, anecdotal evidence suggests such a system compels use, while news filters on standalone computer-based systems do not. For example newspapers appear unable to spark much interest, not to mention turn a profit, with their online products. Most online newspaper systems seemed to be used primarily as archival clipping services, rather than as a primary source for breaking news. • Second, the hybrid system embodies feedback between human agents, while news filters do not. This second point harks back to the importance of the link between news assessment and feedback. In a mass media system, the user is essentially passive in the news selection process, making only coarse decisions within a rigid, predetermined framework. Attempts to build a computer-based user model seem motivated by a philosophy of more or less replicating that experience, only increasing its efficiency through automation. A key assumption of this approach is that automation will reduce user effort, and that such a reduction is desirable. However, if the value added by a computer-based system has to do with its interactivity, it may by its very nature require increased user effort. This may be especially apparent in the shift in news assessment from the news provider to the user. This focuses on the special problem of the dynamic nature of news, and of the user's need for it. This suggests modeling an online communication technology would best be done in the context of a client-server paradigm, and not as a database problem. This study looks at a group of NBC Nightly News viewers who responded to an Internet e-mail address shown during a week-long series of programs dealing with new technology. What NBC
Feedbackand news
587
appeared to be doing in this program was to patch together its highly effective message delivery system with the Internet, a cost-effective feedback channel) The NBC experience under study here was, from an engineering standpoint, clumsy at best. But it does offer the opportunity to take a closer look at a new breed of news consumers who are wired into the Internet and actively using it.
2. METHOD
2.1. Sample selection A mail survey was executed in two waves, the first to a sample of NBC Nightly News viewers who responded to the show via e-mail, and the second to a random probability sample administered nationally. The first wave sample was drawn from a universe of e-mail messages written in response to the 'Almost 2001' series broadcast by NBC Nightly News. The series began during the last week of December 1993, and the actual messages were sent to NBC from 31 December to 14 February 1994. NBC's archive contained 3459 messages. However, examination of the message set revealed that some were not written in response to the series, and others were duplicates, bringing the actual universe of audience response messages to about 3200. Of that universe, 1000 messages were selected randomly. The authors of those messages were invited via e-mail to participate in a survey by returning their postal address. Of the 289 who agreed to take part, 206 completed and returned the survey, for a response rate of 71%. The second part of the sample was a national mail survey. Executed during September 1994, it generated a total of 1129 valid, completed surveys. The sample was randomly drawn from a database with a 30-million-household mailout developed by Donnelly Marketing, Inc. The sampling universe represents an address list of 17 million households generated by a rolling master database updated on a monthly basis. A total of 3000 addresses were randomly selected by zip code and state, and with 1129 completed questionnaires returned, the mailing had an overall response rate of over 30%.
2.2. Instrument construction The questionnaire included a section measuring respondents' assessment of the importance, quality, and credibility of information from newspapers, television news, and in computer discussion groups. While information in computer discussion groups is not news in the strict journalistic sense, it does provide a point of contrast. It also provides an interesting comparison with mass media in that it represents user-generated information about the environment not available in traditional media systems. Respondents were asked to make ratings using a thermometer scale, where 0 indicated the lowest rating and 100 indicated the highest rating. Respondents also rated their ability to understand and use computers on a 7-point Likert-like scale. They also were asked about their age, education, profession, and income.
3. RESULTS
3.1. Group differences There were no marked demographic differences between the two sample groups. For instance, there w e r e no statistically significant differences b e t w e e n the N B C group and the national 3When asked about the project Vice President for NBC Nightly News, William Wbeatley, told a group of journalism educators, "We're on the on-ramp of the Information Superhighway---the engine's running-but we don't really know where we are going yet" (comments given to The Freedom Forum Media Studies Center Technology Studies Seminar, New York, 25 October 1994). Since then NBC and Microsoft Corp. have entered into a joint venture, MSNBC, in an attempt to create the very kind of hybrid system described here. Results on the success of that system are still not available.
588
John E. Newhagen
sample for income or profession. The two groups did differ according to their education, t(1324)=5.56, P<0.001. The NBC group was better educated (M=3.09) than the national sample (M=2.66). 4
3.2. Efficacy and computer use Indexes were created to test differences between the NBC group and the national sample on their use of computers to communicate, and their sense of self efficacy in using computers. First, items related to computer use were combined into indexes, using factor loadings for questions conceming access to the Internet and to commercial online services. Variables were standardized and an index was created by combining scores from the two variables. Similarly, factor loadings for questions concerning how well respondents felt they understood how computers work, and how sure of themselves they felt when using computers to communicate, were combined. Figure 1 shows that the NBC group scored higher (M=0.26) than the national sample ( M = - 0.05) on the computer usage index, t(1229)=9.89, P<0.001. Similarly, the NBC group scored higher (M=0.41) than the national sample ( M = - 0.07) on the computer efficacy index, t(1312) = 13.95, P < 0.001. A total of 25.4% of the national sample reported having access to the Internet. However, Internet access is frequently reported to be just below 20% nationally (Barkow, 1996). The difference might be accounted for by the fact that the question looked at both home and work site access. 5
0.5
[] 0.4
National
[]NeC
0.3-
Standardized Index Score
0.2
0.1
0.0.
-0.1 Internet Access
Computer Efficacy
Fig. I. Intemet access and computer efficacyindexes and sample group. 42= Some college, 3 =college degree. ~Amongthose who said they did have Intemet access, the two groups also differed, X2(3)=525.36, P<0.001. Of those reporting access to the lnternet at home only, the national sample scored higher (59.4%) than the NBC group (40.6%). Similarly, of those reporting Internet access at work only, the national sample (82.5%) topped the NBC group (17.5%). However. for those reporting Internet access at both work and home, the NBC group scored higher (68.5%) than the national sample (31.5%).
Feedback and news
589
3.3. Information assessment, interactivity definition, and sample group Respondent interactivity definition and sample group are used as independent variables, while assessment of information across media systems is used as a dependent variable. Rafaeli's definition (Rafaeli, 1988) of interactivity in computer-mediated communication was employed to code an open-ended question asking respondents to comment on what they thought the word meant. He categorizes interactivity as either one-way, reactive, or truly interactive. One-way communication offers the message receiver no opportunity for a response. The content of a message in reactive, or two-way, communication is affected by the content of the previous message only. Truly interactive communication is cybernetic feedback in the sense that message content depends on the substance of multiple messages in a dynamic communication stream. Coding the open-ended response as either two-way or feedback proved effective, because most respondents explicitly mention one or the other. 3.3.1. Media system interactivity. Analysis of variance shows that respondents in the national sample rated newspapers to be more interactive (M=2.16) than the NBC group (M=1.53), regardless of respondent definition of interactivity, F(1,783)=21.52, P<0.001. 6 Similarly, respondents in the national sample rated television news to be more interactive (M=2.3) than the NBC group (M= 1.61), regardless of respondent definition of interactivity, F(1,783)=20.13, P<0.001. However, the NBC group rated e-mail to be more interactive (M=5.22) than the national sample (M = 4.96), regardless of their definition of interactivity, F( 1,730) = 6.10, P < 0.006. As a point of reference, results for the interactivity of face-to-face communication were very similar to e-mail, with the NBC group (M=6.82) scoring above the national sample (M=6.58), F(1,783)=6.10, P<0.01. Results are summarized in Fig. 2.
7.
I~ NationalSample IINBC
Interactivity 4 Rating 3,
Face-to-Face
E-mall
/ i/ Television N e w s
Newspapers
Medium
Fig. 2. Media interactivityratings and samplegroup. 6Interactivitywas measured on a 7-point Likert-likescale, where 1= not interactive,and 7 = very interactive.
590
John E. Newhagen
3.3.2. Information importance. Respondents in the national sample rated the importance of information in newspapers to be higher (M=71.5) than the NBC group (M=62.2), regardless of respondent definition of interactivity, F(1,785) = 13.3, P < 0.001.7 Analysis of variance shows an interaction between group and interactivity definition for the importance of television news, F(I, 1,776) = 3.04, P<0.01. The NBC group defining interactivity as two-way communication rated the importance of television news at about the same level (M=64.19) as the national sample (M=66.5). The NBC group, which defined interactivity as feedback, rated television news importance lower (M=57.4) than the national sample (M=67.57). The NBC group also rated importance of information in computer discussion groups higher (M=53) than the national sample (M=39.9), regardless of their definition of interactivity, F(1,692)=22.99, P<0.03. Results are summarized in Fig. 3. 3.3.3. Information quality. Analysis of variance shows an interaction approaching statistical significance between sample group and interactivity definition for the quality of newspaper news, F(I, 1,778) = 2.78, P <0.09. The NBC group defining interactivity as two-way communication rated the quality of television news at about the same level (M= 61.8) as the national sample (M=60). However, NBC group respondents who defined interactivity as feedback rated newspaper news quality lower (M=56.8) while the rating for the national sample increased (M=68.4). Analysis also shows an interaction that approaches statistical significance between sample group and interactivity definition for the quality of television news, F(1,1,778)=3.26, P<0.07. Members of the NBC group defining interactivity as two-way communication rated the quality of television news at about the same level (M= 63) as the national sample (M= 63.89). However, the NBC group who defined interactivity as feedback rated television news quality lower (M=56.12) than the national sample (M=64.3). Finally, the NBC group rated the quality of information in computer discussion groups to be higher (M=63.05) than the national sample (M=52.45), regardless of their definition of interactivity, F(1,693)= 3.26, P< 0.03. Results are summarized in Fig. 4. 3.3.4. Information credibility. There were no statistically significant differences in credibility ratings for either newspaper or television news. However, credibility ratings for information in computer discussion groups differed for the national sample and the NBC group, F(1,575)=4.95, P<0.02. There also was a main effect for interactivity definition, F(1,575)= 13.48, P < 0.001. For respondents who defined interactivity as two-way communication, the NBC group was higher (M=54.5) than the national sample (M=44.4). Credibility ratings rose for both groups when respondents defined interactivity as feedback, with the NBC group higher (M=56.8) than the national sample (M=48.6). Results are summarized in Fig. 5.
4. DISCUSSION This study set out to see if people's assessment of news varies according to their perception of a media system's interactivity. First, respondents were classified according to the way they defined interactivity, as either two-way communication or as cybernetic feedback. The study was further enhanced by the opportunity to look at a group that actually used the Internet as a feedback channel to a mass media news source. This enabled categorization of respondent interactivity according to both their own definition of the concept and by their behavior. First, the NBC group consistently rated mass media lower and computer communication higher on both information importance and quality than the national sample. In some tests that difference was heightened by differences in the respondents' definition of interactivity. Those in the NBC group who defined interactivity as feedback rated mass media much more critically, and computer communication more positively, than other respondents. 7Respondents rated information importance, quality, usefulness, and credibility on lO0-degree thermometer scales, where O=very low, and lO0=very high.
Feedback and news
591
Neither measure of interactivity made a difference in how respondents rated the credibility of mass media. This give credence to the proposition a mass media system that excludes interactive feedback limits or distorts the natural information assessment processes. However, the NBC group did rate computer discussion groups to be more credible than the national sample, and ratings increased for members of both groups who defined interactivity as feedback. Thus, when a media system includes a feedback channel, interactivity becomes a factor in the credibility assessment equation. While drawing causal inference from cross-sectional surveys is problematic, these trends do lead to a number of interesting conclusions: first, people seem to conceptualize two levels of interactivity in media, and they employ those levels in their quality and importance assessment of news. Not surprisingly, those who defined interactivity more powerfully, that is as feedback,
Newspapers
73" 71 69
"'0"
67-
Two-Way Feedback
65' 63 61 59
I
National Sample
M~C 60"
Computer Discussion Groups
I
50
J
Two-Way
40
Feedback
Television News 30
|
National Sample
70
I
NBC
56,
666462-
58 56
54
I
J
National Sample
NBC
Fig. 3. Information importance, interactivity definition, and sample group.
592
John E. Newhagen
rated computer communication higher and mass media lower than those who did not. This trend was especially heightened when the respondent had actually communicated with a mass media news source via the Internet. However, user interactivity only predicted credibility assessments for computer information, and not for mass media. It might be that credibility and feedback are associated in ways previous research into the topic have not revealed. Newhagen and Nass (1989) point out that news consumers sometimes rate the credibility of the news source or the medium rather than actual news content. The lack of association between credibility and interactivity for mass media found here serves to underscore the ambiguity of such assessments made in the absence of feedback. We are only beginning to recognize the computer as a communication technology. Its early use by scientific, engineering, and business communities led to its conceptualization as a calculating machine. When the task was defined in terms of language, the metaphor for the computer became that of information processor. For instance, the computer as typewriter is a
Newspapers
62 61
6O 68
56
Two-Way
ss £
F~ck
54
!
Computer Discussion
National Sample
Groups
70
65
55 ¸
50,
Television News
70
45 68
|
I
l~k.l¢mal Sample
NBC
68 6462-
C•
0
80$856,
--'El-"
F~I~N:k
54 52
|
National Sample
NBC
Fig. 4. Information quality, interactivity definition, and sample group.
Feedback and news
593
60
55 n" q)
E
50
o
a) I-
45
-,,,040
Two-Way F ~
!
National Sample
P~C
Fig. 5, Information credibility, interactivity definition, and sample group.
'word processor'. Attempts to construct news filters are really built on this informationprocessing metaphor. These engines might best be thought of as preprocessing filters, pre-selecting information for users in the same way newspaper editors do (see Beniger & Nass, 1986 for a discussion of the importance of pre-processing in a complex information environment). However, news has proved to be a tough nut to crack and filters have been able to only operate at the level of the sections of a daily newspaper. This study suggests progress might be made if designers conceptualize the computer as an interaction machine rather than as a calculating machine, that is, as an agent in communication. This position is based on a theory of news assessment where user needs and the nature of the information itself are both extremely fluid, sometimes changing from moment to moment. Given the dynamic nature of this relationship, it ought not to be surprising that user models employing past performance to predict behavior come up short. Conceptualizing the computer as a communication device opens the door to its use as a feedback channel between journalists and news consumers. This study demonstrates that the existence of such a channel does make a difference in the way people assess news. The implicit goal of building computer filters for news is to make use 'easier' for the consumer. However, shifting from a calculating machine metaphor to a communication metaphor might mean a shift in the goal of the system itself, from 'easier' to 'more useful'. In the first case, 'easier' is conceptualized in terms of data reduction, or preprocessing. However, mass media systems have already proven themselves to be extremely effective at this task, and improving on them might be problematic. Accessing news in an online environment might actually be inherently more effortful due to the complexity of both the technology and the news itself. Notice the shift that takes place when we move from a discussion of the television news consumer as a 'viewer', that is, as a passive recipient, to a computer 'user', an active agent. Working from that premise, the true value of the computer as a news delivery agent may be as a control apparatus, communicating information about a constantly changing environment to the user, while providing a channel back from the user to the news source.
REFERENCES Allen, R. B. (1990). User models: Theory, method, and practice. International Journal of Man-Machine Studies, 32, 511-543. Barkow, T. (1996). Raw data. Wired, 4(10), 82 Beniger, J., & Nass, C. (1986). Preprocessing: The neglected component in sociocybernetic models. In R. E Geyer, & J. van der Zauwen (Eds) Sociocybernetic paradoxes. Beverly Hills, CA: Sage Publications. Blumler, J., & Katz, E. (1974). The uses of mass communications. Current Perspectives on gratifications research. Beverly Hills, CA: Sage Publications. Dechert, C. R. (1966). The development of cybernetics. In Charles R. Dechert (Ed.) The social impact of cybernetics (pp. 11-37). Notre Dame, IN: University of Notre Dame Press.
594
John E. Newhagen
Emery, E., & Emery, M. (1988). The press and America: An interpretive history of the mass media (6th edn). Englewood Cliffs, NJ: Prentice-Hall. Frijda, N. (1988). The laws of emotion. American Psychologist, 43(5), 349-358. Garramone, G. (1984). Candidate image formation. In L. L. Kaid., D. Nimmo, and K. R. Sanders (Eds), New perspectives on political advertising (pp. 235-247). Carbondale, IL: Southern Illinois University Press. Geiger, S., & Newhagen, J. E. (1993). Revealing the black box: An information processing approach to understanding media effects. Journal of Communication, 43(4), 42-50. Gilbert, D. (1991). How mental systems believe. American Psychologist, 46(2), 107-119. Hall, S. (1980). Encoding/decoding. In S. Hall (Ed.), Culture, media, language: Working papers in cultural studies 1972-79 (pp. 128-138). London: Hutchinson. Lanzetta, J., Sullivan, D., Masters, R., & McHugo, G. (1985). Emotional and cognitive responses to televised images of political leaders. In S. S. Krans, and R. Perloff (Eds), Mass media and political thought: An information processing approach. Beverly Hills, CA: Sage Publishing. Newhagen, J. E., & Levy, M. R. (1997). The future of journalism in a distributed communication architecture. In D. L. Borden and K. Harvey (Eds), The electronic grapevine: Rumor, reputation and reporting in the new online environment (in press). Hillsdale, NJ: Lawrence Erlbaum Associates. Newhagen, J. E., & Nass, C. (1989). Differential criteria for evaluating credibility of newspapers and TV news. Journalism Quarterly, 66(2), 277-284. Newhagen, J. E., & Reeves, B. (1992). This evenings bad news: Effects of compelling negative television news images on memory. Journal of Communication, 42(2), 25-41. Newhagen, J. E., Cordes, J. W., & Levy, M. R. (1995). [email protected]: Audience mail and the perception of inter'activity. Journal of Communication, 45(2), 168-179. Rafaeli, S. (1988). Interactivity: From new media to communication. In R. P. Hawkins, J. M. Wiemann, and S. Pingree (Eds), Advancing communication science: Merging mass and interpersonal processes (pp. 110-134). Newbury Park, CA: Sage Publications. Rosenblueth, A., Wiener, R., & Bigelow, J. (1943). Behavior, purpose and teleology. Philosophy of Science, 10, 18-24. Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257-285.