Visual perception at work: Lessons from the world of meteorology

Available online at www.sciencedirect.com

Poetics 38 (2010) 150–164 www.elsevier.com/locate/poetic

Visual perception at work: Lessons from the world of meteorology Phaedra Daipha Department of Sociology, Rutgers University, 54 Joyce Kilmer Avenue, Piscataway, NJ 08854, United States Available online 6 January 2010

Abstract Drawing on fieldwork at a forecasting office of the National Weather Service, this paper analyzes the sociocognitive organization of meteorological perception and expertise via the notion of ‘‘screenwork’’ in the hopes that it may provide a common constructivist ground on which sociologists and neuroscientists will be able to comfortably study visual perception at work. Three key intersecting areas of research are examined in the process: the practice of looking, visual expertise, and visual decision-making. The paper concludes by employing the metaphor of the collage to further elaborate on the link between screenwork and expert visualization. # 2009 Elsevier B.V. All rights reserved.

1. Introduction Why is it that when we think of weather forecasters we envision them in front of computer screens, even as we say that they would be better served looking out the window? What accounts for the link between expert authority and the use of visualization tools? Sociologists of science and of the professions have long established that images play a central role in how expertise is acquired, produced, disseminated, and consumed.1 Images are intimately implicated in the practice and manufacture of knowledge within the laboratory (e.g., Latour and Woolgar, 1986; Lynch, 1985; Lynch and Edgerton, 1988; Knorr Cetina and Amann, 1990; Henderson, 1999) and out in the field (e.g., Law and Lynch, 1988; Latour, 1999: 24–79). They serve to order, quantify, formalize, and discipline our relation to the world (e.g., Foucault, 1970; Lynch, 1990; Turnbull, 1994; Rudwick, 1995; Traweek, 1997). And they prove to be powerful allies in credibility contests (e.g., Shapin and Schaffer, 1985: 61–62; Gieryn, 1999: 201–203) and across epistemic domains (Star and Griesemer, 1989). From drawings, graphs, and blueprints to photographs, E-mail address: [email protected]. The diversity and expansiveness of scholarship on visual representations in science and medicine prohibits a review here. For a recent appraisal, see Burri and Dumit (2008). 1

0304-422X/$ – see front matter # 2009 Elsevier B.V. All rights reserved. doi:10.1016/j.poetic.2009.11.007

P. Daipha / Poetics 38 (2010) 150–164

151

brain scans, simulations, and computer-generated maps, images are flooding the sites where expertise is performed and proclaimed. Traditionally, social studies of expert visualization have tended to bracket out mentalistic considerations and concentrate on the sociologically observable aspects of expertise (Latour, 1987, 1990; Lynch, 1990). Recent advances in computational models of visual perception, however, have prompted efforts to forge a dialogue between sociology and cognitive neuroscience. Now, one may see arguments for the cultural and historical embeddedness of expert visualization alongside arguments for the existence of shared cognitive capacities across epistemic realms and representational traditions (Gooding, 2004, 2006). This paper means to continue and expand the dialogue between sociology and neuroscience on visual perception by identifying three key intersecting areas of research: (1) practices of looking; (2) visual expertise; and (3) visual decision-making. As foreshadowed above, the case study that will be used to illustrate and flesh out this position comes from the world of weather forecasting. Weather forecasting constitutes an ideal site for evaluating visual perception at work and, consequently, the relationship between culture and cognition. Weather forecasters are tasked with predicting the behavior of a phenomenon that, while perceptibly present outside, cannot be physically manipulated and studied under controlled conditions ‘inside.’ Hence, the development of the meteorological profession has been contingent on finding ways to reconstruct the weather indoors, essentially co-evolving with the development of visualization technologies. It is no accident that atmospheric modeling was one of the first applications of computer visualization (McCormick et al., 1987). Weather forecasters have been gaining in experience and authority by mastering the weather on progressively more dynamic, more weather-like maps (Monmonier, 1999). By following the forecasting task, one effectively beholds visual perception in action. In what follows, I begin by conceptualizing expert visualization as screenwork and then outline meteorological screenwork at the National Weather Service (NWS). The longest section of this paper is devoted to exploring through the NWS case what I consider to be three promising, intersecting areas of research between cognitive sociology and neuroscience, namely the practice of looking, visual expertise, and visual decision-making. I conclude by offering the metaphor of the collage as a further elaboration on the link between screenwork and expert visualization. 2. Visualization in expert settings Despite the great variety of visual representations flooding the places of expert activity, there is no question that, today, they mostly reach their destination through computer screens. In effect, it is computer screens that have come to colonize the spaces of knowledge production, and it is screenwork that now forms the backbone of expertise. Conceptualizing expert perception as screenwork reframes it as an externalized, technologically mediated process, thus making it readily accessible to sociological analysis. Substantively, this operationalization is consistent with the notion that expertise emerges out of the sociocognitive synchronization of human and non-human actors mobilized into a distributed web of information processing (e.g., Latour, 1987; Resnik, 1991; Hutchins, 1995; Goodwin, 1995; Gorman, 1997; Giere and Moffatt, 2003; Viseu, 2003). Yet, it is not only cognitive sociology that stands to benefit from the shift to computer visualization as a proxy for expert perception at work; this is exactly the premise behind current research in neuroscience. In our visual culture, the convergence of scholarship on cognition and on culture has arguably found its most prominent technological consummation in the design of computer visualization

152

P. Daipha / Poetics 38 (2010) 150–164

tools. Building on the sociocognitively oriented framework of ‘‘external cognition’’ (Scaife and Rogers, 1996), computer visualization aims to develop semi-automated, interactive representations of physical and/or abstract data that enhance human performance. Naturally, it has sought to incorporate the latest in experimental psychology on visual and cognitive processing, in turn fueling the development of a vision science proper by affording new techniques for the clinical testing of visual perception models. At the same time, the realities of technological implementation and the recent calls for a ‘user-centered’ design have prompted a turn to a more sociological understanding of computer visualization. Although a systematic examination of the role of information technology in visual perception is beyond the limits of this paper, I this paper would like to suggest that computer visualization – operationalized as screenwork – can serve as a hinge to articulate the cultural and the cognitive, in meteorological perception in particular and in expert perception more broadly. Along with evidence for the cultural and historical embeddedness of scientific visualization (e.g., Lynch and Woolgar, 1990; Pickering, 1992; Galison, 1997; Hacking, 1999; Knorr Cetina, 1999), the historiography of modern science has provided us with ample case studies of instruments of visualization (e.g., Cambrosio et al., 1993; Monmonier, 1999; Rasmussen, 1999; Breidbach, 2002; Francoeur, 2002), and there is now a sizable literature on new visualization technologies in medical practice (e.g., Cohn, 2004; Dumit, 2004; Prasad, 2005; Prentice, 2005; Joyce, 2005, 2006). Central in the literature on visualism in science, and key for appreciating the success of computer visualization, is the process of ‘‘mathematization’’ (Lynch, 1990) through which images, selected for their convincing rendering of the ‘raw’ phenomenon under study, become quantified along certain parametric conventions and blended with other similarly processed images into hyperreal representations, thus vested with additional revelatory and explanatory power. What makes these images good candidates for mathematization is their eidetic quality, achieved through a parallel filtering out and enhancement of their features in the form of diagrams and multidimensional models (Lynch, 1990; see also Gooding, 2004). The achievement of visual pertinence is, in effect, the result of a skillful conversion of idiosyncratic and transient impressions into universalized, durable, and communicable knowledge artifacts. Recent years have seen a greater attention to screenwork proper as science studies scholars broadened their purview to include many more arenas of expert knowledge production, arenas where computer visualization takes center stage in image work. Noteworthy here, given its similarities to weather forecasting practice, is research on financial forecasting. In their study of foreign exchange traders, Knorr Cetina and Bruegger (2002) adapt the Husserlian/Schu¨tzian concept of ‘‘appresentation’’ to suggest that the market-on-the-screen does not reflect back mere representations of local markets but rather conjures a new market, the electronic market, by bringing local markets near, delivering them literally at traders’ fingertips. Meanwhile, in their work with arbitrage traders, Daniel Beunza and David Stark (2004) note the paramount importance of the ecology of the trading room floor for the process of recognizing profit opportunities. Traders, they argue, must truly arbitrate between workstations, each corresponding to a different mode of pattern recognition, in the hopes of arriving at a profitable, innovative synthesis of the available appresented information. Studies of screenwork, then, alert to the tension or dual orientation inherent in augmented human perception. On the one hand, the materiality of the information-on-the-screen tends to become disembedded from its local, physical settings, to be bounded instead by the temporal logic of information exchange flows. On the other, the decision-making process is inexorably grounded in the physical space of a given laboratory of calculation designed to facilitate a particular (sociocognitive) experience of augmented reality. Differently put, screens are set up as

P. Daipha / Poetics 38 (2010) 150–164

153

conduits for resources and communication exchange across a visualization network. But they have also come to acquire a recentering, ‘‘scopic’’ quality (Knorr Cetina, 2003), collecting and focusing information on their surface and projecting it outward. It is due in part to the modulatory combinatorial properties of screenwork that computer visualization tools, rather than simply streamlining perception, confer new skills and professional authority. In the examination of forecasting practice at the NWS that follows, I unpack meteorological perception by foregrounding screenwork, or the ways weather forecasters process and coordinate visual information on their screens and with each other. Computer visualization externalizes the diagnostic and prognostic task by presenting information in a synoptic fashion, by distributing cognition across forecasters and interfaces, by directing attention and conversation to certain aspects of the screen/task, by embodying time-tested principles and rules of thumb. Moreover, it shapes forecasting practice itself by defining the standards and the limits of meteorological skill and expertise, by regulating access to material, epistemic, and social resources, and by establishing itself as the most appropriate medium for not only conveying but actually composing the forecast message. The notion of screenwork, therefore, opens up further opportunities to expose the limits of either strictly cultural or strictly biological explanations of the process of visual perception. 3. Screenwork at the NWS To examine meteorological perception at work, I draw on twenty-two months of fieldwork at a forecasting office of the NWS I shall henceforth call ‘‘the Neborough office.’’ Similarly to the other 121 NWS offices around the country charged with providing local weather and warning services, the Neborough office employs ten forecasters2 working in shifts of two and is operational around the clock. Forecasting life revolves around the operations deck, an artificially lit, open-plan space dominated by computer screens and assorted communication devices. Screens are arranged in L-shaped clusters of five to form a total of seven workstations. As is characteristic in control room settings, forecasting workstations are all exactly identical, the high redundancy in information access meant to foster interdependence among the staff. The outmost left screen provides access to the Internet and the weather data floating therein; next comes the ‘‘text screen,’’ where forecasters receive alerts and type out forecasts and warnings; the remaining three screens, sharing the same mouse and keyboard, serve as the ‘‘graphics screens.’’ It is to these last that forecasters give most of their attention – to study the weather of course, but also to digitally put together the new NWS graphical forecast. As screenwork, meteorological perception presupposes skillful handling of information technology. In addition to making every effort to maximize the ‘‘real estate,’’ as they call it, of their three graphics screens, Neborough forecasters are experts at capitalizing on their screens’ visualization capabilities. The aim, always, is to grasp the weather all at once, to embrace it synoptically (Latour, 1999: 51) so as to hopefully truly see it. They will thus use the zoom in/out function to slide through the atmosphere. And rarely do they examine weather maps as static, two-dimensional images. Instead, they exploit the 32-frame loop function to bring the weather to life right there inside the operations deck. More often than not, in fact, looping graphics are variously superimposed to form composite weather images, or they are juxtaposed in four-panel displays, each forecaster having a different tolerance level for the number of variables to be displayed at a time. What to the uninitiated comes across as a multicolored two-dimensional 2

All forecasters have a bachelor’s in meteorology, a few have a master’s degree. Most are white males in their forties and have typically been with the NWS for about 15 years.

154

P. Daipha / Poetics 38 (2010) 150–164

jumble of information becomes transformed into a three-dimensional view of the atmosphere under a forecaster’s knowing gaze. To be sure, forecasters must then see beyond the three-dimensional interaction of weather elements and behold it forward in time, in four dimensions. In effect, they must somehow propel themselves out of the loop of recent weather conditions forever pulsating on their screens and into the future. No wonder that the initial immersion into the weather situation of the day constitutes one of the very few quiet times on the Neborough operations deck. Forecasters will spend several minutes at a time literally staring at an animated weather map, the only sound the clicking of the mouse as they move between frames. Yet, after studying their screens for a while in silence, forecasters inevitably start uttering short comments about what they are seeing, comments that will usually elicit some sort of response from the other forecaster on shift.3 Such exchanges, more often than not, start off with a statement and not a question, a forecaster literally bouncing ideas around. And they always occur while forecasters remain seated at their respective desks. After a couple of back-and-forths, the discussion generally exhausts itself. Even so, the other forecaster will often pull up relevant data on his4 screens before responding. The redundant access to information governing NWS workstations lets forecasters share not only a real but also a virtual space. As a result, the responding forecaster does not need to look away from his screens, even though the ecology of the operations deck has been expressly designed to provide direct visual access to the graphics screens of the other workstation. He has the same data, he is familiar with at least the general features of the weather picture by virtue of working the same shift, and he is cognizant of the rhythm of the forecasting routine on the other desk. Indeed, if he does look away from his screens to the screens of the other workstation, this is bound to be a more involved exchange. And were he to end up wheeling himself over to the other workstation to deliberate the issue while looking at the same screens, this is a strong indication that something extraordinary is going on, something meteorologically complex and/or plain fascinating. The cognitive and/or emotional weight of the situation is such that a physical sharing of virtual space, a literal putting of heads together, is called for. ‘‘Everyone does it differently’’ is a phrase Neborough forecasters often use when asked to describe the weather forecasting process. True enough, a given forecaster may prefer particular maps and graphical analyses of current meteorological conditions in order to establish the problem of the day, and he may follow a particular sequence while doing so. Yet, forecasting the weather at the NWS is not a solitary achievement—it is an entirely collective enterprise, predicated on coordinated resources (Edwards, 2006) and shared, situated practices of looking and reasoning. 3.1. Seeing as situated activity It is rare today to encounter a purely biological explanation of human cognition—just as it is rare to encounter a radically constructionist approach to human experience. Recent arguments for the presence of shared cognitive processes across scientific disciplines consistently – albeit not always explicitly – rest on a socialized understanding of visual perception (e.g., Feist and Gorman, 1998; Carruthers et al., 2002; Beaulieu, 2003; Gooding, 2004). At least in principle, 3

Due to space limitations, I had to forgo an examination of the conversation punctuating forecasting practice, but’’ I consider it very much part of screenwork. What can been seen is inextricably tied to what can be said (Goodwin and Goodwin, 1996; Lynch, 1985). 4 My choice of pronoun is not arbitrary. NWS meteorology remains an overwhelmingly white male-dominated field.

P. Daipha / Poetics 38 (2010) 150–164

155

there is widespread agreement that ways of looking arise from and give rise to particular constellations of practice, i.e., particular sociotechnical systems of ordering the world. Although mostly prompted by separate concerns, furthermore, both cognitive sociology and neuroscience have much at stake in unearthing the mechanisms that transform visual stimuli into recognizable, meaningful information. The situatedness of seeing, therefore, offers a first meeting point for these two research traditions. 3.1.1. Repertoires of visual perception The first thing a forecaster will do when taking over a shift is switch the screens to the display settings stored under his profile, thus properly taking control of the workstation and the weather. While to some extent symbolic, this gesture has instrumental value as well. Much like a writer’s attachment to a particular font or zoom level, such seemingly trivial aesthetic choices become important for getting into ‘‘flow’’ (Csikszentmihalyi, 1990). In a profession that claims to be as much an art as it is a science, personalized forecasting routines, if not expressly encouraged, are nonetheless considered part of the process. And this is precisely why the exclamation ‘‘I was wondering why the weather is all off today!’’ I heard once uttered by a forecaster upon realizing he had neglected to switch display profiles does not ring false on the Neborough operations deck. Also stored under a forecaster’s profile are his so-called ‘‘procedures,’’ a set of personal ‘best practices’ for looking at weather information, accumulated over one’s career and updated as necessary. To study the wind along the atmospheric column, for example, one forecaster might prefer the 850, the 500, the 250, and the surface millibar height charts, while another might find the 700, the 500, the 200, and the surface charts more useful. But where the procedures become truly useful – and the variation among forecaster profiles infinite – is in recalling particular composites of data graphics: in a couple of mouse clicks, one may load multiple variables onto a screen. Forecasters like to claim that relying on their procedures is only a matter of expediency. In practice, however, temporal constraints and the threat of information overload lead to a nearabsolute dependency on such preset templates for studying the weather. These procedural shortcuts, then, do not simply represent a handy meteorological toolkit of sorts but the very embodiment of each forecaster’s skill set. More precisely, they are technological externalizations of the cognitive schemata governing forecasting practice at the NWS, as already distilled in particular techniques and know-how. Their seemingly idiosyncratic nature notwithstanding furthermore, forecaster profiles reflect eminently social decisions, the result of apprenticing at particular meteorology programs and forecasting offices, under particular mentors, with particular technologies, and so on. Says one forecaster. The habit of using four-panel displays is engrained in me from being an old timer from the days of wet paper facsimiles, with their wonderful smells (laughter). That’s the way that the model data was presented and, ah, I just continue that tradition out of familiarity with it. I mean, the basic four-panel that I use, that I start with, is similar to but not completely identical to the traditional four-panel from the facsimile days. There are several of the same things, like 500 millibar heights and vorticity, but lately I’ve started thinking, I’ve started testing out substituting the, instead of vorticity, having the ah, 250 millibar wind fields. [. . .] Plus I’ve got another four-panel where, ah, I take one of those values from each of the different models and I ensemble them as we like to say, just toss them all on top of each

156

P. Daipha / Poetics 38 (2010) 150–164

other to see what happens. And I use different color curves to, to kind of raise those levels of awareness to me. A lot of tailored stuff for my, for mine eyes! (laughter) Yeah, and I do that over a few minutes because it takes time to kinda see where the trend is. In contrast, younger forecasters tend to start their shift with one panel, the size of the whole graphic screen, because ‘‘there’s so much to see and it’s really hard to see it all initially.’’ Unless, that is, they are like Dennis, who got into the habit of working with the four-panel display through one of his mentors at a previous office, from whom he adopted many of the procedures he is using now. Whereas other forecasters are certain to be overwhelmed, drowning in information, Dennis will examine five or six parameters at a time on each of the four-panels on each of his three graphics screens. He says this may put a little strain on his eyes sometimes, but he likes the four-panel display ‘‘because you can see everything. I can look at a lot of parameters real quick and, like you said, there’s a lot to look at. You are looking at instability, this, that. . .. The reason I look at the four-panels is it’s kind of a security blanket, I guess. To make sure I’m not missing anything.’’ Clearly, the perceptual selection, focus, and amplification that procedures allow are as much driven by social as by biological considerations. What gets included, featured, or screened out is a sociocognitive process through and through, continuously renegotiated in forecasters’ struggle over professional expertise and authority, and exquisitely facilitated by the seemingly limitless flexibility of virtuality. 3.1.2. Regimes of visual perception Not only does screenwork at the Neborough office vary across forecasters, it varies across seasons as well. What drives this temporal variation is the differential atmospheric physics behind winter and summer weather. Whereas winter weather depends on large-scale meteorological conditions, summer weather is driven by localized parameters: winter storms evolve, summer storms erupt. Computer-generated models are, therefore, of little help under the threat of a severe summer storm. They may at best provide an indication of increased threat potential, but the where and when are matters left entirely to the micro-dynamics of the moment and forecasters’ agility at pattern recognition. The most valuable forecasting tool now becomes the radar, and a forecaster will simultaneously monitor information from a variety of radars on his screens, zooming in on an evolving thunderstorm to look for suspiciously colored cells, pondering on its three-dimensional structure from a variety of angles, zooming further in and running the mouse over the cells to get their reflectivity value. Unlike with other weather graphics, he will not layer radar graphics, however. The blocky, starkly contrasting color scheme of the severe mode radar display is meant to provide easily identifiable visual cues of real-time information, immediately recallable in their consequences. Indeed, observing Neborough forecasters in severe weather mode, one is reminded of marksmen or snipers: the spatiotemporal elusiveness of summer weather promotes a ‘one shot, one kill’ mentality, and forecasters have been primed to quickly identify their target before it has had a chance to do any damage. Keeping these pattern recognition muscles flexible is certainly the primary goal of the Severe Weather Training Meeting, conducted early every spring at the Neborough office. In contrast to the Winter Training Meeting, which focuses on conceptual analysis of model graphics, severe weather training revolves around the visual analysis of radar graphics. Specifically, during the severe weather meeting I attended in 2004, the weather was discussed in terms of ‘‘hook echo’’ and ‘‘bow echo’’ radar signatures or ‘‘kidney bean-shaped’’ and ‘‘comma-shaped’’ supercells, and forecasters were advised to not hesitate to warn if they saw a bow echo complex acquire a ‘‘seahorse’’ signature, regardless of the intensity of the reflectivity.

P. Daipha / Poetics 38 (2010) 150–164

157

Any analysis of this sharp disjunction in the visualization techniques forecasters use to process weather information would be incomplete without insights from experiments on object recognition. Neuroscientists have long settled on a hierarchically structured model of the visual cortex where an image becomes processed and subsequently fed to increasingly more sophisticated cells, tuned to more complex stimuli (Riesenhuber and Poggio, 1999). It is this nested architecture of the visual cortex that accounts for the relative ease and speed with which we can identify an object. Applied to the case at hand, it is the crude visual profiling of a complex and unstable weather phenomenon through the radar graphics – as opposed to the typical, highly graduated color scheme of meteorological graphics – that allows forecasters to quickly recognize danger and warn accordingly. Beyond appropriately designed displays and training sessions, however, an organization charged with protecting life and property is bound to surround forecasters’ screenwork with additional checks against missed warnings and false alarms—and here neuroscience stands to benefit from lessons ‘on the ground.’ The rapid decision-making character of severe summer weather thus militates for and is explicitly structured around a ‘‘severe weather team.’’ At no other time is the redundant access to weather information characteristic of the NWS operations deck more crucial. For, in the absence of the temporal (visual) continuity of meteorological data that allows their forecast to slowly grow in certainty over consecutive shifts, forecasters can resort to concurrent yet independent evaluations of the situation.5 They can – in fact, are explicitly instructed to – avail themselves of ‘‘another pair of eyes’’ with which to interact, brainstorm, and strategize. Space limitations prevent me from providing a taste of the controlled –visual, but also auditory – chaos reigning on the operations deck during a severe summer weather episode (see Daipha, 2007: 156–216; Fine, 2007; Pirtle Tarp, 2001). Suffice it to say, however, that it constitutes an excellent example of how the visual cortex may be externalized, technologically adapted and enhanced, socially distributed, and ultimately disciplined by the specific organizational logic of a system of expertise. 3.2. Visual expertise More than any other, the topic of expertise has proven a fertile ground for exploring the promise and limits of a rapprochement between culture and cognition (for an overview, see Cerulo, 2010). While there has been little cross-fertilization between sociological and cognitive approaches to the acquisition of visual expertise thus far, the growing research on the connections between brain plasticity and visual priming (Grill-Spector, 2008), the ongoing debate on the existence of a differential benchmark of object recognition between experts and novices (Johnson and Mervis, 1997; Tanaka, 2001), and, more generally, the current revisionist tendency within neuroscience of classical models of visual learning make a methodological and substantive contribution from sociology timely, if not necessary. NWS forecasting is an exceptionally good case for examining issues of expertise acquisition. Meteorologists are considered among the few highly calibrated professionals due to the great frequency of feedback they receive on their performance and the shortness of the link between their decisions and weather outcomes (Allwood and Granhag, 1999; Oreskes, 2003). Equally as important, NWS forecasting operations are founded on the notion of apprenticeship: one cannot 5 Although it is not uncommon for forecasters charged with radar warning duty to share a workstation, when given the option, they all prefer to study the radar on their own screens and terms, occasionally wheeling their chairs together to decide on a particularly thorny case.

158

P. Daipha / Poetics 38 (2010) 150–164

be a Senior Forecaster without previously having been a General Forecaster, and one cannot be a General Forecaster without previously having been an Intern. To be a good NWS forecaster effectively means to have mastered forecasting the NWS way. Weather forecasters, at the NWS and elsewhere, say what they value most in a forecaster are local meteorological knowledge and forecasting experience. Visualization skills, however, consistently appear near the top of that list, as per the interview excerpt below. Phaedra: What are some of the skills of a good forecaster? Forecaster: I think there are. . ..I think it comes down to brain. P: What do you mean by brain? F: Being able to. . ..like I said, seeing. Looking at a screen and seeing ten different things overlaid would drive my brain nuts. But to other people that’s like: ‘‘Oh! I see this, this, this, and this.’’ [. . .] If you can keep a lot of different kinds of data straight, like exactly when you saw it, and what it showed, and if you can keep mental images like that. Like -what is it called?- having good visual memory, that’s a very important skill to have. At any rate, in practice, good screenwork has become indistinguishable from good forecasting. And junior forecasters, always paired with a senior forecaster on a shift, learn how to foresee the weather by learning what to look for on their screens. A Neborough forecaster, recently promoted from Intern to General Forecaster, recalls: F: We always discussed, we’d always go over the models together, like over the forecast together, in the beginning: ‘‘Ok, the major features. I’m seeing this, do you see this?’’ And then we would talk. P: Would you see what he was seeing? F: Yeah, but not as much! He sees a lot more. I’m nowhere as good as Phil is at forecasting. He’s excellent at forecasting, and so. . ..but I would see the general trend. P: You are saying he’s a better forecaster because he is seeing more? F: He picks up on subtle diff. . ..subtleties. Like he would see a little subtlety like: ‘‘Oh, there’s a little disturbance coming through. It might spark a few snow showers or squalls. . .’’ So, yeah, picking up the little things. And experience, too, like knowing that it’s this kind of wind and with this kind of instability, you know, generally you can expect blah. Training for the upcoming season, briefings from one shift to the next, ‘‘post-mortems’’ of what are considered tricky or significant weather events, all occur in front of computer screens. The acquisition of visual expertise is intimately tied to the acquisition of forecasting skill. To be sure, veteran forecasters have a clear advantage over their less experienced colleagues and are therefore assured a higher position of authority in the deference structure of the operations deck. Nonetheless, the times where ‘raw talent’ succeeded where experience failed were not infrequent during my stay at Neborough. After all, years of experience and wisdom are great resources for analogically reasoning one’s way out of a complex weather scenario, but, according to intelligence theory, the ability at pattern recognition is fundamentally grounded on ‘‘fluid intelligence’’ (Horn and Cattell, 1967), i.e., intuitive and abstract reasoning, which arguably peaks in early adulthood. Indeed, together with leadership skills and level-headedness under pressure, it is the ability to rise above one’s (biological) limitations by surrounding oneself with talented forecasters that is the sign of a good senior forecaster at the Neborough office. Thus it is told of Wayne, revered regionwide for his ability to identify the weather pattern several days

P. Daipha / Poetics 38 (2010) 150–164

159

in advance, that ‘‘he plays to his strengths. He brings other people in who have a great skill set at the micro-scale, and that’s why he is such a phenomenal forecaster.’’ 3.3. Visual perception and decision-making Advances in our knowledge of the neural mechanisms of the visual cortex only serve to underscore the complexity of the visualization process, and how much work lies ahead. Not only are we just beginning to uncover the dynamics between human capacity, experience, and learning, we also have barely made any progress in understanding how visual perception becomes actionable, how it informs the decision-making process. Models of bounded rationality (Simon, 1982) and prospect theory (Kahneman and Tversky, 1979) have been uniquely successful in debunking the myth of the ‘‘rational man,’’ but they have yet to be systematically incorporated into the study of visual decision-making. More importantly, rely as they preponderantly do on experimental design settings, they often fall prey to multiple other essentializing assumptions and biases. Sorely needed are sociological theories of judgment and decision-making that can shed light on the cultural context of action. Due to the prototypically chaotic nature of its subject matter and the relatively high calibration of its performance measures, the screenwork and decision-making process of weather forecasting, including NWS forecasting, has been extensively studied by psychologists and computer scientists (e.g., Heideman et al., 1993; Hoffman et al., 1993, 2000; Trafton et al., 2000; Pliske et al., 2004). The ultimate objective of all these studies, however, has been to identify and recommend strategies for optimizing task performance. As a result, they consistently have a blind spot for the (micro) politics, administrative pressures, and traditions of forecasting life. But, of course, to properly evaluate the forecasting task, such dynamics cannot be treated as ‘noise.’ As I elaborate below, they are crucial – arguably, the most crucial – aspects of what drives, motivates, and thwarts forecasting performance. The quest for autonomy and control over nature and machine is a highly resonant notion among weather forecasters, intimately tied to the dual mission of serving science and serving the public that appears to define their identity. To be professionally acknowledged as a forecaster, one is expected to counter one’s skills at pattern recognition against those of the computer. At the Neborough office at least, the credo that computer models are ‘‘just guidance’’ runs deep, and the one or two forecasters reputed to consistently stray from that path and treat computer-generated forecasts as the final word are held in low regard. The following interview excerpt wonderfully captures the prevailing mood: You have to have a framework of how to approach the day starting with the big picture, understanding what’s driving the atmosphere this week, where are we at. Be able to take that into where the sizeable weather is happening. Some people are very very good at that. Others unfortunately have been slaves to the model. [. . .] And they’re going to mimic what the model gives them, and you wind up with a very safe forecast with very little value added. The good forecasters, the excellent forecasters, they don’t stop there. They start higher in the atmosphere. They look at the global pattern to begin to assimilate what’s driving the bus. Where are we? And bring yourself down then into the lower atmosphere and begin to assimilate your own model within your head of where things are. And you can use the ensemble models to help you couch this. You begin to sense that, OK, we’re in this type of regime, we’re in a transition. Transitions usually bring excitement. What kind of excitement are we in for? You come down into the models then and you start looking at the

160

P. Daipha / Poetics 38 (2010) 150–164

fronts and jets, the vorticity advection, and you begin to look and view how the model graphics are simulating what the atmosphere is going to do, and you begin to see where your weather makers are going to be, and you come down and bring that down to the surface to where we’re really forecasting, the human impact, the impact on the surface of the planet. And you’re assimilating tons of information. Where are we? What type of system are we going to be dealing with for the rest of week? [. . .] And then you finally get into a point where conceptually you have an idea of what you’re going to forecast. Communicate that to your [neighboring offices] and then you go into the Graphical Forecast Editor and you assimilate those pieces of information to best fit what you’re going to forecast. . . Others, others, the ones who don’t have that innate ability struggle, especially in times of transition in patterns. They’re going to go on old faithful: surface thicknesses and precip[itation]. And they’re going to play it safe, and a fair percentage of the time they won’t be that bad. But they won’t be adding that value. They won’t be making that big call that could have a significant impact. This almost lyrical description of the meteorologically correct way to work the screens is saturated with what at first blush may appear to be extraneous assumptions: that a good forecast requires ‘‘beating the models,’’ and that a forecaster proves his worth by the risks he is willing to take. To be sure, proper screenwork helps guard against the tendency for extremes. Yet these assumptions are important ingredients that, under the right circumstances, can make or break a forecast. Along with external pressures to meet personal and office verification targets, perceived past performance, and the small-group dynamics of an office, they provide critical insights into the process of visual perception and decision-making. A solid grasp of the context, in addition to the task at hand, proves instrumental for implementing technological change, as illustrated by the recent NWS operational transition to a graphical forecast. What initially seemed like a natural extension of meteorological screenwork beyond the study of the weather to its production was met with prolonged, ferocious resistance by forecasters until NWS administration began to properly embed the new routine into the forecasting environment by framing it in organizationally meaningful terms (Daipha, 2007: 89– 123). Only then were forecasters able to see the potential of the new forecasting technique, and only then did they start to work their screens in such a way as to increase their visual-tactile thinking (Baird, 2004) and enhance their forecasting performance in accordance with the new NWS standards. 4. Discussion: screenwork and the heuristic of the collage I have examined the sociocognitive organization of meteorological perception and expertise via the notion of screenwork in the hopes that it may provide a common constructivist ground on which sociologists and neuroscientists will be able to comfortably study visual perception at work. I conclude by considering the heuristic value of the metaphor of the ‘collage’ for the study of visual perception at work. It was Tversky (1993) who first linked perception to the metaphor of the collage by coining the term ‘‘cognitive collage’’ in order to suggest an alternative mental model of spatial knowledge to the conventional one of the ‘‘cognitive map,’’ arguing that it can better account for the ad hoc multi-modal alignment of information in the face of the systematic distortions inherent in working memory during route navigation. Since then, the notion of the cognitive collage has been greatly influential in Geographic Information System design.

P. Daipha / Poetics 38 (2010) 150–164

161

Yet, beyond capturing the mentalistic process of perception, the collage metaphor can be fruitfully employed to suggest how one goes about crafting visual coherence. Elsewhere (Daipha, 2006, 2007), I have used the heuristic of the collage to examine the distillation process of highly complex information. I define collage as a multidimensional synthesis of diverse fragments, pictorial and otherwise, of different scale and texture, arranged in such a way as to create a new whole image. As a technique, collage is improvisational, open-ended, welcoming of surprising or paradoxical encounters. It accommodates notions of relationality, multiplicity, performativity, reflexivity, material heterogeneity, uncertainty, emergence. As a finished product, a collage’s appeal lies in its poetics of order out of seemingly chaotic and disparate elements. Collage is at its best when ‘‘things relate but don’t add up’’ (Law and Mol, 2002: 1). Applied to the case of meteorology, the heuristic of the collage reveals weather forecasting as a process of assembling, appropriating, juxtaposing, and blurring of information. Neborough forecasters assemble reports of snow or wind damage from volunteer weather observers to supplement the reports from automated weather stations already assembled and plotted on their screens. And they garner further information about the weather by appropriating seemingly extraneous information pieces: road conditions reports faxed over hourly from the Department of Transportation or keyword-filtered newspaper clippings set to arrive daily at the office. As already discussed, they exploit the capabilities of their screens by blurring together layers of observational and/or model information into a composite spatiotemporal animated continuum. These live weather compositions will be then juxtaposed in a four-panel display with other such compositions as forecasters proceed with their visual hermeneutics of the weather. The collage is thus an ongoing cognitive process, partially articulated in the ways forecasters arrange the weather information on their screens or in their discussions with colleagues on the operations deck and online. But it also (momentarily) assumes a fixed form, to become the NWS forecast. By cutting and pasting weather graphics, drawing contours to adjust weather values, assigning algorithmic functions, and generally manipulating the color-coded fields of their graphics screens with their mouse, NWS forecasters actively enlist science, technology, art, experience, and intuition into the construction of a future reality that endeavors to stand up to the complexity of the weather, the needs of their publics, and the demands for professional accountability. In short, the metaphor of the collage, as a heuristic for how coherence is attained in the face of complex information, provides a further articulation of the link between screenwork and expert visual perception. Acknowledgment I am grateful to Karen Cerulo and two anonymous reviewers for their very helpful comments and suggestions. References Allwood, C., Granhag, P., 1999. Feelings of confidence and the realism of confidence judgments in everyday life. In: Juslin, P., Montgomery, H. (Eds.), Judgment and Decision Making: Neo-Brunswikian and Process-Tracing Approaches. Lawrence Erlbaum Associates Inc., Mahwah, pp. 123–146. Baird, D., 2004. Thing Knowledge: A Philosophy of Scientific Instruments. University of California Press, Berkeley. Beaulieu, A., 2003. Brains, maps and the new territory of psychology. Theory and Psychology 13, 561–568. Beunza, D., Stark, D., 2004. Tools of the trade: The socio-technology of arbitrage in a Wall Street trading room. Industrial and Corporate Change 13, 369–400.

162

P. Daipha / Poetics 38 (2010) 150–164

Breidbach, O., 2002. Representation of the microcosm: the claim for objectivity in 19th century scientific microphotography. Journal of the History of Biology 35, 221–250. Burri, R., Dumit, J., 2008. Social studies of scientific imaging and visualization. In: Hackett, E., Amsterdamska, O., Lynch, M., Wajcman, J. (Eds.), The Handbook of Science and Technology Studies. MIT Press, Cambridge, pp. 297– 318. Cambrosio, A., Daniel, J., Keating, P., 1993. Erlich’s ‘‘Beautiful Pictures’’ and the controversial beginnings of immunological imagery. Isis 84, 662–699. Carruthers, P., Stich, S., Siegal, M. (Eds.), 2002. The Cognitive Basis of Science. Cambridge University Press, Cambridge. Cerulo, K., 2010. Mining the intersections of cognitive sociology and neuroscience. Poetics 38, 115–132. Cohn, S., 2004. Increasing resolution, intensifying ambiguity: an ethnographic account of seeing life in brain scans. Economy and Society 33, 52–76. Csikszentmihalyi, M., 1990. Flow: The Psychology of Optimal Experience. Harper and Row, New York. Daipha, P., 2006. The total observation collage: weather forecasting and the search for ground truth. In: American Sociological Association Conference, Montreal, QE. Daipha, P., 2007. Masters of Uncertainty: Weather Forecasters and the Quest for Ground Truth. Unpublished Dissertation, University of Chicago. Dumit, J., 2004. Picturing Personhood: Brain Scans and Biomedical Identity. Princeton University Press, Princeton. Edwards, P., 2006. Meteorology as infrastructural globalism. Osiris 21, 229–250. Feist, G., Gorman, M., 1998. The psychology of science: review and integration of a nascent discipline. Review of General Psychology 2, 3–47. Fine, G., 2007. Authors of the Storm: Meteorologists and the Culture of Prediction. University of Chicago Press, Chicago. Francoeur, E., 2002. Cyrus levinthal, the kluge and the origins of interactive molecular graphics. Endeavour 26, 127–131. Foucault, M., 1970. The Order of Things: An Archaeology of the Human Sciences. Routledge & Kegan Paul, London. Galison, P., 1997. Image and Logic: A Material Culture of Microphysics. Chicago University Press, Chicago. Giere, R., Moffatt, B., 2003. Distributed cognition: where the cognitive and the social merge. Social Studies of Science 33, 301–310. Gieryn, T., 1999. Cultural Boundaries of Science: Credibility on the Line. University of Chicago Press, Chicago. Gorman, M., 1997. Mind in the world: cognition and practice in the invention of the telephone. Social Studies of Science 27, 583–624. Gooding, D., 2004. Cognition, construction and culture: visual theories in the sciences. Journal of Culture and Cognition 4, 551–593. Gooding, D., 2006. Visual cognition: where cognition and culture meet. Philosophy of Science 73, 688–698. Goodwin, C., 1995. Seeing in depth. Social Studies of Science 25, 237–274. Goodwin, C., Goodwin, M., 1996. Seeing as situated activity: formulating planes. In: Engestro¨m, Middleton, (Eds.), Cognition and Communication at Work. Cambridge University Press, Cambridge, pp. 61–95. Grill-Spector, K., 2008. Visual priming. In: Eichenbaum, H., Bryne, J. (Eds.), Learning and Memory: A Comprehensive Reference. Section 3. Elsevier, Oxford, pp. 219–236. Hacking, I., 1999. The Social Construction of What? Harvard University Press, Cambridge, MA. Heideman, K., Stewart, T., Moninger, W., Reagan-Cirincione, P., 1993. The weather information and skill experiment (WISE): the effect of varying levels of information on forecast skill. Weather and Forecasting 8, 25–36. Henderson, K., 1999. On Line and on Paper: Visual Representations, Visual Culture, and Computer Graphics in Design Engineering. MIT Press, Cambridge, MA. Hoffman, R., Detweiler, M., Lipton, K., Conway, J., 1993. Considerations in the use of color in meteorological displays. Weather and Forecasting 8, 505–518. Hoffman, R., Coffey, J., Ford, K., 2000. A Case Study in the Research Paradigm of Human-centered Computing: Local Expertise in Weather Forecasting. National Technology Alliance, Arlington, VA. Horn, J., Cattell, R., 1967. Age differences in fluid and crystallized intelligence. Acta Psychologica 26, 107–129. Hutchins, E., 1995. Cognition in the Wild. MIT Press, Cambridge, MA. Johnson, K., Mervis, C., 1997. Effects of varying levels of expertise on the basic level of categorization. Journal of Experimental Psychology 126, 248–277. Joyce, K., 2005. Appealing images: magnetic resonance imaging and the production of authoritative knowledge. Social Studies of Science 35, 437–462. Joyce, K., 2006. From numbers to pictures: the development of magnetic resonance imaging and the visual turn in medicine. Science as Culture 15, 1–22.

P. Daipha / Poetics 38 (2010) 150–164

163

Kahneman, D., Tversky, A., 1979. Prospect theory: an analysis of decisions under risk. Econometrica 47, 313–327. Knorr Cetina, K., 1999. Epistemic Cultures: How the Sciences Make Knowledge. Harvard University Press, Cambridge, MA. Knorr Cetina, K., 2003. From pipes to scopes: the flow architecture of financial markets. Distinction 7, 7–23. Knorr Cetina, K., Amann, K., 1990. Image dissection in natural scientific inquiry. Science, Technology, and Human Values 15, 259–283. Knorr Cetina, K., Bruegger, U., 2002. Global microstructures: the interaction practices of financial markets. American Journal of Sociology 107, 905–950. Latour, B., 1987. Science in Action: How to Follow Scientists and Engineers through Society. Harvard University Press, Cambridge, MA. Latour, B., 1990. Visualization and cognition: thinking with eyes and hands. Knowledge and Society 6, 1–40. Latour, B., 1999. Pandora’s Hope: Essays on the Reality of Science Studies. Harvard University Press, Cambridge, MA. Latour, B., Woolgar, S., 1986 (1979). Laboratory Life: The Construction of Scientific Facts. Princeton University Press, Princeton, NJ. Law, J., Lynch, M., 1988. Lists, field guides, and the descriptive organization of seeing: birdwatching as an exemplary observational activity. Human Studies 11, 271–304. Law, J., Mol, A. (Eds.), 2002. Complexities: Social Studies of Knowledge Practices. Duke University Press, Durham. Lynch, M., 1985. Art and Artifact in Laboratory Science. Routledge & Kegan Paul, London. Lynch, M., 1990. The externalized retina: selection and mathematization in the visual documentation of objects in the life sciences. In: Lynch, M., Woolgar, S. (Eds.), Representation in Scientific Practice. MIT Press, Cambridge, MA, pp. 153– 186. Lynch, M., Edgerton, S., 1988. Aesthetics and digital image processing: representational craft in contemporary astronomy. In: Fyfe, J., Law, J. (Eds.), Picturing Power: Visual Depiction and Social Relations. Routledge & Kegan Paul, London, pp. 184–220. Lynch, M., Woolgar, S. (Eds.), 1990. Representation in Scientific Practice. MIT Press, Cambridge, MA. McCormick, B., DeFanti, T., Brown, M. (Eds.), 1987. Visualization in Scientific Computing. Computer Graphics, p. 21. Monmonier, M., 1999. Air Apparent: How Meteorologists Learned to Map, Predict, and Dramatize Weather. University of Chicago Press, Chicago. Oreskes, N., 2003. The role of quantitative models in science. In: Canham, C., Cole, J., Lauenroth, W. (Eds.), Models in Ecosystem Science. Princeton University Press, Princeton, pp. 13–31. Pickering, A. (Ed.), 1992. Science as Practice and Culture. Chicago University Press, Chicago. Pirtle Tarp, K., 2001. Communication in the Distributed Cognition Framework: An Ethnographic Study of a National Weather Service Forecast Office. Unpublished Manuscript, University of Oklahoma. Pliske, R., Crandall, B., Klein, G., 2004. Competence in weather forecasting. In: Smith, K., Shanteau, J., Johnson, P. (Eds.), Psychological Investigations of Competence in Decision Making. Cambridge University Press, Cambridge, pp. 40–70. Prasad, A., 2005. Making images/making bodies: visibilizing and disciplining through magnetic resonance imaging (MRI). Science, Technology & Human Values 30, 291–316. Prentice, R., 2005. The anatomy of a surgical simulation: the mutual articulation of bodies in and through the machine. Social Studies of Science 35, 837–866. Rasmussen, N., 1999. Picture Control: The Electron Microscope and the Transformation of Biology in America, 1940– 1960. Stanford University Press, Stanford. Resnik, L., 1991. Shared cognition: thinking as a social practice. In: Resnik, L., Levine, J., Teasley, S. (Eds.), Perspectives on Socially Shared Cognition. American Psychology Society, Washington, DC, pp. 1–20. Riesenhuber, M., Poggio, T., 1999. Hierarchical models of object recognition in cortex. Nature Neuroscience 2, 1019–1025. Rudwick, M., 1995. Scenes from Deep Time: Early Pictorial Representations of the Prehistoric World. Chicago University Press, Chicago. Scaife, M., Rogers, Y., 1996. External cognition: how do graphical representations work? International Journal of Human-Computer Studies 45, 185–213. Shapin, S., Schaffer, S., 1985. Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life. Princeton University Press, Princeton. Simon, H., 1982. Models of Bounded Rationality. MIT Press, Cambridge, MA. Star, S., Griesemer, J., 1989. Institutional ecology, ‘translations’ and boundary objects: amateurs and professionals in Berkeley’s museum of vertebrate zoology, 1907–39. Social Studies of Science 19, 387–420. Trafton, J., Kirschenbaum, S., Tsui, T., Miyamoto, R., Ballas, J., Raymond, P., 2000. Turning pictures into numbers: extracting and generating information from complex visualizations. International Journal of Human Computer Studies 53, 827–850.

164

P. Daipha / Poetics 38 (2010) 150–164

Tanaka, J., 2001. The entry point of face recognition: evidence for face expertise. Journal of Experimental Psychology 130, 534–543. Traweek, S., 1997. Iconic devices: toward an ethnography of physical images. In: Downey, G., Dumit, J. (Eds.), Cyborgs & Citadels: Anthropological Interventions in Emerging Sciences and Technologies. School of American Research Press, Santa Fe, pp. 103–115. Turnbull, D., 1994. Maps are Territories, Science is an Atlas. University of Chicago Press, Chicago. Tversky, B., 1993. Cognitive maps, cognitive collages, and spatial mental model. In: Frank, A., Campari, I. (Eds.), Spatial Information Theory: Theoretical Basis for GIS. Springer Verlag, Heidelberg-Berlin, pp. 14–24. Viseu, A., 2003. Simulation and augmentation: issues of wearable computers. Ethics and Information Technology 5, 17– 26. Phaedra Daipha is Assistant Professor of Sociology at Rutgers, The State University of New Jersey. Her research interests primarily revolve around expertise, prediction, and technology use. She is currently working on a book manuscript that analyzes the process of diagnosis and prognosis under regimes of high complexity, based on an ethnographic study of forecasting practice at the National Weather Service.