Influence of film and monitor display luminance on observer performance and visual search

Influence of Film and Monitor Display Luminance on Observer Performance and Visual SearcW Elizabeth Krupinski, PhD, Hans Roehrig, PhD, Toshihiko Furukawa


2/:::¸

::

:::~;-: i.=!5::

<~/i

....

~ ::

: <:

:

.::

"

Rationale andObj~tives2 The p u r p o s e o f ~ : siudy was tion p e r f o ~ c e

and Visual: Search behavior, The :reSults:

i

::~cceptable display Conditions :for Viewing radiographs o n :(cathode-ray tube (CRT)monitors. ~: i ::: i: : : :

MaterialS:and Methods. Two groupS6f six radi0iogists each ~iewed 50 pairs of mammograms: One group:viewed : :: film images o n a standard marmnographiC View bo•; the:= 5:other viewed images on a::high4resolufiog CRT monJtori : Two luminance levels Were studied for each:display type. : : ~servers: reported on the presence or: absencei:;f:maSses ior: microcalcification clusters and on their C0nfidenCe:in: : ithat decision.: Confidence dat~ were analyzed by ~ing: . a l ; :ternative free-response receiver overating: ch~cteristic::: : i :(AFROC): techniques;iEye ~ i t i e n als0 was::recorded ::a S : obServersviewed:t,he~ages~ : i : : :: : : : : : : Results, For both the:film andlmonitorstudie~ de~cfion : : feeted:si~ficandy :by display ! ~ n a n c e ; ~ ¢ search behavior was, Total:vieWing:~d:deci~ion dwCd times were ::shorter With the hlg,her-la~ancedisptays, ~eeially:for :true-negativedeciSionS; Significantly morefiX~on dus- = : :ters were :generated during the Search of lesion2fr~ : of !esion~containing imageswith the:lower;lun-,in~ dis-: :plays. : i i : : i Conclusion:Di~iay lUmifi~ceaffec~)vi~ ~arehper-::: £ormance with:both fitm and mon!tc~=displays ~thout feefing detection performance Ngnific~tly: Uigher-lu~= i :i:nance displays Yield:mo e efficient::searchperf~anc;.:: i i The :h'~aeSnegativedwe!l times: and numb~;0f clasters ::i :suggestive that Ioweizlumhnance leveiS prolong the search : and recognition~flaormal~ies!bn-free areas ~ompaxed::: : iii with:lesion-cOntaining areas~: : i : : i : : :: :::

K . wo s. sptay:l na.Ce; awoli es obse performancei

; ::::::

::::::::i :::

;

:' :;

Digital image acquisition and display technologies are undergoing rapid development and increasing use in radiology departments. This is especially true for mammography, in which full-field digital mammographic systems may soon replace traditional film acquisition and display systems (1-3). One main advantage of digital images is that tools such as image enhancement and computer-aided detection schemes can be readily implemented on the display workstation to improve lesion detectability and recognition. One potential impediment to regular use of digital systems, however, is that optimal display design and performance factors are not yet completely understood for reading radiographs from cathode-ray tube (CRT) monitors. Numerous perceptual, learning, and ergonomic factors must be considered during the transition from film to filmless radiographic reading (4). The question of how best to display digitally acquired mammograms on a computer workstation is being addressed by several groups (5-8). In fact, a recent workshop of the Working Group on Digital Mammography (9) dedicated an entire meeting to Digital Displays and Workstation Design. In addition to issues such as designing a navigation system for viewing digitally displayed mammograms, one important and basic factor considered at this meeting was display luminance. Radiographic view boxes for reading film images are bright compared with CRT

Aead Radio! 1999; 6:411--418 1 From the Department of Radiology, PO Box 245067, University of Arizona, Tucson, AZ 85724 (E.K., H.R.); and DataRay Corporation, Westminster, Colo (T.F.). Received October 29, 1998; revision requested December 17; revision received January 4, 1999; accepted January 19. Supported in part by DataRay Corporation and Toshiba Medical Systems, Tokyo, Japan. Address reprint requests to E.K. ©AUR, 1999

411

Table 1 Basic Specifiications for the DR110 Monitors

160.00--

E

120.00

o o v (1) c0 r-

80.00

E _J

"5 c~ "5

O

40.00

0.00

' 100.00

I 200.00

'

I 300.00

~

I 400.00

Beam Current (gA) Light output versus anode peak current for the 140- and 80-footlambert DataRay DR110 CRT monitors.

monitors. A luminance of more than 1,000 foot-lambert (3,500 nit) is recommended for mammography (10). Monitors at approximately 200 foot-lambert are available today, but their cost and useful lifetime are important factors when considering use of such monitors. Several studies (11-14) have demonstrated how changes in view-box luminance can affect the detection of targets in mammographic images. In general, the combination of higher-luminance view boxes with proper image masking has yielded the highest detection levels. The question, however, is whether these results can be generalized to CRT monitors. Few, if any, experimental studies have addressed this question. The g0al of this study was to determine if changes in luminance affect the detection accuracy of radiologists viewing mammograms on highresolution CRT monitors. In addition to evaluating detection performance, we also were interested in determining if changes in luminance affect the visual search patterns of radiologists as evaluated by recording eye position.

Two studies were conducted. The first study examined the effect of changes in luminance on detection accuracy and visual search behavior when film images were used. As noted elsewhere, changes in view-box luminance (1114) affect detection performance. To our knowledge, no studies have been performed to examine whether changes in view-box luminance affect visual search patterns. The

412

Parameter

Setting

Display type CRT size Deflection angle Active display area Phosphor Bulb transmission Panel transmission Resolution Refresh rate

Portrait mode 21-inch full screen 90 ° 11.5 x 15.5 inches Phosphor-45 52% 92% and 62% 1,728 x 2,304 70 Hz

second study paralleled the first, examining changes in detection accuracy and visual search performance as a function of luminance differences with CRT monitors. Images and Displays The same general paradigm was used for both studies. A series of 50 pairs of mammograms (craniocaudal and mediolateral views, right or left breast) was collected from the Tucson Breast Center. Eighteen cases had a single, subtle mass, and 18 had a single, subtle cluster of microcalcifications. Fourteen were lesion free. All lesions were judged to be subtle by an experienced mammographer who did not otherwise participate in the study. All cases of masses or microcalcifications were biopsy proved, and the lesion-free cases had a confirmed, lesion-free status for at least 3 years. Only malignant lesions were included in the study, because the goal was to measure search and detection performance, not classification performance. The classification of microcalcifications, in particular, often requires the radiologist to use a magnifying glass or to move close to the display. Both of these factors can disrupt the eye-position recording system, so we used only a detection paradigm. The film images were digitized by using a digitizer (Lumiscan 150; Lumisys, Sunnyvale, Calif) with a spot size of 80 gm and a contrast resolution of 12 bit. For the film study, images were displayed on a standard mammographic view box (1,100 foot-lambert). In one condition, the film images were simply displayed on the view box as they normally are (at 1,100 foot-lambert). In the second condition, a piece of radiographic film was uniformly exposed (fiat field) and placed behind the view-box diffuser panel. This resulted in a uniform drop in display luminance to 60% of original (660 foot-lambert). The mammograms were then placed on the front of the view box as normally done. A 60% drop in luminance

Vol 6, No 7, July 1999

DISPLAY L U M I N A N C E

Table 2 AFROC A1 Values for Observers in the Film Study

Table 3 AFROC A1 Values for Observers in the Monitor Study

Observer

1,100 foot-lambert

660 foot-lambert

Difference

Observer

140 foot-lambert

80 foot-lambert

Difference

1 2 3 4 5 6 Mean

.9466 .9837 .9514 .9563 .8850 .9772 .9500

.9398 .9894 .9548 .9517 .8784 .9691 .9472

+.0068 -.0057 -.0034 +.0046 +.0066 +.0081 +.0028

1 2 3 4 5 6 Mean

.9655 .9843 .9538 .9744 .9727 .9663 .9695

.9603 .9837 .9498 .9379 .9551 .9695 .9594

+.0052 +.0006 +.0040 +.0365 +.0176 -.0032 +.0101

was chosen because that was the luminance difference for the two monitors used in the second study. All extraneous view-box light was masked. For the monitor study, images were displayed on two monitors (DR110; DataRay, Westminster, Colo). One monitor had a maximum luminance of 140 foot-lambert, and the other had a maximum luminance of 80 foot-lambert. The monitors were the same, however, in every other respect except for the front panel, which accounted for the 60% difference in luminance. Table 1 lists the basic specifications of the monitors, and the Figure shows the light output versus anode peak current for the two monitors. A display controller board (MD4; Dome Imaging Systems, Waltham, Mass) was used in both monitors. In both studies, ambient room lights were turned off to make viewing conditions as similar as possible. The brightness and contrast controls of the monitors were set with the pattern of the Society of Motion Pictures and Television Engineers, or SMPTE (15,16), so that the 95% versus 100% and 0% versus 5% contrast objects could be seen equally well.

Viewing Procedure Six radiologists participated as observers in each study. The same six radiologists did not participate in both studies, however, because in each, they had to view every image in two conditions. If the radiologists had participated in both studies, they would have seen each image a total of four times, thus increasing the likelihood they would recall some of the images and confound the results. The radiologists in both studies were matched as closely as possible regarding years of radiology experience and experience reading mammograms. In each group of observers, one read mammograms on a daily basis, two read at least once a month, and three were 3rd- or 4th-year residents experienced in reading mammograms.

A counterbalanced, randomized experimental design was used for each study. In the film image study, three of the radiologists saw half the images on the regular, higherluminance view box during the first session and the other half on the view box with the flat-field film behind the diffuser panel during the second session. After a period of at least 6 months (to promote forgetting of the images), the observers came back and viewed the images in the opposite conditions. The other three observers viewed the images in the reverse order. The protocol was the same for the six observers in the monitor study. Each session lasted approximately 1 hour. The pairs of images appeared side by side (craniocaudal on the left, mediolateral on the right) on the view box or CRT monitor, No image-processing functions (eg, window, level) were available during the monitor-reading sessions. A previous study (17) showed that the presence and use of an image-processing menu significantly (P = .0001) affects visual search times and viewing patterns. We wanted to eliminate this variable from the study to make viewing conditions, except for the displays themselves, as similar as possible. During the image-selection process, we chose only those images that looked as similar as possible to their film counterparts. Rejected images had digitization artifacts or individual microcalcifications that appeared too small to be sampled sufficiently by the digitizer and thus were rendered invisible. Observers could view images for as long as desired. Viewing time per case was recorded, and for each case, the observers had to report their decision in two parts. In the first part, observers decided if the case was lesion free or contained a mass or microcalcification cluster. Observers then had to report their confidence in that decision by using a sixlevel rating scale (1 = no lesion, definite; 2 = no lesion, probable; 3 = no lesion, possible; 4 = lesion, possible; 5 = lesion, probable; 6 = lesion present, definite). If a mass or

413

Table 4 Decisions Made in the Higher- and Lower-Luminance Conditions of the Film and Monitor Studies Film (%) Decision True-positive False-negative False-positive True-negative

Monitor (%)

1,100 foot-lambert

660 foot-lambert

140 foot-lambert

80 foot-lambert

88 12 4 90

84 16 5 89

85 15 4 88

83 17 5 86

(191/216) (25/216) (12/300) (76/84)

(182/216) (34/216) (14/300) (75/84)

(183/216) (33/216) (12/300) (74/84)

(179/216) (37/216) (14/300) (72/84)

Note.--Numbers in parentheses are raw data.

microcalcification cluster was reported, observers also had to indicate its position on an outline of the breast provided for that purpose. Observers could indicate more than one finding per case. The readers were told that all lesion-containing cases were malignant and that no classification was required. Eye-Position Recording and Analysis Eye position was recorded by using an Eye-Tracker 4000SU system with head tracker (Applied Science Laboratories, Waltham, Mass). Complete details of the procedures used to record eye position and to calibrate the position of the observers have been described elsewhere (18). For initial calibration purposes in this study, the observers were seated 45 cm from the display. After calibration, observers were free to move closer to the display and to move their heads. A detailed account of the methods used to analyze the x,y-fixation data can be found in Nodine et al (18). For this

study, if 50% of a fixation cluster overlapped a mass or microcalcification cluster, it was considered to be a "hit." A 5 ° circle was used to define the area of a fixation cluster, because focal attention is assumed to extend to structures as much as 2.5 ° from the center of a cluster (19). The same criterion was used for a false-positive decision, except the cluster overlapped the erroneously reported image location. True-negative decisions constituted those areas with fixation clusters that were lesion free. True-negative clusters were measured on both lesion-free and lesion-containing images, because both types of images have fixated, lesionfree areas. Statistical Analysis The confidence data were analyzed by using alternative free response receiver operating characteristic (AFROC) analysis (20) techniques, because observers could indicate

414

more than one finding per case. Again, this was a detection task only, without classification of lesions as either benign or malignant. A t test for paired observations was used to analyze the differences in conditions statistically. The data were also analyzed by looking at the actual number of truepositive, false-negative, false-positive, and true-negative decisions made in each viewing condition. The viewing time data were analyzed with an analysis of variance (ANOVA). The dwell data were anlayzed with a median test for two populations, since the distributions were skewed.

Observer Performance The area under the curve (A1) values for the six observers in the film image and monitor studies are presented in Tables 2 and 3, respectively. For the film image study, there was no statistically significant difference in A1 values between the 1,100-foot-lambert (mean, .9500; standard deviation, .032) and 660-foot-lambert (mean, .9472; standard deviation, .034) conditions (t = 1.182, d f = 5, P = .2904). For the monitor study, there was also no statistically significant difference in A1 values between the 140foot-lambert (mean, .9695; standard deviation, .009) and the 80-foot-lambert (mean, .9594; standard deviation, .015) conditions (t = 1.685, d f = 5, P = .1528). The decision results are presented in Table 4. As mentioned, observers were allowed to report more than one lesion per image, so the false-positive rate is based on the total number of images in the study, not just the number of lesion-free images. On average, there were 13.25 falsepositive reports (4.5%) across observers in each of the four conditions (high and low luminance, film and monitor). There was an average of 8.5 false-positive reports on the lesion-free images and 5.0 on the lesion-containing images across all conditions.

Table 5 Viewing Times for the Film and Monitor Studies Condition Film 1,100 foot-lambert, mass 1,100 foot-lambert, microcalcification 1,100 foot-lambert, lesion free 660 foot-lambert, mass 660 foot-lambert, microcalcification 660 foot-lambert, lesion free Monitor 140 foot-lambert, mass 140 foot-lambert, microcalcification 140 foot-lambert, lesion free 80 foot-lambert, mass 80 foot-lambert, microcalcification 80 foot-lambert, lesion free

Table 6 Dwell Times in the Film Image Study

Viewing Time (sec)

Dwell Time (msec) Decision

23.45 23.73 30.04 24.19 27.48 36.55

_+ 15.41 _+ 16.38 _+ 15.95 +_ 18.31 _+ 18.92 _+ 17.47

48.08 45.50 54.66 48.89 49.97 61.14

_+ 22.62 _+ 21.49 _+ 21.49 _+ 24.31 _+ 22.36 _+ 24.11

Note.--Data presented as mean + standard deviation.

Viewing Time

Table 5 presents the total viewing time results. In the film image study, overall viewing times differed significantly for the 1,100- versus the 660-foot-lambert condition (t = -1.968, df= 299, P = .048), with the viewing time lasting 2.82 seconds longer on average with the 660-foot-lambert display. Viewing times ranged from 14.83 to 39.17 seconds with the 1,100-foot-lambert display and from 17.30 to 43.67 seconds with the 660-footlambert display. Overall viewing time also differed significantly in the monitor study (t = 1.994, df = 299, P = .046), with the viewing time lasting 3.71 seconds longer on average with the 80-foot-lambert monitor than on the 140-foot-lambert monitor. Viewing times ranged from 6 to 99 seconds with the 80-foot-lambert monitor and from 10 to 99 seconds with the 140-foot-lambert monitor. As Table 5 shows, the viewing times were also categorized according to image type. For monitor viewing, an ANOVA revealed a significant effect on viewing time as a function of display (F = 5.61, df= 5, P < .0001). Post hoc analyses (Fisher protected least squares difference test) revealed that for both the 140- and 80-footlambert conditions, the lesion-free images had significantly longer viewing times than both the mass and microcalcification images (P < .05). All three image types had longer viewing times in the lower-luminance conditions, but only with the lesion-free images did this approach statistical significance (P = .06). The same pattern of results was observed during film image viewing (F = 6.32, df= 5, P < .001); lesion-free images had

True-positive False-negative False-positive True-negative

1,100 foot-lambert 1,549 1,273 1,505 511

_+ 908 + 1,116 + 1,032 + 344

660 foot-lambert 1,695 1,607 1,736 603

+ 1,026 + 943 _+ 987 + 416

Note.--Data presented as median _+ interquartile range.

Table 7 Dwell Times in the Monitor Study Dwell Time (msec) Decision True-positive False-negative False-positive True-negative

140 foot-lambert 1,684 1,329 1,527 604

_+ 1,145 _+ 1,283 + 848 + 477

80 foot-lambert 1,737 1,647 1,741 627

+ 1,210 + 804 + 1,025 _+ 475

Note.--Data presented as median + interquartile range.

longer viewing times than those with a mass or microcalcification under both luminance conditions. D e c i s i o n Dwell T i m e s a n d F i x a t i o n C l u s t e r s

The median dwell times for true-positive, true-negative, false-positive, and false-negative decisions in the film and monitor studies are presented in Tables 6 and 7, respectively. The median values for each decision were compared statistically for the two conditions in each study by using a median test (21). The median decision dwell time values did not differ significantly in either the film or monitor study for the true-positive, false-negative, or false-positive decisions. In both studies, however, the median dwell time values for the true-negative decisions were significantly greater for the lower- than for the higher-luminance displays (film: %2 = 4.6, df= 1, P < .05; monitor: X2 = 4.08, df= 1, P < .05). Analysis of the eye-position data also included the number of fixation clusters generated during the search of lesion-containing versus lesion-free cases. Tables 8 and 9 show the results of this analysis for film image and monitor viewing, respectively. In both studies, there were no significant differences in the number of fixation clusters generated during the search of lesion-containing images under higher- versus lower-luminance conditions. For lesion-free images, however, significantly

415

Table 8 Fixation Clusters Generated per Image for Lesion-containing and Lesion-free Film Images Displayed on a View Box

Table 9 Fixation Clusters Generated per Image for Lesion-containing and Lesion-free Film Images Displayed on a CRT Monitor

Condition

No. of Clusters

Condition

No. of Clusters

1,100 foot-lambert Mass Microcalcification Lesion free

7.98 + 1.96 7.63 + 2.05 9.27 _+ 2.45

140 foot-lambert Mass Microcalcification Lesion free

8.52 + 2.12 9.18 + 2.11 10.55 + 2.86

660 foot-lambert Mass Microcalcification Lesion free

80 foot-lambert 8.08 _+ 2.19 7.92 + 2.12 10.84 + 2.71

N o t e . - - D a t a presented as a mean + standard deviation.

more fixation clusters were generated during the search with the lower- than with the higher-luminance displays (monitor: t = 2.83, df= 166, P < .01; film: t = 3.13, df= 166, P < .01).

The main findings of this study can be summarized as follows. Changes in display luminance, for both film and monitor viewing, do not significantly affect lesion detection performance. Some aspects of visual search behavior, however, are significantly affected by changes in display luminance. The main effects of luminance on search behavior are prolonged dwell time associated with truenegative decisions on the lower-luminance displays and a greater number of fixation clusters generated on lesionfree images with the lower-luminance displays. These two effects result in longer total viewing times on the lower-luminance displays, especially for lesion-free images, which have predominantly true-negative fixation clusters. The results suggest that one benefit of higher-luminance displays is more efficient search and recognition of normal image features. Regarding detection accuracy, the AFROC analysis showed no statistically significant difference in performance with higher- versus higher-luminance displays in either study. In fact, even though the lesions selected for the study were judged to be subtle by an experienced mammographer, AFROC A1 values were high for all observers in all conditions. In part, this is because the readers were asked to perform only a detection task. If we had included benign and malignant lesions and asked the readers to classify the lesions on detection, the classification performance may not have been as high as the detection performance.

416

Mass Microcalcification Lesion free

8.89 + 2.40 9.30 + 2.01 12.05 + 3.92

N o t e . - - D a t a presented as a mean + standard deviation.

The breakdown of each type of decision (Table 4) is suggestive that performance was so high in part because the false-positive rate was only 4%-5% for all conditions. On the other hand, the false-negative rates ranged from 12% to 17%, indicating the radiologists missed a fair number of lesions. It is interesting that even though the differences were not statistically significant, the true-positive rates were higher and the false-positive rates were lower with the higher-luminance displays compared to the lowerluminance displays in both film image and monitor reading. In fact, the true-positive rate for the higher-luminance film condition was 5% higher than that for the higher-luminance monitor condition. At the very least, this demonstrates that compared with the standard, higher-luminance mammographic view box, diagnostic performance with a lower-luminance CRT monitor is likely to be poorer. Performance with the higher-luminance monitor was nearly equal to that with the lower-luminance view box, but it fell short of the performance with the higher-luminance mammographic view box. The viewing time and eye-position data revealed some very interesting differences in visual search behavior for the different conditions. Overall, the observers spent significantly more time (approximately 20 seconds longer) viewing images on the CRT monitors than on film. This result also has been found in previous studies (17,22) and therefore is not surprising. Total viewing time, however, also was a function of display luminance. In both the film and monitor conditions, viewing time was significantly longer with the lower-luminance display. This is suggestive that one factor contributing to the general finding that viewing time is longer when using a monitor than when using film is that the luminance levels for film displays are significantly higher than those for monitors. There may be other factors involved, as well, such as contrast differences

and experience using monitors to view images, but these results are suggestive that luminance is at least an important contributor to the phenomenon. The prolonged total view times also were a function of the type of film image. Lesion-free images were more affected by changes in luminance than either the mass or the microcalcification images, and the microcalcification images were more affected than the mass images. The eyeposition data provide one possible reason why this should occur. Two aspects of visual search were observed to be affected by luminance: true-negative decision dwell times, and the number of clusters generated when viewing lesionfree images. Overall, the median dwell times, associated with each possible decision (true-positive, false-negative, false-positive, and true-negative) were longer with the lower-luminance displays for both film and monitor viewing (Tables 6, 7). Only the differences for true-negative dwell times, however, were statistically significant. Because dwell time is assumed to be a reflection of information-processing operations (18,23,24), we can conclude that observers spent more time processing normal image feature information on the lower-luminance than on the higher-luminance displays. The increased number of predominantly true-negative fixation clusters generated with lesion-free images (Tables 8, 9) also is indicative that observers spent more time processing truly normal image features with the lower-luminance displays. It is interesting that even though there were significant differences in true-negative dwell times and number of clusters on lesion-free images as a function of luminance, the percentage of false-positive and true-negative decisions did not differ significantly with luminance. This is indicative that observers allocated more image-processing resources to reach the same decision with the lower-luminance than with the higher-luminance displays. Visual search and processing were more efficient in terms of time and resource allocation for the higher-luminance displays. The results of this study demonstrate that display luminance affects the visual search behavior of radiologists. As mentioned, other factors, such as display contrast and experience with monitor displays (25), probably could also contribute to the general finding that it takes longer to read images from a monitor than from film. We demonstrated with this 'study, however, that luminance changes affect behavior not only when film is compared with a monitor but also when luminance is changed for a given display medium (film or monitor). The observed differences in detection accuracy as a function of luminance were relatively small and not statistically significant.

The question of whether the differences in dwell time and number of clusters as a function of luminance are clinically important needs more research. If these prolonged viewing times persist as more and more images are read from monitors, however, fewer cases will be read during a given period of time compared with film viewing--especially if a high percentage are lesion free, which is the case for mammographic screening. This could reduce workflow and substantially affect the overall productivity of a radiology department. To help maximize diagnostic accuracy rates and workflow efficiency, we need to understand further the influence of display parameters such as luminance on diagnostic and search behaviors. If we understand how and why certain displays (view box or CRT monitor) are better than others, we can make informed and confident decisions regarding the types of displays used in the clinical reading environment. REFERENCE5 1. Schilling RB, Cox JD, Sharma SRD. Advanced digital mammegraphy. J Digit Imaging 1998; 11:163-165. 2. Moore RH, Kopans DB, Niklason LT, et at. Initial clinical experience with full-field digital mammography (abstr). Radiology 1997; 205(P):274. 3. Fajardo LL, Williams MB. The clinical potential of digital mammography. In: Doi K, Giger ML, Nishikawa RM, Schmidt RA, eds. Digital mammography '96. New York, NY: Elsevier, 1996; 43-52. 4. Wang J, Langer S. A brief review of human perception factors digital displays for picture archiving and communications systems. J Digit Imaging 1997; 10:158-168. 5. Williams MB, Mangiafico PA, Simoni PU, Stanton MJ, Phillips W, Rosen D. Workstation display of images from a prototype detector for digital mammography (abstr). Radiology 1997; 205(P):742. 6. Zeffiro T, Sterling VA, VanMeter J. Digital mammography review station (abstr). Radiology 1997; 205(P):742. 7. Hemminger BM, Dillon A, Pisano ED, Johnston RE. Demonstration of a softcopy display system for digital mammography (abstr). Radiology 1997; 205(P):743. 8. Jahangiri Mi, Lou SL, Hoogstrate DR, Huang HK. Full-field direct digital mammogram (FFDM) display workstation (DWS) (abstr). Radiology 1997; 205(P):745. 9. Working Group on Digital Mammography. Digital displays and workstation design. Presented at the meeting on digital displays and workstation design, Washington, DC, March 9-10, 1998. 10. American College of Radiology. Recommended specifications for new mammography equipment: screen-film systems, image receptors, and film processors. Reston, Va: American College of Radiology, 1995; 4345. 11. Kimme-Smith C, Haus AG, DeBruhl N, Bassett LW. Effects of ambient light and view box luminance on the detection of calcifications in mammography. AJ R 1997; 168:775-778. 12. Waynant RW, Chakrabarti K, Kaczmerak R, Suieiman O, Rowberg A. Improved sensitivity and specificity of mammograms by producing uniform luminance from viewboxes. J Digit Imaging 1998; 11:189-191. 13. Wang J, Gray JE. Detection of small low-contrast objects in mammography: effect of viewbox masking and luminance. AJR 1998; 170:105108. 14. Robson KJ, Kotre CJ, Faulkner K. An experimental investigation of the effect of light-box luminance on the detection of low contrast objects in mammography. Br J Radiol 1996; 69:153-159.

417

15. Gray JE, Lisk KG, Haddick DH, Harshbarger JH, Oosterhof A, Schwenker R. Test pattern for video displays and hard-copy camera. Radiology 1985; 154:519-527. 16. Lisk KG. SMPTE test pattern for certification of medical diagnostic display devices. Proc SPIE 1984; 486:79-82. 17. Krupinski EA, Lund PJ. Differences in time to interpretation for evaluation of bone radiographs with monitor and film viewing. Acad Radiol 1997; 4:177-182. 18. Nodine CF, Kundel HL, Toto LC, Krupinski EA. Recording and analyzing eye-position data using a microcomputer workstation. Behav Res Methods Instrum Comput 1992; 24:475-485. 19. Kundel HL, Nodine CF, Thickman DT, Toto LC. Searching for lung nodules: a comparison of human performance with random and systematic scanning models. Invest Radiol 1987; 22:417-422. 20. Chakraborty DP, Winter LHL. Free-response methodology: alternate analysis and a new observer-performance experiment. Radiology 1990; 174:873-881.

418

21. Ferguson GA. Statistical analysis in psychology and education. New York, NY: McGraw-Hill, 1981;400-419. 22. Krupinski EA, Maloney K, Bessen SC, et al. Receiver operating characteristic evaluation of computer display of adult portable chest radiographs. Invest Radiol 1994; 29:141-146. 23. Just MA, Carpenter PA. Eye fixations and cognitive processes, Cognitive Psycho11976; 8:441-480. 24. Just MA, Carpenter PA. Reading and spatial cognition: reflections from eye fixations. In: Luer G, Lass U, Shallo-Hoffman J, eds. Eye movement research: physiological and psychological aspects. Lewiston, NY: Hogrefe, 1988; 193-213. 25. Krupinski EA, Weinstein RS, Rozek LS. Experience-related differences in diagnosis from medical images displayed on monitors. Telemedicine J 1996; 2:101-108.