- Email: [email protected]
Image quality preferences among radiographers and radiologists. A conjoint analysis
Radiography (2005) 11, 191e197
Image quality preferences among radiographers and radiologists. A conjoint analysis Borgny Ween a,b,*, Doris Tove Kristoffersen c, Glenys A. Hamilton b, Dag Rune Olsen d,e a
Department of Radiology, Rikshospitalet University Hospital, Oslo, Norway Center for Shared Decision Making and Nursing Research, Rikshospitalet University Hospital, Oslo, Norway c Informed Choice Research Department, Norwegian Health Services Research Centre, Oslo, Norway d Institute of Cancer Research, The Norwegian Radium Hospital, Oslo, Norway e Department of Medical Physics, University of Oslo, Oslo, Norway b
Received 9 November 2004; accepted 8 March 2005 Available online 11 May 2005
KEYWORDS Experimental study; Digital imaging; Digital radiography; Measuring image quality; Post-processing; Radiography (diagnostic)
Abstract Purpose The aim of this study was to investigate the image quality preferences among radiographers and radiologists. The radiographers’ preferences are mainly related to technical parameters, whereas radiologists assess image quality based on diagnostic value. Methods A conjoint analysis was undertaken to survey image quality preferences; the study included 37 respondents: 19 radiographers and 18 radiologists. Digital urograms were post-processed into 8 images with different properties of image quality for 3 different patients. The respondents were asked to rank the images according to their personally perceived subjective image quality. Results Nearly half of the radiographers and radiologists were consistent in their ranking of the image characterised as ‘very best image quality’. The analysis showed, moreover, that chosen filtration level and image intensity were responsible for 72% and 28% of the preferences, respectively. The corresponding figures for each of the two professions were 76% and 24% for the radiographers, and 68% and 32% for the radiologists. In addition, there were larger variations in image preferences among the radiologists, as compared to the radiographers. Conclusions Radiographers revealed a more consistent preference than the radiologists with respect to image quality. There is a potential for image quality improvement by developing sets of image property criteria. ª 2005 The College of Radiographers. Published by Elsevier Ltd. All rights reserved.
* Corresponding author. Tel.: C47 23072600; fax: C47 23072610. E-mail address: [email protected] (B. Ween). 1078-8174/$ - see front matter ª 2005 The College of Radiographers. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.radi.2005.03.002
192
Introduction Digital healthcare has demonstrated a major impact on the structure and organisation of health services. In Norway, nation wide broad band communication is being implemented, integration in all hospitals, and most clinics have advanced toward fully digital radiology services.1 Also internationally remote evaluation and reporting of diagnostic images increases dramatically.2 A distinct feature of digital diagnostic images is the possibility of image processing using a vast number of different algorithms. Image processing as filtering and scaling operations that adjusts the overall density and contrast of the image, affects image quality and thus diagnostic value. Although postprocessing now is an integrated part of remote evaluation and reporting undertaken by the radiologist, inherent image properties laid down during image acquisition by the radiographers are equally important with respect to image quality.3 The common goal of the two professions is the provision of diagnostic images of high quality ensuring a precise diagnosis. Radiographers and radiologists therefore ought to share the same standards for subjective perceived image quality. Image quality relies on physical and technical factors, as well as human perception.4 Good practice criteria5,6 have been developed and represent standards for particular anatomical region, for standard-sized groups of patients, addressing any clinical situation. Radiographers choose exposure technique, according to good practice criteria, ensuring sufficient image quality, taking into account individual patient factors. Post-processing is thus used to enhance clinical and diagnostic information and value. Preferences of image quality among radiographers and radiologists were investigated by conjoint analysis in this study. Conjoint analysis (CA) is a technique to elicit what combination of a limited number of factors and their levels (attributes) is most preferred. In this study filter and intensity were the factors investigated. Image quality was assessed by analysing rankings between individuals’ subjective response to images. Ranking images imply choosing combinations of properties and weighting attributes of a product in an individual clinical context. Conjoint analysis is traditionally performed as a part of making a business evaluation of a new product idea. The method was introduced in the mid-1970s, when modern marketing research was developed. CA is a study design and a regression-type statistical analysis. CA may be useful for different methodological questions in health care.7
B. Ween et al. The characteristics of CA are described in detail elsewhere.8e10
Methods Respondents All 41 radiographers and radiologists actively involved in routine radiological diagnostics at The Norwegian Radium Hospital, Department of Radiology were invited to participate. Of these, 37 (90%) participated, 19 (86%) of the radiographers and 18 (95%) of the radiologists. They were 16 senior and 21 junior staff members; 26 worked fulltime and 11 part-time, 13 were men and 24 women, and age ranged between 24 and 63 years, with a mean age of 42.5 years.
Images Intravenous urography (IVUs) was conducted on a series of patients using Prestige VH Digital Radiography Spot 3.0, with software version 3.37 (General Electric, Milwaukee, USA). Of 20 IVUs, three met the following inclusion criteria: adults of normal weight where the initial AP projection image was acquired without use of contrast media. The three included patients were all females, 50e 60 years of age, and admitted to the hospital with a confirmed cancer diagnosis. Their AP projection images were made with very similar exposure factors. By visual inspection, the images were found to be similar with respect to noise and contrast. All images were stored without file compression on a Sun workstation (Preston Technology, Preston, WA, USA). Copies of each of the 3 patient images were further post-processed with filters and windowing using Advantage Windows 2.0 (General Electric, Milwaukee, USA). The availability in the present study was 8 filters (0e7). Filters are characterised by the array of convolution coefficient: the larger the array, the more dramatic is the effect of changing frequency. Their array is squared (52) and found in all filter numbers. Filter number 0 (gains 1) makes neither smoothing nor edges enhancement. Filter numbers 1e7 pass increasingly higher wave frequencies, due to gradually more negative values of the array. Image intensity is equivalent to the amplitude of light. The intensity scale varies with the actual bit system, in this study 8-bit (-255). Window range value and centre window value cause grey level distribution over the entire dynamic range of the display monitor. In this study, 8 different
Image quality preferences
193
combinations of software filtration numbers and window intensity levels were used (Table 1). Patient identifications were erased before printing to hard copies with 8-bits contrast resolution (Laser Imager, Model 1417 Helios, Sterling Medical Products, Holland). Hard copies were made because, at the time of the study, the radiographers and the radiologists were mainly familiar with evaluating urograms displayed on light boxes. The images were circular with a diameter of 21 cm, using magnification factor 1.0. Images were given a randomised code for identification.
Image scoring procedure The 8 images (Fig. 1) of each of the three patients were presented with an identification tag (A, B and C) in a random order on separate light boxes. Image quality was scored according to a preference rank of 1e8, for each patient, and where a score of 1 implied ‘the very best image quality’, and a score of 8 implied ‘the worst image quality’. Respondents were not allowed to give several images equal scores. Ranking was carried out without time limitations. Image readings were made under identical conditions.
Statistical design and analysis Two fractions of a 42 factorial design were used to specify the 8 attribution levels to be used. A full factorial design with four filters and four window levels would have provided 16 images. However, 16 images would have been too many to rank, giving low reproducibility. Four images were considered as too few. Eight images to rank for each patient were sufficient to obtain a satisfactory discriminatory effect. A metric conjoint analysis was performed by SAS version 6.12. The dependent variable was ‘the preference rank of image quality’ and the independent variables were both ordinal: filter numbers and intensity levels. Background variables were analysed by SPSS version 10.0. A p-value less than 0.05 was considered statistically significant.
Results Time consumption during image reading was registered for 35 participants. Radiographers used 3e 16 min (mean 8), while radiologists used 3e25 min (mean 9). Altogether, 888 scores were available for further analysis: 111 preference scores were available for each image for further analysis, and each preference rank number was given a score from respondents 111 times. Image number 2 was perceived as having the very best image quality by both the radiographers and the radiologists and received a total of 52 preference scores (47%), as shown in Table 2. Image number 2 scored best for each of the three patients (not shown in tables). The post-processing properties of image number 2 (Table 1) were the lowest filter setting combined with a low intensity setting. The rest of the ‘best’ scores (53%) were divided between five other images. Two experienced ones, one radiographer and one radiologist, ranked image number 2 as having the worst image quality. The rest of their scoring did not differ considerably from the other participants. The worst image quality, given rank 8 by 48 (43%) of the 111 scores, was image number 5. Image 5 was characterised by the high number of filtering. The total preference structure of images calculated, for both professions using conjoint analysis, shows that filtration was responsible for 72% of the preferences, whereas intensity was only responsible for 28% of the preferences. The corresponding figures for each of the two categories of professionals were 76% and 24% for the radiographers, and 68% and 32% for the radiologists, respectively (not shown in tables). The preference ‘structure’ was further analysed for patient A, B and C separately. Table 3 shows that for radiographers the relative importance of both attributes was similar for all patients (ranges 73e80% and 20e 27%) while the corresponding figures for the radiologists showed larger variation (48e84% and 16e52%).
Table 1 Fractional, factorial design of combinations in filter numbers and window level (wl)/window width (ww) used for conjoint analysis of subjective image quality preferences Image numbers
1
2
3
4
5
6
7
8
Filter numbers Intensity levels (wl/ww)
0 75/150
0 90/182
3 110/220
3 133/255
5 75/150
5 90/182
7 110/220
7 133/255
194
B. Ween et al.
Figure 1
The 8 images from one of the patients, patient A.
Discussion Image quality is a vague term that is difficult to assess quantitatively.4 This study addresses subjective observer performance of clinical images. The term diagnostic performance, on the other hand, focuses on patient diagnosis. Despite its lack of generalisability, the preference method is useful in certain cases when different professions’ use of imaging technology is investigated.11
Influence of post-processing on image properties Post-processing apply filters, i.e. mathematical algorithms that improve the perceptibility of image features.12 Filters applied during acquisition cannot be changed by post-processing by the radiologists, whilst windowing can be altered.
Filtering was found to be 2e3 times more important, with respect to image preferences, than windowing. The higher filtration numbers applied, the more edge enhancement, and thus contrast is increased. Increased contrast allows the observer to perceive more details. This is, however, purely perception, as the information inherent in the image does not alter. Moreover, as contrast is enhanced, noise will be more predominant. Noise, seen as ‘salt and pepper pattern’, makes a dramatic impact on image quality. Quality decreases very quickly as noise increases, especially when signals of interest have low contrast, as in kidney tumours. Intensity is the least important property. In this study low window setting were generally preferred. The central value of the window (wl) is selected according to the attenuation characteristics of the structure under examination. Image preference is a trade-off between an acceptable amount of noise weighed against a sufficient level of intensity. Lower levels of noise may not be seen, but still
Table 2 Summed preference scores for all three patients (888 scores) Preference rankings Image numbers 1 1 2 3 4 5 6 7 8
2
3
4
5
6
6 52 26 4 0 22 8 24 37 14 7 11 15 13 19 19 5 19 9 9 13 10 8 25 9 5 7 12 13 11 10 4 8 16 10 18 40 2 1 15 20 2 14 2 0 21 48 3
7
8
1 6 13 21 29 24 8 9
0 4 8 16 25 21 23 14
Table 3 Relative importance of attributes, explaining image preferences among the radiographers compared to the radiologists for the three patients, measured by conjoint analysis
Patient A Patient B Patient C
Radiographers
Radiologists
Filtering (%)
Intensity (%)
Filtering (%)
Intensity (%)
80 73 73
20 27 27
84 48 63
16 52 37
Image quality preferences reduce spatial resolution dramatically, especially when structures are in the light grey colour. Low levels of noise were indeed preferred in this study, avoiding ‘drowning’ of low-contrast targets. However, it should be pointed out that the individuals’ noise acceptance may vary substantially. While abdominal radiography generally requires low sharpness and medium contrast due to larger structures, urography requires a much higher sharpness for the same contrast because point targets and line targets need to be identified.13 This is characteristic of ureteric stones of only 1mm size5 as well as linear calcifications that can be a sign of neoplasm, particularly in the kidney.14 Noise and contrast combined with the human visual system’s poor response to high spatial frequencies are limiting factors in determining how small an object in a given imaging system that can be demonstrated. Kundel14 state that contrast and resolution cannot be seen, whereas observers perceive objects. Although contrast and resolution can be dealt with as experimental abstractions, their relationship to the real world images is difficult to determine.
Are radiographers and radiologists similar in reading images? The findings in this study cannot easily be compared to other results comparing radiographers and radiologists image preferences, due to lack of comparable studies. Ma ˚nsson11 asked a group of radiologists the single question ‘Which image do you prefer’? He found an acceptable agreement within the group of radiologists. Ciantar15 considered both subjective assessments and objective measures in a study of chest X-rays of pigs. He concluded that the radiologists agreed amongst themselves quite well, and that ‘subjective’ image quality criteria have some validity, at least for approximate assessments and that objective image quality criteria were achievable. The number of radiographers reporting in the United Kingdom has increased, as has the scope of examinations being reported.16 Inter-reader agreement has been found to be comparable between radiographers and radiologists in image readings on fracture radiographs,17e18 while radiographers with specialised training can report barium enema examinations to a high standard.19 The present study addresses subjective preferences, without any ‘gold standard’ as in ROC studies.20 The fact that observers perceive differently when reading images is well known.21 Kundel14 stated that as radiographers gain clinical experience, they begin to apply adequate-for-diagnosis
195 criteria, resulting in a more individualised way of working. Dreyfus & Dreyfus22 maintain that professionals might not automatically develop their skills to a professional level even after years of practise. Regular discussions among radiographers and radiologists may prevent heterogeneous image quality standards and may pave the way for a common consensus between the two professional groups with respect to image quality.
Quality of the study High participation rates imply that the results should be regarded as representative. Randomised presentation of images for each patient and blinding of the post-processed data prevents observer bias. The technical properties of the images were chosen based on a pilot study including 4 radiographers. The 4 agreed on ‘the very best’ and ‘the worst image quality’. The final study was then made with the same range of image properties. The 4 radiographers in the pilot study were later included in the main study, but images from different patients were included to prevent bias. Patient images were selected from those present in the archives of the hospital. Of course, patients who undergo urography might not be representative of patients with other specific diagnoses or symptoms.23 The image preferences obtained in this study are thus limited to the specific radiological procedure performed. Also the number of patients included was limited; however, the qualities of the images were considered to be representative of images seen in everyday practise. Explained statistical variance (R2 as well as the adjusted R2) would have been identical to 1.0 if all respondents had identical preferences, whereas no correlation at all yield R2 Z 0.0. The sums of the preferences for the 888 film evaluations showed the highest value images having the very best (52 scores, 47%) and the worst image quality (48 scores, 43%). This implies a high degree of agreement with respect to what was the best and the worst image qualities. Also, the variance values were sufficient to enable quality discrimination. The background variables of the respondents were in general found to be representative for Norwegian radiographers and radiologists. The radiographers’ mean age was 38.7 years. Comparable data on Norwegian radiographers’ ages were, however, not available. Mean age for radiologists was 46.5 years, e.g. the same as for Norwegian radiologists.24 With respect to other background variables like occupational grade and gender, the study population was found to be representative of Norwegian radiographers and radiologists.
196
B. Ween et al.
Table 4
Stability of conjoint analysis model for the two professional groups (see Table 3) Radiographers
Patient A Patient B Patient C a b
Radiologists
R2
Adjusted R2a
p-valueb
R2
Adjusted R2
p-value
0.32 0.24 0.17
0.30 0.20 0.20
0.001 0.0001 0.0001
0.050 0.062 0.16
0.080 0.095 0.13
0.03 0.02 0.0002
Adjusted for degrees of freedom. p-values for adjusted R2, statistical significance level: p Z 0.05.
However, we did not collected data on the participants’ years of experience, but this is likely to closely co-vary with age.
Confounders Other factors than profession may affect preferences among radiographers and radiologists. Experience rather than profession might significantly influence image quality preferences. Seniors may appreciate other image characteristics than junior staff members and might have developed a consensus with respect to image quality preferences that junior staff member’s lack. When analysing the background variables, there were statistically significant differences in occupational seniority (p Z 0.005) between the two professions, and also an age difference (p Z 0.01). The difference in gender (p Z 0.065) was close to reaching statistical significance. In this conjoint analysis, we have adjusted for age and gender, but not seniority (Table 4). In the adjusted model, practically the same preferences were found. Seniority was closely related to age, and therefore not included in the adjusted model. Time consumption for reading was slightly different between the two groups of professions, but the difference was so small that we considered it unlikely to be a confounder.
Conclusions This study revealed that radiographers showed a more consistent preference as compared to the radiologists with respect to image quality. The findings indicate a potential for image quality improvement by development and implementation of image property criteria. Development of quality criteria for examination in digital radiography is needed. The professionals have to rely on their own experience and local procedures, and validated local procedures are seldom employed. In the absence of objective image quality criteria, local consensus based on experience has to be adopted.
Acknowledgements We acknowledge the participation of respondents at The Norwegian Radium Hospital, Department of ˚ se Radiology, and help from radiographer Bjørg A Rue. The Norwegian Radium Hospitals’ Research Foundation and Rikshospitalet University Hospital funded this project.
References 1. Bergstrom R, Størmer J. Norway advances toward fully digital healthcare. Norwegian hospitals linking up with PACS and RIS can use their experiences to teach other nations. Diagnostic Imaging Europe 2003;19:16e23. 2. Gould, P. Mobile reading devices give clinicians free range. 20th EuroPACS Conference, 5e7 September 2002. Diagnostic Imaging Europe. Conference Proceedings. 3. Huang HK. PACS basic principles and applications. New York: Wiley-Liss; 1999. 4. International Commission on Radiation Units and Measurements, Inc. In: Medical imaging e the assessment of image quality. Vol. 54. Bethesda, Maryland, USA: International Commission on Radiation Units and Measurements; 1996. 5. European Guidelines for Quality Criteria for Diagnostic Radiographic Images. European Commission and Directorate-General XII: science, research and development. EUR 16260. Luxembourg: Office for Official Publications of the European Communities; 1996. 6. European Guidelines for Quality Criteria for Diagnostic Radiographic Images in Paediatrics. European Commission and Directorate-General XII: science, research and development. EUR 16261. Luxembourg: Office for Official Publications of the European Communities; 1996. 7. Ryan M, McIntosh E, Shackley P. Methodological issues in the application of conjoint analysis in health care. Health Economics 1998;7:373e8. 8. Green PE, Srinivasan V. Conjoint analysis in consumer’s research: issues and outlook. Journal of Consumers Research 1978;5:103e23. 9. Green PE, Srinivasan V. Conjoint analysis in marketing: new developments with implications for research and practice. Journal of Marketing 1990:October 3e19. 10. Green PE, Krieger AM, Wind Y. Thirty years of conjoint analysis: reflections and properties. Interfaces 2001;31:56e73. 11. Ma ˚nsson LG. Methods for the evaluation of image quality: a review. Radiation Protection Dosimetry 2000;90:89e99. 12. Crane R. A simplified approach to image processing. Classical and modern techniques in C. New Jersey: Prentice Hall PTR; 1996.
Image quality preferences 13. Vyborny CJ. Image quality and the clinical radiographic examination. Radiographics 1997;17:479e98. 14. Kundel HL. Images, image quality and observer performance: new horizons in radiology lecture. Radiology 1979; 132:265e71. 15. Ciantar D, Fitzgerald M, Cotterill AD, Pettett A, Cook V, Beluffi G, et al. Correlation between quantitative and subjective assessment of image quality in paediatric radiology. Radiation Protection Dosimetry 2000;90:185e8. 16. Paterson AM, Price RC, Thomas A, Nuttall L. Reporting by radiographers: a policy and practice guide. Radiography 2004;10:205e12. 17. Loughran CF. Reporting of fracture radiographs by radiographers: the impact of a training programme. British Journal of Radiology 1994;67:945e50. 18. Piper KJ, Paterson AM, Godfrey RC. Accuracy of radiographers’ reports in the interpretation of radiographic
197
19.
20. 21. 22.
23. 24.
examinations of the skeletal system: a review of 6796 cases. Radiography 2004;11:27e34. Murphy M, Loughran CF, Birchenough H, Savage J, Sutcliffe C. A comparison of radiographer and radiologist reports on radiographer conducted barium enemas. Radiography 2002;8:215e21. Obuchowski NA. Receiver operating characteristic curves and their use in radiology. Radiology 2003;229:3e8. Kundel HL, Polansky M. Measurement of observer agreement. Radiology 2003;228:303e8. Dreyfus HL, Dreyfus SE, Athanasiou T. Mind over machine: the power of human intuition and expertise in the era of the computer. New York: Free Press; 1988. Kazerooni EA. Population and sample. American Journal of Roentgenology 2001;177:993e9. Den norske lægeforening. Legestatistikk 2002. Mean age for radiologists, !http://www.legeforeningen.noO.
Image quality preferences among radiographers and radiologists. A conjoint analysis Borgny Ween a,b,*, Doris Tove Kristoffersen c, Glenys A. Hamilton b, Dag Rune Olsen d,e a
Department of Radiology, Rikshospitalet University Hospital, Oslo, Norway Center for Shared Decision Making and Nursing Research, Rikshospitalet University Hospital, Oslo, Norway c Informed Choice Research Department, Norwegian Health Services Research Centre, Oslo, Norway d Institute of Cancer Research, The Norwegian Radium Hospital, Oslo, Norway e Department of Medical Physics, University of Oslo, Oslo, Norway b
Received 9 November 2004; accepted 8 March 2005 Available online 11 May 2005
KEYWORDS Experimental study; Digital imaging; Digital radiography; Measuring image quality; Post-processing; Radiography (diagnostic)
Abstract Purpose The aim of this study was to investigate the image quality preferences among radiographers and radiologists. The radiographers’ preferences are mainly related to technical parameters, whereas radiologists assess image quality based on diagnostic value. Methods A conjoint analysis was undertaken to survey image quality preferences; the study included 37 respondents: 19 radiographers and 18 radiologists. Digital urograms were post-processed into 8 images with different properties of image quality for 3 different patients. The respondents were asked to rank the images according to their personally perceived subjective image quality. Results Nearly half of the radiographers and radiologists were consistent in their ranking of the image characterised as ‘very best image quality’. The analysis showed, moreover, that chosen filtration level and image intensity were responsible for 72% and 28% of the preferences, respectively. The corresponding figures for each of the two professions were 76% and 24% for the radiographers, and 68% and 32% for the radiologists. In addition, there were larger variations in image preferences among the radiologists, as compared to the radiographers. Conclusions Radiographers revealed a more consistent preference than the radiologists with respect to image quality. There is a potential for image quality improvement by developing sets of image property criteria. ª 2005 The College of Radiographers. Published by Elsevier Ltd. All rights reserved.
* Corresponding author. Tel.: C47 23072600; fax: C47 23072610. E-mail address: [email protected] (B. Ween). 1078-8174/$ - see front matter ª 2005 The College of Radiographers. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.radi.2005.03.002
192
Introduction Digital healthcare has demonstrated a major impact on the structure and organisation of health services. In Norway, nation wide broad band communication is being implemented, integration in all hospitals, and most clinics have advanced toward fully digital radiology services.1 Also internationally remote evaluation and reporting of diagnostic images increases dramatically.2 A distinct feature of digital diagnostic images is the possibility of image processing using a vast number of different algorithms. Image processing as filtering and scaling operations that adjusts the overall density and contrast of the image, affects image quality and thus diagnostic value. Although postprocessing now is an integrated part of remote evaluation and reporting undertaken by the radiologist, inherent image properties laid down during image acquisition by the radiographers are equally important with respect to image quality.3 The common goal of the two professions is the provision of diagnostic images of high quality ensuring a precise diagnosis. Radiographers and radiologists therefore ought to share the same standards for subjective perceived image quality. Image quality relies on physical and technical factors, as well as human perception.4 Good practice criteria5,6 have been developed and represent standards for particular anatomical region, for standard-sized groups of patients, addressing any clinical situation. Radiographers choose exposure technique, according to good practice criteria, ensuring sufficient image quality, taking into account individual patient factors. Post-processing is thus used to enhance clinical and diagnostic information and value. Preferences of image quality among radiographers and radiologists were investigated by conjoint analysis in this study. Conjoint analysis (CA) is a technique to elicit what combination of a limited number of factors and their levels (attributes) is most preferred. In this study filter and intensity were the factors investigated. Image quality was assessed by analysing rankings between individuals’ subjective response to images. Ranking images imply choosing combinations of properties and weighting attributes of a product in an individual clinical context. Conjoint analysis is traditionally performed as a part of making a business evaluation of a new product idea. The method was introduced in the mid-1970s, when modern marketing research was developed. CA is a study design and a regression-type statistical analysis. CA may be useful for different methodological questions in health care.7
B. Ween et al. The characteristics of CA are described in detail elsewhere.8e10
Methods Respondents All 41 radiographers and radiologists actively involved in routine radiological diagnostics at The Norwegian Radium Hospital, Department of Radiology were invited to participate. Of these, 37 (90%) participated, 19 (86%) of the radiographers and 18 (95%) of the radiologists. They were 16 senior and 21 junior staff members; 26 worked fulltime and 11 part-time, 13 were men and 24 women, and age ranged between 24 and 63 years, with a mean age of 42.5 years.
Images Intravenous urography (IVUs) was conducted on a series of patients using Prestige VH Digital Radiography Spot 3.0, with software version 3.37 (General Electric, Milwaukee, USA). Of 20 IVUs, three met the following inclusion criteria: adults of normal weight where the initial AP projection image was acquired without use of contrast media. The three included patients were all females, 50e 60 years of age, and admitted to the hospital with a confirmed cancer diagnosis. Their AP projection images were made with very similar exposure factors. By visual inspection, the images were found to be similar with respect to noise and contrast. All images were stored without file compression on a Sun workstation (Preston Technology, Preston, WA, USA). Copies of each of the 3 patient images were further post-processed with filters and windowing using Advantage Windows 2.0 (General Electric, Milwaukee, USA). The availability in the present study was 8 filters (0e7). Filters are characterised by the array of convolution coefficient: the larger the array, the more dramatic is the effect of changing frequency. Their array is squared (52) and found in all filter numbers. Filter number 0 (gains 1) makes neither smoothing nor edges enhancement. Filter numbers 1e7 pass increasingly higher wave frequencies, due to gradually more negative values of the array. Image intensity is equivalent to the amplitude of light. The intensity scale varies with the actual bit system, in this study 8-bit (-255). Window range value and centre window value cause grey level distribution over the entire dynamic range of the display monitor. In this study, 8 different
Image quality preferences
193
combinations of software filtration numbers and window intensity levels were used (Table 1). Patient identifications were erased before printing to hard copies with 8-bits contrast resolution (Laser Imager, Model 1417 Helios, Sterling Medical Products, Holland). Hard copies were made because, at the time of the study, the radiographers and the radiologists were mainly familiar with evaluating urograms displayed on light boxes. The images were circular with a diameter of 21 cm, using magnification factor 1.0. Images were given a randomised code for identification.
Image scoring procedure The 8 images (Fig. 1) of each of the three patients were presented with an identification tag (A, B and C) in a random order on separate light boxes. Image quality was scored according to a preference rank of 1e8, for each patient, and where a score of 1 implied ‘the very best image quality’, and a score of 8 implied ‘the worst image quality’. Respondents were not allowed to give several images equal scores. Ranking was carried out without time limitations. Image readings were made under identical conditions.
Statistical design and analysis Two fractions of a 42 factorial design were used to specify the 8 attribution levels to be used. A full factorial design with four filters and four window levels would have provided 16 images. However, 16 images would have been too many to rank, giving low reproducibility. Four images were considered as too few. Eight images to rank for each patient were sufficient to obtain a satisfactory discriminatory effect. A metric conjoint analysis was performed by SAS version 6.12. The dependent variable was ‘the preference rank of image quality’ and the independent variables were both ordinal: filter numbers and intensity levels. Background variables were analysed by SPSS version 10.0. A p-value less than 0.05 was considered statistically significant.
Results Time consumption during image reading was registered for 35 participants. Radiographers used 3e 16 min (mean 8), while radiologists used 3e25 min (mean 9). Altogether, 888 scores were available for further analysis: 111 preference scores were available for each image for further analysis, and each preference rank number was given a score from respondents 111 times. Image number 2 was perceived as having the very best image quality by both the radiographers and the radiologists and received a total of 52 preference scores (47%), as shown in Table 2. Image number 2 scored best for each of the three patients (not shown in tables). The post-processing properties of image number 2 (Table 1) were the lowest filter setting combined with a low intensity setting. The rest of the ‘best’ scores (53%) were divided between five other images. Two experienced ones, one radiographer and one radiologist, ranked image number 2 as having the worst image quality. The rest of their scoring did not differ considerably from the other participants. The worst image quality, given rank 8 by 48 (43%) of the 111 scores, was image number 5. Image 5 was characterised by the high number of filtering. The total preference structure of images calculated, for both professions using conjoint analysis, shows that filtration was responsible for 72% of the preferences, whereas intensity was only responsible for 28% of the preferences. The corresponding figures for each of the two categories of professionals were 76% and 24% for the radiographers, and 68% and 32% for the radiologists, respectively (not shown in tables). The preference ‘structure’ was further analysed for patient A, B and C separately. Table 3 shows that for radiographers the relative importance of both attributes was similar for all patients (ranges 73e80% and 20e 27%) while the corresponding figures for the radiologists showed larger variation (48e84% and 16e52%).
Table 1 Fractional, factorial design of combinations in filter numbers and window level (wl)/window width (ww) used for conjoint analysis of subjective image quality preferences Image numbers
1
2
3
4
5
6
7
8
Filter numbers Intensity levels (wl/ww)
0 75/150
0 90/182
3 110/220
3 133/255
5 75/150
5 90/182
7 110/220
7 133/255
194
B. Ween et al.
Figure 1
The 8 images from one of the patients, patient A.
Discussion Image quality is a vague term that is difficult to assess quantitatively.4 This study addresses subjective observer performance of clinical images. The term diagnostic performance, on the other hand, focuses on patient diagnosis. Despite its lack of generalisability, the preference method is useful in certain cases when different professions’ use of imaging technology is investigated.11
Influence of post-processing on image properties Post-processing apply filters, i.e. mathematical algorithms that improve the perceptibility of image features.12 Filters applied during acquisition cannot be changed by post-processing by the radiologists, whilst windowing can be altered.
Filtering was found to be 2e3 times more important, with respect to image preferences, than windowing. The higher filtration numbers applied, the more edge enhancement, and thus contrast is increased. Increased contrast allows the observer to perceive more details. This is, however, purely perception, as the information inherent in the image does not alter. Moreover, as contrast is enhanced, noise will be more predominant. Noise, seen as ‘salt and pepper pattern’, makes a dramatic impact on image quality. Quality decreases very quickly as noise increases, especially when signals of interest have low contrast, as in kidney tumours. Intensity is the least important property. In this study low window setting were generally preferred. The central value of the window (wl) is selected according to the attenuation characteristics of the structure under examination. Image preference is a trade-off between an acceptable amount of noise weighed against a sufficient level of intensity. Lower levels of noise may not be seen, but still
Table 2 Summed preference scores for all three patients (888 scores) Preference rankings Image numbers 1 1 2 3 4 5 6 7 8
2
3
4
5
6
6 52 26 4 0 22 8 24 37 14 7 11 15 13 19 19 5 19 9 9 13 10 8 25 9 5 7 12 13 11 10 4 8 16 10 18 40 2 1 15 20 2 14 2 0 21 48 3
7
8
1 6 13 21 29 24 8 9
0 4 8 16 25 21 23 14
Table 3 Relative importance of attributes, explaining image preferences among the radiographers compared to the radiologists for the three patients, measured by conjoint analysis
Patient A Patient B Patient C
Radiographers
Radiologists
Filtering (%)
Intensity (%)
Filtering (%)
Intensity (%)
80 73 73
20 27 27
84 48 63
16 52 37
Image quality preferences reduce spatial resolution dramatically, especially when structures are in the light grey colour. Low levels of noise were indeed preferred in this study, avoiding ‘drowning’ of low-contrast targets. However, it should be pointed out that the individuals’ noise acceptance may vary substantially. While abdominal radiography generally requires low sharpness and medium contrast due to larger structures, urography requires a much higher sharpness for the same contrast because point targets and line targets need to be identified.13 This is characteristic of ureteric stones of only 1mm size5 as well as linear calcifications that can be a sign of neoplasm, particularly in the kidney.14 Noise and contrast combined with the human visual system’s poor response to high spatial frequencies are limiting factors in determining how small an object in a given imaging system that can be demonstrated. Kundel14 state that contrast and resolution cannot be seen, whereas observers perceive objects. Although contrast and resolution can be dealt with as experimental abstractions, their relationship to the real world images is difficult to determine.
Are radiographers and radiologists similar in reading images? The findings in this study cannot easily be compared to other results comparing radiographers and radiologists image preferences, due to lack of comparable studies. Ma ˚nsson11 asked a group of radiologists the single question ‘Which image do you prefer’? He found an acceptable agreement within the group of radiologists. Ciantar15 considered both subjective assessments and objective measures in a study of chest X-rays of pigs. He concluded that the radiologists agreed amongst themselves quite well, and that ‘subjective’ image quality criteria have some validity, at least for approximate assessments and that objective image quality criteria were achievable. The number of radiographers reporting in the United Kingdom has increased, as has the scope of examinations being reported.16 Inter-reader agreement has been found to be comparable between radiographers and radiologists in image readings on fracture radiographs,17e18 while radiographers with specialised training can report barium enema examinations to a high standard.19 The present study addresses subjective preferences, without any ‘gold standard’ as in ROC studies.20 The fact that observers perceive differently when reading images is well known.21 Kundel14 stated that as radiographers gain clinical experience, they begin to apply adequate-for-diagnosis
195 criteria, resulting in a more individualised way of working. Dreyfus & Dreyfus22 maintain that professionals might not automatically develop their skills to a professional level even after years of practise. Regular discussions among radiographers and radiologists may prevent heterogeneous image quality standards and may pave the way for a common consensus between the two professional groups with respect to image quality.
Quality of the study High participation rates imply that the results should be regarded as representative. Randomised presentation of images for each patient and blinding of the post-processed data prevents observer bias. The technical properties of the images were chosen based on a pilot study including 4 radiographers. The 4 agreed on ‘the very best’ and ‘the worst image quality’. The final study was then made with the same range of image properties. The 4 radiographers in the pilot study were later included in the main study, but images from different patients were included to prevent bias. Patient images were selected from those present in the archives of the hospital. Of course, patients who undergo urography might not be representative of patients with other specific diagnoses or symptoms.23 The image preferences obtained in this study are thus limited to the specific radiological procedure performed. Also the number of patients included was limited; however, the qualities of the images were considered to be representative of images seen in everyday practise. Explained statistical variance (R2 as well as the adjusted R2) would have been identical to 1.0 if all respondents had identical preferences, whereas no correlation at all yield R2 Z 0.0. The sums of the preferences for the 888 film evaluations showed the highest value images having the very best (52 scores, 47%) and the worst image quality (48 scores, 43%). This implies a high degree of agreement with respect to what was the best and the worst image qualities. Also, the variance values were sufficient to enable quality discrimination. The background variables of the respondents were in general found to be representative for Norwegian radiographers and radiologists. The radiographers’ mean age was 38.7 years. Comparable data on Norwegian radiographers’ ages were, however, not available. Mean age for radiologists was 46.5 years, e.g. the same as for Norwegian radiologists.24 With respect to other background variables like occupational grade and gender, the study population was found to be representative of Norwegian radiographers and radiologists.
196
B. Ween et al.
Table 4
Stability of conjoint analysis model for the two professional groups (see Table 3) Radiographers
Patient A Patient B Patient C a b
Radiologists
R2
Adjusted R2a
p-valueb
R2
Adjusted R2
p-value
0.32 0.24 0.17
0.30 0.20 0.20
0.001 0.0001 0.0001
0.050 0.062 0.16
0.080 0.095 0.13
0.03 0.02 0.0002
Adjusted for degrees of freedom. p-values for adjusted R2, statistical significance level: p Z 0.05.
However, we did not collected data on the participants’ years of experience, but this is likely to closely co-vary with age.
Confounders Other factors than profession may affect preferences among radiographers and radiologists. Experience rather than profession might significantly influence image quality preferences. Seniors may appreciate other image characteristics than junior staff members and might have developed a consensus with respect to image quality preferences that junior staff member’s lack. When analysing the background variables, there were statistically significant differences in occupational seniority (p Z 0.005) between the two professions, and also an age difference (p Z 0.01). The difference in gender (p Z 0.065) was close to reaching statistical significance. In this conjoint analysis, we have adjusted for age and gender, but not seniority (Table 4). In the adjusted model, practically the same preferences were found. Seniority was closely related to age, and therefore not included in the adjusted model. Time consumption for reading was slightly different between the two groups of professions, but the difference was so small that we considered it unlikely to be a confounder.
Conclusions This study revealed that radiographers showed a more consistent preference as compared to the radiologists with respect to image quality. The findings indicate a potential for image quality improvement by development and implementation of image property criteria. Development of quality criteria for examination in digital radiography is needed. The professionals have to rely on their own experience and local procedures, and validated local procedures are seldom employed. In the absence of objective image quality criteria, local consensus based on experience has to be adopted.
Acknowledgements We acknowledge the participation of respondents at The Norwegian Radium Hospital, Department of ˚ se Radiology, and help from radiographer Bjørg A Rue. The Norwegian Radium Hospitals’ Research Foundation and Rikshospitalet University Hospital funded this project.
References 1. Bergstrom R, Størmer J. Norway advances toward fully digital healthcare. Norwegian hospitals linking up with PACS and RIS can use their experiences to teach other nations. Diagnostic Imaging Europe 2003;19:16e23. 2. Gould, P. Mobile reading devices give clinicians free range. 20th EuroPACS Conference, 5e7 September 2002. Diagnostic Imaging Europe. Conference Proceedings. 3. Huang HK. PACS basic principles and applications. New York: Wiley-Liss; 1999. 4. International Commission on Radiation Units and Measurements, Inc. In: Medical imaging e the assessment of image quality. Vol. 54. Bethesda, Maryland, USA: International Commission on Radiation Units and Measurements; 1996. 5. European Guidelines for Quality Criteria for Diagnostic Radiographic Images. European Commission and Directorate-General XII: science, research and development. EUR 16260. Luxembourg: Office for Official Publications of the European Communities; 1996. 6. European Guidelines for Quality Criteria for Diagnostic Radiographic Images in Paediatrics. European Commission and Directorate-General XII: science, research and development. EUR 16261. Luxembourg: Office for Official Publications of the European Communities; 1996. 7. Ryan M, McIntosh E, Shackley P. Methodological issues in the application of conjoint analysis in health care. Health Economics 1998;7:373e8. 8. Green PE, Srinivasan V. Conjoint analysis in consumer’s research: issues and outlook. Journal of Consumers Research 1978;5:103e23. 9. Green PE, Srinivasan V. Conjoint analysis in marketing: new developments with implications for research and practice. Journal of Marketing 1990:October 3e19. 10. Green PE, Krieger AM, Wind Y. Thirty years of conjoint analysis: reflections and properties. Interfaces 2001;31:56e73. 11. Ma ˚nsson LG. Methods for the evaluation of image quality: a review. Radiation Protection Dosimetry 2000;90:89e99. 12. Crane R. A simplified approach to image processing. Classical and modern techniques in C. New Jersey: Prentice Hall PTR; 1996.
Image quality preferences 13. Vyborny CJ. Image quality and the clinical radiographic examination. Radiographics 1997;17:479e98. 14. Kundel HL. Images, image quality and observer performance: new horizons in radiology lecture. Radiology 1979; 132:265e71. 15. Ciantar D, Fitzgerald M, Cotterill AD, Pettett A, Cook V, Beluffi G, et al. Correlation between quantitative and subjective assessment of image quality in paediatric radiology. Radiation Protection Dosimetry 2000;90:185e8. 16. Paterson AM, Price RC, Thomas A, Nuttall L. Reporting by radiographers: a policy and practice guide. Radiography 2004;10:205e12. 17. Loughran CF. Reporting of fracture radiographs by radiographers: the impact of a training programme. British Journal of Radiology 1994;67:945e50. 18. Piper KJ, Paterson AM, Godfrey RC. Accuracy of radiographers’ reports in the interpretation of radiographic
197
19.
20. 21. 22.
23. 24.
examinations of the skeletal system: a review of 6796 cases. Radiography 2004;11:27e34. Murphy M, Loughran CF, Birchenough H, Savage J, Sutcliffe C. A comparison of radiographer and radiologist reports on radiographer conducted barium enemas. Radiography 2002;8:215e21. Obuchowski NA. Receiver operating characteristic curves and their use in radiology. Radiology 2003;229:3e8. Kundel HL, Polansky M. Measurement of observer agreement. Radiology 2003;228:303e8. Dreyfus HL, Dreyfus SE, Athanasiou T. Mind over machine: the power of human intuition and expertise in the era of the computer. New York: Free Press; 1988. Kazerooni EA. Population and sample. American Journal of Roentgenology 2001;177:993e9. Den norske lægeforening. Legestatistikk 2002. Mean age for radiologists, !http://www.legeforeningen.noO.