- Email: [email protected]
Computer-Aided Diagnosis in Radiology
Guest Editorial
Computer-aided Diagnosis in Radiology1 Maryellen L. Giger, PhD
The quality of a medical imaging examination depends on both image acquisition and image interpretation. In the past, radiology and, subsequently, patient care have benefited greatly from improved and standardized imaging programs (such as those for screening mammography) and from new imaging modalities (such as computed tomography and magnetic resonance imaging) made possible by advances in computer technology. Presently, various methods of computer-aided diagnosis (CAD) are being developed to aid in the interpretation of the increasing amounts of medical image data and clinical information, and various reviews on these methods have already been published (1– 4). This issue of Academic Radiology contains three scientific articles on different aspects of the use of computer vision and artificial intelligence to ultimately aid in the task of image interpretation. The authors of one article attempted to modify and extend CAD methods from digitized screen-film mammography to be used with full-field digital mammography (FFDM) (5), while investigators from another study focused on the benefit of computerextracted size features for the characterization of malignant and benign microcalcifications (6). Authors of the third article considered the use of artificial neural networks for merging both human-extracted radiologic features and clinical information into a diagnostic decision on multiple interstitial lung diseases (7). The purpose of Acad Radiol 2002; 9:1–3 1
From the Department of Radiology (MC2026), University of Chicago, 5841 S Maryland Ave, Chicago, IL 60637. Received and accepted November 1, 2001. Supported in part by the National Institutes of Health (National Cancer Institute and National Institute of Arthritis and Musculoskeletal and Skin Disorders), the U.S. Army Medical Research and Materiel Command, and the American Cancer Society. Address correspondence to the author.
The author is a shareholder in R2 Technology, Los Altos, Calif. It is the University of Chicago Conflict of Interest Policy that investigators publicly disclose actual or potential significant financial interests that would reasonably appear to be affected by the research activities. ©
AUR, 2002
all three studies was to extend the scope of the field and to bring the implementation of CAD closer to the clinical arena. Humans are limited in their ability to detect and diagnose disease during image interpretation because of their nonsystematic search patterns and the presence of structure noise that camouflages the normal anatomic background. In addition, the vast amounts of image data generated by some imaging devices makes detection of potential disease a burdensome task and may cause oversight errors. Also, the similar characteristics of some abnormal and normal lesions, as well as overlap in clinical information, may cause interpretational errors. Developments in computer vision and artificial intelligence in medical image interpretation have shown the potential for computers as providers of a “second opinion” in image interpretation. CAD systems leave the final diagnosis to the radiologist. Two main areas of radiology that are targeted for use of CAD include mammography and thoracic radiography. Screening programs for asymptomatic people involve the visual search for a specific abnormality in mostly normal images. Although mammography is currently the best method for the detection of breast cancer, 10%–30% of women who have breast cancer and who undergo mammography have negative mammographic results. In approximately two-thirds of these false-negative results, the radiologist failed to detect the cancer that was evident retrospectively (8). Missed detections may be due to the subtle nature of the radiographic findings (ie, low conspicuity of the lesion), poor image quality, or eye fatigue or oversight by the radiologist. In addition, it has been demonstrated that interpretation by two radiologists can increase sensitivity (9,10). Various investigators have shown that CAD can increase the effectiveness of screening procedures by using a computer system as a “second reader” (like a spell checker), aiding the radiologist by indicating locations of
1
GIGER
suspicious abnormalities in mammograms and leaving the final decisions regarding the possible presence of cancer and patient care to the radiologist. CAD of digitized screening mammograms is now used routinely in many radiology practices, and one commercial system has been approved by the Food and Drug Administration since 1998. The development of FFDM also promises to improve the early detection and diagnosis of breast cancer because of its intrinsically higher image quality (11). Two commercial FFDM systems currently have Food and Drug Administration approval. Further potential is made possible by combining FFDM and CAD to increase detection in digitized images because digital imaging systems can have higher detective quantum efficiency than do digital screen-film mammography systems. Just as with human observers, the relationship between physical image quality and performance levels has not yet been fully established for CAD. The question remains, though, on how much additional work will be necessary to convert current CAD methods for screen-film mammography to those needed for FFDM. It is likely that such conversions will include some means for “standardizing” both the image data between “machines” and the conversions specific to the task, such as mass versus microcalcifications and detection versus characterization. These conversions could be developed from known physical image-quality indices of the various digital imaging units, or they could be empirically derived, given a sufficient database. Li et al partially address this question in their article, “Computer-aided Diagnosis of Masses with Full-Field Digital Mammography” (5). The authors modified their wavelet-based mass-detection method, previously developed for use with digitized screen-film mammograms (12), for the analysis of a limited data set of images obtained with a FFDM machine (DMR 2000D; GE Medical Systems). The method was developed and trained on 60 cases (36 normal and 24 abnormal images with 34 masses) and tested on an independent data set of 24 normal and 10 abnormal digital images with 10 masses. The authors used the processed FFDM image (which had a reduced range of gray levels from intensity mapping and thickness compensation), since such an image had an appearance and an intensity distribution similar to those of digitized screen-film mammograms. During optimization, parameters involving multiscale filtering, segmentation, and classification were modified. With this partial optimization, the CAD method achieved a sensitivity of 91%, with an average of 3.2 false detections per image on the
2
Academic Radiology, Vol 9, No 1, January 2002
training data set. This performance, however, was inferior to that previously achieved by the authors with digitized screen-film mammograms, as referenced in their article (12). In the future, it is expected that inclusion of imagequality characteristics, such as contrast and noise, will be incorporated into such conversions. Readers should expect more articles on the performance of CAD in digital images, especially with the advent of more FFDM machines and digital chest radiography units. Once a suspect lesion is found by a radiologist, patient care depends on the radiologist’s level of suspicion regarding the abnormality. Questions involving the likelihood of disease and patient care arise whenever a suspect lesion (eg, a mammographic lesion, a liver lesion, a lung nodule, or a colonic polyp) is found in an initial image. For instance, if a suspicious region is detected at mammography by a radiologist, he or she must then visually assess various radiographic characteristics and determine the recommended course of action (ie, return for screening, follow-up, or biopsy). Many patients are referred for surgical biopsy on the basis of a radiographically detected mass lesion or a cluster of microcalcifications. Although general rules for the differentiation between benign and malignant breast lesions exist, considerable variability occurs in the interpretation of lesions by radiologists with current radiographic techniques (13). Thus, a role for computers exists in the characterization of such lesions to aid radiologists. Buchbinder et al focus on the role of size in mammographic decision making in their article, “Can the Size of Microcalcifications Predict Malignancy of Clusters at Mammography?” (6). Three quantitative features that described the dimensions of individual calcifications in the breast—length, area, and brightness—were automatically extracted from digitized screen-film mammograms (42-m pixels) by a computer system. The average values within each cluster (as well as the average of the two extreme values) were then compared with pathologic findings. The authors concluded that the average length and area of calcifications in benign clusters were significantly smaller than those in malignant clusters. The authors note that distinguishing malignant from benign cases based on a difference of 0.06 mm in average individual calcification length cannot be performed visually (ie, manually) and can be performed only by using CAD of the image data. Better performance was obtained by using the average of the extreme values rather than the standard average calculation; however, the average of the extreme values was more sensitive to variations in size measurements
GUEST EDITORIAL
Academic Radiology, Vol 9, No 1, January 2002
than was the average of all values. In this study, as with studies of many other CAD methods for lesion characterization, the robustness of a method depends on the detection and segmentation of all relevant individual calcifications in a cluster and the feature analysis method used; that is, different segmentation methods yield different results. The ultimate role of computers in the task of image interpretation will mimic the role of the radiologist in the sense that the computer will take image data and clinical information about the patient as input prior to outputting a diagnosis. The potential of CAD will be most noted in decision making in complex situations, such as with multiple disease states, when clinical information is especially important. It is not reasonable to expect humans to retain complex clinical profiles of multiple and similar disease states. Abe et al report on the usefulness of radiologic and clinical features in their article, “Use of an Artificial Neural Network to Determine the Diagnostic Value of Specific Clinical and Radiologic Parameters in the Diagnosis of Interstitial Lung Disease on Chest Radiographs” (7). Their artificial neural network was designed to differentiate between 11 types of interstitial lung diseases by using up to 10 clinical parameters and 16 radiologic findings. The artificial neural network was evaluated with roundrobin analysis and 370 cases (150 actual cases, 110 published cases, and 110 hypothetical cases). The authors concluded that clinical parameters can be as important as or more important than radiologist-rated image findings in the specific task of diagnosing interstitial lung disease. It is expected that researchers in future studies will further investigate the relative roles of clinical and image information in a variety of patient examinations. The information technology revolution is now changing methods for the interpretation of medical images. With each new issue of medical imaging journals come new imaging techniques, new applications, or new results from observer studies and the preclinical or clinical test-
ing of such methods. In the future, it is likely that all medical images will have some form of CAD performed on them to benefit patient care and outcome. ACKNOWLEDGMENTS
The author is grateful to various members of the Department of Radiology at the University of Chicago for useful discussions. REFERENCES 1. Giger ML, MacMahon H. Image processing and computer-aided diagnosis. Radiol Clin North Am 1996; 34:565–596. 2. Giger ML. Computer-aided diagnosis. In: Haus A, Yaffe M, eds. Syllabus: a categorical course in diagnostic radiology physics: physical aspects of breast imaging— current and future considerations. Oak Brook, Ill: Radiological Society of North America, 1999; 249 –272. 3. Giger ML, Huo Z, Kupinski MA, Vyborny CJ. Computer-aided diagnosis in mammography. In: Sonka M, Fitzpatrick MJ, eds. Handbook of medical imaging. Vol 2. Medical imaging processing and analysis. Bellingham, Wash: Society of Photo-Optical Instrumentation Engineers, 2000; 915–1004. 4. Vyborny CJ, Giger ML, Nishikawa RM. Computer-aided detection and diagnosis. Radiol Clin North Am 2000; 38:725–740. 5. Li L, Clark RA, Thomas J. Computer-aided diagnosis of masses with full-field digital mammography. Acad Radiol 2002; 9:4 –12. 6. Buchbinder S, Leichter I, Lederman R, et al. Can the size of microcalcifications predict malignancy of clusters at mammography? Acad Radiol 2002; 9:18 –25. 7. Abe H, Ashizawa K, Katsuragawa S, et al. Use of an artificial neural network to determine the diagnostic value of specific clinical and radiologic parameters in the diagnosis of interstitial lung disease on chest radiographs. Acad Radiol 2002; 9:13–17. 8. Harvey JA, Fajardo LL, Inis CA. Previous mammograms in patients with impalpable breast carcinoma: retrospective vs blinded interpretation. AJR Am J Roentgenol 1993; 161:1167–1172. 9. Tabar L, Fagerberg G, Duffy SW, Day NE, Gad A, Grontoft O. Update of the Swedish two-county program of mammographic screening for breast cancer. Radiol Clin North Am 1992; 30:187–210. 10. Thurfjell EL, Lernevall KA, Taube AA. Benefit of independent double reading in a population-based mammography screening program. Radiology 1994; 191:241–244. 11. Yaffe M. Digital mammography. In: Haus A, Yaffe M, eds. Syllabus: categorical course in diagnostic radiology physics: physical aspects of breast imaging— current and future considerations. Oak Brook, Ill: Radiological Society of North America, 1999; 229 –238. 12. Li L, Qian W, Clarke LP. Digital mammography: computer-assisted diagnosis method for mass detection with multiorientation and multiresolution wavelet transforms. Acad Radiol 1997; 4:724 –731. 13. Elmore JG, Wells CK, Lee CH, Howard DH, Feinstein AR. Variability in radiologists’ interpretations of mammograms. N Engl J Med 1994; 331:1493–1499.
3
Computer-aided Diagnosis in Radiology1 Maryellen L. Giger, PhD
The quality of a medical imaging examination depends on both image acquisition and image interpretation. In the past, radiology and, subsequently, patient care have benefited greatly from improved and standardized imaging programs (such as those for screening mammography) and from new imaging modalities (such as computed tomography and magnetic resonance imaging) made possible by advances in computer technology. Presently, various methods of computer-aided diagnosis (CAD) are being developed to aid in the interpretation of the increasing amounts of medical image data and clinical information, and various reviews on these methods have already been published (1– 4). This issue of Academic Radiology contains three scientific articles on different aspects of the use of computer vision and artificial intelligence to ultimately aid in the task of image interpretation. The authors of one article attempted to modify and extend CAD methods from digitized screen-film mammography to be used with full-field digital mammography (FFDM) (5), while investigators from another study focused on the benefit of computerextracted size features for the characterization of malignant and benign microcalcifications (6). Authors of the third article considered the use of artificial neural networks for merging both human-extracted radiologic features and clinical information into a diagnostic decision on multiple interstitial lung diseases (7). The purpose of Acad Radiol 2002; 9:1–3 1
From the Department of Radiology (MC2026), University of Chicago, 5841 S Maryland Ave, Chicago, IL 60637. Received and accepted November 1, 2001. Supported in part by the National Institutes of Health (National Cancer Institute and National Institute of Arthritis and Musculoskeletal and Skin Disorders), the U.S. Army Medical Research and Materiel Command, and the American Cancer Society. Address correspondence to the author.
The author is a shareholder in R2 Technology, Los Altos, Calif. It is the University of Chicago Conflict of Interest Policy that investigators publicly disclose actual or potential significant financial interests that would reasonably appear to be affected by the research activities. ©
AUR, 2002
all three studies was to extend the scope of the field and to bring the implementation of CAD closer to the clinical arena. Humans are limited in their ability to detect and diagnose disease during image interpretation because of their nonsystematic search patterns and the presence of structure noise that camouflages the normal anatomic background. In addition, the vast amounts of image data generated by some imaging devices makes detection of potential disease a burdensome task and may cause oversight errors. Also, the similar characteristics of some abnormal and normal lesions, as well as overlap in clinical information, may cause interpretational errors. Developments in computer vision and artificial intelligence in medical image interpretation have shown the potential for computers as providers of a “second opinion” in image interpretation. CAD systems leave the final diagnosis to the radiologist. Two main areas of radiology that are targeted for use of CAD include mammography and thoracic radiography. Screening programs for asymptomatic people involve the visual search for a specific abnormality in mostly normal images. Although mammography is currently the best method for the detection of breast cancer, 10%–30% of women who have breast cancer and who undergo mammography have negative mammographic results. In approximately two-thirds of these false-negative results, the radiologist failed to detect the cancer that was evident retrospectively (8). Missed detections may be due to the subtle nature of the radiographic findings (ie, low conspicuity of the lesion), poor image quality, or eye fatigue or oversight by the radiologist. In addition, it has been demonstrated that interpretation by two radiologists can increase sensitivity (9,10). Various investigators have shown that CAD can increase the effectiveness of screening procedures by using a computer system as a “second reader” (like a spell checker), aiding the radiologist by indicating locations of
1
GIGER
suspicious abnormalities in mammograms and leaving the final decisions regarding the possible presence of cancer and patient care to the radiologist. CAD of digitized screening mammograms is now used routinely in many radiology practices, and one commercial system has been approved by the Food and Drug Administration since 1998. The development of FFDM also promises to improve the early detection and diagnosis of breast cancer because of its intrinsically higher image quality (11). Two commercial FFDM systems currently have Food and Drug Administration approval. Further potential is made possible by combining FFDM and CAD to increase detection in digitized images because digital imaging systems can have higher detective quantum efficiency than do digital screen-film mammography systems. Just as with human observers, the relationship between physical image quality and performance levels has not yet been fully established for CAD. The question remains, though, on how much additional work will be necessary to convert current CAD methods for screen-film mammography to those needed for FFDM. It is likely that such conversions will include some means for “standardizing” both the image data between “machines” and the conversions specific to the task, such as mass versus microcalcifications and detection versus characterization. These conversions could be developed from known physical image-quality indices of the various digital imaging units, or they could be empirically derived, given a sufficient database. Li et al partially address this question in their article, “Computer-aided Diagnosis of Masses with Full-Field Digital Mammography” (5). The authors modified their wavelet-based mass-detection method, previously developed for use with digitized screen-film mammograms (12), for the analysis of a limited data set of images obtained with a FFDM machine (DMR 2000D; GE Medical Systems). The method was developed and trained on 60 cases (36 normal and 24 abnormal images with 34 masses) and tested on an independent data set of 24 normal and 10 abnormal digital images with 10 masses. The authors used the processed FFDM image (which had a reduced range of gray levels from intensity mapping and thickness compensation), since such an image had an appearance and an intensity distribution similar to those of digitized screen-film mammograms. During optimization, parameters involving multiscale filtering, segmentation, and classification were modified. With this partial optimization, the CAD method achieved a sensitivity of 91%, with an average of 3.2 false detections per image on the
2
Academic Radiology, Vol 9, No 1, January 2002
training data set. This performance, however, was inferior to that previously achieved by the authors with digitized screen-film mammograms, as referenced in their article (12). In the future, it is expected that inclusion of imagequality characteristics, such as contrast and noise, will be incorporated into such conversions. Readers should expect more articles on the performance of CAD in digital images, especially with the advent of more FFDM machines and digital chest radiography units. Once a suspect lesion is found by a radiologist, patient care depends on the radiologist’s level of suspicion regarding the abnormality. Questions involving the likelihood of disease and patient care arise whenever a suspect lesion (eg, a mammographic lesion, a liver lesion, a lung nodule, or a colonic polyp) is found in an initial image. For instance, if a suspicious region is detected at mammography by a radiologist, he or she must then visually assess various radiographic characteristics and determine the recommended course of action (ie, return for screening, follow-up, or biopsy). Many patients are referred for surgical biopsy on the basis of a radiographically detected mass lesion or a cluster of microcalcifications. Although general rules for the differentiation between benign and malignant breast lesions exist, considerable variability occurs in the interpretation of lesions by radiologists with current radiographic techniques (13). Thus, a role for computers exists in the characterization of such lesions to aid radiologists. Buchbinder et al focus on the role of size in mammographic decision making in their article, “Can the Size of Microcalcifications Predict Malignancy of Clusters at Mammography?” (6). Three quantitative features that described the dimensions of individual calcifications in the breast—length, area, and brightness—were automatically extracted from digitized screen-film mammograms (42-m pixels) by a computer system. The average values within each cluster (as well as the average of the two extreme values) were then compared with pathologic findings. The authors concluded that the average length and area of calcifications in benign clusters were significantly smaller than those in malignant clusters. The authors note that distinguishing malignant from benign cases based on a difference of 0.06 mm in average individual calcification length cannot be performed visually (ie, manually) and can be performed only by using CAD of the image data. Better performance was obtained by using the average of the extreme values rather than the standard average calculation; however, the average of the extreme values was more sensitive to variations in size measurements
GUEST EDITORIAL
Academic Radiology, Vol 9, No 1, January 2002
than was the average of all values. In this study, as with studies of many other CAD methods for lesion characterization, the robustness of a method depends on the detection and segmentation of all relevant individual calcifications in a cluster and the feature analysis method used; that is, different segmentation methods yield different results. The ultimate role of computers in the task of image interpretation will mimic the role of the radiologist in the sense that the computer will take image data and clinical information about the patient as input prior to outputting a diagnosis. The potential of CAD will be most noted in decision making in complex situations, such as with multiple disease states, when clinical information is especially important. It is not reasonable to expect humans to retain complex clinical profiles of multiple and similar disease states. Abe et al report on the usefulness of radiologic and clinical features in their article, “Use of an Artificial Neural Network to Determine the Diagnostic Value of Specific Clinical and Radiologic Parameters in the Diagnosis of Interstitial Lung Disease on Chest Radiographs” (7). Their artificial neural network was designed to differentiate between 11 types of interstitial lung diseases by using up to 10 clinical parameters and 16 radiologic findings. The artificial neural network was evaluated with roundrobin analysis and 370 cases (150 actual cases, 110 published cases, and 110 hypothetical cases). The authors concluded that clinical parameters can be as important as or more important than radiologist-rated image findings in the specific task of diagnosing interstitial lung disease. It is expected that researchers in future studies will further investigate the relative roles of clinical and image information in a variety of patient examinations. The information technology revolution is now changing methods for the interpretation of medical images. With each new issue of medical imaging journals come new imaging techniques, new applications, or new results from observer studies and the preclinical or clinical test-
ing of such methods. In the future, it is likely that all medical images will have some form of CAD performed on them to benefit patient care and outcome. ACKNOWLEDGMENTS
The author is grateful to various members of the Department of Radiology at the University of Chicago for useful discussions. REFERENCES 1. Giger ML, MacMahon H. Image processing and computer-aided diagnosis. Radiol Clin North Am 1996; 34:565–596. 2. Giger ML. Computer-aided diagnosis. In: Haus A, Yaffe M, eds. Syllabus: a categorical course in diagnostic radiology physics: physical aspects of breast imaging— current and future considerations. Oak Brook, Ill: Radiological Society of North America, 1999; 249 –272. 3. Giger ML, Huo Z, Kupinski MA, Vyborny CJ. Computer-aided diagnosis in mammography. In: Sonka M, Fitzpatrick MJ, eds. Handbook of medical imaging. Vol 2. Medical imaging processing and analysis. Bellingham, Wash: Society of Photo-Optical Instrumentation Engineers, 2000; 915–1004. 4. Vyborny CJ, Giger ML, Nishikawa RM. Computer-aided detection and diagnosis. Radiol Clin North Am 2000; 38:725–740. 5. Li L, Clark RA, Thomas J. Computer-aided diagnosis of masses with full-field digital mammography. Acad Radiol 2002; 9:4 –12. 6. Buchbinder S, Leichter I, Lederman R, et al. Can the size of microcalcifications predict malignancy of clusters at mammography? Acad Radiol 2002; 9:18 –25. 7. Abe H, Ashizawa K, Katsuragawa S, et al. Use of an artificial neural network to determine the diagnostic value of specific clinical and radiologic parameters in the diagnosis of interstitial lung disease on chest radiographs. Acad Radiol 2002; 9:13–17. 8. Harvey JA, Fajardo LL, Inis CA. Previous mammograms in patients with impalpable breast carcinoma: retrospective vs blinded interpretation. AJR Am J Roentgenol 1993; 161:1167–1172. 9. Tabar L, Fagerberg G, Duffy SW, Day NE, Gad A, Grontoft O. Update of the Swedish two-county program of mammographic screening for breast cancer. Radiol Clin North Am 1992; 30:187–210. 10. Thurfjell EL, Lernevall KA, Taube AA. Benefit of independent double reading in a population-based mammography screening program. Radiology 1994; 191:241–244. 11. Yaffe M. Digital mammography. In: Haus A, Yaffe M, eds. Syllabus: categorical course in diagnostic radiology physics: physical aspects of breast imaging— current and future considerations. Oak Brook, Ill: Radiological Society of North America, 1999; 229 –238. 12. Li L, Qian W, Clarke LP. Digital mammography: computer-assisted diagnosis method for mass detection with multiorientation and multiresolution wavelet transforms. Acad Radiol 1997; 4:724 –731. 13. Elmore JG, Wells CK, Lee CH, Howard DH, Feinstein AR. Variability in radiologists’ interpretations of mammograms. N Engl J Med 1994; 331:1493–1499.
3