- Email: [email protected]
Computer image processing techniques for automated breast thermogram interpretation
COMPUTERS
AND
BIOMEDICAL
Computer Automated
RESEAKCH
Image Breast
6,522-529
(1973)
Processing Thermogram
Techniques for Interpretation
JAMES WINTER AND MARK ALLEN STEIN Department qf Radiological Sciences, UCLA School qj’ Medic&, Center for the Health Sciences, Los Angeles, California 90024
The development of an inexpensive technique for automated interpretation of breast thermograms by computer has important implications for the mass screening of breast carcinoma. Success in disclosing the thermographic abnormality is demonstrated using three computer image processing techniques: (1) spatial signature analysis: (2) symmetry measurement using thermal density distributions; and (3) image coding by contour map data structure.
Breast carcinoma is the most common causeof female carcinoma deaths, asit has been since 1947(I). Approximately one woman in I7 will develop carcinoma of the breast (2). The five-year survival is related to the stageat the time of diagnosis(3). In 1968, breast cancer caused 28 816 (19.2 T/,) of the female carcinoma deaths out of 144 853total deathsrelated to carcinoma in women (1). The importance of screening for breast carcinoma has been frequently emphasizedin the literature (5). Thermography, a method for producing an infrared image of the surface of an anatomic part, is based upon the physical principle that the amount of radiation emitted by an object dependsupon its absolute temperature. The image obtained by an infrared sensitivedetector asit scansover a patient representssurface temperature variation. The observation that carcinoma, as well as other pathologic mammary conditions, increases surface temperature resulted in the development of breast thermography as a clinically useful tool in the detection of breast malignancy (4). Thermography is somewhat nonspecific but may be effectively usedasa screening technique (4-7). Mammography or breast xeroradiography, requiring radiation exposure and considerable expense, may then be limited to patients with positive thermograms to further selectthose patients suitable for breast biopsy. Many investigators do not make their criteria for thermographjc diagnosis explicit. Absolute temperature has been regarded as having no diagnostic value (8). Symmetry and relative temperature are the basisfrom which most diagnostic criteria are developed, such as: (I) temperature asymmetry between corresponding regions of each breast: (2) temperature differences between a reference point (e.g., suprasternal notch) and each breast: (3) presenceof an asymmetrical venous pattern; Copyright 0 1973 by Academic Press, Inc. All rights of reproduction in any form reserved. Printed in Great Britain
522
AUTOMATEDBREASTTHERMOGRAMINTERPRETATION
523
(4) presence of a “hot spot”; (5) localized periareolar heat; (6) diffuse unilateral elevation of temperature. The human visual system is excellent at detecting local variations in picture brightness. However, it is poorly adapted for quantitative image brightness estimation and non-contiguous area comparisons. This suppression of gradual variations in brightness by the human visual system (9) allows us to perceive the world without being disturbed by variations in illumination, but makes visual inspection somewhat unsuitable for the subtle symmetry comparisons required in thermogram interpretation. Thermography is a valid screening procedure despite the lack of perfect sensitivity and specificity if it maximizes the yield of malignancies for a given expenditure. Many inaccuracies in thermographic interpretation are attributable to inconsistent application of poorly defined diagnostic criteria and to the limitations of the human visual system. Other shortcomings of manual interpretation for mass screening include cost and boredom. If low cost automation of the thermographic examination is realized, then systematic screening of the entire population at risk becomes feasible for the first time. Although large pilot studies have been conducted (4, 6, IO), no mass screening of women at risk has yet been achieved using the modalities currently available, due to limited resources both in dollars and personnel. FEASIBILITY
The physician’s diagnostic abilities exceed by far those of current computer technology, especially when visual interpretation is required. The useful role of computers in medical diagnosis in the immediate future is in the assistance of the physician’s diagnostic effort, especially in stylized, highly repetitive tasks where the sheer bulk of the work load makes it reasonable to attempt automation. The most difficult task in meaningful application of computers to medicine is the selection of significant problems which have practical importance and which are amenable to solution using current technology. Breast thermograms are suitable for automated interpretation because of their relative simplicity compared with other radiological images. The diagnostic criteria involve essentially a measure of the symmetry and relative temperature. Relatively few features are required. Only rudimentary knowledge of anatomy and pathology is necessary for thermographic interpretation. In thermography, the object studied is opaque, unlike the overlapping objects present in radiographs. Classification is limited to two classes (normal versus abnormal). The image is inherently quantitative in nature due to the direct representation of the physical temperature of the breasts. The low spatial resolution and limited dynamic range of image brightness minimize thermographic image processing costs. It is possible to calibrate the image to eliminate distortions introduced by the imaging system. The availability of the
524
WINTERANDSTEIK
temperature signal in electrical form, directly from the thermographic equipment, eliminates the need for the difficult and expensive process of optical scanning of an intermediate photograph. This permits a thermographic instrument to be directly connected to a computer analog-to-digital converter for image data acquisition. The desirability of thermographic screening for breast malignancy. the motivation for automation, and the general utility of a low cost method for automating this examination are clear. This paper demonstrates some techniques that show promise for successful automation of breast thermogram interpretation. METHODS
A model 700 Thermlscope (Texas Instruments Incorporated) is used to produce the thermogram which is permanently recorded on Polaroid type 5.5positive/negative film. This instrument has a mercury cadmium telluride (HgCdTe) detector cooled by liquid nitrogen. The infrared spectral sensitivity range is 8-14 micrometers. The thermograph scans 525 horizontal fines using mirrors with 525 picture elements per line over a 33” x 33” field of view. Thermal sensitivity is at least 0.07’C. A gray shade bar and digital temperature indicator are recorded on the film for calibration. Thermograms are obtained after the skin temperature has been allowed to equilibrate with room air at 70°F for ten minutes. Anterior and both oblique views are recorded. A repeat anterior view is obtained following topical application of alcohol to each breast. Currently, each of the patients also receives a physical examination and almost all undergo mammography immediately following the lhermographic examination. A representative case of biopsy proven breast carcinoma is presented to illustrate the use of automated techniques. Illustrative CaseStudy
A 50-year-old white female presented with a four month history of a lump in the right breast. Physical examination revealed a 2-3 cm firm movable nontender mass in the upper outer quadrant of the right breast. The clinical impression was fibrocystic disease. The thermogram (Fig. 1S) was positive on the right. The mammogram showed minute stippled calcifications on the right, compatible with those resulting from carcinoma. Excisional biopsy of the upper outer quadrant of the right breast showed intraductal (comedo) carcinoma with focal stromal infiltration. A right radical mastectomy was performed and pathologic examination revealed widespread extension of carcinoma into adjoining lobules, but no evidence of axillary node metastasis. Mammary dysplasia was also present. Thermogram transparencies were optically scanned using an IBM 2282 recorder/ scanner. Low resolution digital images were obtained on a 64 x 64 square grid using eigh t shades of gray. Image processing employed the APL interactive computer language, using an IBM 360, Model 91 computer.
AUTOMATEDBREASTTHERMOGRAMINTERPRETATION
(ROW
525
MEAN) I
bx,;
ttl”, TEMPERATURE
COLUMN
B’ack
NUMBER
FIG. 1. Thermographic spatial signature analysis and thermal density distributions. The anterior thermogram (B) is positive on the right due to temperature elevation by a carcinoma of the right breast. Vertical (A) and horizontal (F) spatial signatures, consisting of row and column means, respectively, detect the abnormality. Better demonstration of the asymmetry is obtained by recomputing the horizontal spatial signature after deleting the inframammary region (D). A temperature frequency histogram shows the percentage of picture area at each temperature (E). Comparison of histograms of each breast (C) also shows detection of the thermal asymmetry produced by the cancer. RESULTS
Spatial Signature Analysis Spatial signature analysis (II) is a technique which may be used to develop symmetry criteria. Figure 1F is the horizontal spatial signature of the breast thermogram shown in Fig. IB. Each point represents the projection of the mean temperature of the corresponding column of the picture. The central trough in the
526
WINTER
AND
STEIN
signature corresponds to the midline of the patient, and the two peaks correspond to the breasts. Symmetric points about the midline represent corresponding areas of each breast. Similarly, the vertical spatial signature (Fig. IA) represents the projection of the mean temperature of each row (horizontal mean) of the picture. The combined thermographic effect of both breasts is shown here. The thermogram (Fig. 1B) discloses a localized increase in the temperature of the right breast, a result of an increase in heat emanating from adjacent veins. This thermographic abnormality reflects an underlying carcinoma. There is marked bilateral asymmetry of temperature. The hot area appears as a large asymmetrical peak in the portion of the horizontal spatial signature which corresponds to the right breast (Fig. 1F). The lesion can also be seen as a prominent peak in the upper portion of the vertical spatial signature (Fig. 1A). A second smaller peak, localized further down in the vertical spatial signature, corresponds to the inframammary fold. Although the inframammary temperature elevation is roughly bilaterally symmetrical, it does contribute to the horizontal spatial signature. The asymmetry of temperature resulting from the presence of the carcinoma is more apparent in the horizontal spatial signature when the temperature effect of the warm inframammary region is ignored (Fig. I D). This example demonstrates the use of spatial signature analysis in detecting and localizing dominant hot spots, in estimating picture symmetry, and in detecting the inframammary folds. However, normal variations in breast temperature may occasionally mask a small abnormal hot spot in both horizontal and vertical projections. Density Distributions Symmetry criteria may also be based upon thermal density distributions. The histogram in Fig. 1E shows the percentage area of the picture at each temperature (gray level). Temperature distributions computed for one breast may be compared with those for the contralateral breast (Fig. IC) to provide a measure of thermal asymmetry which is not dependent upon local spatial distribution. This example illustrates that the right breast has a greater proportion of its area at a higher temperature and a lesser proportion of its area at a lower temperature, compared with the left breast. Computations of spatial signatures and density distributions are extremely easy and rapid. However, each of these techniques discards a considerable amount of two-dimensional spatial information. The identification of subtle hot spots may require preservation of more spatial information. Contour Coding A new method of contour coding pictures that does not lose any spatial information has been developed and applied to thermograms. The concept of contour lines is most familiar in the context of topographic maps (12) where each contour line represents a level of elevation of terrain. By analogy, the contour lines of the
AUTOMATED
BREAST
THERMOGRAM
INTERPRETATION
527
thermogram represent levels of constant temperature (isotherms). A contour line is constructed from a series of directional line segments arranged in a chain (13). In contrast to the previous methods of contour coding (14) this new method produces a contour map data structure formed by a family of concentric or adjacent closed curves, preserving both the information content and structure of the original picture. The relationship among the contour lines (insidedness, adjacency, etc.) is the topology of the contour map (15) and corresponds to the structure of the thermal variations.
FIG. 2. Contour coding for localization of “hot spots.” A sequence of contour levels from cold (A) to hot (E) shows progressive delineation of the abnormal temperature in the right breast by the computer generated contour lines. The coldest level (A) outlines both breasts and the inframammary folds. At intermediate temperatures (B and C), the normal pattern in the left breast and inframammary regions begins to break-up into separate contour lines, but the abnormal hot area on the right remains unified. Only the contour lines outlining the asymmetrical “hot spots” due to the carcinoma are displayed at the highest temperatures @ and E). The normal mottled venous pattern has been suppressed by excluding contour lines with small perimeters and those surrounding regions of colder temperature. Compare with the unprocessed thermogram (F), where hot is displayed as black.
Because of the simplicity of topological representation, hot spots may be identified on the thermogram using simple rules that select combinations of contour lines, using their elementary size and shape properties. In this way, the extremely versatile methods of linguistic analysis (16,17) may be applied to picture processing (18) at an earlier stage of the computation than has previously been achieved. Figure 2 shows the progressive delineation of the right sided lesion by successive, selected, concentric contour lines. This demonstrates the utility of contour coding for thermogram analysis in the identification of localized hot spots.
528
WINTERANDSTEIN CONCLUSION
The literature supports the contentions that early detection of breast carcinoma results in a decreased mortality and that thermography is valuable as a detection method. In view of this and in view of the high incidence of breast carcinoma, inexpensive mass screening utilizing thermography is extremely desirable. However, limitations of financial and skilled human resources prohibit this with any of the currently available screening methods. Three automated techniques have been presented that view the breast thermogram without human intervention. All three ofthese techniques identify the thermographic abnormality corresponding to the carcinoma. This paves the way for future research endeavors in this area to refine these techniques, to perform classification experiments for establishing the precise diagnostic criteria, and to conduct a mass screening pilot study with automated interpretation of the breast thermogram. ACKNOWLEDGMENTS The authors acknowledge the advice of Dr. Moses A. Greenfield, Dr. Leo G. Rigler, and Dr. Richard Gold. The technical assistance of Jochen Haber, Kuniye Yahata, Gloria Conroy, Lynnzee Zelden, C.R.T., Charlene Moore, C.R.T., and Ellen Feldon is greatly appreciated. Computer assistance was obtained from the Campus Computing Network, UCLA, and from the Health Sciences Computing Facility, UCLA, sponsored by NIH Special Research Resources Grant RR-3. REFERENCES 1. SILVERBERG,E. AND HOLLEB, A. I. Cancer statistics 1972. Ca 22,2-20 (1972). 2. SEIDMAN, H. Cancer of the breast: statistical and epidemiological data. Cancer 24,1355-1378 (1969). 3. JAMES,A. G. “Cancer Prognosis Manual,” 2nd ed., American Cancer Society, New York, 1966. 4. LILIENFELD, A. M., BARNES,J. M., BARNES,R. B., BRASFIELD,R., CONNELL, J. F., DIAMOND, E., GERSHON-COHEN, J., HABERMAN, J., ISARD, H. J., LANE, W. Z., LATTES, R., MILLER, J., SEAMAN, W., AND SHERMAN, R. An evaluation of thermography in the detection of breast cancer. A cooperative pilot study. Cancer 24,1206-1211 (1969). 5. FREUNDLICH, I. M. Thermography. New Engl. J. Med. 287,880-881 (1972). 6. ISARD, H. J., BECKER, W., SHILO, R., AND OSTRUM, B. J. Breast thermography after four years and 10,000 studies. Amer. J. Roentgenol. 115, 811-821 (1972). 7. HABERMAN, J. D. The present status of mammary thermography. Ca l&315-321 (1968). 8. DODD, G. D., WALLACE, J. D., FREUNDLICH, I. M., MARSH, L., AND ZERMINO, A. Thermography and cancer of the breast. Cancer 23,797-802 (1969). 9. STOCKHAM, T. G. Image processing in the context of a visual model. Proc. IEEE 60,828~842 (1972). 10. SHAPIRO, S., STRAX, P., AND VENET, L. Periodic breast cancer screening in reducing mortality from breast cancer. JAMA 215,1777-1785 (1971). II. HALL, E. L., KRUGER, R. P., DWYER, S. J., III, HALL, D. L., MCLAKEN, R. W., ANI) LODWICK, G. S. A survey of preprocessing and feature extraction techniques for radiographic images. IEEE Trans. Comput. C-20, 1032-1044 (1971).
AUTOMATED
BREAST
THERMOGRAM
INTERPRETATION
529
12. BATCHA, J. P. AND REESE,J. R. Surface determination and automatic contouring for mineral exploration, extraction and processing. Quurt. Colorado School ofMines 59,1-14 (1964). 13. FREEMAN, H. Techniques for the digital computer analysis of chain-encoded arbitrary plane curves. Proc. National Electronics Conf. 17,421432 (1961). 14. WILKINS, L. C. AND WINTZ, P. A. Studies on data compression, Part I: Picture coding by contours. Technical Report TR-EE 70-17, School of Electrical Engineering, Purdue University, Lafayette, In, 1970. 15. MORSE, S. P. Concepts of use in contour map processing. Commun. ACM 12, 147-152 (1969). 16. MILLER, W. F. AND SHAW, A. C. Linguistic methods in picture processing-A survey. AFIPS Cot& Proc., 1968 Fall Joint Computer Conference 33 (part l), 279-290 (1968). 17. ROSENFELD,A. Picture processing by computer. Computing Surveys 1,147-174 (1969). 18. BRICE, C. R. AND FENNEMA, C. L. Scene analysis using regions. Artificial Intelligence 1,205226 (1970).
19
AND
BIOMEDICAL
Computer Automated
RESEAKCH
Image Breast
6,522-529
(1973)
Processing Thermogram
Techniques for Interpretation
JAMES WINTER AND MARK ALLEN STEIN Department qf Radiological Sciences, UCLA School qj’ Medic&, Center for the Health Sciences, Los Angeles, California 90024
The development of an inexpensive technique for automated interpretation of breast thermograms by computer has important implications for the mass screening of breast carcinoma. Success in disclosing the thermographic abnormality is demonstrated using three computer image processing techniques: (1) spatial signature analysis: (2) symmetry measurement using thermal density distributions; and (3) image coding by contour map data structure.
Breast carcinoma is the most common causeof female carcinoma deaths, asit has been since 1947(I). Approximately one woman in I7 will develop carcinoma of the breast (2). The five-year survival is related to the stageat the time of diagnosis(3). In 1968, breast cancer caused 28 816 (19.2 T/,) of the female carcinoma deaths out of 144 853total deathsrelated to carcinoma in women (1). The importance of screening for breast carcinoma has been frequently emphasizedin the literature (5). Thermography, a method for producing an infrared image of the surface of an anatomic part, is based upon the physical principle that the amount of radiation emitted by an object dependsupon its absolute temperature. The image obtained by an infrared sensitivedetector asit scansover a patient representssurface temperature variation. The observation that carcinoma, as well as other pathologic mammary conditions, increases surface temperature resulted in the development of breast thermography as a clinically useful tool in the detection of breast malignancy (4). Thermography is somewhat nonspecific but may be effectively usedasa screening technique (4-7). Mammography or breast xeroradiography, requiring radiation exposure and considerable expense, may then be limited to patients with positive thermograms to further selectthose patients suitable for breast biopsy. Many investigators do not make their criteria for thermographjc diagnosis explicit. Absolute temperature has been regarded as having no diagnostic value (8). Symmetry and relative temperature are the basisfrom which most diagnostic criteria are developed, such as: (I) temperature asymmetry between corresponding regions of each breast: (2) temperature differences between a reference point (e.g., suprasternal notch) and each breast: (3) presenceof an asymmetrical venous pattern; Copyright 0 1973 by Academic Press, Inc. All rights of reproduction in any form reserved. Printed in Great Britain
522
AUTOMATEDBREASTTHERMOGRAMINTERPRETATION
523
(4) presence of a “hot spot”; (5) localized periareolar heat; (6) diffuse unilateral elevation of temperature. The human visual system is excellent at detecting local variations in picture brightness. However, it is poorly adapted for quantitative image brightness estimation and non-contiguous area comparisons. This suppression of gradual variations in brightness by the human visual system (9) allows us to perceive the world without being disturbed by variations in illumination, but makes visual inspection somewhat unsuitable for the subtle symmetry comparisons required in thermogram interpretation. Thermography is a valid screening procedure despite the lack of perfect sensitivity and specificity if it maximizes the yield of malignancies for a given expenditure. Many inaccuracies in thermographic interpretation are attributable to inconsistent application of poorly defined diagnostic criteria and to the limitations of the human visual system. Other shortcomings of manual interpretation for mass screening include cost and boredom. If low cost automation of the thermographic examination is realized, then systematic screening of the entire population at risk becomes feasible for the first time. Although large pilot studies have been conducted (4, 6, IO), no mass screening of women at risk has yet been achieved using the modalities currently available, due to limited resources both in dollars and personnel. FEASIBILITY
The physician’s diagnostic abilities exceed by far those of current computer technology, especially when visual interpretation is required. The useful role of computers in medical diagnosis in the immediate future is in the assistance of the physician’s diagnostic effort, especially in stylized, highly repetitive tasks where the sheer bulk of the work load makes it reasonable to attempt automation. The most difficult task in meaningful application of computers to medicine is the selection of significant problems which have practical importance and which are amenable to solution using current technology. Breast thermograms are suitable for automated interpretation because of their relative simplicity compared with other radiological images. The diagnostic criteria involve essentially a measure of the symmetry and relative temperature. Relatively few features are required. Only rudimentary knowledge of anatomy and pathology is necessary for thermographic interpretation. In thermography, the object studied is opaque, unlike the overlapping objects present in radiographs. Classification is limited to two classes (normal versus abnormal). The image is inherently quantitative in nature due to the direct representation of the physical temperature of the breasts. The low spatial resolution and limited dynamic range of image brightness minimize thermographic image processing costs. It is possible to calibrate the image to eliminate distortions introduced by the imaging system. The availability of the
524
WINTERANDSTEIK
temperature signal in electrical form, directly from the thermographic equipment, eliminates the need for the difficult and expensive process of optical scanning of an intermediate photograph. This permits a thermographic instrument to be directly connected to a computer analog-to-digital converter for image data acquisition. The desirability of thermographic screening for breast malignancy. the motivation for automation, and the general utility of a low cost method for automating this examination are clear. This paper demonstrates some techniques that show promise for successful automation of breast thermogram interpretation. METHODS
A model 700 Thermlscope (Texas Instruments Incorporated) is used to produce the thermogram which is permanently recorded on Polaroid type 5.5positive/negative film. This instrument has a mercury cadmium telluride (HgCdTe) detector cooled by liquid nitrogen. The infrared spectral sensitivity range is 8-14 micrometers. The thermograph scans 525 horizontal fines using mirrors with 525 picture elements per line over a 33” x 33” field of view. Thermal sensitivity is at least 0.07’C. A gray shade bar and digital temperature indicator are recorded on the film for calibration. Thermograms are obtained after the skin temperature has been allowed to equilibrate with room air at 70°F for ten minutes. Anterior and both oblique views are recorded. A repeat anterior view is obtained following topical application of alcohol to each breast. Currently, each of the patients also receives a physical examination and almost all undergo mammography immediately following the lhermographic examination. A representative case of biopsy proven breast carcinoma is presented to illustrate the use of automated techniques. Illustrative CaseStudy
A 50-year-old white female presented with a four month history of a lump in the right breast. Physical examination revealed a 2-3 cm firm movable nontender mass in the upper outer quadrant of the right breast. The clinical impression was fibrocystic disease. The thermogram (Fig. 1S) was positive on the right. The mammogram showed minute stippled calcifications on the right, compatible with those resulting from carcinoma. Excisional biopsy of the upper outer quadrant of the right breast showed intraductal (comedo) carcinoma with focal stromal infiltration. A right radical mastectomy was performed and pathologic examination revealed widespread extension of carcinoma into adjoining lobules, but no evidence of axillary node metastasis. Mammary dysplasia was also present. Thermogram transparencies were optically scanned using an IBM 2282 recorder/ scanner. Low resolution digital images were obtained on a 64 x 64 square grid using eigh t shades of gray. Image processing employed the APL interactive computer language, using an IBM 360, Model 91 computer.
AUTOMATEDBREASTTHERMOGRAMINTERPRETATION
(ROW
525
MEAN) I
bx,;
ttl”, TEMPERATURE
COLUMN
B’ack
NUMBER
FIG. 1. Thermographic spatial signature analysis and thermal density distributions. The anterior thermogram (B) is positive on the right due to temperature elevation by a carcinoma of the right breast. Vertical (A) and horizontal (F) spatial signatures, consisting of row and column means, respectively, detect the abnormality. Better demonstration of the asymmetry is obtained by recomputing the horizontal spatial signature after deleting the inframammary region (D). A temperature frequency histogram shows the percentage of picture area at each temperature (E). Comparison of histograms of each breast (C) also shows detection of the thermal asymmetry produced by the cancer. RESULTS
Spatial Signature Analysis Spatial signature analysis (II) is a technique which may be used to develop symmetry criteria. Figure 1F is the horizontal spatial signature of the breast thermogram shown in Fig. IB. Each point represents the projection of the mean temperature of the corresponding column of the picture. The central trough in the
526
WINTER
AND
STEIN
signature corresponds to the midline of the patient, and the two peaks correspond to the breasts. Symmetric points about the midline represent corresponding areas of each breast. Similarly, the vertical spatial signature (Fig. IA) represents the projection of the mean temperature of each row (horizontal mean) of the picture. The combined thermographic effect of both breasts is shown here. The thermogram (Fig. 1B) discloses a localized increase in the temperature of the right breast, a result of an increase in heat emanating from adjacent veins. This thermographic abnormality reflects an underlying carcinoma. There is marked bilateral asymmetry of temperature. The hot area appears as a large asymmetrical peak in the portion of the horizontal spatial signature which corresponds to the right breast (Fig. 1F). The lesion can also be seen as a prominent peak in the upper portion of the vertical spatial signature (Fig. 1A). A second smaller peak, localized further down in the vertical spatial signature, corresponds to the inframammary fold. Although the inframammary temperature elevation is roughly bilaterally symmetrical, it does contribute to the horizontal spatial signature. The asymmetry of temperature resulting from the presence of the carcinoma is more apparent in the horizontal spatial signature when the temperature effect of the warm inframammary region is ignored (Fig. I D). This example demonstrates the use of spatial signature analysis in detecting and localizing dominant hot spots, in estimating picture symmetry, and in detecting the inframammary folds. However, normal variations in breast temperature may occasionally mask a small abnormal hot spot in both horizontal and vertical projections. Density Distributions Symmetry criteria may also be based upon thermal density distributions. The histogram in Fig. 1E shows the percentage area of the picture at each temperature (gray level). Temperature distributions computed for one breast may be compared with those for the contralateral breast (Fig. IC) to provide a measure of thermal asymmetry which is not dependent upon local spatial distribution. This example illustrates that the right breast has a greater proportion of its area at a higher temperature and a lesser proportion of its area at a lower temperature, compared with the left breast. Computations of spatial signatures and density distributions are extremely easy and rapid. However, each of these techniques discards a considerable amount of two-dimensional spatial information. The identification of subtle hot spots may require preservation of more spatial information. Contour Coding A new method of contour coding pictures that does not lose any spatial information has been developed and applied to thermograms. The concept of contour lines is most familiar in the context of topographic maps (12) where each contour line represents a level of elevation of terrain. By analogy, the contour lines of the
AUTOMATED
BREAST
THERMOGRAM
INTERPRETATION
527
thermogram represent levels of constant temperature (isotherms). A contour line is constructed from a series of directional line segments arranged in a chain (13). In contrast to the previous methods of contour coding (14) this new method produces a contour map data structure formed by a family of concentric or adjacent closed curves, preserving both the information content and structure of the original picture. The relationship among the contour lines (insidedness, adjacency, etc.) is the topology of the contour map (15) and corresponds to the structure of the thermal variations.
FIG. 2. Contour coding for localization of “hot spots.” A sequence of contour levels from cold (A) to hot (E) shows progressive delineation of the abnormal temperature in the right breast by the computer generated contour lines. The coldest level (A) outlines both breasts and the inframammary folds. At intermediate temperatures (B and C), the normal pattern in the left breast and inframammary regions begins to break-up into separate contour lines, but the abnormal hot area on the right remains unified. Only the contour lines outlining the asymmetrical “hot spots” due to the carcinoma are displayed at the highest temperatures @ and E). The normal mottled venous pattern has been suppressed by excluding contour lines with small perimeters and those surrounding regions of colder temperature. Compare with the unprocessed thermogram (F), where hot is displayed as black.
Because of the simplicity of topological representation, hot spots may be identified on the thermogram using simple rules that select combinations of contour lines, using their elementary size and shape properties. In this way, the extremely versatile methods of linguistic analysis (16,17) may be applied to picture processing (18) at an earlier stage of the computation than has previously been achieved. Figure 2 shows the progressive delineation of the right sided lesion by successive, selected, concentric contour lines. This demonstrates the utility of contour coding for thermogram analysis in the identification of localized hot spots.
528
WINTERANDSTEIN CONCLUSION
The literature supports the contentions that early detection of breast carcinoma results in a decreased mortality and that thermography is valuable as a detection method. In view of this and in view of the high incidence of breast carcinoma, inexpensive mass screening utilizing thermography is extremely desirable. However, limitations of financial and skilled human resources prohibit this with any of the currently available screening methods. Three automated techniques have been presented that view the breast thermogram without human intervention. All three ofthese techniques identify the thermographic abnormality corresponding to the carcinoma. This paves the way for future research endeavors in this area to refine these techniques, to perform classification experiments for establishing the precise diagnostic criteria, and to conduct a mass screening pilot study with automated interpretation of the breast thermogram. ACKNOWLEDGMENTS The authors acknowledge the advice of Dr. Moses A. Greenfield, Dr. Leo G. Rigler, and Dr. Richard Gold. The technical assistance of Jochen Haber, Kuniye Yahata, Gloria Conroy, Lynnzee Zelden, C.R.T., Charlene Moore, C.R.T., and Ellen Feldon is greatly appreciated. Computer assistance was obtained from the Campus Computing Network, UCLA, and from the Health Sciences Computing Facility, UCLA, sponsored by NIH Special Research Resources Grant RR-3. REFERENCES 1. SILVERBERG,E. AND HOLLEB, A. I. Cancer statistics 1972. Ca 22,2-20 (1972). 2. SEIDMAN, H. Cancer of the breast: statistical and epidemiological data. Cancer 24,1355-1378 (1969). 3. JAMES,A. G. “Cancer Prognosis Manual,” 2nd ed., American Cancer Society, New York, 1966. 4. LILIENFELD, A. M., BARNES,J. M., BARNES,R. B., BRASFIELD,R., CONNELL, J. F., DIAMOND, E., GERSHON-COHEN, J., HABERMAN, J., ISARD, H. J., LANE, W. Z., LATTES, R., MILLER, J., SEAMAN, W., AND SHERMAN, R. An evaluation of thermography in the detection of breast cancer. A cooperative pilot study. Cancer 24,1206-1211 (1969). 5. FREUNDLICH, I. M. Thermography. New Engl. J. Med. 287,880-881 (1972). 6. ISARD, H. J., BECKER, W., SHILO, R., AND OSTRUM, B. J. Breast thermography after four years and 10,000 studies. Amer. J. Roentgenol. 115, 811-821 (1972). 7. HABERMAN, J. D. The present status of mammary thermography. Ca l&315-321 (1968). 8. DODD, G. D., WALLACE, J. D., FREUNDLICH, I. M., MARSH, L., AND ZERMINO, A. Thermography and cancer of the breast. Cancer 23,797-802 (1969). 9. STOCKHAM, T. G. Image processing in the context of a visual model. Proc. IEEE 60,828~842 (1972). 10. SHAPIRO, S., STRAX, P., AND VENET, L. Periodic breast cancer screening in reducing mortality from breast cancer. JAMA 215,1777-1785 (1971). II. HALL, E. L., KRUGER, R. P., DWYER, S. J., III, HALL, D. L., MCLAKEN, R. W., ANI) LODWICK, G. S. A survey of preprocessing and feature extraction techniques for radiographic images. IEEE Trans. Comput. C-20, 1032-1044 (1971).
AUTOMATED
BREAST
THERMOGRAM
INTERPRETATION
529
12. BATCHA, J. P. AND REESE,J. R. Surface determination and automatic contouring for mineral exploration, extraction and processing. Quurt. Colorado School ofMines 59,1-14 (1964). 13. FREEMAN, H. Techniques for the digital computer analysis of chain-encoded arbitrary plane curves. Proc. National Electronics Conf. 17,421432 (1961). 14. WILKINS, L. C. AND WINTZ, P. A. Studies on data compression, Part I: Picture coding by contours. Technical Report TR-EE 70-17, School of Electrical Engineering, Purdue University, Lafayette, In, 1970. 15. MORSE, S. P. Concepts of use in contour map processing. Commun. ACM 12, 147-152 (1969). 16. MILLER, W. F. AND SHAW, A. C. Linguistic methods in picture processing-A survey. AFIPS Cot& Proc., 1968 Fall Joint Computer Conference 33 (part l), 279-290 (1968). 17. ROSENFELD,A. Picture processing by computer. Computing Surveys 1,147-174 (1969). 18. BRICE, C. R. AND FENNEMA, C. L. Scene analysis using regions. Artificial Intelligence 1,205226 (1970).
19