- Email: [email protected]
Application of CAD systems for the automatic detection of lung nodules
Accepted Manuscript Application of CAD systems for the automatic detection of lung nodules Faridoddin Shariaty, Mojtaba Mousavi PII:
S2352-9148(18)30138-2
DOI:
https://doi.org/10.1016/j.imu.2019.100173
Article Number: 100173 Reference:
IMU 100173
To appear in:
Informatics in Medicine Unlocked
Received Date: 29 June 2018 Revised Date:
18 March 2019
Accepted Date: 24 March 2019
Please cite this article as: Shariaty F, Mousavi M, Application of CAD systems for the automatic detection of lung nodules, Informatics in Medicine Unlocked (2019), doi: https://doi.org/10.1016/ j.imu.2019.100173. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
Application of CAD Systems for the Automatic Detection of Lung Nodules Faridoddin Shariaty 1, Mojtaba Mousavi 2 Institute of Physics, Nanotechnology and Telecommunications, Peter the Great St. Petersburg Polytechnic University (SPbPU), St. Petersburg, Russia, E-mail: [email protected] Department of Electrical Engineering, University of Qom, Qom, Iran, E-mail: [email protected]
SC
RI PT
ABSTRACT: Lung cancer is a common type of cancer that requires early diagnosis due to its often fatal consequences. Computer image processing techniques may be useful to increase the speed and accuracy of lung cancer detection. In order to process medical images, computerized tomography images usually are incorporated due to their high resolution and low noise level. In this paper, the application of Computer-Aided Detection systems for the diagnosis of lung cancer has been studied - including preprocessing and segmentation methods, and data analysis techniques. The main goal was to investigate the latest technology for the development of computational diagnostic tools, so as to assist with the acquisition, processing, and analysis of the medical imagery data. However, there are aspects that still require further attention, such as to increase the sensitivity, reduce false positives, and optimize the detection of each type of nodule, even for differing size and shape.
M AN U
KEYWORDS: Computer-Aided Detection (CAD) systems; Lung Nodule detection; Medical image processing; Computed Tomography (CT) I. INTRODUCTION
AC C
EP
TE D
Lung cancer is one of the most devastating diseases. Worldwide, more than 1.6 million new patients are diagnosed with lung cancer each year [1]. This cancer is the second most common type of cancer among men and women, and is responsible for the death of about a quarter of the people who suffer from this type of cancer [2, 3]. In recent decades, lung cancer has caused more deaths than breast cancer among women in developed countries [4]. According to the US Cancer Society, in 2017, 116,990 men and 10,510 women were suffering from lung cancer. 80% of these people die within 5 years [5]. Delay in the diagnosis of lung cancer is an important factor in the mortality of this disease. Common symptoms of lung cancer include coughing, sore throat, chest pain, fatigue, chest infection, hemoptysis, and weight loss [4]. Excessive delay in detecting or triggering lung cancer cases could cause the patient's affliction. This explains the need for effectiveness and timely implementation of diagnostic and therapeutic procedures. In medical facilities, diverse factors can lead to inefficiencies in out-patient care for cancer patients: the problem of access to specialized medical care, patient referral problems, and the absence of specific treatment. There is a need for out-patient care within a public health system to be restored so as to expedite counseling, respond to requests for diagnostic tests, and reduce the delay in the diagnosis and treatment of the tumor. However, the effect of delay in the treatment of lung cancer on the survival rate of the patient is still a challenging issue [6, 7]. Current clinical methods acquire thousands of images per patient, and it would be very difficult for a physician to accurately analyze all the photos in detail. In addition, human interpretation of medical images has inherent error, and is prone to failing to discover all data and imaging information [8]. With the advancement of computer systems, the expertise of radiologists can be used computationally to extract information from medical pictures. Human analysis is usually subjective and qualitative, and carelessness may occur. Furthermore, a comparative analysis is required between an image with nodule and another nodule pattern, and the human observer usually provides a qualitative response. The usage and extraction of quantitative or numerical features of images certainly require the utilization of computers. Given that most analyses which are conducted by humans are based on qualitative judgment, they will depend on time spent for a particular observer, or from one observer to another. This can be due to the lack of diligence or because of inadequate knowledge and also because of the diversity of education and the level of understanding or competence. Computers can apply a given procedure frequently in a short time and with high precision. In addition, the knowledge of many experts can be implemented computationally; thus computers are often trained in a specific field by a team of several human experts [9]. In this regard, the movement in medicine toward CAD systems, which is due to a quantitative analysis of CT lung images, can improve CT image analysis (which may contain pulmonary nodules), diagnosis of the disease, detection of small cancerous nodules (which with much difficulty can be detected by a physician) and reduce diagnostic time [10, 11]. In this paper, a variety of CAD systems and the steps to access lung CT databases are described. After mentioning the types of a nodule, the methods of extraction of lung and their importance investigated. In the next step, the diagnosis of the nodules and their classification described. Finally, the existing challenges are discussed. In the survey, we have attempted to present comprehensive but brief and easy to understand content, while highlighting the concept. II. Computer-Aided Detection
ACCEPTED MANUSCRIPT
RI PT
In the early 1980s at Kurt Rossmann Laboratories for Radiology Research at the University of Chicago's Department of Radiology, extensive and systematic studies of various Computer Aided Design (CAD) began. Recently, CAD has become an integral part of the clinical work to detect lung cancer in CT images, in many screening sites and hospitals [12]. Computer Aided Detection (CADe) and Computer Aided Diagnosis (CADx) systems are two types of CAD systems. CADe is a system for locating lesions in medical pictures, while the CADx system performs a diagnosis of lesions, for example, to make the distinction between benign and malignant tumors. CADe and CADx systems for the diagnosis of lung cancer in recent decades have been important areas of research. The CADe system does not show the radiological profile of tumors, and the CADx system does not detect nodules and does not have a good level of automation; as a result, these systems are not currently widely used in clinical settings [13]. Some CAD systems can be both CADx and CADe; in large part, this paper examines CADe systems. The diagnosis of a nodule by the CAD system consists of five important steps: database access, preprocessing, segmentation, analysis, and classification. The block diagram of these steps shown in Fig. 1. Analysis
Segmentation operation Extraction of the lungs
False positive reduction
Preprocessing
Classification
Nodule segmentation
SC
Database Acquisition
Feature extraction
M AN U
Fig. 1. Five important steps of CAD systems
1. Database Acquisition
AC C
EP
TE D
The development of CAD techniques for the diagnosis, classification, and evaluation of lung nodules can be facilitated through CT image databases. Some available databases include LIDC, LIDC-IDRI, ELCAP, LISS, and ANODE09. The Lung Image Database Consortium (LIDC) began to develop a web-accessible research resource in 2001. The database was combined with the Infectious Disease Research Institute (IDRI) database and created a general reference for the medical research community. The LIDC/IDRI database was completed in 2009 and resulted in a publicly available database, including 1018 chest CT scans with diagnostic and size reports, with four radiologists having examined each image [14]. The ELCAP database was established in 1991 by a group of doctors at the Cornell University Medical Center (now Weill Medical College of Cornell University), in collaboration with other institute specialists, and contains 50 low-dose lung CT images [15]. LISS is a public database containing 271 CT scans with 677 abnormal regions in them. In this database, all private data in the CT scans are eliminated or replaced with provisioned values in order to make it publically available, and it is promising to apply to CADe, diagnosis research, and medical education. This database is divided into two parts: 2-D and 3-D. The advantage of the LISS database is that it focuses on CT imaging signs instead of commonly considered lung nodules [16]. ANODE09 is an initiative from 2009 by Medical Center of Utrecht University and other organizations, to compare systems that automatically detect pulmonary nodules in lung CT scans in a common database with a single assessment protocol. Since the main purpose of ANODE09 is algorithm evaluation, it contains only 55 anonymous CT scans with the lung nodule specified in five images, and the rest of the images are for CADe system testing. The results of CADe systems on CT test images, including a list of nodule locations, are sent to this database and after being processed, the result of the test is published on the results pages of [17, 18]. 2. Preprocessing
Lung CT image preprocessing is performed to improve their quality and to achieve better results in the diagnosis of the lung nodule. The lungs contain several structures that can be confused with nodules, and the importance of this step is to enhance the image. The more common preprocessing methods in the literature are as follows. In [19], the following preprocessing methods are considered: Adaptive Median Filter, Alpha-Trimmed Mean Filter, Gaussian Filter, Gabor Filter, High Pass Filter, Laplacian Filter, and Bilateral Filter. H. Kim et al. [20] used the Median Filter (the kernel size is 3X3) and Top-hat filter to reduce image noise and to smooth it. Also, W. Suiyuan et al. [21] employed linear interpolation for preprocessing of lung CT image. Fig. 2 illustrates the preprocessing operation.
RI PT
ACCEPTED MANUSCRIPT
Fig. 2. Preprocessing CT Lung Image Processing. A) Original image B) Image with changes in opacity, color and gradient, and applying a gradient filter to enhance image quality. [22][20]
SC
A. Teramoto et al. [23], in order to differentiate the nodule from the normal lung tissue, used an erosion filter in preprocessing operations that shrink the image nodules and blood vessels. The erosion filter is one of the morphological filters that extend the distinction between image objects. 3. Segmentation Operation
TE D
M AN U
Image segmentation is the process of partitioning a digital image into multiple segments. The goal of segmentation is to simplify and change the representation of an image into something that is more meaningful and easier to analyze. The segmentation process in the CAD system for lung CT images consists of two parts: lung extraction and nodule segmentation. 3.1. Lung Extraction Since the lung is a complex organ that includes various structures such as vessels, gaps, and bronchus or pleura, which can be close to the lung nodule, lung extraction is one of the most important steps in CAD systems, which speeds up operations in the system, as well as increasing the accuracy of nodule diagnosis and segmentation. The nodules are classified into two categories in terms of their location in the lung [24]: non-attached nodules (for example wellcircumscribed nodules) and attached nodules (juxta-pleural, vascularized and pleural-tail).
EP
Fig. 3. Four examples of CT images that contain nodules: (a) Well-Circumscribed, (b) Pleural-Tail, (c) Juxta-Pleural, and (d) Vascularized. A black circle is drawn around the nodule center [24].
=
AC C
One of the important goals of lung imagery extraction is the separation of attached nodules to the lung wall, which may be removed during the processing operation. Accordingly, for this purpose, various algorithms are presented, which are classified as follows: thresholding, deformable models, shaped-based models, edge-based models [25]. M. Alilou et al. [26] employed a multiple thresholding method for lung extraction. To determine the desired threshold, assumed and be the mean gray-levels of the body and non-body voxels that segmented with threshold Ti. The new threshold (Ti+1) calculates via: .
The iterative updating of the new threshold is repeated until Ti+1 = Ti. The Hounsfield value of the air is chosen as the value of the initial threshold (T0 = -1000 HU). The lung segmentation process is shown in Fig. 4.
A)Main Image
B)Tresholding
C)Invert
D)Hole filling , Extract largest hole
E)AND(B,D)
F)Hole filling
ACCEPTED MANUSCRIPT Fig. 4. Lung segmentation process and the result
M AN U
SC
RI PT
M. Keshani et al. [27], introduced a lung extraction method, which segmented the attached nodules from lung wall. In this work, adaptive fuzzy thresholding was employed in the first step. In the second stage, 5×5 and 23×23 windows were applied to obtain a hole-free lung mask. In the third stage, 45 degrees rotated windows with dimensions 50×50 and 25×25 were applied to segment large and small sized non-isolated nodules connected to the chest wall. In the next step, a typical threshold was used to exclude areas outside of the thorax pixels, and bone parts pixels, which are located in the mid region of the original image. Finally, the created mask was given to the Active Cantor model as the input mask. R. Shojaii et al. [28] presented a lung segmentation technique based on the watershed algorithm, which is characterized by applying the internal and external markers to the gradient image of the lung region. In this work, the Rolling Ball filter was used to smooth the lines and fill the holes while maintaining the original boundaries. E. M. van Rikxoort et al. [29] used a hybrid method to segment lung from surrounding tissue. This method consists of four steps. The lung is separated in the first step using an automated 3D algorithm such as Region Growing and some morphological techniques. This step is fast, but it may be accompanied by some errors that are detected and corrected by using an error detection method. A more complex algorithm based on multi-atlas segmentation extracts lung regions. Finally, the results of the algorithm are evaluated by an error detection method to prevent possible errors. In another work, S. Shimoyama et al. [30] used an Active Contour algorithm for lung segmentation. For optimal lung segmentation, the previous lung slice, which will be similar to the current slice, is used as an input mask for the Active Contour algorithm, and it causes a better segmentation result. 3.2. Nodule segmentation Nodule segmentation algorithms have been used to determine the size of the nodule and its growth, as well as to determine the type of nodule [31]. The similarity of the nodule to structures such as vessels, chest, and surrounding tissue has been one of the nodule segmentation problems for CAD systems. Additionally, size, location, and contrast of nodules may make some problems for CAD systems. The categorization of nodules in terms of size, location, and density can be seen in Fig. 5 [32]. In CAD systems, the detached nodules and solid nodules are segmented with high accuracy, but for other nodules, their segmentation is associated with the false-positive increase.
Nodule
Location Deattached
Solid
GGO
Size Micronodule
TE D
Attached
Density
Mass
Fig. 5. Categorization of nodules in terms of size, location and density
AC C
EP
Nodule segmentation methods are classified in different approaches. A. Mittal [33] divided segmentation methods into four branches: Edge Detection, Region-Based Methods, Thresholding Technique, and Clustering Technique. The complete categorization of this article is illustrated in Fig. 6. All these segmentation methods have been used for lung nodule segmentation in the literature [34-38]. Edge Detection [34] Region Growing [33]
Segmentation Methods
Region Based Methods Splitting & Merging
Global Thresholding Thresholding Technique [35] Local Thresholding
K-mean Clustering [36] Clustering Technique C-mean Clustering [37] Fig. 6. Segmentation methods categorization as proposed in [33]
ACCEPTED MANUSCRIPT
Based on Discontinuity
Edge Based [34]
Threshold Based [35] Region Growing [33] Region Based
RI PT
Segmentation methods
In [39, 40], the segmentation methods are first divided into two groups, based on similarity and based on discontinuity, respectively, and then in [39], other segmentation methods are classified, as depicted in the Fig. 7. The segmentation methods have used for lung nodule segmentation in the literature [41-43].
Splitting & Merging
Based on Similarity
Neural Network Based [40, 41]
Wavelets Based [42]
SC
Special Theory
Clustering Based [36, 37]
Fig. 7. Segmentation methods categorization proposed in [39]
AC C
EP
TE D
M AN U
The importance of nodule segmentation with high accuracy has been considered in the literature. For instance, J. Dehmeshkiet al. [44] proposed a nodule segmentation technique based on a region growing algorithm. This region growing is operated at intervals of a volumetric mask that is created by first applying a district adaptive segmentation algorithm that identifies foreground and background regions at intervals using a precise window size. The foreground objects are then crammed to eliminate any holes, and a property map is generated to create a 3D mask. Then the mask is enlarged to contain the background, while excluding unwanted foreground regions. By excluding and confining the search volume, the mask is helpful to estimate parameters for subsequent region growing, and to verify dependableness. This technique was tested on 815 lung nodules, and a radiologist confirmed 84% of the results. M. Keshani et al. [45] identified nodules using extraction 2-D statistical characteristics and the 3-D anatomy via a SVM classifier. Then the nodule boundaries were extracted by the Active Contour algorithm. The proposed algorithm can segment solid and cavitary nodules which are connected to the lung wall. The Active Contour model which was used in this paper, presented in [46], employed a robust and efficient model for detecting thin edges and the boundary between nodule and lung. Results presented in [47] show the strength of this Active Contour algorithm versus noisy images. S. Diciotti et al. [48] presented a method based on local shape analysis which isolates and refines the boundary between nodule and vessel without any modification. In various papers, the shape-based method has been investigated and, in this paper, the power of this method is presented. Other nodule segmentation techniques such as Watershed [49] have been used and results have been presented. A comparison between three segmentation algorithms (Markov random field [50], Watershed, active contour) in nodule segmentation is illustrated in Fig. 8.
Fig. 8. Comparison between the three algorithms in nodule segmentation: a) Markov random field (yellow) b) Watershed (red) c) active contour (green).
4. Analysis After segmentation of candidate nodules, we are required to examine them. The study of nodule-like structures consists of two parts: false positive reduction and feature extraction. 4.1. False positive reduction:
ACCEPTED MANUSCRIPT
RI PT
One of the most important characteristics of CAD systems is the false positive rate, which indicates the power and reliability of a CAD system. This stage is one of the important components of the lung nodule detection system, which plays an important role in detecting lung cancer and early treatment [51]. A. Setio et al. [52] presented a false positive reduction method using multi-view ConvNets. One of the methods of false positive reduction is to remove structures with less similarity to the nodules, which is done by examining the features of each candidate nodule, such as the method used in [53], which specified the 2-D and 3-D features to eliminate false nodule candidates. Q. Dou et al. [51] proposed a new method based on CNN to reduce false positives. 4.2. Feature extraction For detecting cancerous nodules using apparent characteristics, candidate nodules feature extraction should be done. For instance, J. Lee et al. [53] calculates Multiple geometric parameters for each suspected lung lesion, including its shape, elongation, size, speculation, density, and other features, for rating each candidate nodule, which indicates its likelihood of actually being a lung nodule. N. Gurcan et al. [55] extracted the following features for each 3D object in order to classify candidate nodules: volume, surface area, average gray value, standard deviation, skewness, and kurtosis of the gray value histogram. Alilou et al. [26] computed a set of 17 2D and 3D features for each segmented and labeled candidate nodule. They grouped these features into the following four types: 3D geometrical, 3D intensitybased, 2D geometrical and 2D intensity-based features. 5. Classification
M AN U
SC
Many studies have been presented in the field of automatic nodule extraction by computer systems, which mainly use classification methods to divide nodule candidates into nodule and non-nodule. Classification algorithms typically use features, as described in the previous step, to detect nodules. The most common methods which are used in the literature include SVM [54, 55], Neural Networks [56], and CNN [57, 58]. III. Conclusions
TE D
Using computer systems and image processing for medical CT images analysis has progressed substantially in recent years, and in general, many published works have the potential to be used in medical practice. In this context, physicians need to learn more about operational computer systems in medical images processing, which would lead to using these systems for the early detection of lung cancer. However, in order to facilitate acceptance and use of these systems, the deficiencies of them must be addressed and eliminated. For this reason, close contact of developers and analysts with the medical community is necessary, so that awareness of the specific needs in CAD systems can be utilized to extend these systems. This will be accomplished with joint efforts between doctors, patients, engineers, and scientists. In this work, the use of CAD systems in lung CT image processing has been investigated, and the stages of processing in order for diagnostics and to extract lung nodules are presented. This information should be useful to researchers in this field and also to encourage physicians to use these systems. REFERENCES
1. Tsao, A. S., Scagliotti, G. V., Bunn Jr, P. A., Carbone, D. P., Warren, G. W., Bai, C., ... & Adusumilli, P. S.
EP
(2016). Scientific advances in lung cancer 2015. Journal of Thoracic Oncology, 11(5), 613-638.
AC C
2. J. website : www.cancer.org , 2017 , https://www.cancer.org/cancer/non-small-cell-lungcancer/about/key-statistics.html.". 3. Spiro, S. G., & Porter, J. C. (2002). Lung cancer—where are we today? Current advances in staging and nonsurgical treatment. American journal of respiratory and critical care medicine, 166(9), 1166-1196. 4. Ellis, P. M., & Vandermeer, R. (2011). Delays in the diagnosis of lung cancer. Journal of thoracic disease, 3(3), 183.
5. "American Cancer Society. (2017). Cancer fact and figures [Online]. Available https://www.cancer.org/cancer/non-small-cell-lung cancer/about/key-statistics.html pdf." 6. Knorst, M. M., Dienstmann, R., & Fagundes, L. P. (2003). Delay in the diagnosis and surgical treatment of lung cancer. Jornal de Pneumologia, 29(6), 358-364.
7. Bozcuk, H., & Martin, C. (2001). Does treatment delay affect survival in non-small cell lung cancer? A retrospective analysis from a single UK centre. Lung cancer, 34(2), 243-252. 8. Zhang, Y., Oikonomou, A., Wong, A., Haider, M. A., & Khalvati, F. (2017). Radiomics-based prognosis analysis for non-small cell lung cancer. Scientific reports, 7, 46349. 9. Sprindzuk, M. V., Kovalev, V. A., Snezhko, E. V., & Kharuzhyk, S. A. (2010). Lung cancer differential diagnosis based on the computer assisted radiology: The state of the art. Polish journal of radiology, 75(1), 67. 10. Doi, K. (2005). Current status and future potential of computer-aided diagnosis in medical imaging. The British journal of radiology, 78(suppl_1), s3-s19.
ACCEPTED MANUSCRIPT
11. van Ginneken, B., Schaefer-Prokop, C. M., & Prokop, M. (2011). Computer-aided diagnosis: how to move from the laboratory to the clinic. Radiology, 261(3), 719-732. 12. Doi, K. (2007). Computer-aided diagnosis in medical imaging: historical review, current status and future potential. Computerized medical imaging and graphics, 31(4-5), 198-211. 13. Firmino, M., Angelo, G., Morais, H., Dantas, M. R., & Valentim, R. (2016). Computer-aided detection (CADe) and diagnosis (CADx) system for lung cancer with likelihood of malignancy. Biomedical engineering online, 15(1), 2. 14. Armato, S. G., McLennan, G., Bidaut, L., McNitt Gray, M. F., Meyer, C. R., Reeves, A. P., ... & Kazerooni, E. A. (2011). The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Medical physics, 38(2), 915-931.
RI PT
15. "http://www.ielcap.org/ielcap.". 16. Han, G., Liu, X., Han, F., Santika, I. N. T., Zhao, Y., Zhao, X., & Zhou, C. (2015). The LISS—a public database of common imaging signs of lung diseases for computer-aided detection and diagnosis research and medical education. IEEE Transactions on Biomedical Engineering, 62(2), 648-656.
17. "https://anode09.grand-challenge.org/.". 18. Van Ginneken, B., Armato III, S. G., de Hoop, B., van Amelsvoort-van de Vorst, S., Duindam, T., Niemeijer, M.,
23. 24. 25. 26. 27. 28. 29.
30.
31. 32. 33.
34.
SC
M AN U
22.
TE D
21.
EP
20.
AC C
19.
... & Camarlinghi, N. (2010). Comparing and combining algorithms for computer-aided detection of pulmonary nodules in computed tomography scans: the ANODE09 study. Medical image analysis, 14(6), 707-722. Vijaya, G., & Suhasini, A. (2014). An adaptive preprocessing of lung CT images with various filters for better enhancement. Academic Journal of Cancer Research, 7(3), 179-184. Kim, H., Nakashima, T., Itai, Y., Maeda, S., kooi Tan, J., & Ishikawa, S. (2007, October). Automatic detection of ground glass opacity from the thoracic MDCT images by using density features. In 2007 International Conference on Control, Automation and Systems (pp. 1274-1277). IEEE. Suiyuan, W., & Junfeng, W. (2012, November). Pulmonary nodules 3D detection on serial CT scans. In 2012 Third Global Congress on Intelligent Systems (pp. 257-260). IEEE. Firmino, M., Morais, A. H., Mendoça, R. M., Dantas, M. R., Hekis, H. R., & Valentim, R. (2014). Computer-aided detection system for lung cancer in computed tomography scans: review and future prospects. Biomedical engineering online, 13(1), 41. Teramoto, A., & Fujita, H. (2013). Fast lung nodule detection in chest CT images using cylindrical noduleenhancement filter. International journal of computer assisted radiology and surgery, 8(2), 193-205. Farag, A. A., El Munim, H. E. A., Graham, J. H., & Farag, A. A. (2013). A novel approach for lung nodules segmentation in chest CT using level sets. IEEE Transactions on Image Processing, 22(12), 5202-5213. El-Baz, A., Elnakib, A., Abou El-Ghar, M., Gimel'farb, G., Falk, R., & Farag, A. (2013). Automatic detection of 2D and 3D lung nodules in chest spiral CT scans. International journal of biomedical imaging, 2013. Alilou, M., Kovalev, V., Snezhko, E., & Taimouri, V. (2014). A comprehensive framework for automatic detection of pulmonary nodules in lung CT images. Image Analysis & Stereology, 33(1), 13-27. Keshani, M., Azimifar, Z., Boostani, R., & Shakibafar, A. (2010, October). Lung nodule segmentation using active contour modeling. In 2010 6th Iranian Conference on Machine Vision and Image Processing (pp. 1-6). IEEE. Shojaii, R., Alirezaie, J., & Babyn, P. (2005, September). Automatic lung segmentation in CT images using watershed transform. In IEEE International Conference on Image Processing 2005 (Vol. 2, pp. II-1270). IEEE. van Rikxoort, E. M., de Hoop, B., Viergever, M. A., Prokop, M., & van Ginneken, B. (2009). Automatic lung segmentation from thoracic computed tomography scans using a hybrid approach with error detection. Medical physics, 36(7), 2934-2947. Shimoyama, S., Homma, N., Sakai, M., Ishibashi, T., & Yoshizawa, M. (2009, August). Auto-detection of nonisolated pulmonary nodules connected to the chest walls in X-ray CT images. In 2009 ICCAS-SICE (pp. 36723675). IEEE. Sluimer, I., Schilham, A., Prokop, M., & Van Ginneken, B. (2006). Computer analysis of computed tomography scans of the lung: a survey. IEEE transactions on medical imaging, 25(4), 385-405. Ye, X., Lin, X., Dehmeshki, J., Slabaugh, G., & Beddoe, G. (2009). Shape-based computer-aided detection of lung nodules in thoracic CT images. IEEE Transactions on Biomedical Engineering, 56(7), 1810-1820. Aman Mittal, (2016). A Review on Image Segmentation Techniques. International Journal of Recent Advances in Engineering & Technology (IJRAET), (pp. 61-65). Yang, B., Xiang, D., Yu, F., & Chen, X. (2018, March). Lung tumor segmentation based on multi-scale template matching and region growing. In Medical Imaging 2018: Biomedical Applications in Molecular, Structural, and Functional Imaging(Vol. 10578, p. 105782Q). International Society for Optics and Photonics.
ACCEPTED MANUSCRIPT
35. Singh, A. K., Saini, V., & Saini, L. M. (2017, June). ROI based detection of abnormalities in lungs using medical
42. 43.
44. 45.
46. 47. 48.
49. 50. 51. 52.
53.
54. 55.
56.
RI PT
41.
SC
40.
M AN U
39.
TE D
38.
EP
37.
AC C
36.
image processing. In 2017 International Conference on Intelligent Computing and Control Systems (ICICCS) (pp. 876-880). IEEE. Huang, S., Liu, X., Han, G., Zhao, X., Zhao, Y., & Zhou, C. (2018, February). 3D GGO candidate extraction in lung CT images using multilevel thresholding on supervoxels. In Medical Imaging 2018: Computer-Aided Diagnosis (Vol. 10575, p. 1057533). International Society for Optics and Photonics. Sangamithraa, P. B., & Govindaraju, S. (2016, March). Lung tumour detection and classification using EK-Mean clustering. In 2016 International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET) (pp. 2201-2206). IEEE. Nithila, E. E., & Kumar, S. S. (2016). Segmentation of lung nodule in CT data using active contour model and Fuzzy C-mean clustering. Alexandria Engineering Journal, 55(3), 2583-2588. Sivakumar, P., & Meenakshi, S. (2016). A review on image segmentation techniques. International Journal of Advanced Research in Computer Engineering & Technology (IJARCET), 5(3). Kandwal, R., Kumar, A., & Bhargava, S. (2014). Existing image segmentation techniques. International Journal of Advanced Research in Computer Science and Software Engineering, 4(4). Messay, T., Hardie, R. C., & Tuinstra, T. R. (2015). Segmentation of pulmonary nodules in computed tomography using a regression neural network approach and its application to the lung image database consortium and image database resource initiative dataset. Medical image analysis, 22(1), 48-62. Skourt, B. A., El Hassani, A., & Majda, A. (2018). Lung CT Image Segmentation using deep neural networks. Procedia Computer Science, 127, 109-113. Ceylan, M., ÖZBAY, Y., UÇAN, O. N., & Yildirim, E. (2010). A novel method for lung segmentation on chest CT images: complex-valued artificial neural network with complex wavelet transform. Turkish Journal of Electrical Engineering & Computer Sciences, 18(4), 613-624. Dehmeshki, J., Amin, H., Valdivieso, M., & Ye, X. (2008). Segmentation of pulmonary nodules in thoracic CT scans: a region growing approach. IEEE transactions on medical imaging, 27(4), 467-480. Keshani, M., Azimifar, Z., Tajeripour, F., & Boostani, R. (2013). Lung nodule segmentation and recognition using SVM classifier and active contour modeling: A complete intelligent system. Computers in biology and medicine, 43(4), 287-300. Lankton, S., & Tannenbaum, A. (2008). Localizing region-based active contours. IEEE transactions on image processing, 17(11), 2029-2039. Yezzi Jr, A., Tsai, A., & Willsky, A. (2002). A fully global approach to image segmentation via coupled curve evolution equations. Journal of Visual Communication and Image Representation, 13(1-2), 195-216. Diciotti, S., Lombardo, S., Falchini, M., Picozzi, G., & Mascalchi, M. (2011). Automated segmentation refinement of small lung nodules in CT scans by local shape analysis. IEEE Transactions on Biomedical Engineering, 58(12), 3418-3428. Tan, Y., Schwartz, L. H., & Zhao, B. (2013). Segmentation of lung lesions on CT scans using watershed, active contours, and Markov random field. Medical physics, 40(4). Tan, Y., Schwartz, L. H., & Zhao, B. (2013). Segmentation of lung lesions on CT scans using watershed, active contours, and Markov random field. Medical physics, 40(4). Dou, Q., Chen, H., Yu, L., Qin, J., & Heng, P. A. (2017). Multilevel contextual 3-D CNNs for false positive reduction in pulmonary nodule detection. IEEE Transactions on Biomedical Engineering, 64(7), 1558-1567. Setio, A. A. A., Ciompi, F., Litjens, G., Gerke, P., Jacobs, C., Van Riel, S. J., ... & van Ginneken, B. (2016). Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE transactions on medical imaging, 35(5), 1160-1169. Krishnamurthy, S., Narasimhan, G., & Rengasamy, U. (2016, December). An automatic computerized model for cancerous lung nodule detection from computed tomography images with reduced false positives. In International Conference on Recent Trends in Image Processing and Pattern Recognition (pp. 343-355). Springer, Singapore. Lee, I. J., Gamsu, G., Czum, J., Wu, N., Johnson, R., & Chakrapani, S. (2005). Lung nodule detection on chest CT: evaluation of a computer-aided detection (CAD) system. Korean journal of radiology, 6(2), 89-93. Orozco, H. M., Villegas, O. O. V., Sánchez, V. G. C., Domínguez, H. D. J. O., & Alfaro, M. D. J. N. (2015). Automated system for lung nodules classification based on wavelet feature descriptor and support vector machine. Biomedical engineering online, 14(1), 9. Shen, W., Zhou, M., Yang, F., Yang, C., & Tian, J. (2015, June). Multi-scale convolutional neural networks for lung nodule classification. In International Conference on Information Processing in Medical Imaging (pp. 588599). Springer, Cham.
ACCEPTED MANUSCRIPT
57. Pérez, G., & Arbeláez, P. (2017, November). Automated detection of lung nodules with three-dimensional
AC C
EP
TE D
M AN U
SC
RI PT
convolutional neural networks. In 13th International Conference on Medical Information Processing and Analysis (Vol. 10572, p. 1057218). International Society for Optics and Photonics. 58. Aliferis, C. F., Tsamardinos, I., Massion, P. P., Statnikov, A. R., Fananapazir, N., & Hardin, D. P. (2003, May). Machine Learning Models for Classification of Lung Cancer and Selection of Genomic Markers Using Array Gene Expression Data. In FLAIRS Conference (pp. 67-71).
S2352-9148(18)30138-2
DOI:
https://doi.org/10.1016/j.imu.2019.100173
Article Number: 100173 Reference:
IMU 100173
To appear in:
Informatics in Medicine Unlocked
Received Date: 29 June 2018 Revised Date:
18 March 2019
Accepted Date: 24 March 2019
Please cite this article as: Shariaty F, Mousavi M, Application of CAD systems for the automatic detection of lung nodules, Informatics in Medicine Unlocked (2019), doi: https://doi.org/10.1016/ j.imu.2019.100173. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
Application of CAD Systems for the Automatic Detection of Lung Nodules Faridoddin Shariaty 1, Mojtaba Mousavi 2 Institute of Physics, Nanotechnology and Telecommunications, Peter the Great St. Petersburg Polytechnic University (SPbPU), St. Petersburg, Russia, E-mail: [email protected] Department of Electrical Engineering, University of Qom, Qom, Iran, E-mail: [email protected]
SC
RI PT
ABSTRACT: Lung cancer is a common type of cancer that requires early diagnosis due to its often fatal consequences. Computer image processing techniques may be useful to increase the speed and accuracy of lung cancer detection. In order to process medical images, computerized tomography images usually are incorporated due to their high resolution and low noise level. In this paper, the application of Computer-Aided Detection systems for the diagnosis of lung cancer has been studied - including preprocessing and segmentation methods, and data analysis techniques. The main goal was to investigate the latest technology for the development of computational diagnostic tools, so as to assist with the acquisition, processing, and analysis of the medical imagery data. However, there are aspects that still require further attention, such as to increase the sensitivity, reduce false positives, and optimize the detection of each type of nodule, even for differing size and shape.
M AN U
KEYWORDS: Computer-Aided Detection (CAD) systems; Lung Nodule detection; Medical image processing; Computed Tomography (CT) I. INTRODUCTION
AC C
EP
TE D
Lung cancer is one of the most devastating diseases. Worldwide, more than 1.6 million new patients are diagnosed with lung cancer each year [1]. This cancer is the second most common type of cancer among men and women, and is responsible for the death of about a quarter of the people who suffer from this type of cancer [2, 3]. In recent decades, lung cancer has caused more deaths than breast cancer among women in developed countries [4]. According to the US Cancer Society, in 2017, 116,990 men and 10,510 women were suffering from lung cancer. 80% of these people die within 5 years [5]. Delay in the diagnosis of lung cancer is an important factor in the mortality of this disease. Common symptoms of lung cancer include coughing, sore throat, chest pain, fatigue, chest infection, hemoptysis, and weight loss [4]. Excessive delay in detecting or triggering lung cancer cases could cause the patient's affliction. This explains the need for effectiveness and timely implementation of diagnostic and therapeutic procedures. In medical facilities, diverse factors can lead to inefficiencies in out-patient care for cancer patients: the problem of access to specialized medical care, patient referral problems, and the absence of specific treatment. There is a need for out-patient care within a public health system to be restored so as to expedite counseling, respond to requests for diagnostic tests, and reduce the delay in the diagnosis and treatment of the tumor. However, the effect of delay in the treatment of lung cancer on the survival rate of the patient is still a challenging issue [6, 7]. Current clinical methods acquire thousands of images per patient, and it would be very difficult for a physician to accurately analyze all the photos in detail. In addition, human interpretation of medical images has inherent error, and is prone to failing to discover all data and imaging information [8]. With the advancement of computer systems, the expertise of radiologists can be used computationally to extract information from medical pictures. Human analysis is usually subjective and qualitative, and carelessness may occur. Furthermore, a comparative analysis is required between an image with nodule and another nodule pattern, and the human observer usually provides a qualitative response. The usage and extraction of quantitative or numerical features of images certainly require the utilization of computers. Given that most analyses which are conducted by humans are based on qualitative judgment, they will depend on time spent for a particular observer, or from one observer to another. This can be due to the lack of diligence or because of inadequate knowledge and also because of the diversity of education and the level of understanding or competence. Computers can apply a given procedure frequently in a short time and with high precision. In addition, the knowledge of many experts can be implemented computationally; thus computers are often trained in a specific field by a team of several human experts [9]. In this regard, the movement in medicine toward CAD systems, which is due to a quantitative analysis of CT lung images, can improve CT image analysis (which may contain pulmonary nodules), diagnosis of the disease, detection of small cancerous nodules (which with much difficulty can be detected by a physician) and reduce diagnostic time [10, 11]. In this paper, a variety of CAD systems and the steps to access lung CT databases are described. After mentioning the types of a nodule, the methods of extraction of lung and their importance investigated. In the next step, the diagnosis of the nodules and their classification described. Finally, the existing challenges are discussed. In the survey, we have attempted to present comprehensive but brief and easy to understand content, while highlighting the concept. II. Computer-Aided Detection
ACCEPTED MANUSCRIPT
RI PT
In the early 1980s at Kurt Rossmann Laboratories for Radiology Research at the University of Chicago's Department of Radiology, extensive and systematic studies of various Computer Aided Design (CAD) began. Recently, CAD has become an integral part of the clinical work to detect lung cancer in CT images, in many screening sites and hospitals [12]. Computer Aided Detection (CADe) and Computer Aided Diagnosis (CADx) systems are two types of CAD systems. CADe is a system for locating lesions in medical pictures, while the CADx system performs a diagnosis of lesions, for example, to make the distinction between benign and malignant tumors. CADe and CADx systems for the diagnosis of lung cancer in recent decades have been important areas of research. The CADe system does not show the radiological profile of tumors, and the CADx system does not detect nodules and does not have a good level of automation; as a result, these systems are not currently widely used in clinical settings [13]. Some CAD systems can be both CADx and CADe; in large part, this paper examines CADe systems. The diagnosis of a nodule by the CAD system consists of five important steps: database access, preprocessing, segmentation, analysis, and classification. The block diagram of these steps shown in Fig. 1. Analysis
Segmentation operation Extraction of the lungs
False positive reduction
Preprocessing
Classification
Nodule segmentation
SC
Database Acquisition
Feature extraction
M AN U
Fig. 1. Five important steps of CAD systems
1. Database Acquisition
AC C
EP
TE D
The development of CAD techniques for the diagnosis, classification, and evaluation of lung nodules can be facilitated through CT image databases. Some available databases include LIDC, LIDC-IDRI, ELCAP, LISS, and ANODE09. The Lung Image Database Consortium (LIDC) began to develop a web-accessible research resource in 2001. The database was combined with the Infectious Disease Research Institute (IDRI) database and created a general reference for the medical research community. The LIDC/IDRI database was completed in 2009 and resulted in a publicly available database, including 1018 chest CT scans with diagnostic and size reports, with four radiologists having examined each image [14]. The ELCAP database was established in 1991 by a group of doctors at the Cornell University Medical Center (now Weill Medical College of Cornell University), in collaboration with other institute specialists, and contains 50 low-dose lung CT images [15]. LISS is a public database containing 271 CT scans with 677 abnormal regions in them. In this database, all private data in the CT scans are eliminated or replaced with provisioned values in order to make it publically available, and it is promising to apply to CADe, diagnosis research, and medical education. This database is divided into two parts: 2-D and 3-D. The advantage of the LISS database is that it focuses on CT imaging signs instead of commonly considered lung nodules [16]. ANODE09 is an initiative from 2009 by Medical Center of Utrecht University and other organizations, to compare systems that automatically detect pulmonary nodules in lung CT scans in a common database with a single assessment protocol. Since the main purpose of ANODE09 is algorithm evaluation, it contains only 55 anonymous CT scans with the lung nodule specified in five images, and the rest of the images are for CADe system testing. The results of CADe systems on CT test images, including a list of nodule locations, are sent to this database and after being processed, the result of the test is published on the results pages of [17, 18]. 2. Preprocessing
Lung CT image preprocessing is performed to improve their quality and to achieve better results in the diagnosis of the lung nodule. The lungs contain several structures that can be confused with nodules, and the importance of this step is to enhance the image. The more common preprocessing methods in the literature are as follows. In [19], the following preprocessing methods are considered: Adaptive Median Filter, Alpha-Trimmed Mean Filter, Gaussian Filter, Gabor Filter, High Pass Filter, Laplacian Filter, and Bilateral Filter. H. Kim et al. [20] used the Median Filter (the kernel size is 3X3) and Top-hat filter to reduce image noise and to smooth it. Also, W. Suiyuan et al. [21] employed linear interpolation for preprocessing of lung CT image. Fig. 2 illustrates the preprocessing operation.
RI PT
ACCEPTED MANUSCRIPT
Fig. 2. Preprocessing CT Lung Image Processing. A) Original image B) Image with changes in opacity, color and gradient, and applying a gradient filter to enhance image quality. [22][20]
SC
A. Teramoto et al. [23], in order to differentiate the nodule from the normal lung tissue, used an erosion filter in preprocessing operations that shrink the image nodules and blood vessels. The erosion filter is one of the morphological filters that extend the distinction between image objects. 3. Segmentation Operation
TE D
M AN U
Image segmentation is the process of partitioning a digital image into multiple segments. The goal of segmentation is to simplify and change the representation of an image into something that is more meaningful and easier to analyze. The segmentation process in the CAD system for lung CT images consists of two parts: lung extraction and nodule segmentation. 3.1. Lung Extraction Since the lung is a complex organ that includes various structures such as vessels, gaps, and bronchus or pleura, which can be close to the lung nodule, lung extraction is one of the most important steps in CAD systems, which speeds up operations in the system, as well as increasing the accuracy of nodule diagnosis and segmentation. The nodules are classified into two categories in terms of their location in the lung [24]: non-attached nodules (for example wellcircumscribed nodules) and attached nodules (juxta-pleural, vascularized and pleural-tail).
EP
Fig. 3. Four examples of CT images that contain nodules: (a) Well-Circumscribed, (b) Pleural-Tail, (c) Juxta-Pleural, and (d) Vascularized. A black circle is drawn around the nodule center [24].
=
AC C
One of the important goals of lung imagery extraction is the separation of attached nodules to the lung wall, which may be removed during the processing operation. Accordingly, for this purpose, various algorithms are presented, which are classified as follows: thresholding, deformable models, shaped-based models, edge-based models [25]. M. Alilou et al. [26] employed a multiple thresholding method for lung extraction. To determine the desired threshold, assumed and be the mean gray-levels of the body and non-body voxels that segmented with threshold Ti. The new threshold (Ti+1) calculates via: .
The iterative updating of the new threshold is repeated until Ti+1 = Ti. The Hounsfield value of the air is chosen as the value of the initial threshold (T0 = -1000 HU). The lung segmentation process is shown in Fig. 4.
A)Main Image
B)Tresholding
C)Invert
D)Hole filling , Extract largest hole
E)AND(B,D)
F)Hole filling
ACCEPTED MANUSCRIPT Fig. 4. Lung segmentation process and the result
M AN U
SC
RI PT
M. Keshani et al. [27], introduced a lung extraction method, which segmented the attached nodules from lung wall. In this work, adaptive fuzzy thresholding was employed in the first step. In the second stage, 5×5 and 23×23 windows were applied to obtain a hole-free lung mask. In the third stage, 45 degrees rotated windows with dimensions 50×50 and 25×25 were applied to segment large and small sized non-isolated nodules connected to the chest wall. In the next step, a typical threshold was used to exclude areas outside of the thorax pixels, and bone parts pixels, which are located in the mid region of the original image. Finally, the created mask was given to the Active Cantor model as the input mask. R. Shojaii et al. [28] presented a lung segmentation technique based on the watershed algorithm, which is characterized by applying the internal and external markers to the gradient image of the lung region. In this work, the Rolling Ball filter was used to smooth the lines and fill the holes while maintaining the original boundaries. E. M. van Rikxoort et al. [29] used a hybrid method to segment lung from surrounding tissue. This method consists of four steps. The lung is separated in the first step using an automated 3D algorithm such as Region Growing and some morphological techniques. This step is fast, but it may be accompanied by some errors that are detected and corrected by using an error detection method. A more complex algorithm based on multi-atlas segmentation extracts lung regions. Finally, the results of the algorithm are evaluated by an error detection method to prevent possible errors. In another work, S. Shimoyama et al. [30] used an Active Contour algorithm for lung segmentation. For optimal lung segmentation, the previous lung slice, which will be similar to the current slice, is used as an input mask for the Active Contour algorithm, and it causes a better segmentation result. 3.2. Nodule segmentation Nodule segmentation algorithms have been used to determine the size of the nodule and its growth, as well as to determine the type of nodule [31]. The similarity of the nodule to structures such as vessels, chest, and surrounding tissue has been one of the nodule segmentation problems for CAD systems. Additionally, size, location, and contrast of nodules may make some problems for CAD systems. The categorization of nodules in terms of size, location, and density can be seen in Fig. 5 [32]. In CAD systems, the detached nodules and solid nodules are segmented with high accuracy, but for other nodules, their segmentation is associated with the false-positive increase.
Nodule
Location Deattached
Solid
GGO
Size Micronodule
TE D
Attached
Density
Mass
Fig. 5. Categorization of nodules in terms of size, location and density
AC C
EP
Nodule segmentation methods are classified in different approaches. A. Mittal [33] divided segmentation methods into four branches: Edge Detection, Region-Based Methods, Thresholding Technique, and Clustering Technique. The complete categorization of this article is illustrated in Fig. 6. All these segmentation methods have been used for lung nodule segmentation in the literature [34-38]. Edge Detection [34] Region Growing [33]
Segmentation Methods
Region Based Methods Splitting & Merging
Global Thresholding Thresholding Technique [35] Local Thresholding
K-mean Clustering [36] Clustering Technique C-mean Clustering [37] Fig. 6. Segmentation methods categorization as proposed in [33]
ACCEPTED MANUSCRIPT
Based on Discontinuity
Edge Based [34]
Threshold Based [35] Region Growing [33] Region Based
RI PT
Segmentation methods
In [39, 40], the segmentation methods are first divided into two groups, based on similarity and based on discontinuity, respectively, and then in [39], other segmentation methods are classified, as depicted in the Fig. 7. The segmentation methods have used for lung nodule segmentation in the literature [41-43].
Splitting & Merging
Based on Similarity
Neural Network Based [40, 41]
Wavelets Based [42]
SC
Special Theory
Clustering Based [36, 37]
Fig. 7. Segmentation methods categorization proposed in [39]
AC C
EP
TE D
M AN U
The importance of nodule segmentation with high accuracy has been considered in the literature. For instance, J. Dehmeshkiet al. [44] proposed a nodule segmentation technique based on a region growing algorithm. This region growing is operated at intervals of a volumetric mask that is created by first applying a district adaptive segmentation algorithm that identifies foreground and background regions at intervals using a precise window size. The foreground objects are then crammed to eliminate any holes, and a property map is generated to create a 3D mask. Then the mask is enlarged to contain the background, while excluding unwanted foreground regions. By excluding and confining the search volume, the mask is helpful to estimate parameters for subsequent region growing, and to verify dependableness. This technique was tested on 815 lung nodules, and a radiologist confirmed 84% of the results. M. Keshani et al. [45] identified nodules using extraction 2-D statistical characteristics and the 3-D anatomy via a SVM classifier. Then the nodule boundaries were extracted by the Active Contour algorithm. The proposed algorithm can segment solid and cavitary nodules which are connected to the lung wall. The Active Contour model which was used in this paper, presented in [46], employed a robust and efficient model for detecting thin edges and the boundary between nodule and lung. Results presented in [47] show the strength of this Active Contour algorithm versus noisy images. S. Diciotti et al. [48] presented a method based on local shape analysis which isolates and refines the boundary between nodule and vessel without any modification. In various papers, the shape-based method has been investigated and, in this paper, the power of this method is presented. Other nodule segmentation techniques such as Watershed [49] have been used and results have been presented. A comparison between three segmentation algorithms (Markov random field [50], Watershed, active contour) in nodule segmentation is illustrated in Fig. 8.
Fig. 8. Comparison between the three algorithms in nodule segmentation: a) Markov random field (yellow) b) Watershed (red) c) active contour (green).
4. Analysis After segmentation of candidate nodules, we are required to examine them. The study of nodule-like structures consists of two parts: false positive reduction and feature extraction. 4.1. False positive reduction:
ACCEPTED MANUSCRIPT
RI PT
One of the most important characteristics of CAD systems is the false positive rate, which indicates the power and reliability of a CAD system. This stage is one of the important components of the lung nodule detection system, which plays an important role in detecting lung cancer and early treatment [51]. A. Setio et al. [52] presented a false positive reduction method using multi-view ConvNets. One of the methods of false positive reduction is to remove structures with less similarity to the nodules, which is done by examining the features of each candidate nodule, such as the method used in [53], which specified the 2-D and 3-D features to eliminate false nodule candidates. Q. Dou et al. [51] proposed a new method based on CNN to reduce false positives. 4.2. Feature extraction For detecting cancerous nodules using apparent characteristics, candidate nodules feature extraction should be done. For instance, J. Lee et al. [53] calculates Multiple geometric parameters for each suspected lung lesion, including its shape, elongation, size, speculation, density, and other features, for rating each candidate nodule, which indicates its likelihood of actually being a lung nodule. N. Gurcan et al. [55] extracted the following features for each 3D object in order to classify candidate nodules: volume, surface area, average gray value, standard deviation, skewness, and kurtosis of the gray value histogram. Alilou et al. [26] computed a set of 17 2D and 3D features for each segmented and labeled candidate nodule. They grouped these features into the following four types: 3D geometrical, 3D intensitybased, 2D geometrical and 2D intensity-based features. 5. Classification
M AN U
SC
Many studies have been presented in the field of automatic nodule extraction by computer systems, which mainly use classification methods to divide nodule candidates into nodule and non-nodule. Classification algorithms typically use features, as described in the previous step, to detect nodules. The most common methods which are used in the literature include SVM [54, 55], Neural Networks [56], and CNN [57, 58]. III. Conclusions
TE D
Using computer systems and image processing for medical CT images analysis has progressed substantially in recent years, and in general, many published works have the potential to be used in medical practice. In this context, physicians need to learn more about operational computer systems in medical images processing, which would lead to using these systems for the early detection of lung cancer. However, in order to facilitate acceptance and use of these systems, the deficiencies of them must be addressed and eliminated. For this reason, close contact of developers and analysts with the medical community is necessary, so that awareness of the specific needs in CAD systems can be utilized to extend these systems. This will be accomplished with joint efforts between doctors, patients, engineers, and scientists. In this work, the use of CAD systems in lung CT image processing has been investigated, and the stages of processing in order for diagnostics and to extract lung nodules are presented. This information should be useful to researchers in this field and also to encourage physicians to use these systems. REFERENCES
1. Tsao, A. S., Scagliotti, G. V., Bunn Jr, P. A., Carbone, D. P., Warren, G. W., Bai, C., ... & Adusumilli, P. S.
EP
(2016). Scientific advances in lung cancer 2015. Journal of Thoracic Oncology, 11(5), 613-638.
AC C
2. J. website : www.cancer.org , 2017 , https://www.cancer.org/cancer/non-small-cell-lungcancer/about/key-statistics.html.". 3. Spiro, S. G., & Porter, J. C. (2002). Lung cancer—where are we today? Current advances in staging and nonsurgical treatment. American journal of respiratory and critical care medicine, 166(9), 1166-1196. 4. Ellis, P. M., & Vandermeer, R. (2011). Delays in the diagnosis of lung cancer. Journal of thoracic disease, 3(3), 183.
5. "American Cancer Society. (2017). Cancer fact and figures [Online]. Available https://www.cancer.org/cancer/non-small-cell-lung cancer/about/key-statistics.html pdf." 6. Knorst, M. M., Dienstmann, R., & Fagundes, L. P. (2003). Delay in the diagnosis and surgical treatment of lung cancer. Jornal de Pneumologia, 29(6), 358-364.
7. Bozcuk, H., & Martin, C. (2001). Does treatment delay affect survival in non-small cell lung cancer? A retrospective analysis from a single UK centre. Lung cancer, 34(2), 243-252. 8. Zhang, Y., Oikonomou, A., Wong, A., Haider, M. A., & Khalvati, F. (2017). Radiomics-based prognosis analysis for non-small cell lung cancer. Scientific reports, 7, 46349. 9. Sprindzuk, M. V., Kovalev, V. A., Snezhko, E. V., & Kharuzhyk, S. A. (2010). Lung cancer differential diagnosis based on the computer assisted radiology: The state of the art. Polish journal of radiology, 75(1), 67. 10. Doi, K. (2005). Current status and future potential of computer-aided diagnosis in medical imaging. The British journal of radiology, 78(suppl_1), s3-s19.
ACCEPTED MANUSCRIPT
11. van Ginneken, B., Schaefer-Prokop, C. M., & Prokop, M. (2011). Computer-aided diagnosis: how to move from the laboratory to the clinic. Radiology, 261(3), 719-732. 12. Doi, K. (2007). Computer-aided diagnosis in medical imaging: historical review, current status and future potential. Computerized medical imaging and graphics, 31(4-5), 198-211. 13. Firmino, M., Angelo, G., Morais, H., Dantas, M. R., & Valentim, R. (2016). Computer-aided detection (CADe) and diagnosis (CADx) system for lung cancer with likelihood of malignancy. Biomedical engineering online, 15(1), 2. 14. Armato, S. G., McLennan, G., Bidaut, L., McNitt Gray, M. F., Meyer, C. R., Reeves, A. P., ... & Kazerooni, E. A. (2011). The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Medical physics, 38(2), 915-931.
RI PT
15. "http://www.ielcap.org/ielcap.". 16. Han, G., Liu, X., Han, F., Santika, I. N. T., Zhao, Y., Zhao, X., & Zhou, C. (2015). The LISS—a public database of common imaging signs of lung diseases for computer-aided detection and diagnosis research and medical education. IEEE Transactions on Biomedical Engineering, 62(2), 648-656.
17. "https://anode09.grand-challenge.org/.". 18. Van Ginneken, B., Armato III, S. G., de Hoop, B., van Amelsvoort-van de Vorst, S., Duindam, T., Niemeijer, M.,
23. 24. 25. 26. 27. 28. 29.
30.
31. 32. 33.
34.
SC
M AN U
22.
TE D
21.
EP
20.
AC C
19.
... & Camarlinghi, N. (2010). Comparing and combining algorithms for computer-aided detection of pulmonary nodules in computed tomography scans: the ANODE09 study. Medical image analysis, 14(6), 707-722. Vijaya, G., & Suhasini, A. (2014). An adaptive preprocessing of lung CT images with various filters for better enhancement. Academic Journal of Cancer Research, 7(3), 179-184. Kim, H., Nakashima, T., Itai, Y., Maeda, S., kooi Tan, J., & Ishikawa, S. (2007, October). Automatic detection of ground glass opacity from the thoracic MDCT images by using density features. In 2007 International Conference on Control, Automation and Systems (pp. 1274-1277). IEEE. Suiyuan, W., & Junfeng, W. (2012, November). Pulmonary nodules 3D detection on serial CT scans. In 2012 Third Global Congress on Intelligent Systems (pp. 257-260). IEEE. Firmino, M., Morais, A. H., Mendoça, R. M., Dantas, M. R., Hekis, H. R., & Valentim, R. (2014). Computer-aided detection system for lung cancer in computed tomography scans: review and future prospects. Biomedical engineering online, 13(1), 41. Teramoto, A., & Fujita, H. (2013). Fast lung nodule detection in chest CT images using cylindrical noduleenhancement filter. International journal of computer assisted radiology and surgery, 8(2), 193-205. Farag, A. A., El Munim, H. E. A., Graham, J. H., & Farag, A. A. (2013). A novel approach for lung nodules segmentation in chest CT using level sets. IEEE Transactions on Image Processing, 22(12), 5202-5213. El-Baz, A., Elnakib, A., Abou El-Ghar, M., Gimel'farb, G., Falk, R., & Farag, A. (2013). Automatic detection of 2D and 3D lung nodules in chest spiral CT scans. International journal of biomedical imaging, 2013. Alilou, M., Kovalev, V., Snezhko, E., & Taimouri, V. (2014). A comprehensive framework for automatic detection of pulmonary nodules in lung CT images. Image Analysis & Stereology, 33(1), 13-27. Keshani, M., Azimifar, Z., Boostani, R., & Shakibafar, A. (2010, October). Lung nodule segmentation using active contour modeling. In 2010 6th Iranian Conference on Machine Vision and Image Processing (pp. 1-6). IEEE. Shojaii, R., Alirezaie, J., & Babyn, P. (2005, September). Automatic lung segmentation in CT images using watershed transform. In IEEE International Conference on Image Processing 2005 (Vol. 2, pp. II-1270). IEEE. van Rikxoort, E. M., de Hoop, B., Viergever, M. A., Prokop, M., & van Ginneken, B. (2009). Automatic lung segmentation from thoracic computed tomography scans using a hybrid approach with error detection. Medical physics, 36(7), 2934-2947. Shimoyama, S., Homma, N., Sakai, M., Ishibashi, T., & Yoshizawa, M. (2009, August). Auto-detection of nonisolated pulmonary nodules connected to the chest walls in X-ray CT images. In 2009 ICCAS-SICE (pp. 36723675). IEEE. Sluimer, I., Schilham, A., Prokop, M., & Van Ginneken, B. (2006). Computer analysis of computed tomography scans of the lung: a survey. IEEE transactions on medical imaging, 25(4), 385-405. Ye, X., Lin, X., Dehmeshki, J., Slabaugh, G., & Beddoe, G. (2009). Shape-based computer-aided detection of lung nodules in thoracic CT images. IEEE Transactions on Biomedical Engineering, 56(7), 1810-1820. Aman Mittal, (2016). A Review on Image Segmentation Techniques. International Journal of Recent Advances in Engineering & Technology (IJRAET), (pp. 61-65). Yang, B., Xiang, D., Yu, F., & Chen, X. (2018, March). Lung tumor segmentation based on multi-scale template matching and region growing. In Medical Imaging 2018: Biomedical Applications in Molecular, Structural, and Functional Imaging(Vol. 10578, p. 105782Q). International Society for Optics and Photonics.
ACCEPTED MANUSCRIPT
35. Singh, A. K., Saini, V., & Saini, L. M. (2017, June). ROI based detection of abnormalities in lungs using medical
42. 43.
44. 45.
46. 47. 48.
49. 50. 51. 52.
53.
54. 55.
56.
RI PT
41.
SC
40.
M AN U
39.
TE D
38.
EP
37.
AC C
36.
image processing. In 2017 International Conference on Intelligent Computing and Control Systems (ICICCS) (pp. 876-880). IEEE. Huang, S., Liu, X., Han, G., Zhao, X., Zhao, Y., & Zhou, C. (2018, February). 3D GGO candidate extraction in lung CT images using multilevel thresholding on supervoxels. In Medical Imaging 2018: Computer-Aided Diagnosis (Vol. 10575, p. 1057533). International Society for Optics and Photonics. Sangamithraa, P. B., & Govindaraju, S. (2016, March). Lung tumour detection and classification using EK-Mean clustering. In 2016 International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET) (pp. 2201-2206). IEEE. Nithila, E. E., & Kumar, S. S. (2016). Segmentation of lung nodule in CT data using active contour model and Fuzzy C-mean clustering. Alexandria Engineering Journal, 55(3), 2583-2588. Sivakumar, P., & Meenakshi, S. (2016). A review on image segmentation techniques. International Journal of Advanced Research in Computer Engineering & Technology (IJARCET), 5(3). Kandwal, R., Kumar, A., & Bhargava, S. (2014). Existing image segmentation techniques. International Journal of Advanced Research in Computer Science and Software Engineering, 4(4). Messay, T., Hardie, R. C., & Tuinstra, T. R. (2015). Segmentation of pulmonary nodules in computed tomography using a regression neural network approach and its application to the lung image database consortium and image database resource initiative dataset. Medical image analysis, 22(1), 48-62. Skourt, B. A., El Hassani, A., & Majda, A. (2018). Lung CT Image Segmentation using deep neural networks. Procedia Computer Science, 127, 109-113. Ceylan, M., ÖZBAY, Y., UÇAN, O. N., & Yildirim, E. (2010). A novel method for lung segmentation on chest CT images: complex-valued artificial neural network with complex wavelet transform. Turkish Journal of Electrical Engineering & Computer Sciences, 18(4), 613-624. Dehmeshki, J., Amin, H., Valdivieso, M., & Ye, X. (2008). Segmentation of pulmonary nodules in thoracic CT scans: a region growing approach. IEEE transactions on medical imaging, 27(4), 467-480. Keshani, M., Azimifar, Z., Tajeripour, F., & Boostani, R. (2013). Lung nodule segmentation and recognition using SVM classifier and active contour modeling: A complete intelligent system. Computers in biology and medicine, 43(4), 287-300. Lankton, S., & Tannenbaum, A. (2008). Localizing region-based active contours. IEEE transactions on image processing, 17(11), 2029-2039. Yezzi Jr, A., Tsai, A., & Willsky, A. (2002). A fully global approach to image segmentation via coupled curve evolution equations. Journal of Visual Communication and Image Representation, 13(1-2), 195-216. Diciotti, S., Lombardo, S., Falchini, M., Picozzi, G., & Mascalchi, M. (2011). Automated segmentation refinement of small lung nodules in CT scans by local shape analysis. IEEE Transactions on Biomedical Engineering, 58(12), 3418-3428. Tan, Y., Schwartz, L. H., & Zhao, B. (2013). Segmentation of lung lesions on CT scans using watershed, active contours, and Markov random field. Medical physics, 40(4). Tan, Y., Schwartz, L. H., & Zhao, B. (2013). Segmentation of lung lesions on CT scans using watershed, active contours, and Markov random field. Medical physics, 40(4). Dou, Q., Chen, H., Yu, L., Qin, J., & Heng, P. A. (2017). Multilevel contextual 3-D CNNs for false positive reduction in pulmonary nodule detection. IEEE Transactions on Biomedical Engineering, 64(7), 1558-1567. Setio, A. A. A., Ciompi, F., Litjens, G., Gerke, P., Jacobs, C., Van Riel, S. J., ... & van Ginneken, B. (2016). Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE transactions on medical imaging, 35(5), 1160-1169. Krishnamurthy, S., Narasimhan, G., & Rengasamy, U. (2016, December). An automatic computerized model for cancerous lung nodule detection from computed tomography images with reduced false positives. In International Conference on Recent Trends in Image Processing and Pattern Recognition (pp. 343-355). Springer, Singapore. Lee, I. J., Gamsu, G., Czum, J., Wu, N., Johnson, R., & Chakrapani, S. (2005). Lung nodule detection on chest CT: evaluation of a computer-aided detection (CAD) system. Korean journal of radiology, 6(2), 89-93. Orozco, H. M., Villegas, O. O. V., Sánchez, V. G. C., Domínguez, H. D. J. O., & Alfaro, M. D. J. N. (2015). Automated system for lung nodules classification based on wavelet feature descriptor and support vector machine. Biomedical engineering online, 14(1), 9. Shen, W., Zhou, M., Yang, F., Yang, C., & Tian, J. (2015, June). Multi-scale convolutional neural networks for lung nodule classification. In International Conference on Information Processing in Medical Imaging (pp. 588599). Springer, Cham.
ACCEPTED MANUSCRIPT
57. Pérez, G., & Arbeláez, P. (2017, November). Automated detection of lung nodules with three-dimensional
AC C
EP
TE D
M AN U
SC
RI PT
convolutional neural networks. In 13th International Conference on Medical Information Processing and Analysis (Vol. 10572, p. 1057218). International Society for Optics and Photonics. 58. Aliferis, C. F., Tsamardinos, I., Massion, P. P., Statnikov, A. R., Fananapazir, N., & Hardin, D. P. (2003, May). Machine Learning Models for Classification of Lung Cancer and Selection of Genomic Markers Using Array Gene Expression Data. In FLAIRS Conference (pp. 67-71).