Pulmonary nodule detection in medical images: A survey

Biomedical Signal Processing and Control 43 (2018) 138–147

Contents lists available at ScienceDirect

Biomedical Signal Processing and Control journal homepage: www.elsevier.com/locate/bspc

Review

Pulmonary nodule detection in medical images: A survey Junjie Zhang a , Yong Xia a,b,∗ , Hengfei Cui a,b , Yanning Zhang a a Shaanxi Key Lab of Speech & Image Information Processing (SAIIP), School of Computer Science and Engineering, Northwestern Polytechnical University, Xi’an 710072, China b Centre for Multidisciplinary Convergence Computing (CMCC), School of Computer Science and Engineering, Northwestern Polytechnical University, Xi’an 710072, China

a r t i c l e

i n f o

Article history: Received 2 May 2017 Received in revised form 24 November 2017 Accepted 21 January 2018 Keywords: Lung cancer Pulmonary nodule detection Computer-aided detection Medical image analysis

a b s t r a c t Malignant nodules may be due to primary tumors or a metastasis and, given the importance of diagnosing early primary lung tumors, the detection of pulmonary nodules is critical. Therefore, a lot of research efforts have been devoted to the research on computer-aided detection (CADe) schemes for pulmonary nodule detection. This survey sheds light on what CADe schemes are really implementing to detect pulmonary nodules and which will in turn assist radiologist for better diagnosis. This paper provides a systematic depiction of both feature engineering- and deep learning-based CADe schemes, including the categories of pulmonary nodules, modalities of chest medical imaging, commonly used datasets with nodule annotations, and related publications in recent years. A comprehensive comparison and analyses of pulmonary nodule detection schemes proposed in the last three years are also presented. © 2018 Elsevier Ltd. All rights reserved.

Contents 1. 2. 3. 4.

5. 6. 7.

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 Publications analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 CADe schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 Feature engineering based schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 4.1. Data acquisition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 4.2. Image preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 4.3. Lung parenchyma segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 4.4. Candidate nodules detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 4.5. False positive reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 Deep learning-based methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 Comparison and discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 Conclusion and future prospects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

1. Introduction Lung cancer, also known as lung carcinoma, is the number one cause of cancer deaths in both men and women worldwide with an incidence of 13% and death rate of 19.5% across all cancers [1,2]. About 70% of patients are diagnosed at the advanced stages of lung cancer and have a 5-year survival rate of approximately 16%.

∗ Corresponding author. E-mail address: [email protected] (Y. Xia). https://doi.org/10.1016/j.bspc.2018.01.011 1746-8094/© 2018 Elsevier Ltd. All rights reserved.

However, there is a greater chance that lung cancer can be treated effectively if it can be diagnosed at an early stage, accompanying with the 5-year survival rate of 70% [3]. Since malignant pulmonary nodules may be primary lung tumors or metastases, a reliable clinical treatment of lung cancer lies in the early and accurate detection of pulmonary nodules [4]. On chest CT scans, a pulmonary nodule usually refers to a “spot” of less than 3 cm in diameter on the lung [5,6]. Big irregular nodules have the risk of being cancerous. According to their spatial locations, pulmonary nodules can be mainly categorized into three groups [7,8]: solitary pulmonary nodules (SPNs), juxtapleural nodules (JPNs), and juxta-vascular nodules (JVNs). Seen

J. Zhang et al. / Biomedical Signal Processing and Control 43 (2018) 138–147

139

Fig. 1. Sample images of the three nodule categories. Solitary pulmonary nodules (SPNs) are rounded and independent, juxta-pleural nodules (JPNs) are adhere to the neighboring pleural surface, and juxta-vascular nodules (JVNs) adhere to vessels.

from the Transaxial view, SPNs are rounded, solid or subsolid, opaque, and isolated, JPNs usually adhere to the neighboring pleural surface, and JVNs adhere to vessels. Therefore, the detection of JVNs is more difficult due to their complex surroundings. A sample image for each type of pulmonary nodules is shown in Fig. 1. Medical imaging is non-invasive and effective for pulmonary nodule detection. Several imaging modalities, including the X-ray imaging, magnetic resonance imaging (MRI), computerized tomography (CT), positron emission tomography (PET), single-photon emission computed tomography (SPECT) and PET-CT, have been used to diagnose lung cancer [2,9]. Each imaging modality has its pros and cons. For instance, PET and SPECT can characterize the glucose metabolism, and hence are able to unveil the functional abnormality even before any structural changes become evident. MRI, CT and X-ray imaging are structural imaging modalities, which provide anatomical information of the human body. The dualmodality PET-CT imaging combines two scanners and can produce both functional PET images and structural CT images in a single scanning session [10,11]. Among these imaging modalities, CT is much simpler and less expensive than PET and MRI and can produce much higher sensitivity than X-ray imaging in lung nodule detection [2,12]. Therefore, most research efforts in this area have been devoted to lung nodule detection on chest CT scans. Fast-growing medical image data has become a useful resource to facilitate faster clinical solutions, however, which also leads to an intense pressure for radiologists. Therefore, computer-aided detection (CADe) schemes have been implemented to carry out the preliminary screening of pulmonary nodules by spotting suspicious lesions in medical images [13], and thereby help radiologists to make the discrimination of potential abnormalities. Such strategy can efficiently reduce radiologists workload and notably improve the accuracy of pulmonary nodule diagnosis [14]. Although there are several review papers on CADe schemes for pulmonary nodules, these reviews mainly focus on the CADe schemes proposed before 2014 [13]. This paper provides a more comprehensive survey of the state-of-art works and also helps researchers get a whole picture of CADe schemes.

2. Publications analysis In this paper, varied keywords are jointly used to carry out a comprehensive collection of CADe schemes proposed from January 2006 to March 2017. Logical expressions are (“lung” or “pulmonary” or “lungs”) and (“nodule” or “nodules” or “cancer” or “tumor” or “tumors”) and (“detection” or “detect” or “detected” or “detecting”

Fig. 2. Number of papers published annually on CADs schemes for pulmonary nodules.

Fig. 3. Authors and the related number of papers (1st author).

or “classification” or “segmentation” or “CAD” or “CADe”). These items are used as a part of the title or abstract of an academic paper, selected from the Science Direct, Wiley, IEEE Xplore, Ebsco, Springer, Science, Nature, PubMed, ACM, NCBI, BioMed central, Web of Science, and Engineering Village database. In the initial survey, we found 876 items in total. After illuminating duplicated and non-English items, books, academic dissertations, and patents, we obtained 302 journal and conference papers for analyses. The number of papers published annually is shown in Fig. 2, which reveals that the CADe scheme for pulmonary nodules is a hot research topic during the past 11 years. The top nine productive authors in this area are listed in Fig. 3. The top 10 conferences and journals, including the Proceedings of the International Society for Optical Engineering-Medical Imaging (SPIE-MI), Radiological Society of North America (RSNA), Medical Physics (MP), American Journal of Roentgenology (AJR), IEEE International Symposium on Biomedical Imaging (ISBI), Academic Radiology (AR), Medical Image Analysis (MIA), Computer Methods and Programs in Biomedicine (CMPB), European Journal of Radiology (EJR), and

140

J. Zhang et al. / Biomedical Signal Processing and Control 43 (2018) 138–147

Fig. 4. Top 10 conferences and journals and the corresponding number of papers they published.

IEEE International Conference on Image Processing (ICIP) and the corresponding number of papers they published are shown in Fig. 4. With the explosive growth of artificial intelligence algorithms and remarkable breakthroughs in machine learning, it is foreseeable that this area will draw more research attention in the computer vision and medical imaging communities. 3. CADe schemes According to the way of feature extraction, most CADe schemes for pulmonary nodules can be roughly categorized into two groups: feature engineering-based and deep learning-based schemes. Feature engineering-based schemes are based on handcrafted visual features and hence usually need empirically determine what kind of features should be extracted to better represent the underlying problem [15]. Furthermore, these schemes can be mainly divided into two successive stages: the segmentation-based and feature-based stages [11,16]. At the segmentation-based stage, lung parenchyma is first delineated, and then segmentation algorithms such as thresholding, region growing and clustering are used to detect possible nodules on the lung parenchyma. This stage contains a lot of false positive nodules [11,17–19]. At the feature-based stage, visual features are extracted to characterize each nodule candidate, and then a classifier is trained and used to differentiate nodules from non-nodule tissues [15,20–22]. Recently, deep learning techniques have been widely applied to many medical image analysis tasks [17–19,23], including pulmonary nodule detection [24]. Deep learning can be traced back to the seminal work done by Hinton in 2006 [25], which is inspired by the functioning of brain and designed by simulating the interaction of different neurons [26,27]. Since then, various deep learning techniques have been proposed in the literature, such as the deep belief network (DBN), deep convolutional neural networks (CNNs), and deep recurrent neural networks (DRNNs) [28,29]. They have distinct advantages over traditional methods in providing a uniform feature extraction-classification framework to free users from the troublesome handcrafted feature extraction [30–33]. 4. Feature engineering based schemes Feature engineering-based schemes for pulmonary nodules usually consists of five steps: data acquisition, image preprocessing, lung parenchyma segmentation, candidate nodules detection and false positives reduction. A diagram that summarizes these schemes is shown in Fig. 5, in which optional components are drawn in dot line. For example, chest CT scans are widely used by current CADe schemes, and PET scans are also used as auxiliary materials [22,34]. Therefore, both “PET scan” and “PET-CT scan” are drawn in dot lines. 4.1. Data acquisition Medical images acquisition is considered as the preliminary foundation to design an effective CADe system. There are several

publically available CT datasets, such as the Lung Image Datasets Consortium (LIDC) [35] and LUng Nodule Analysis 2016 (LUNA16) [36], for the research on pulmonary nodule detection. LIDC:As the most commonly used dataset for early diagnosis of lung cancer, LIDC currently contains 1010 documented patient records collected from five universities including Weill Cornell Medical College, University of California at Los Angeles, University of Chicago, University of Iowa, and University of Michigan. Each patient record contains at least one lung CT or X-ray scan with four sets of annotations from four radiologists. Nodules with a diameter larger than 3 mm were further annotated by the radiologist to provide the identification of malignancy [35,37]. LUNA16:The LUNA16 dataset was created by the challenge of LUng Nodule Analysis 2016. It contains 888 CT scans with relevant annotations, which were collected during a two-phase annotation procedure from four experienced radiologists. Each slice thickness is greater than 2.5 mm. Each radiologist marked lesions they identified as non-nodules, nodule <3 mm, and nodules ≥3 mm. The annotations that are not included in the reference standard are referred as irrelevant findings [36]. However, there are neither PET nor PET-CT image datasets available for this research. Therefore, many results are reported on private chest PET or PET-CT datasets [22,34]. 4.2. Image preprocessing Image preprocessing is performed to improve both the quality and interpretability of the original chest images. The most commonly used image preprocessing methods include the histogram equalization [38,39], median filters [40–42], Gaussian filter [43,44], Laplacian of Gaussian filter (LOG) [45,46], Wiener filter [47] and combined methods. Ogul et al. [48] combined gray-level transform with unsharp filtering for noise removal. Hong et al. [49] used an adaptive wiener filter which can adjust filtering effect to smoothing images. Ashwin et al. [50] combined the adaptive median filter with adaptive histogram equalization to denoise images. Teramoto et al. [22] proposed a novel active contour filter (ACF) to enhance the contrast of CT images. However, although image preprocessing is a critical step in intensity-based image analysis [51], the CADe schemes that use only geometric or texture features do not need to perform preprocessing [52–55]. 4.3. Lung parenchyma segmentation Lung parenchyma segmentation aims to extract the volume of lung parenchyma on preprocessed medical images while eliminating the impurities including image artefacts, trachea, bronchi, sternum, fat, and muscle. It is crucial for improving the reliability, accuracy and precision as well as decreasing computational complexity of CADe systems [3]. Many image segmentation methods can be used for lung parenchyma segmentation, including thresholding, shape based methods, edge based methods, morphological methods [2,47,48], and combined methods [56]. For instance, the lung parenchyma segmentation result shown in Fig. 6 was obtained in four steps: (a) each chest CT scan is re-sliced to a uniform voxel size of 1.0 mm × 1.0 mm × 1.0 mm; (b) each re-sliced CT scan is binarized using the OTSU algorithm on a slice-by-slice basis; (c) morphological closing is used to fill holes, and the morphological dilation with a disk structure element of size 5 is then carried out to produce a lung mask that covers as much lung tissues as possible; and (d) this mask is applied to the re-sliced CT scan to get the volume of lung. Zhao et al. [34] first adopted a dynamic threshold method to extract lung parenchyma, and then used an advanced watershed algorithm to obtain SPNs on co-registered CT and PET slices. Javaid et al. [39] first combined thresholding and morphological closing

J. Zhang et al. / Biomedical Signal Processing and Control 43 (2018) 138–147

141

Fig. 5. Flowchart of CAD schemes.

to extract lung regions on contrast enhanced chest CT scans, and then removed the bronchi attached to nodules using a thresholding method. John et al. [40] used a thoracic mask to deal with the thoracic cavity, and extracted the pulmonary parenchyma by designing a lung mask. Elsayed et al. [56] first adopted a 2D gray thresholding to obtain binary image, and then combined filling operations and mask operations to extract lung filed. Wang et al. [57] used an energy-based active contour model (ACM) to remove thoracic wall, and incorporated customized constraint energy items into the model architecture to discriminate lesions on chest CT scans. Leemput et al. [58] firstly generated two image masks using thresholding, and then combined morphological operations and connected regions to get the lung tissue. Novo et al. [44] first used

an enhancement filter generated by 3D Hessian matrix to identify round lesions, and then combined the shape index, curvedness approach and central adaptive mean approach to detect JPNs and small nodules effectively. Mehre et al. [59] first implemented both thresholding and 3D labeled connected components to extract lung volumes, then performed morphological closing to fill up holes on CT images, and finally used the Gaussian-filtered chain code for contour correction. Lakshmi et al. [41] adopted an image thresholding method to get lung parenchyma. Liu et al. [60] used region growing to segment pulmonary parenchyma. Han et al. [61] combined the high-level vector quantization (HVQ), connected components and morphological closing operation to extract lung parenchyma. Krishnamurthy et al. [8] jointly used region growing mask and lung

142

J. Zhang et al. / Biomedical Signal Processing and Control 43 (2018) 138–147

Fig. 6. Segmentation of lung parenchyma.

lobe mask to obtain lung volumes. Akram et al. [62] proposed a hybrid method that combines the thresholding, 3D region connection, morphological operations and lung mask for lung parenchyma segmentation. Teramoto et al. [63] used an erosion filter to shrink the chest CT images containing vessels and pulmonary nodules, and also enlarged the gaps among different objects. Santos et al. [64] first adopted both thresholding and connected regions to eliminate trachea and main bronchi firstly, and then used Gaussian mixture models (GMM) to segment internal lung tissues. Filho et al. [65] designed a novel segmentation method to extract, reconstruct and isolate lung parenchyma, and used a quadratic enhancement filter to highlight the thoracic structure. Choi et al. [66] first applied the optimal threshold method to get the initial region, and then applied both 3D connected regions and contour correction method to filter unrelated regions in turn to reserve lung lobes. Zhai et al. [20] performed lung parenchyma segmentation in four steps: (1) using morphological opening to remove image noise, (2) adopting the Otsu thresholding algorithm to segment the lung volumes, (3) employing the border tracing method to get the initial lung contour, and (4) using the adaptive border marching (ABM) algorithm to refine the segmentation results. These pulmonary parenchyma segmentation methods share many similarities, including (1) using thresholding to binarize images, (2) using 2D or 3D regions connection to remove thorax, and (3) using morphological operations to fill holes generated in the segmentation process and to reserve JPNs attached to pleura [64,67–72]. 4.4. Candidate nodules detection Candidate nodules detection is to extract pulmonary nodules from lung parenchyma by removing other lung tissues such as alveolar sacs, vessels, trachea, bronchi, fat, muscle, etc. [40]. The success of this procedure mainly depends on the accurate segmentation of lung parenchyma, and which will lay a great impact on the result of false positive reduction. As well known, this procedure is extremely complicated due to the complex inner structures and similar intensity of varied lung tissues [2,7]. The segmentation for SPNs is relatively easier than pulmonary nodules of two other types due to the simple surroundings. At present, some methods such as image thresholding, statistical and matching models are usually applied in CADe schemes to obtain candidate nodules [49,73,74] whats more, there are also other schemes to adopt such shape models as morphological approaches, template matching for the same purpose [75,76], and assembled algorithms are the most commonly used methods for candidate

nodules detection. Latest related works with an effective result are selected and analyzed as follows. Teramoto et al. [22] firstly used a type of contrast-enhancement filter named active contour filter to enhance lung regions on chest CT images, then the candidate nodules were extracted by a thresholding of enhanced images. At the same time, PET images were subsequently binarized using a pretrained thresholding to detect pulmonary nodules. Lu et al. [16] adopted a hybrid method combining by morphological operation, dot-enhancement using Hessian matrix, segmentation of fuzzy relationship, maximum algorithm of local density and geodesic distance map to detect candidate pulmonary nodules of different types, additionally, Hessian matrix is also used in other works to enhance nodular contour [56,77,78]. Similarly, Guilherme et al. [79] used three methods including region-growing, refined morphological operations and active contours to segment JPNs. Solid nodules were detected with an appropriate thresholding which was included in a representative sliding window, and sub-solid and non-solid nodules were enhanced with a multiscale LoG filter prior to candidate nodules detection. Javaid et al. [39] first broken off the connection between candidate nodules and vessels or thorax, and K-means algorithm was used for clustering the candidate pulmonary nodules in a way of 3D connections. John and Mini [40] first located candidate pulmonary nodules by using intermediate thresholding, and then obtained initial nodule candidates by combing both micro-level thresholding and binary image mask. Ogul et al. [48] firstly proposed a detection scheme combining by intensity and multi-scale for candidate nodules, then the LoG algorithm was implemented to extract blobs by using scale-space maxima. Leemput et al. [58] proposed a special LoG filter, in which an empirically established threshold was used to extract candidate pulmonary nodules. Yanagihara et al. [80] constructed three novel filters including radial suppression filter, moment-of-inertia filter and center displacement filter to get candidate nodules jointly, and the result shows that lung impurities were eliminated successfully. Liu et al. [60] proposed a model matching method named selective enhancement filter to extract candidate nodules. The 2D CT sequences were stacked for 3D representation using Hidden Conditional Random Field (HCRF) to detect candidate nodules. Mehre et al. [59] proposed a novel method using connected regions and a rule-based selection method combining by thresholds and morphological opening operation were proposed respectively to detect initial candidate nodules. Han et al. [61] first used HVQ to segment lung parenchyma, and then implemented the low-level VQ (LVQ) to extract the initial candidate nodule regions. Krishnamurthy et al. [8] adopted an automatic region-growing method using morphologi-

J. Zhang et al. / Biomedical Signal Processing and Control 43 (2018) 138–147

cal operation to detected well-circumscribed candidate nodules, and then processed JPNs and JVNs by using edge bridge successively, fill technique and 3D centroid-shift analysis. Akram et al. [62] first extracted region of interest (ROI) extracted by using a method of optimal thresholding, and then performed elongation for candidate nodules pruning. Zhai et al. [20] proposed a novel method combining by region-growth algorithm and inclusion rule was proposed to detect candidate nodules. Santos et al. [64] proposed a novel method on the basis of shape structure, the authors firstly eliminated vessels and bronchi of which the diameter is more than or equals to 30 mm, and then the Hessian matrix was used to detect lung tissues of blob-shaped structures. Yokota et al. [70] removed vessel regions removed by using a 3D line and dot enhancement filter respectively, then the candidate groundglass nodules (GGNs) were segmented by a suitable threshold. Nomura et al. [70] adopted a multi-scale method based on Hessian eigenvalues for the detection of pulmonary nodules due to the different nodule size. Hasanabadi et al. [71] applied a novel template matching algorithm in processed CT slices by thresholding for the pulmonary nodule detection. Manikandan and Devi [81] acquired local structure information of each voxel with the eigenvalue decomposition of Hessian matrix, and then detected candidate nodules by multi-scale dot enhancement filtering. Ogul et al. [48] adopted five convergence indexes including convergence index, adaptive ring filter, sliding band filter, iris filter and weighted convergence filter to lower the error rate of the detection for candidate nodules. Filho et al. [20] applied quality threshold (QT) cluster to segment lung parenchymal, and then used region-growing algorithm to detect candidate nodules. Choi and Choi [66] proposed a multi-scale dot enhancement filter to detect candidate nodules. According to their scheme, both eigenvalue decompositions of Hessian matrix and Gaussian image smoothing in multiple scales were carried out sequentially. Jacobs et al. [82] first proposed a method combining by double-threshold intensity mask, morphological erosion operation and connected regions for coarse candidate nodules detection, and then implemented another hybrid method based on both cubic volume of interest and morphological operations to extract candidate nodules robustly. 4.5. False positive reduction False positives reduction (FP-reduction) can be regarded as another complex procedure we should analyze and extract features of the candidate nodules, reduce the dimensionality of feature sets by using varied methods of feature selection, classify pulmonary nodules and non-nodules using machine learning techniques. The key purpose of the procedure is to further eliminate the false samples from candidate nodules. With respect to features, they can be categorized into three groups including intensity features, shape features, and texture features [2,3,13]. The classifiers mostly used for classification of candidate nodules are rule based method, artificial neural network (ANN) based classifiers, support vector machine (SVM), linear discriminant analysis (LDA) based classifiers, decision tree based methods, association based classifiers, boosting based methods. Then, some selected works are elaborated as follows. Teramoto et al. [22] first used shape features extracted from CT images and metabolic features extracted from PET images to eliminate the false positives (FPs), and then employed a rule-based SVM classifier for the classification of nodules from non-nodule tissues. Lu et al. [16] proposed a regression tree-based classifier to discriminate nodules and non-nodules. Han et al. [60] used a group of features, including the mean intensity value, intensity variance, kurtosis, minimum intensity value, maximum intensity value, size, overlap value, circular shape descriptor value and compactness, to improve the performance of FP-reduction. Jacobs et al. [82] used reserved candidates to train a SVM classifier with the radial basis

143

function (RBF) for further reducing the number of false positives. John and Mini [40] extracted the features, including the weighted centroid distance and mean intensity difference, to train a SVM classifier with the RBF-kernel and reported their performance of FP-reduction in the 10-fold cross validation. Lakshmi and Jeeva [41] extracted another group of features, including the area, solidity, eccentricity, extent, equivalent diameter, perimeter, major axis length, minor axis length, minimum intensity, maximum intensity and mean intensity, to describe pulmonary nodules, and then designed a novel feature-based classifier to remove non-nodule candidates. Sivakumar and Chandrasekar [42] calculated geometric and intensity features, including the area, diameter, perimeter, irregularity index, mean, equivalent circle diameter, centroid and eccentricity, and used an ANN as a classifier to eliminate false positive nodules, which reduced the false positive rate to 0.9%. Geometrical features such as size, compactness, circularity and sphericity were extracted to describe the candidate nodules, SVM classifier was trained by 4-fold cross-validation and then used to identify the performance [59]. Han et al. [61] used both rule-based filter and feature-based SVM for double screening of pulmonary nodules, and the most interesting trait is that 2D and 3D features were jointly extracted to reduce false positive nodules, similar multi-dimensionality features were also used in [62,72,83]. Akram et al. [62] extracted 2D and 3D features including intensity features and geometry features, then trained a SVM classifier to reduce the number of candidate nodules. Santos et al. [64] extracted six features including Shannon and Tsallis entropy and used a libSVM software based SVM classifier for the reduction of false positive nodules. Yokota et al. [70] extracted 7 density features, 5 shape features, GLCM (Gray Level Co-occurrence Matrix) features to describe lung tissues, and then an ANN was used to classify GGO regions and normal regions. Manikandan and Devi [81] extracted shape features based on surface constituent, and then trained an evolutionary SVM (ESVM) to classify candidate nodules into nodules and non-nodules. Except for the detection of SPNs, the method combining by Hessian matrix and dot filter has been proved that they can also be used to reduce the false positive nodules [77]. Ogul et al. [48] constructed eight feature sets including position features, texture features, intensity features, Gaussian features, detector features, gradient features and two new features which based on texture and gradient entropy were extracted, and then implemented dimensionality reduction to remove redundant features, finally, used SVM classifier to conduct the experimental verification. Filho et al. [65] used a step-wise discriminant analysis technique to select features, the most effective features selected by genetic algorithm (GA) was integrated into SVM for training and testing. Choi et al. [66] proposed a novel shape-based feature method to describe pulmonary nodules, and then applied SVM classifier to detect non-isolated nodules effectively. Jacobs et al. [82] extracted a set of 128 features including intensity, shape and a novel set of context features for the detection of subsolid candidate nodules, then free-response operating characteristic (FROC) and 10 GentleBoost classifiers (GB10) were used for double optimization of candidate nodules classification. Liu et al. [84] compared random forest (RF) with SVM classifier and the final result showed that RF can reduce false positive nodules more effectively. 5. Deep learning-based methods Except for natural images, deep learning techniques have also been applied in medical image analysis [56]. Data acquisition and image preprocessing are the fundamental procedures before feature extraction using DNNs, since the first two steps of feature engineering- and deep learning-based schemes are in common, the following parts will not elaborate them anymore. Whats more,

144

J. Zhang et al. / Biomedical Signal Processing and Control 43 (2018) 138–147

deep learning techniques are being developed, and there are no clear detailed theories to explain the availability and reliability, and different teams are trying to design their own suitable models, therefore, we cannot give a generic framework for deep learningbased schemes for pulmonary nodules detection. Teramoto et al. [22] carried out a CNN to extract both anatomical and metabolic features with CT-PET images, then combined with hand-crafted features to reduce false positive, the experimental results showed that more deep features can be effectively extracted using ensemble methods. Shen et al. [85] presented a novel CADe system for pulmonary nodules using a multi-crop convolutional neural network (MC-CNN). Salient information of pulmonary nodules is automatically extracted from training data, and different regions from convolutional feature maps are cropped and then applies max-pooling for several times. Setio et al. [86] proposed a multi-view convolutional network, it is fed with nodule candidates obtained by combining three candidate detectors which are respectively designed for solid, subsolid, and large nodules, to automatically extract discriminative features. The proposed architecture cascaded multiple streams of 2-D CNNs, and the outputs are combined using a dedicated fusion method to get the final classification. Anirudh et al. [87] presented that it would be better that employ 3D CNNs to learn highly discriminative features for nodule detection instead of hand-engineered ones such as geometric shape or texture. Ginneken et al. [21] combined fully connected layers of CNN and SVM to elevate the performance of nodule classification. Hua et al. [37] have proved that CAD schemes using DBNs or CNNs can achieve a better performance. Except for classification, some deep learning based schemes for pulmonary nodule detection have also been put forward recently. Nima and Kenji [88] carried out a comprehensive comparison between massive-training artificial neural networks (MTANNs) and different CNNs. The experimental results showed that with limited training data, MTANNs would be a suitable end-to-end machinelearning method for detection and classification of lesions that do not require high-level semantic features. Ypsilantis and Montana [89] designed a novel RNN named ReCTnet for the fully-automated detection of pulmonary nodules with CT scans. The paper indicated that the network can learn to distinguish nodules and normal structures at pixel level, and it simultaneously generate threedimensional probability maps highlighting areas that are likely to cover the objects of interest. Fu et al. [90] trained three CNNs using three different image types: lung CT images, the noduleenhanced images, and the blood vessel-enhanced images, and then hand-crafted features were assembled to train a SVM classifier for nodules detection. Hamidian et al. [91] trained a 3D CNN using volumes of interest (VOI) extracted from the LIDC dataset to automatically detect pulmonary nodules in chest CT images, and then converted the 3D CNN which has a fixed field of view to a 3D fully convolutional network (FCN) which can generate the score map for the entire volume efficiently in a single pass. And this screening FCN is used to generate difficult negative examples which are used to train a new discriminant CNN. 6. Comparison and discussion In Table 1, we compared 22 CADe schemes for pulmonary nodule detection published since 2014. The information of each scheme includes the publication year, first author, imaging modality, dataset, number of studies and nodules, 2D or 3D processing, methods and performance measured by accuracy, sensitivity, specificity, and FP/scan are commonly used. Accuracy gives the percentage of correctly detected samples, sensitivity gives the true positive rate, specificity gives the true negative rate, and FP/scan is the abbreviation of false positive per scan. Since CT can provide clearer anatomical structures of the lung than X-ray imaging and it is

less expensive and more assessable than PET, most researchers designed their algorithms for chest CT scans, except that Teramoto et al. [22] used both CT and PET slices and Ogul et al. [48] used a private dataset selected from the Japanese Society of Radiological Technology (JSRT) database [92]. Moreover, the LIDC dataset has been most commonly used due to its detailed annotation and a relatively large number of studies. The number of data is also important for designing CADe schemes, since it always requires a large number of data to train a classifier robustly. For instance, Ypsilantis et al. [89] used 1080 CT scans as training samples and resulted in a more reliable system than the one trained by Zhai et al. [20] on 19 CT scans. Although most schemes were designed to detect any type of nodules, some schemes focus more on a specific nodule type. For instance, Aresta et al. [79] aimed to detect juxta-pleural lung nodules only, and hence achieved a low sensitivity. Most early CADe schemes were designed to process chest images on a slice-by-slice basis, which are easy to implement but hard to use the spatial information among slices. In recent years, 3D and 2D–3D inputs have been accepted by more and more CADe schemes, such as those proposed by Fu et al. [90], Aresta et al. [79], Javaid et al. [39], and Mehre et al. [59], respectively. Traditionally, thresholding, morphological operation and region growing are often used for lung segmentation and candidate nodule detection [39,60,72,89,90,93], SVM and its variants are the most commonly used classifiers. In the last two years, deep learning techniques have been gradually applied to CADe schemes for pulmonary nodules. Teramoto et al. [22] only used deep CNN to extract deep features, and then input these features to a rule-based SVM classifier the final nodule and non-nodule classification; whereas Hamidian et al. [91], Fu et al. [90] and Ypsilantis et al. [89] applied deep CNN to both feature extraction and nodule identification. We believe that the next-generation of CADe systems will continue to advance the paradigm based on deep learning, due to the outstanding performance of deep learning in object localization and detection [94–97]. Meanwhile, dimensionality reduction [98] and data augmentation [86,88] have also been adopted to boost the performance of these schemes. Cascaded systems have been demonstrated to have better performance than a single classifier, since such systems use an ensemble of multiple classifiers [22,85]. Generally, CADe aims to screen chest images, end thus enables radiologists focus only on positive cases. In this scenario, false positive cases may be corrected by radiologists during the manual discrimination, whereas there may not be another opportunity to correct false negative cases. Therefore, CADe schemes are expected to achieve a high sensitivity, i.e. detecting as many pulmonary nodules as possible. It shows in Table 1 that the schemes proposed by Akram et al. [62], Manikandan and Devi [81] and Choi et al. [66] achieved a sensitivity of 95.31%, 98.3%, 97.5%, respectively. In addition, the CADe scheme designed by Manikandan et al. [81] achieved higher accuracy and sensitivity but lower specificity than the scheme developed by Filho et al. [65], and hence is more suitable for clinical applications. However, a too low specificity means that the scheme may produce too many false positive cases, which need manual inspection. A well-performed CADe scheme should maximize the specificity while keeping a high sensitivity. 7. Conclusion and future prospects This survey presents a comprehensive review of automated CADe schemes for pulmonary nodule detection on medical images. The cited articles are selected from several popular academic databases, which were published during the past 10 years, from January 2006 to March 2017. Both handcrafted feature- and deep learning-based CADe schemes are elucidated in great detail, and their advantages and disadvantages are analyzed and compared. In addition, several commonly used datasets and performance metrics

Table 1 Performance comparison of CADe schemes for pulmonary nodules. Authors

Modality

Datasets

Sample size

2D/3D

Main methods

2017

Liu [84]

CT

LIDC

80 scans, 978 nodules

2D

2017 2017

Hamidian [91] Fu [90]

CT CT

LIDC LIDC

534 scans 1366 nodules

2D 2D, 3D

2017

Aresta [79]

CT

LIDC

315 scans, 510 nodules

3D

2016 2016

Teramoto [22] Javaid [39]

CT, PET CT

Private LIDC

104 slices 1302 slices

2D, 3D 2D, 3D

2016 2016

Mehre [59] Ypsilantis [89]

CT CT

LIDC LIDC

97 scans 1080 scans

2D, 3D 3D

2015

Lu [16]

CT

LIDC

294 scans, 631 nodules

3D

2015

Liu [60]

CT

LIDC

59 nodules

3D

2015 2015 2015 2014 2014

Lakshmi [41] Han [61] Akram [62] Zhai [20] Santos [64]

CT CT CT CT CT

LIDC LIDC LIDC LIDC LIDC

279 scans, 72 nodules 490 nodules 84 scans 19 scans 28 scans

2D 2D 2D 2D, 3D 3D

2014 2014

Yokota [70] Alilou [72]

CT CT

LIDC LIDC

– 60 patients, 211 nodules

3D 2D,3D

2014 2014

Manikandan [81] Oul [48]

CT X-ray

LIDC JSRT, private

– 140 nodules; 417 nodules

3D 2D

2014

Filho [65]

CT

LIDC

140 scans

3D

2014

Choi [66]

CT

LIDC

–

3D

2014

Jacobs [82]

CT

LIDC

20,000 slices

2D

Thresholding, fuzzy C-means, hand-crafted feature extraction, random forest 3D FCN, 3D CNN Thresholding, 2D CNN, SVMMorphological operation, 3D region growing, hand-crafted feature extraction Region growing, morphological operation, active contour, multiscale LoG, rule-based classifier, SVM Active contour filter, SVM, 2D CNN, rule-based classifier Thresholding, SVM, morphological operation, K-means clustering Thresholding, SVM, morphological operation Thresholding, flood filling, morphological operation, RNN-CNN Morphological operation, dot enhancement, regression tree, local density maximum algorithm, fuzzy connection Region growing, thresholding, hidden conditional random field Thresholding, ANN, morphological operation Hierarchical vector quantization, SVM Thresholding, hole filling, contour correction, SVM Region growing, fuzzy min–max, K-means clustering, ANN Region growing, Hessian matrix, Gaussian mixture models, SVM 3D line filter, thresholding, ANN Region growing, thresholding, morphological operation, SVM Hessian matrix, dot enhancement, evolutionary SVM Local contrast enhancement, local normalization, LoG filter, convergence index filter Quadratic enhancement, Gaussian filter, median filter, quality threshold cluster, region growing, SVM Thresholding, Hessian matrix, 3-D connected component, SVM multi-scale dot enhancement Connected component, morphological operation, linear discriminant classifier

Performance metrics Acc

Sen

Spe

FP/s

93.20%

92.40%

94.80%

4.5

– 90.90%

80% –

– –

15.28 4

–

57.40%

–

4

– 96.22%

90.10% 91.65%

– –

4.9 3.19

– –

92.91% 90.50%

– –

3 4.5

–

85.20%

–

3.13

89.30%

1.2

92.20% – 97.52% – –

– 82.70% 95.31% 84% 80.50%

– – 99.73% – –

– 4 – 2.6 1.17

–

82.10% 80%

–

6.7 3.9

87.50% –

98.30% 80%;76%

77.60% –

11 6.4; 7.6

97.55%

85.91%

97.70%

1.82

–

97.50%

–

6.76

–

80%

–

1

J. Zhang et al. / Biomedical Signal Processing and Control 43 (2018) 138–147

Year

145

146

J. Zhang et al. / Biomedical Signal Processing and Control 43 (2018) 138–147

are also elaborated. In the future, the research on pulmonary nodule detection should mainly focus on: (1) developing novel CADe schemes based on the latest advances in artificial intelligence, such as deep learning, deep reinforce learning and transfer learning; (2) offering a clinically relevant explanation to the features discovered by various learning algorithms; (3) improving the detection of micronodules, whose diameters are less than 3 mm, and ground glass pulmonary opacity characterized by a slight increase in lung density; and (4) Jointly using clinical records and medical images for multimodality analysis. We believe that the fast progress in this area will enable automated, reliable and accurate pulmonary nodule detection and thus deliver early diagnosis and treatment to improve survival of patients with lung cancer. Acknowledgements This work was supported in part by the National Natural Science Foundation of China under Grants 61771397, 61471297 and 61401209, in part by the Key Projects of National Natural Science Foundation of China under Grants 61231016, and in part by the China 863 Program under Grants 2015AA016402. References [1] J. Ferlay, I. Soerjomataram, R. Dikshit, S. Eser, C. Mathers, M. Rebelo, D.M. Parkin, D. Forman, F. Bray, Cancer incidence and mortality worldwide: sources, methods and major patterns in globocan 2012, Int. J. Cancer 136 (5) (2015) E359–386. [2] K. Bhavanishankar, M.V. Sudhamani, Techniques for detection of solitary pulmonary nodules in human lung and their classifications – a survey, Int. J. Cybern. Informat. 4 (1) (2015) 27–40. [3] D.R. Baldwin, Prediction of risk of lung cancer in populations and in pulmonary nodules: significant progress to drive changes in paradigms, Lung Cancer 89 (1) (2015) 1–3. [4] A.A. Kohan, J.A. Kolthammer, J.L. Vercher-Conejero, C. Rubbert, S. Partovi, R. Jones, K.A. Herrmann, P. Faulhaber, N staging of lung cancer patients with PET/MRI using a three-segment model attenuation correction algorithm: initial experience, Eur. Radiol. 23 (11) (2013) 3161–3169. [5] M.E. Callister, D.R. Baldwin, How should pulmonary nodules be optimally investigated and managed? Lung Cancer 91 (2016) 48–55. [6] A. Asija, R. Manickam, W.S. Aronow, D. Chandy, Pulmonary nodule: a comprehensive review and update, Hosp. Pract. 42 (3) (2014) 7–16. [7] S.L.A. Lee, A.Z. Kouzani, E.J. Hu, Automated detection of lung nodules in computed tomography images: a review, Mach. Vision Appl. 23 (1) (2012) 151–163. [8] K. Senthilkumar, N. Ganesh, R. Umamaheswari, Three-dimensional lung nodule segmentation and shape variance analysis to detect lung cancer with reduced false positives, Proc. Inst. Mech. Engs. Part H J. Eng. Med. 230 (1) (2016) 58–70. [9] J. Conway, Lung imaging – two dimensional gamma scintigraphy, SPECT, CT and PET, Adv. Drug Deliv. Rev. 64 (4) (2012) 357–368. [10] A. Teramoto, H. Fujita, Proceedings of SPIE - The International Society for Optical Engineering 83152V, Pulmonary nodule detection in PET/CT images: improved approach using combined nodule detection and hybrid FP reduction (2012). [11] V. Ambrosini, S. Nicolini, P. Caroli, C. Nanni, A. Massaro, M.C. Marzola, D. Rubello, S. Fanti, PET/CT imaging in different types of lung cancer: an overview, Eur. J. Radiol. 81 (5) (2012) 988–1001. [12] A. Cieszanowski, A. Lisowska, M. Dabrowska, P. Korczynski, M. Zukowska, I.P. Grudzinski, R. Pacho, O. Rowinski, R. Krenke, MR imaging of pulmonary nodules: detection rate and accuracy of size estimation in comparison to computed tomography, PLOS ONE 11 (6) (2016) e0156272. [13] I.R.S. Valente, E.C. Neto, V.H.C.D. Albuquerque, Automatic 3d pulmonary nodule detection in CT images, Comput. Methods Programs Biomed. 124 (2016) 91–107. [14] B. Sahiner, H.P. Chan, L.M. Hadjiiski, P.N. Cascade, E.A. Kazerooni, A.R. Chughtai, C. Poopat, T. Song, L. Frank, J. Stojanovska, Effect of CAD on radiologists’ detection of lung nodules on thoracic CT scans: analysis of an observer performance study by nodule size, Acad. Radiol. 16 (12) (2009) 1518–1530. [15] I.D. Trier, A.K. Jain, T. Taxt, Feature extraction methods for character recognition – a survey, Pattern Recogn. 29 (4) (1996) 641–662. [16] L. Lu, Y. Tan, L.H. Schwartz, B. Zhao, Hybrid detection of lung nodules on CT scan images, Med. Phys. 42 (9) (2015) 5042–5054. [17] G. Litjens, T. Kooi, B.E. Bejnordi, S. Aaa, F. Ciompi, M. Ghafoorian, V.D.L. Jawm, G.B. Van, C.I. Snchez, A survey on deep learning in medical image analysis, Med. Image Anal. 42 (9) (2017) 60–88. [18] M. Lai, Deep Learning for Medical Image Segmentation, 2015 arXiv:1505.02000.

[19] A. Esteva, B. Kuprel, R.A. Novoa, J. Ko, S.M. Swetter, H.M. Blau, S. Thrun, Dermatologist-level classification of skin cancer with deep neural networks, Nature 542 (7639) (2017) 115–118. [20] Z. Zhai, D. Shi, Y. Cheng, H. Guo, Computer-aided detection of lung nodules with fuzzy min–max neural network for false positive reduction, International Conference on Intelligent Human–Machine Systems and Cybernetics (2014) 66–69. [21] B.V. Ginneken, A.A.A. Setio, C. Jacobs, F. Ciompi, Off-the-shelf convolutional neural network features for pulmonary nodule detection in computed tomography scans, IEEE International Symposium on Biomedical Imaging (2015) 286–289. [22] A. Teramoto, H. Fujita, O. Yamamuro, T. Tamaki, Automated detection of pulmonary nodules in PET/CT images: ensemble false-positive reduction using a convolutional neural network technique, Med. Phys. 43 (6) (2016) 2821–2827. [23] O. Ronneberger, P. Fischer, T. Brox, U-net: convolutional networks for biomedical image segmentation, International Conference on Medical Image Computing and Computer-Assisted Intervention (2015) 234–241. [24] M. Braverman, Poly-logarithmic independence fools bounded-depth boolean circuits, Commun. ACM 54 (4) (2011) 108–115. [25] G.E. Hinton, R.R. Salakhutdinov, Reducing the dimensionality of data with neural networks, Science 313 (5786) (2006) 504–507. [26] H. Wang, B. Raj, ***On the origin of deep learning, arXiv:1702.07800. [27] J. Schmidhuber, Deep learning in neural networks: an overview, Neural Netw. 61 (2015) 85–117. [28] V. Sze, Y. H. Chen, T. J. Yang, J. Emer, ***Efficient processing of deep neural networks: A tutorial and survey, arXiv:1703.09039. [29] Y. Lecun, Y. Bengio, G. Hinton, Deep learning, Nature 521 (7553) (2015) 436–444. [30] G. Hinton, L. Deng, D. Yu, G.E. Dahl, A.R. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T.N. Sainath, Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups, IEEE Signal Process. Mag. 29 (6) (2012) 82–97. [31] D. Yu, L. Deng, F. Seide, The deep tensor neural network with applications to large vocabulary speech recognition, IEEE Trans. Audio Speech Lang. Process. 21 (2) (2013) 388–396. [32] K. Yu, Large-scale deep learning at Baidu, ACM International Conference on Conference on Information and Knowledge Management (2013) 2211–2212. [33] Y. Bengio, Deep learning of representations: looking forward, International Conference on Statistical Language and Speech Processing (2013) 1–37. [34] J. Zhao, G. Ji, Y. Qiang, X. Han, B. Pei, Z. Shi, ***A new method of detecting pulmonary nodules with pet/ct based on an improved watershed algorithm, Plos One 10 (4). [35] A.S. 3Rd, G. Mclennan, L. Bidaut, M.F. Mcnitt-Gray, C.R. Meyer, A.P. Reeves, B. Zhao, D.R. Aberle, C.I. Henschke, E.A. Hoffman, The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on ct scans, Med. Phys. 38 (2) (2011) 915–931. [36] S. Aaa, A. Traverso, B.T. De, B. Msn, B. Cvd, P. Cerello, H. Chen, Q. Dou, M.E. Fantacci, B. Geurts, Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the luna16 challenge, Med. Image Anal. 42 (2017) 1–13. [37] K.L. Hua, C.H. Hsu, S.C. Hidayati, W.H. Cheng, Y.J. Chen, Computer-aided classification of lung nodules on computed tomography images via deep learning technique, Oncotargets Therapy 8 (2015) 2015–2022. [38] B. Nemade, D. Chaudhari, C. B. Lahoti, ***Advanced computerized scheme for detection of lung nodules by incorporating vde image, International Journal of Advanced Research in Computer and Communication Engineering 4 (2). [39] M. Javaid, M. Javid, M.Z. Rehman, S.I. Shah, A novel approach to CAD system for the detection of lung nodules in ct images, Comput. Methods Programs Biomed. 135 (2016) 125–139. [40] J. John, M.G. Mini, Multilevel thresholding based segmentation and feature extraction for pulmonary nodule detection ? Proc. Technol. 24 (2016) 957–963. [41] N.A. Lakshmi, J.B. Jeeva, A computer aided diagnosis for detection and classification of lung nodules, IEEE International Conference on Intelligent Systems and Control (2015) 1–5. [42] S. Sivakumar, D.C. Chandrasekar, Lung nodule detection using fuzzy clustering and support vector machines, Int. J. Eng. Technol. 5 (1) (2013) 179–185. [43] V. Vignesh, K. Kothavari, Classification and detection of lung nodules using virtual dual energy in CXR images, International Conference on Green Computing Communication and Electrical Engineering (2014) 1–6. [44] J. Novo, L. Gonalves, A.M. Mendona, A. Campilho, 3d lung nodule candidate detection in multiple scales, in: International Conference on Machine Vision Applications (2015) 61–64. [45] S.V. Fotin, A. P. Reeves, A. M. Biancardi, D. F. Yankelevitz, C. I. Henschke, ***A multiscale laplacian of gaussian filtering approach to automated pulmonary nodule detection from whole-lung low-dose ct scans, SPIE Medical imaging 72601Q. [46] S. Diciotti, G. Picozzi, M. Falchini, M. Mascalchi, N. Villari, G. Valli, 3-d segmentation algorithm of small lung nodules in spiral ct images, IEEE Trans. Inform. Technol. Biomed. 12 (1) (2008) 7–19. [47] G. S. Nair, S. A. D. Dr., ***Automated lung nodule detection method for surgical preplanning, Journal of Electronics and Communication Engineering 9(3)(2014)19-23.

J. Zhang et al. / Biomedical Signal Processing and Control 43 (2018) 138–147 [48] B.B. Ogul, P. Kosucu, A. zam, S.D. Kanik, Lung nodule detection in X-ray images: a new feature set, Med. Biol. Eng. Comput. 45 (2015) 150–155. [49] S. Hong, C. Li, L. Yang, A detection approach for solitary pulmonary nodules based on CT images, International Conference on Computer Science and Network Technology (2013) 1253–1257. [50] S. Ashwin, J. Ramesh, S.A. Kumar, K. Gunavathi, Efficient and reliable lung nodule detection using a neural network based computer aided diagnosis system, International Conference on Emerging Trends in Electrical Engineering and Energy Management (2013) 135–142. [51] T. Tokisa, N. Miyake, S. Maeda, H. Kim, J.K. Tan, S. Ishikawa, S. Murakami, T. Aoki, Detection of lung nodule on temporal subtraction images based on artificial neural network, Int. J. Fuzzy Logic Intell. Syst. 12 (2) (2012) 137–142. [52] G. Qiao, Y. Hong, T. Liu, J. Ou, Automated pulmonary nodule detection system in computed tomography images: a hierarchical block classification approach, Entropy 15 (2) (2013) 507–523. [53] A. Tartar, N. Kili, A. Akan, A new method for pulmonary nodule detection using decision trees, International Conference of the IEEE Engineering in Medicine and Biology Society (2013) 7355–7359. [54] G. Narasimhan, Texture pattern based lung nodule detection (TPLND) technique in ct images, Int. Rev. Comput. Softw. 9 (3) (2014) 415–426. [55] J. Bai, X. Huang, S. Liu, Q. Song, R. Bhagalia, Learning orientation invariant contextual features for nodule detection in lung CT scans, IEEE International Symposium on Biomedical Imaging (2015) 149–166. [56] O. Elsayed, K. Mahar, M. Kholief, H.A. Khater, Automatic detection of the pulmonary nodules from CT images, SAI Intelligent Systems Conference (2015) 742–746. [57] B. Wang, X. Tian, Q. Wang, Y. Yang, H. Xie, S. Zhang, L. Gu, Pulmonary nodule detection in ct images based on shape constraint CV model, Med. Phys. 42 (3) (2015) 1241–1254. [58] S.V.D. Leemput, F. Dorssers, B.E. Bejnordi, A novel spherical shell filter for reducing false positives in automatic detection of pulmonary nodules in thoracic ct scans, SPIE Medical Imaging, vol. 94142P (2015). [59] S.A. Mehre, S. Mukhopadhyay, A. Dutta, N.C. Harsha, A.K. Dhara, N. Khandelwal, An automated lung nodule detection system for CT images using synthetic minority oversampling, SPIE Medical Imaging, vol. 97850H (2016). [60] Y. Liu, Z. Wang, M. Guo, P. Li, Hidden conditional random field for lung nodule detection, IEEE International Conference on Image Processing (2015) 3518–3521. [61] H. Han, L. Li, F. Han, B. Song, W. Moore, Z. Liang, Fast and adaptive detection of pulmonary nodules in thoracic CT images using a hierarchical vector quantization scheme, IEEE J. Biomed. Health Informat. 19 (2) (2015) 648–659. [62] S. Akram, M.Y. Javed, M.U. Akram, U. Qamar, A. Hassan, Pulmonary nodules detection and classification using hybrid features from computerized tomographic images, J. Med. Imaging Health Informat. 6 (8) (2016) 252–259. [63] A. Teramoto, H. Fujita, Fast lung nodule detection in chest CT images using cylindrical nodule-enhancement filter, Int. J. Comput. Assist. Radiol. Surg. 8 (2) (2013) 193–205. [64] A.M. Santos, A.O.D.C. Filho, A.C.D. Paiva, M. Gattass, Automatic detection of small lung nodules in 3d CT data using Gaussian mixture models, Tsallis entropy and SVM, Eng. Appl. Artif. Intell. 36(C (2014) 27–39. [65] D.C.F. Ao, W.B. de Sampaio, A.C. Silva, A.C. de Paiva, R.A. Nunes, M. Gattass, Automatic detection of solitary lung nodules using quality threshold clustering, genetic algorithm and diversity index, Artif. Intell. Med. 60 (3) (2014) 165–177. [66] W.J. Choi, T.S. Choi, Automated pulmonary nodule detection based on three-dimensional shape-based feature descriptor, Comput. Methods Programs Biomed. 113 (1) (2014) 37–54. [67] Y. Nomura, M. Nemoto, Y. Masutani, S. Hanaoka, T. Yoshikawa, S. Miki, E. Maeda, N. Hayashi, N. Yoshioka, K. Ohtomo, Reduction of false positives at vessel bifurcations in computerized detection of lung nodules, J. Biomed. Graph. Comput. 4 (3) (2014) 887–897. [68] E.E. Nithila, S.S. Kumar, Automatic detection of solitary pulmonary nodules using swarm intelligence optimized neural networks on CT images, Eng. Sci. Technol. 20 (3) (2016) 1374–1377. [69] P. Kamra, R. Vishraj, Kanica, S. Gupta, Performance comparison of image segmentation techniques for lung nodule detection in CT images, International Conference on Signal Processing, Computing and Control (2016) 302–306. [70] K. Yokota, S. Maeda, H. Kim, J.K. Tan, S. Ishikawa, R. Tachibana, Y. Hirano, S. Kido, Automatic detection of GGO regions on CT images in LIDC dataset based on statistical features, International Symposium on Soft Computing and Intelligent Systems (2014) 1374–1377. [71] H. Hasanabadi, M. Zabihi, Q. Mirsharif, Detection of pulmonary nodules in CT images using template matching and neural classifier, J. Adv. Comput. Res. 5 (1) (2014) 19–28. [72] M. Alilou, A comprehensive framework for automatic detection of pulmonary nodules in lung CT images, Image Anal. Stereol. 33 (1) (2014) 13–27.

147

[73] Y. Liu, J. Yang, D. Zhao, J. Liu, ***A study of pulmonary nodule detection in three-dimensional thoracic ct scans (2010) 481–484. [74] W. Ying, J. Tong, M.X. Lin, Autonomous detection of solitary pulmonary nodules on CT images for computer-aided diagnosis, Chinese Control and Decision Conference (2011) 4054–4059. [75] T. Messay, R.C. Hardie, S.K. Rogers, A new computationally efficient CAD system for pulmonary nodule detection in CT imagery, Med. Image Anal. 14 (3) (2010) 390–406. [76] M. Assefa, I. Faye, A.S. Malik, M. Shoaib, Lung nodule detection using multi-resolution analysis, ICME International Conference on Complex Medical Engineering (2013) 457–461. [77] L. Liu, X. Wang, Y. Li, L. Wang, Solitary pulmonary nodules detection based on dot-filter and region growing, J. Chem. Pharmaceut. Res. 6 (6) (2014) 1176–1181. [78] G. Orban, G. Horvath, Lung nodule detection on digital tomosynthesis images: a preliminary study, IEEE International Symposium on Biomedical Imaging (2014) 141–144. [79] G. Aresta, A. Cunha, A. Campilho, Detection of juxtapleural lung nodules in computed tomography images, SPIE Medical Imaging, vol. 101343N (2017). [80] T. Yanagihara, H. Takizawa, Pulmonary nodule detection from X-ray CT images based on region shape analysis and appearance-based clustering, Algorithms 8 (2) (2015) 209–223. [81] N. Manikandan, K.U.K. Devi, Automatic detection and classification of pulmonary nodules on CT images, Int. J. Eng. Techn. Res. 2 (5) (2014) 208–214. [82] C. Jacobs, E.M. van Rikxoort, T. Twellmann, E.T. Scholten, P.A. de Jong, J.M. Kuhnigk, M. Oudkerk, H.J. de Koning, M. Prokop, C. Schaeferprokop, Automatic detection of subsolid pulmonary nodules in thoracic computed tomography images, Med. Image Anal. 18 (2) (2014) 374–384. [83] P. Eskandarian, J. Bagherzadeh, Computer-aided detection of pulmonary nodules based on SVM in thoracic ct images, Information and Knowledge Technology (2015) 1–6. [84] J.K. Liu, H.Y. Jiang, M.D. Gao, C.G. He, Y. Wang, P. Wang, H. Ma, Y. Li, An assisted diagnosis system for detection of early pulmonary nodule in computed tomography images, J. Med. Syst. 41 (2) (2017) 30. [85] W. Shen, M. Zhou, F. Yang, D. Yu, D. Dong, C. Yang, Y. Zang, J. Tian, Multi-crop convolutional neural networks for lung nodule malignancy suspiciousness classification, Pattern Recogn. 61 (2017) 663–673. [86] A.A. Setio, F. Ciompi, G. Litjens, P. Gerke, C. Jacobs, R.S. Van, W.M. Winkler, M. Naqibullah, C. Sanchez, G.B. Van, Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks, IEEE Trans. Med. Imaging 35 (5) (2016) 1160–1169. [87] R. Anirudh, Lung nodule detection using 3d convolutional neural networks trained on weakly labeled data, SPIE Medical Imaging, vol. 978532 (2016). [88] N. Tajbakhsh, K. Suzuki, Comparing two classes of end-to-end machine-learning models in lung nodule detection and classification: Mtanns vs. CNNS, Pattern Recogn. 63 (2016) 476–486. [89] P. P. Ypsilantis, G. Montana, ***Recurrent convolutional networks for pulmonary nodule detection in ct imaging, arXiv:1609.09143. [90] L. Fu, J. Ma, Y. Ren, Y.S. Han, J. Zhao, Automatic detection of lung nodules: false positive reduction using convolution neural networks and handcrafted features, SPIE Medical Imaging, vol. 101340A (2017). [91] S. Hamidian, B. Sahiner, N. Petrick, A. Pezeshk, 3d convolutional neural network for automatic detection of lung nodules in chest ct, SPIE Medical Imaging, vol. 1013409 (2017). [92] J. Shiraishi, S. Katsuragawa, J. Ikezoe, T. Matsumoto, T. Kobayashi, K. Komatsu, M. Matsui, H. Fujita, Y. Kodera, K. Doi, Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists’ detection of pulmonary nodules, AJR Am. J. Roentgenol. 174 (1) (2000) 71–74. [93] S.J.v. Riel, C.I. Snchez, A.A. Bankier, D.P. Naidich, J. Verschakelen, E.T. Scholten, P.A. de Jong, C. JacobsJacobs, R.E. Van, L. Petersbax, Observer variability for classification of pulmonary nodules on low-dose CT images and its effect on nodule management, Radiology 277 (3) (2015) 863–871. [94] R. Girshick, J. Donahue, T. Darrell, J. Malik, Region-based convolutional networks for accurate object detection and segmentation, IEEE Trans. Pattern Anal. Mach. Intell. 38 (1) (2015) 142–158. [95] S. Ren, R. Girshick, R. Girshick, J. Sun, Faster R-CNN: towards real-time object detection with region proposal networks, IEEE Trans. Pattern Anal. Mach. Intell. 39 (6) (2017) 1137–1149. [96] J. Redmon, S. Divvala, R. Girshick, A. Farhadi, You only look once: unified, real-time object detection, IEEE Conference on Computer Vision and Pattern Recognition (2015) 779–788. [97] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.Y. Fu, A.C. Berg, SSD: single shot multibox detector, European Conference on Computer Vision (2016) 21–37. [98] T. Zhou, H. Lu, J. Zhang, H. Shi, Pulmonary nodule detection model based on SVM and CT image feature-level fusion with rough sets, BioMed Res. Int. 2016 (2016) 1–13.