Developed Newton-Raphson based deep features selection framework for skin lesion recognition

Journal Pre-proof

Developed Newton-Raphson Based Deep Features Selection Framework for Skin Lesion Recognition Muhammad Attique Khan , Muhammad Sharif , Tallha Akram , Syed Ahmad Chan Bukhari , Ramesh Sunder Nayak PII: DOI: Reference:

S0167-8655(19)30354-X https://doi.org/10.1016/j.patrec.2019.11.034 PATREC 7718

To appear in:

Pattern Recognition Letters

Received date: Revised date: Accepted date:

20 September 2019 20 November 2019 23 November 2019

Please cite this article as: Muhammad Attique Khan , Muhammad Sharif , Tallha Akram , Syed Ahmad Chan Bukhari , Ramesh Sunder Nayak , Developed Newton-Raphson Based Deep Features Selection Framework for Skin Lesion Recognition, Pattern Recognition Letters (2019), doi: https://doi.org/10.1016/j.patrec.2019.11.034

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. © 2019 Published by Elsevier B.V.

   

Highlights Artificial Bee Colony based contrast stretching is perform Lesion detection through Faster RCNN along with pixels Information Deep Features are extracted using entropy based activation function A Newton Raphson based most discriminate features are selected

Developed Newton-Raphson Based Deep Features Selection Framework for Skin Lesion Recognition Muhammad Attique Khan1, Muhammad Sharif2, Tallha Akram3, Syed Ahmad Chan Bukhari4, Ramesh Sunder Nayak5 1,2,3

Department of Computer Science, COMSATS University Islamabad, Wah Campus, Pakistan Division of Computer Science, Mathematics and Science, College of Professional Studies, St. John’s University, New York, USA Information Science, Canara Engineering College, Mangaluru, Karnataka, India

4 5

ABSTRACT Melanoma is the fatal form of skin cancer; however, its diagnosis at the primary stages significantly reduces the mortality rate. These days, the increasing numbers of skin cancer patients have boosted the requirement for a care decision support system - capable of detecting the lesions with high accuracy. In this work, a method is proposed for skin cancer localization and recognition by implementing a novel combination of a deep learning model and iteration-controlled Newton-Raphson (IcNR) based feature selection method. The proposed framework follows three primary steps - lesion localization through faster region based convolutional neural network (RCNN), deep feature extraction, and feature selection by IcNR approach. In the localization step, a new contrast stretching approach based on bee colony method (ABC) is being followed. The enhanced images along with their ground truths are later plugged into Fast-RCNN to get segmented images. A pre-trained model, DenseNet201, is utilized to extract deep features via transfer learning, which are later subjected to selection step using proposed IcNR approach. The selected most discriminant features are finally utilized for classification using multilayered feed forward neural networks. Tests are performed on ISBI2016 and ISBI2017 datasets to achieving an accuracy of 94.5% and 93.4%, respectively. Simulation results reveal that the proposed technique outperforms existing methods with greater accuracy, and time. Keywords- Skin cancer, contrast stretching, lesion localization, deep features, best features

1. Introduction In skin cancer, melanoma is the deadliest type – responsible for the death of large number of people worldwide [1, 2]. Skin cancer can be treatable if diagnosed at the early stages, otherwise, consequences will be severe [3]. At the early stage, melanoma starts in the melanocyte cells, which seems like a mole having black or brown color [4, 5]. In the year 2017, reported skin cancer cases, in United States (US) only, are 95,360 (57,140 men and 38,220 women), in which melanoma cases are 87,110 (52,170 men and 34,940 women). The estimated deaths occurred in the USA since 2017 are 13,590 (9,250 men and 4,340 women) [6, 7]. In the year 2018, an estimated 99,550 (60,350 men and 39,200 women) cases are reported. From those melanoma cases are 91,270 including 55,150 men and 36,120 women. The death cases, on the other hand, in 2018 are 13,460 including 9,070 men and 4,390 women. In 2019 only, stats show 104,350 cases, including 62,320 men and 42, 030 women. The number of melanoma cases are 96,480, including 57,220 men and 39,260 women. The number of melanoma death cases during 2019 in USA are 7,320 [8]. Figure. 1 provides cancer statistics in brief for the year 2015 to 2019. A conventional method for skin lesion detection is through visual inspection, which is quite a challenging task – clearly depends upon the expert. A dermatologist mostly uses the screening methods such as 7-point checklist [9], ABCDE rule [10], and several other advanced techniques like optical imaging system and light, etc. [11] for the detection of skin lesion. These methods perform well but are time-consuming and are also not free from human error. Due to the recent advancements in the field of computer vision (CV), several computerized systems are utilized in clinics which play helps to doctors for diagnosis at early stage. Most of the existing methods incorporates four primary stages for skin lesion detection from contrast stretching to classification [12, 13]. Preprocessing step is very important for the removal of noises such as hair, bubbles, etc., and also plays a vital role in an accurate segmentation [14]. Feature extraction from the segmented image is a crucial step, as good features lead to an accurate classification and vice versa [15, 16]. Lately, with the advent of deep learning methods [17-19], there is an increasing trend to utilize them in medical domain [20]. By embedding the concept of transfer learning, convolutional neural network (CNN) models [21, 22] - trained on the large image datasets are retrained on the skin datasets. Feature selection is an important research in the area of machine learning and CV [23, 24]. In the medical imaging, the extraction of features from raw images generates various patterns information and few of them are not essential for classification task [25]. The irrelevant information misguides the selected classifiers and reduces the overall performance. 1.1. Motivations and Contributions Inspired from the comparative work by Fernandes et al. [26], in which early skin lesion is detected based on two state-of-the-art techniques, color constancy, and skin lesion analysis. Authors performed a detailed analysis to conclude that color constancy approach is a better choice for skin lesion detection. Additionally, they also concluded that early detection of skin lesion is quite expedient for the treatment of melanoma. In this work, implemented a DenseNet pre-trained CNN model [27] for deep feature extraction and later best most discriminant features are selected by employing a Newton Raphson (NR) method. Our major contributions are- (a) Artificial Bee Colony (ABC) based an efficient contrast stretching method is proposed for an accurate segmentation; (b) Faster RCNN is implemented for lesion detection – utilizing ground truth pixels’ information; (c) An entropy based activation function for deep features extraction is implemented, and (d) A Newton Raphson (IcNR) computational method is implemented for the most discriminant features selection. The remaining manuscript is ordered as follows: Related work is described in Section 2. Proposed DLNR method presented in Section 3. Results and comparison are discussed in Section 4. Finally, Section 5 concludes the overall manuscript. 2.

Related Work

An automatic mechanism for the recognition of skin lesion is an arduous task due to a set of factors including low contrast, irregularity, presence of several artifacts like hairs and bubbles, etc. Manual inspection of a skin lesion is dependent on a qualified specialist, which can’t be available whole time, therefore, machine learning based methods are proposed by a pool of researchers working in this domain. Codella et al. [28] presented a hybrid method for lesion recognition, ensemble deep learning (DL). Three different features extraction methods are fused together in developing a standard approach. The method is tested on ISBI 2016 to achieving an accuracy of 76%. Li et al. [29] introduced a reliable DL method for melanoma recognition which efficiently tackles the problem of visual similarity among lesion types. The introduced method solves the problems of lesion extraction through segmentation technique, feature calculation, and finally recognition. Two fully residual network layers are defined for segmentation and lesion classification which further improved by lesion index calculation unit. The ISIC 2017 dataset is utilized for testing results and achieved accuracy of 91.2%. Soumen et al. [30] introduced a DCNN based method – tested on Dermofit and MEDNOE datasets in two phases. Individual performance is computed at the early phase and later combined both datasets and achieves accuracy of 83.07%. Mahbod et al. [31] introduced a fully automated design for skin lesion recognition using optimized deep features and different saturation levels. Three CNN models such as VGG, AlexNet, and ResNet18 are utilized for feature generators. These generated features are classified by SVM and achieve a melanoma recognition accuracy is 83.33%. Abbas et al. [32] presented a method for pigmented skin lesion recognition using a new method name DermoDeep. The DermoDeep method includes the fusion of visual and DNN features which consists total of five layers of architecture. This model is trained on 2800 ROI images. The performance of DermoDeep model is validated through sensitivity and specificity measures which obtained 93% and 95%, respectively. Despite of the advantages in terms of classification accuracy by the above mentioned techniques, there exists a few shortcomings including lack of generalization due to a variation in the dermoscopy scans, and a low resolution of selected datasets. Moreover, the selections of most discriminant features are not efficiently dealt by the aforementioned methods. Considering these factors, in this work, our primarily focusing on a preprocessing steps, and later selected the most discriminant feature so as to reduce computational complexity and time. 3.

Proposed Methodology

The proposed deep learning and NR (DLNR) based skin lesion recognition system consists of the following primary steps, Figure 2, where contrast stretching is performed to visually improve the lesion area. Later, Faster RCNN (F-RCNN) is applied on contrast stretched images for lesion boundary localization. After localization of lesion boundary, extract the deep features by employing pretrained CNN model name DenseNet. Transfer learning based optimized skin lesion features is extracted that later improved by NR method. The best selected features through NR, are fed to neural network (NN) for final classification in the form of labeled results.

3.1. Contrast Stretching In this work, improve the visual contents of the lesions, artificial bee colony (ABC) algorithm is utilized. Considering RGB image 𝜉(𝑥, 𝑦) ∈ Δ, having dimensions 𝑁 × 𝑀, where the value of 𝑁, 𝑀 is initialized as 512 for each image in the database Δ. A transformation function is defined by utilizing the intensity level of 𝜉(𝑥, 𝑦) as follows: 1 𝑢 𝜉𝑡 = 1 (𝑎−1) × ∫0 𝑍 (𝑎−1) (1 − 𝑍)𝑏−1 𝑑𝑧 (1) 𝑏−1 ∫0 𝑍

(1−𝑍)

𝑑𝑧

Where, 𝑍 is an integration variable, 𝑎 and 𝑏 are adjusted parameters of a given function where a higher value of 𝑎 is required compared to 𝑏. To adjust the value of 𝑎 &𝑏, a fitness function is defined which evaluates the automated criteria of measuring the quality of entire lesion image. A fitness function is defined as follows: 𝐹(𝜉𝐸 (𝑥, 𝑦)) = 𝑙𝑜𝑔(log(∑𝑖=1(𝜓)). 𝑁𝜓 . 𝐻(𝜉𝐸 ). 𝐶(𝜉𝐸 )) (2) ∑ Where, 𝑖=1(𝜓) denotes the sum of edge intensities of an image which are calculated through canny edge detector. The symbol 𝑁𝜓 denotes the number of edge pixels of processed image𝜉𝐸 (𝑥, 𝑦), 𝐻(𝜉𝐸 ) denotes the entropy value of image 𝜉𝐸 (𝑥, 𝑦) which is defined as follows: 𝐻(𝜉𝐸 ) = ∑𝑛𝑖=0 𝑝𝑖 𝑙𝑜𝑔2 (𝑝𝑖 ) (3) Where, 𝑛 = 255 and 𝑝𝑖 denotes the probability value of 𝑖𝑡ℎ intensity pixel and 𝐶(𝜉𝐸 )denotes the contrast value of image 𝜉𝐸 (𝑥, 𝑦) formulated as follows: 𝐶(𝜉𝐸 ) = ∑𝑛𝐵 (4) 𝑖=1 𝐶(𝜉𝐸 )(𝐵𝑖 ) where, 𝐵𝑖 denotes 𝑖𝑡ℎ image blocks and 𝑛𝐵 presents the 𝑛𝑡ℎ image blocks, respectively. As our major goal is to improve the contrast level as compare to original contrast, therefore, the local band limited contrast for each block is computed. The mathematical formulation is defined as. 𝜉𝐿𝑐 (𝐵𝑖 ) = ∑(𝑟,𝑐)∈𝐵 𝐶(𝜉𝐸 )(𝑟, 𝑐) (5) = ∑(𝑟,𝑐)∈𝐵

𝜉𝐸 (𝑟,𝑐)⨂ 𝜙𝑏

(6)

𝜉𝐸 (𝑟,𝑐)⨂ 𝜙𝑙

Where, 𝑟, 𝑐 denotes the rows and columns pixels of each block, 𝜙𝑏 denotes the band pass filter and 𝜙𝑙 denotes the low pass filter, respectively. A final enhanced image is obtained through above expression presented by 𝜉𝐸 (𝑥, 𝑦) which visual effects are shown in Figure 3. In the next step, each enhanced image is pass by FRCNN for lesion boundary localization. 3.2. Lesion Detection In the domain of medical imaging, infection localization is a very critical task, because the complete model is totally dependent on it. In this work, a faster RCNN [33] is utilized for lesion detection, which in general includes three primary blocks, convolutional layers block, regional proposal network block, and prediction block. In faster RCNN, the input image is feed to CNN for deep feature extraction instead of the regional proposal network (RPN). By using deep features, initially ROP is identified prior to bending them into a dynamic mask. Later, ROI pooling layer reshapes them, which are finally processed through FC layer. Through these extracted features, a mask is generated along with the active contour model which produces a lesion boundary on the original image. The loss function of FRCNN is defined as: 1 1 𝜉𝐿𝑝 ({𝑓𝑖 }, {ℎ𝑖 }) = 𝑐 ∑𝑖(𝑓𝑖 , 𝑓𝑖 ∗ ) + 𝜆 𝑟𝑔 ∑𝑖 𝑓𝑖 ∗ 𝜉 𝑟𝑔 (ℎ𝑖 , ℎ𝑖 ∗ ) (7) 𝑁

1

𝑁

1

where the term 𝑐 ∑𝑖(𝑓𝑖 , 𝑓𝑖 ∗ ) recognizes the lesion and background pixels, 𝜆 𝑟𝑔 ∑𝑖 𝑓𝑖 ∗ 𝜉 𝑟𝑔 (ℎ𝑖 , ℎ𝑖 ∗ ) represents the boundary on a lesion 𝑁 𝑁 region. Our main architecture of lesion detection using dermoscopy images through FRCNN is shown in Figure 4. The extracted feature map is compared to the ground truth images by generating a boundary instead of a rectangle on the lesion region. The boundary is drawn on those images which IOU value is more than 0.50, as few IOU values are given in Table. The IOU is computed through following mathematical expression. 𝜉 𝑖𝑜𝑢 = 𝐴𝑂𝐿

𝐴𝑂𝑈

𝜉 𝐴𝑂𝐿

𝜉 𝐴𝑂𝑈

(8)

where 𝜉 and 𝜉 denotes the overlap area of foreground and background. In this case, the IOU is computed between predicted boundary from FRCNN and groundtruth pixels. Mathematically, the predicted boundary is defined by following formulation: 1 𝐿𝑒𝑠𝑖𝑜𝑛𝑃𝑖𝑥𝑒𝑙𝑠 𝜉 𝑔𝑡 (𝑓𝑖 ) = { (9) 0 𝐵𝑎𝑐𝑘𝑔𝑟𝑜𝑢𝑛𝑑𝑃𝑖𝑥𝑒𝑙𝑠 The boundary extraction results are shown in Figure 5. 3.3. Deep Features Extraction In the area of machine learning, CNN shows significant importance for the recognition tasks such as object recognition, character recognition, recognition of medical infections (i.e. skin lesion, brain tumor, and lungs nodules) [34-36], to name but a few [18, 19, 21, 37]. Recently, various deep learning models are introduced by several researchers [38]. The most well-known deep learning pre-trained models are AlexNet [39], VGG [40], GoogleNet [41], ResNet [42], and Yolo [43]. These models are utilized for classification purposes and they obtained significant success. The most recent, a densely connected neural network is introduced name DenseNet by Gao et al. [27]. The DenseNet CNN model is train easily as compare to ResNet CNN model. As compare to ResNet, in the DenseNet, every layer has direct access to the gradient from loss function and original input data. Another advantage of DenseNet is that the regularization effect and this model reduces the problem of over fitting when smaller training samples are available. Let we have a set of binary images of a clear lesion area denoted by 𝚫′ which are feed into CNN model for feature extraction. The CNN includes 𝐿 layers and each layer implements a non-linear transformation function (TF) 𝐻 𝑙 (𝑖), where 𝑙 denotes the indexes of each layer in the network. The transformation function 𝐻 𝑙 (𝑖) can be a batch normalization, pooling, ReLu, or convolution. For feature extraction, the output layer of ResNet is denoted by 𝑂𝑙 and defined as: 𝑂𝑙 = 𝐻 𝑙 (𝑂𝑙−1 ) + 𝑂𝑙−1 (10) As mentioned earlier that in the DenseNet, a direct connection is done among the entire layer as shown in Figure 6. In mathematically, the Output layer of DenseNet is defined as: 𝑂𝑙 = 𝐻 𝑙 ([𝑂0 , 𝑂1 , 𝑂2 , … 𝑂𝑙−1 ]) (11)

Where, [𝑂0 , 𝑂1 , 𝑂2 , … 𝑂𝑙−1 ] denotes the concatenation of feature maps produce in layer 0,1, … 𝑙 − 1. The DenseNet CNN includes total of 709 layers where the input layer size is 224 × 224 × 3. A fully connected layer is utilized for feature extraction and the number of parameters defined in the Table 1 as: The transfer learning based features are mapped from source model to proposed model. This statement defines that the structure of original DenseNet is mapped and trains the model on our own skin datasets. Because, the original model is trained on ImageNet dataset and if we built a new model, then it takes too much time, therefore transfer learning (TL) is performed. The concept of transfer learning is shown in the Figure 7. After TL, the obtained resultant vector is of dimension 𝑁 × 1000, where 𝑁 represent the training and testing samples. A brief stepwise training and testing process is defined as follows: Training & Testing- In the training and testing process, first of all divide our boundary extracted data store in a ratio of 50:50. Then, perform normalization to equal the size of processed data as equal to the input size of convolution layer. After that, load a pre-trained CNN network name DenseNet and perform TL. Through TL, the train and test the original model on Skin lesions datasets. Later, cross entropy activation function is employed for features extraction on FC layer. The extracted features are optimized through a newton Raphson (NR) based computational method. The NR method provides a most optimal value which is initialized in a fitness function for best feature selection. After that, two vectors are obtains- training vector and testing vector. The trained feature vector is utilized for a training a model which later utilized for prediction results. Whereas, the testing feature vector is utilized for final classification accuracy. 3.4. Feature Selection In this article, our major purpose of employ feature selection method is to remove the redundant kind of information, eliminate noisy, and less informative features. This selection process can help in improving system efficiency and accuracy. A new method for feature selection name iteration controlled Newton Raphson (IcNR) is implemented. In the proposed IcNR method, initially select first feature as a guess value. Then, perform convergence until the desired value is not obtained. Later, the desired value is put into a threshold function for binary clustering- best selected and less informative. Mathematically, the proposed IcNR method is defined as follows: The extracted test deep feature vector denoted by Ω, where the dimension of Ω is 𝑁 × 1000. Consider, the NR iteration 𝜑𝑛+1 converges towards 𝜑 ∗ along first derivative 𝐷′ (𝜑 ∗ ) ≠ 0. Then error is compute for 𝑛𝑡ℎ features as: 𝜑 𝑛 = 𝜑 ∗ + 𝜖𝑛 (12) ∗ (𝜑 ) Further, modifying 𝐷 𝑛 regarding 𝜑 ∗ as follows: 1 𝐷(𝜑 𝑛 ) = 𝐷(𝜑 ∗ ) + 𝐷′ (𝜑 ∗ )𝜖𝑛 + 𝐷′′ (𝜑 ∗ )𝜖 2 𝑛 + ⋯ (13) 2

1

= 𝐷 ′ (𝜑 ∗ )𝜖𝑛 + 𝐷′′ (𝜑 ∗ )𝜖 2 𝑛 + ⋯ 2 𝐷′ (𝜑 𝑛 ) = 𝐷′ (𝜑 ∗ ) + 𝐷′′ (𝜑 ∗ )𝜖𝑛 + ⋯ 𝐷(𝜑 ) 𝜖𝑛+1 ≈ 𝜖𝑛 − ′ (𝜑𝑛 )

(14) (15) (16)

𝜖𝑛+1 ≈

(17)

𝐷 𝑛 𝐷′′ (𝜑∗ ) 2 𝜖 𝑛 ′ ∗ 2𝐷 (𝜑 )

Hence, in the last, the final IcNR expression is defined as: 𝐷(𝜑 ) 𝜑𝑛+1 = 𝜑 𝑛 − ′ (𝜑𝑛 ) 𝐷

𝑛

(18)

Where, 𝑛 + 1 and 𝑛 denote the next and previous features, respectively in a matrix Ω. This expression returns an exact value which provides into a final threshold function for feature clustering, where the number of cluster are in number. The first cluster denoted by 𝜉𝑏𝑒𝑠𝑡 (𝐾1 ) is a best feature whereas the second cluster is denoted by 𝜉𝑙𝑜𝑠𝑒 (𝐾2 ) is a cluster of irrelevant features. The threshold function is defined as follows: 𝜉 (𝐾 ) 𝑖𝑓 Ω ≥ 𝜑𝑛+1 𝜑 𝑏𝑒𝑠𝑡 (𝐾1 , 𝐾2 ) = { 𝑏𝑒𝑠𝑡 1 (19) 𝜉𝑒𝑙𝑚𝑡 (𝐾2 ) 𝐸𝑙𝑠𝑒𝑤ℎ𝑒𝑟𝑒 From the above threshold function, we select cluster 𝜉𝑏𝑒𝑠𝑡 (𝐾1 ) which includes best selected features based on iterative newton Raphson value. The best select features are feed to multilayer perceptron neural network for recognition of skin lesions into their relevant class. The labeled recognition results are demonstrated in Figure 8. 4. Simulation Results and Analysis 4.1. Datasets Description The proposed skin lesion recognition method is tested on two datasets, ISBI 2016 and ISBI 2017. The ISBI 2017 dermoscopy dataset includes total of 2,750 RGB images of different resolutions. From total dermoscopy images, 517 images are malignant and 2,223 are benign. The ISBI 2016 dermoscopy dataset includes total of 1,279 images of different resolutions having 273 malignant and 1,006 benign. 4.2. Simulation Procedure In the simulation procedure, images are divided into training and testing images. Two different training/testing ratios are followed in this work for the experimental process- i) 70:30 (70% sample are selected randomly for training and the remaining 30% samples for testing), ii) 50:50. Moreover, the results are computed in two different steps- i) The results are separately computed in the form of performance metrics like accuracy, sensitivity, computational time, etc., ii) Categories wise combined both datasets and computes the results, followed as step 1. The performance results are analyzed through 6 different methods, as given in Table 2. 4.3. Results The proposed skin lesion recognition results in the form of numerical values and graphical is demonstrated in this section. As explained in section 4.2, the two evaluation schemes are followed- 70:30 approach and 50:50 approach for training and testing (Tr&Ts). The recognition results of proposed deep learning method for 70:30 method using ISBI 2016 dataset are given in Table 3. Six different types of classification techniques are utilized as the description is given in Table 3. The neural network (NN) is selected based on the best accuracy performance. The achieve recognition accuracy of CT, LDA, LSVM, FKNN, and WKNN is 89.1%, 92.1%, 93.8%, 89.2%, 88.7%, and 94.5%, respectively, given in Table 3. The NN achieve the best accuracy as compared to other classification methods,

verified through Table 4. In this table, the confusion matrices are given for the top 3 best classifiers. The other calculated performance methods like sensitivity (Sen), precision (Prec), F1-Score (F1-Score), AUC, FPR, accuracy, false negative rate (FNR), and time. For NN, the achieve values of these performance methods is 94%, 94.5%, 94.24%, 0.98, 0.06, 94.5%, 5.6%, and 9.438 seconds. The classification time of WKNN is 6.935 seconds which is good as compare to NN and other methods. In the second phase, the 50:50 method results are demonstrated in Table 5. The maximum achieves accuracy of 94.4% for NN whereas the other methods achieve accuracy of 88.2%, 91.1%, 92.1%, 90.8%, 92.1%, and 94.4%, respectively. The top 3 best accuracies are also verified through Table 6 (left to right). The results demonstrated in Table 3 and Table 5 shows little bit variation after the change in training images and testing images selection. The maximum change has occurred for WKNN in which the accuracy of 88.7% (70:30) is changed to 92.1% (50:50) after the selection of images. In the last, the selection of different strategies of Tr&Ts effects on the proposed recognition time as plotted in Figure 9 and 10. From Fig 9 and 10, it is clearly shown that the proposed recognition method outperforms for 50:50 method of data selection for Tr&Ts.

The recognition results of ISBI 2017 dataset using proposed deep learning method are given in Table 7 for selection strategy of 70:30. As demonstrated in Table 7, the classification accuracy of 6 classification methods like CT, LDA, LSVM, FKNN, WKNN, and NN is 79.9%, 89.4%, 89.3%, 87.7%, 85.6%, and 94.20%, respectively. The accuracies of top 3 best classification methods is also verified through a Table 8 (left to right). From all, NN is outperforms and its other calculated measures are Sen (94.2%), Prec (94.4%), F1 score (94.29%), AUC (0.98), FP rate (0.005), and FNR (5.80%), respectively. Later, the 50:50 selection method of Tr&Ts is adopted and achieve an maximum accuracy of 93.4% for NN, demonstrated in Table 9. The top 3 best accuracies under Table 9 are confirmed by Table 10 in the form of confusion matrix. The recognition performance of Table 7 and 9 shows the change in results after selection of different strategies of Tr&Ts. The maximum change occurred after selection of different strategies is 8.5% for LSVM whereas the lowest change is 0.4% among CT. In the last, the recognition time is also calculated for both selection strategies as shown in Figure 11 whereas the minimum noted time is 10.691 seconds for WKNN.

4.4. Discussion In this section, the proposed method results are discussed in terms of their recognition accuracy, as well as comparison with the existing techniques. As shown in Figure 1, the skin cancer cases are increased each year, therefore the efficient automated system is required in the clinics which help the determatalogiest for quick diagnosis of skin lesions. A new automated technique is proposed in this work, as architecture is presented in Figure 2. The visual results of each step given in Figure 3-8. The recognition accuracy is computed on ISBI2016 and ISBI2017 by employing two different strategies- 50:50 and 70:30. The results of 70:30 for both datasets are given in Table 3 and 7 which confirms by Table 4 and 8. Later, the accuracy is computed in a 50:50 approach and achieves the best performance up to average 4% as compared to the 70:30 approaches. In addition, compute the recognition performance on original features for both datasets and compare the performance with proposed selected features; results are given in Table 11 and 12. From the results, it is show that the proposed method outperforms on selected features for approach 50:50 on both datasets. In addition, it is showed from Figure 9, 10, and 11, the computational time of the proposed system is the change for different strategies like 70:30 and 50:50. From Table 11 and 12, it is clearly show that the computational time of original features is high as compare to the best-selected features. In the last, a comparison is conducted of proposed system with few recent state-of-the-art techniques, given in Table 13. From this table, it is clearly authenticate our proposed results as compare to others. 5.

Conclusion

A new automated system is proposed for skin lesion localization and recognition – utilizing the concept of deep learning and IcNR based feature selection. The proposed IcNr selection method is evaluated on two freely available datasets- ISBI2016 and ISBI2017 to achieving an average accuracy of 94.5% and 93.4%, respectively. From the results, it is concluded that the contrast stretching step increases the segmentation accuracy by enhancing the lesion area compared to the background. Additionally, it is also concluded that the feature extraction via transfer learning consumes less time in comparison to the training from the scratch. Moreover, the results clearly reveal the best classification due to the selection of most discriminant features - making this selection step quite necessary. However, the proposed method has few limitations such as the detection of accurate lesion region and selection of best features. The lesion representation of each image is different to each other therefore sometimes, the detection process not work well especially for border lesions. Moreover, each time, it is not essential the best selected features produce best results due to final selected point. So in future, the proposed method will be modified and tested on the large datasets including ISBI 2018 and HAM10000.

Conflict of Interest Article Title: Developed Newton-Raphson Based Deep Features Selection Framework for Skin Lesion Recognition Muhammad Attique Khan1, Muhammad Sharif2, Tallha Akram3, Syed Ahmad Chan Bukhari4, Ramesh Sunder Nayak5 On the behalf of corresponding author, all authors declare that they have no conflict of interest.

References [1]

[2] [3]

[4] [5] [6]

[7]

[8] [9] [10] [11] [12]

[13] [14] [15] [16] [17] [18]

[19]

[20] [21] [22] [23] [24] [25] [26] [27] [28]

M. A. Khan, T. Akram, M. Sharif, A. Shahzad, K. Aurangzeb, M. Alhussein, et al., "An implementation of normal distribution based segmentation and entropy controlled features selection for skin lesion detection and classification," BMC cancer, vol. 18, p. 638, 2018. T. Akram, M. A. Khan, M. Sharif, and M. Yasmin, "Skin lesion segmentation and recognition using multichannel saliency estimation and M-SVM on selected serially fused features," Journal of Ambient Intelligence and Humanized Computing, pp. 1-20, 2018. M. A. Khan, T. Akram, M. Sharif, K. Javed, M. Rashid, and S. A. C. Bukhari, "An integrated framework of skin lesion detection and recognition through saliency method and optimal deep neural network features selection," Neural Computing and Applications, pp. 120. U. Jamil, S. Khalid, M. U. Akram, A. Ahmad, and S. Jabbar, "Melanocytic and nevus lesion detection from diseased dermoscopic images using fuzzy and wavelet techniques," Soft Computing, vol. 22, pp. 1577-1593, 2018. M. A. Khan, T. Akram, M. Sharif, T. Saba, K. Javed, I. U. Lali, et al., "Construction of saliency map and hybrid set of features for efficient segmentation and classification of skin lesion," Microscopy research and technique, 2019. M. Nasir, M. Attique Khan, M. Sharif, I. U. Lali, T. Saba, and T. Iqbal, "An improved strategy for skin lesion detection and classification using uniform segmentation and feature selection based approach," Microscopy research and technique, vol. 81, pp. 528-543, 2018. N. C. Codella, D. Gutman, M. E. Celebi, B. Helba, M. A. Marchetti, S. W. Dusza, et al., "Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic)," in 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), 2018, pp. 168-172. M. E. Celebi, N. Codella, and A. Halpern, "Dermoscopy Image Analysis: Overview and Future Directions," IEEE journal of biomedical and health informatics, 2019. J. Kawahara, S. Daneshvar, G. Argenziano, and G. Hamarneh, "Seven-Point Checklist and Skin Lesion Classification Using Multitask Multimodal Neural Nets," IEEE journal of biomedical and health informatics, vol. 23, pp. 538-546, 2019. S. Goldsmith and A. Solomon, "A series of melanomas smaller than 4 mm and implications for the ABCDE rule," Journal of the European Academy of Dermatology and Venereology, vol. 21, pp. 929-934, 2007. Z. Turani, E. Fatemizadeh, T. Blumetti, S. Daveluy, A. F. Moraes, W. Chen, et al., "Optical Radiomic Signatures Derived from Optical Coherence Tomography Images to Improve Identification of Melanoma," Cancer research, 2019. A. Liaqat, M. A. Khan, J. H. Shah, M. Sharif, M. Yasmin, and S. L. Fernandes, "Automated ulcer and bleeding classification from WCE images using multiple features fusion and selection," Journal of Mechanics in Medicine and Biology, vol. 18, p. 1850038, 2018. S. Naqi, M. Sharif, M. Yasmin, and S. L. Fernandes, "Lung nodule detection using polygon approximation and hybrid features from CT images," Current Medical Imaging Reviews, vol. 14, pp. 108-117, 2018. F. Bokhari, T. Syedia, M. Sharif, M. Yasmin, and S. L. Fernandes, "Fundus image segmentation and feature extraction for the detection of glaucoma: A new approach," Current Medical Imaging Reviews, vol. 14, pp. 77-87, 2018. A. Mahbod, G. Schaefer, C. Wang, R. Ecker, and I. Ellinge, "Skin lesion classification using hybrid deep neural networks," in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019, pp. 1229-1233. M. A. Khan, S. Rubab, A. Kashif, M. I. Sharif, N. Muhammad, J. H. Shah, et al., "Lungs Cancer Classification from CT Images: An Integrated Design of Contrast based Classical Features Fusion and Selection," Pattern Recognition Letters, 2019. M. Rashid, M. A. Khan, M. Sharif, M. Raza, M. M. Sarfraz, and F. Afza, "Object detection and classification: a joint selection and fusion strategy of deep convolutional neural network and SIFT point features," Multimedia Tools and Applications, pp. 1-27, 2018. M. Sharif, M. Attique Khan, M. Rashid, M. Yasmin, F. Afza, and U. J. Tanik, "Deep CNN and geometric features-based gastrointestinal tract diseases detection and classification from wireless capsule endoscopy images," Journal of Experimental & Theoretical Artificial Intelligence, pp. 1-23, 2019. M. A. Khan, T. Akram, M. Sharif, M. Awais, K. Javed, H. Ali, et al., "CCDF: Automatic system for segmentation and recognition of fruit crops diseases based on correlation coefficient and deep CNN features," Computers and electronics in agriculture, vol. 155, pp. 220-236, 2018. S. Mukherjee, A. Adhikari, and M. Roy, "Malignant Melanoma Classification Using Cross-Platform Dataset with Deep Learning CNN Architecture," in Recent Trends in Signal and Image Processing, ed: Springer, 2019, pp. 31-41. M. Raza, M. Sharif, M. Yasmin, M. A. Khan, T. Saba, and S. L. Fernandes, "Appearance based pedestrians’ gender recognition by employing stacked auto encoders in deep learning," Future Generation Computer Systems, vol. 88, pp. 28-39, 2018. J. Amin, M. Sharif, N. Gul, M. Yasmin, and S. A. Shad, "Brain Tumor Classification based on DWT Fusion of MRI Sequences using Convolutional Neural Network," Pattern Recognition Letters, 2019. M. Sharif, M. A. Khan, M. Faisal, M. Yasmin, and S. L. Fernandes, "A framework for offline signature verification system: Best features selection approach," Pattern Recognition Letters, 2018. M. A. Khan, M. Rashid, M. Sharif, K. Javed, and T. Akram, "Classification of gastrointestinal diseases of stomach from WCE using improved saliency-based method and discriminant features selection," Multimedia Tools and Applications, pp. 1-28, 2019. M. I. Sharif, J. P. Li, M. A. Khan, and M. A. Saleem, "Active Deep neural Network Features Selection for Segmentation and Recognition of Brain Tumors using MRI Images," Pattern Recognition Letters, 2019. S. L. Fernandes, B. Chakraborty, V. P. Gurupur, and G. Prabhu, "Early skin cancer detection using computer aided diagnosis techniques," Journal of Integrated Design and Process Science, vol. 20, pp. 33-43, 2016. G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, "Densely connected convolutional networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4700-4708. N. C. Codella, Q-B. Nguyen, Sharath Pankanti, D. A. Gutman, Brian Helba, A. C. Halpern, and John R. Smith. , ""Deep learning ensembles for melanoma recognition in dermoscopy images." IBM Journal of Research and Development 61, no. 4/5 (2017): 5-1.

," 2017. [29] Y. Li, and Linlin Shen. , ""Skin lesion analysis towards melanoma detection using deep learning network.," " Sensors vol. 18, no. 2 (2018): 556., 2018. [30] S. Mukherjee, Arunabha Adhikari, and Madhusudan Roy. "Malignant Melanoma Classification Using Cross-Platform Dataset with Deep Learning CNN Architecture." In Recent Trends in Signal and Image Processing, pp. 31-41. Springer, Singapore, 2019. [31] A. Mahbod, Gerald Schaefer, Chunliang Wang, Rupert Ecker, and Isabella Ellinge. "Skin lesion classification using hybrid deep neural networks." In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1229-1233. IEEE, 2019.

[32] [33] [34] [35] [36] [37]

[38] [39] [40] [41] [42] [43] [44]

[45] [46] [47] [48] [49]

Q. Abbas, and M. Emre Celebi. "DermoDeep-A classification of melanoma-nevus skin lesions using multi-feature fusion of visual features and deep neural network." Multimedia Tools and Applications (2019): 1-22. R. Girshick, "Fast r-cnn," in Proceedings of the IEEE international conference on computer vision, 2015, pp. 1440-1448. J. Amin, M. Sharif, M. Yasmin, and S. L. Fernandes, "Big data analysis for brain tumor detection: Deep convolutional neural networks," Future Generation Computer Systems, vol. 87, pp. 290-297, 2018. S. L. Fernandes, V. P. Gurupur, H. Lin, and R. J. Martis, "A Novel fusion approach for early lung cancer detection using computer aided diagnosis techniques," Journal of Medical Imaging and Health Informatics, vol. 7, pp. 1841-1850, 2017. J. Amin, M. Sharif, M. Yasmin, H. Ali, and S. L. Fernandes, "A method for the detection and classification of diabetic retinopathy using structural predictors of bright lesions," Journal of Computational Science, vol. 19, pp. 153-164, 2017. M. Rashid, M. A. Khan, M. Sharif, M. Raza, M. M. Sarfraz, and F. Afza, "Object detection and classification: a joint selection and fusion strategy of deep convolutional neural network and SIFT point features," Multimedia Tools and Applications, vol. 78, pp. 15751-15777, 2019. M. A. Khan, M. Sharif, T. Akram, M. Yasmin, and R. S. Nayak, "Stomach Deformities Recognition Using Rank-Based Deep Features Selection," Journal of medical systems, vol. 43, p. 329, 2019. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in neural information processing systems, 2012, pp. 1097-1105. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, et al., "Going deeper with convolutions," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1-9. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778. G. Ning, Z. Zhang, C. Huang, X. Ren, H. Wang, C. Cai, et al., "Spatially supervised recurrent convolutional neural networks for visual object tracking," in 2017 IEEE International Symposium on Circuits and Systems (ISCAS), 2017, pp. 1-4. M. A. Khan, M. Y. Javed, M. Sharif, T. Saba, and A. Rehman, "Multi-Model Deep Neural Network based Features Extraction and Optimal Selection Approach for Skin Lesion Classification," in 2019 International Conference on Computer and Information Sciences (ICCIS), 2019, pp. 1-7. A. R. Lopez, X. Giro-i-Nieto, J. Burdick, and O. Marques, "Skin lesion classification from dermoscopic images using deep learning techniques," in 2017 13th IASTED International Conference on Biomedical Engineering (BioMed), 2017, pp. 49-54. M. A. Khan, T. Akram, M. Sharif, T. Saba, K. Javed, I. U. Lali, et al., "Construction of saliency map and hybrid set of features for efficient segmentation and classification of skin lesion," Microscopy research and technique, vol. 82, pp. 741-763, 2019. F. Afza, M. A. Khan, M. Sharif, and A. Rehman, "Microscopic skin laceration segmentation and classification: A framework of statistical normal distribution and optimal feature selection," Microscopy research and technique, 2019. T. J. Brinker, A. Hekler, A. H. Enk, and C. von Kalle, "Enhanced classifier training to improve precision of a convolutional neural network to identify images of skin lesions," PloS one, vol. 14, p. e0218713, 2019. M. Goyal and M. H. Yap, "Automatic Lesion Boundary Segmentation in Dermoscopic Images with Ensemble Deep Learning Methods," arXiv preprint arXiv:1902.00809, 2019.

Figure 1: Skin cancer statistics of USA for the last five years.

Figure 2: Proposed architecture of Skin lesion boundary localization and recognition.

Figure 3: A few sample images after contrast stretching using proposed method.

Figure 4: Skin lesion detection architecture (FRCNN).

Figure 5: A few sample results of proposed localization method.

Figure 6: Architecture of DenseNet CNN model which shows that each layer access to other layers directly.

Figure 7: An overview of transfer learning utilized in our proposed model.

Figure 8: Sample localization and classification results for a malignant class

Figure 9: Proposed classification time on different classification methods using 70:30 approach.

Figure 10: Proposed classification time on different classification methods using 50:50 approach.

Figure 21: Comparison of proposed classification time on different classification methods using 50:50 and 70:30 approach.

Table 1: Description about input and output layer parameters Input layer

Output Layer

Convolution 2D Filter size: [7,7] Number of channels: 3 Number of filters: 64 Stride: [2,2] Padding Mode: manual Padding Size: [3,3,3,3] Weight learning rate factor: 1

Fully Connected layer Input size: 1920 Weight Learn factor: 1 Activation function: Cross entropy Output Feature vector size: 1000

Table 2: Description of selected classifiers in the form of kernel type, methods etc. Classifier Complex Tree (CT) Discriminant Analysis (LDA) Linear SVM (LSVM) Fine KNN (F-KNN) Weighted KNN (WKNN) Neural Network (NN)

Description 100 splits along with Gini’s diversity criteria LDA type is linear and full covariance structure is utilized. Linear kernel function along with automatic kernel scale and one vs one multi class method. 10 number of neighbors and Euclidean distance along with equal weight is utilized. 10 number of neighbors and Euclidean metric along with Square inverse distance weight. Method Feed Forward and activation is used as a sigmoid function, numbers of hidden layers are 3.

Table 3: Proposed skin lesion recognition performance of ISBI2016 dataset using 70:30 (training & testing) Classifier

CT LDA LSVM FKNN WKNN NN

Sen (%)

Prec (%)

89.0 92.0 93.5 89.0 90.5 94.0

89.0 92.6 93.7 89.2 91.0 94.5

Evaluation Measures F1 AUC FPR Acc Score (%) (%) 89.00 0.91 0.11 89.1 92.29 0.92 0.08 92.1 93.59 0.97 0.065 93.8 89.09 0.89 0.11 89.2 90.74 0.98 0.09 88.7 0.98 94.24 0.06 94.5

FNR (%)

Time (sec)

10.9 7.9 6.2 10.8 13.3 5.6

11.818 10.618 22.745 7.561 6.935 9.438

Table 4: Confusion matrices of top 3 best accuracy recognition results- (Left to Right) Neural network, LSVM, LDA. The M denotes the malignant and B denotes the benign. Class Class M

B

M

96 %

B

7%

5 % 93 %

Clas s

Class M

B

M

94 %

6%

B

7%

93 %

Clas s

Class M

B

M

91 %

9%

B

7%

93%

Table 5: Proposed skin lesion recognition performance of ISBI2016 dataset using 50:50 (training & testing) Classifier

Evaluation Measures AUC FPR Acc (%)

Sen (%)

Prec (%)

CT

88.0

88.6

F1 Score (%) 88.29

FNR (%)

Time (sec)

0.89

0.12

LDA

91.0

91.5

91.24

0.98

0.09

88.2

11.8

7.588

91.1

8.9

7.678

LSVM

92.0

92.8

92.39

0.98

FKNN

91.0

91.4

91.19

0.97

0.08

92.1

7.9

7.229

0.09

90.8

9.2

21.830

WKNN

92.0

92.3

92.14

NN

94.5

94.6

94.54

0.94

0.08

92.1

6.9

16.247

0.98

0.05

94.4

5.6

7.526

Table 6: Confusion matrices of top 3 best accuracy recognition results- (Left to Right) Neural network, LSVM, LDA. Class

Class

Class

Class

Class M

B

M

95%

5%

B

6%

94%

Class M

B

M

95%

5%

B

11%

89%

M

B

M

95%

5%

B

11%

89%

Table 7: Proposed skin lesion recognition performance of ISBI2017 dataset using 70:30 (training & testing) Classifier

Evaluation Measures AUC FPR Acc (%)

Sen (%)

Prec (%)

F1 Score (%)

FNR (%)

Time (sec)

CT

80.0

80.4

80.19

0.82

0.200

79.9

20.1

14.170

LDA

89.5

89.7

89.59

0.94

LSVM

88.0

90.2

89.08

0.93

0.105

89.4

10.6

18.235

0.125

89.3

10.7

15.994

FKNN

87.5

88.0

87.74

0.93

0.125

87.7

12.3

17.896

WKNN

85.5

85.7

85.59

0.92

0.145

85.6

16.4

23.430

NN

94.2

94.4

94.29

0.98

0.005

94.20

5.80

15.941

Table 8: Confusion matrices of top 3 best accuracy recognition results- (Left to Right) Neural network, LDA, and LSVM. Class Class Class Class Class Class M B M B M B M

92.4%

6.6%

M

88%

12%

M

86%

14%

B

4%

96%

B

9%

91%

B

10%

90%

Table 9: Proposed skin lesion recognition performance of ISBI2017 dataset using 50:50 (training & testing) Classifier

Evaluation Measures Sen (%)

Prec (%)

AUC

FPR

Acc (%)

FNR (%)

Time (sec)

80.5

F1 Score (%) 79.99

CT

79.5

LDA

80.5

0.86

0.205

79.5

20.5

18.057

80.7

80.59

0.92

0.195

80.9

19.1

LSVM

15.766

87.0

89.2

88.08

0.93

0.137

88.4

11.6

17.025

FKNN

79.5

79.6

79.54

0.79

0.205

79.3

20.7

11.479

WKNN

80.5

81.5

80.99

0.92

0.195

80.4

19.6

10.691

NN

93.0

93.2

93.09

0.97

0.006

93.4

6.6

12.917

Table 10: Confusion matrices of top 3 best accuracy recognition results- (Left to Right) Neural network, LSVM, and WKNN. Class Class Class Class Class Class M B M B M B M 5% M M 95% 83% 17% 81% 19% B

9%

91%

B

9%

91%

B

20%

80%

Table 11: Features based comparison of recognition accuracy on ISBI2016 dataset. Classifier Original CT



LDA



LSVM



FKNN



WKNN



MLNN



Proposed

     

Performance Metrics Accuracy (%) 84.6 88.2 89.1 91.1 89.7 92.1 88.4 90.8 89.0 92.1 91.2 94.4

FNR (%) 15.4 11.8 10.9 8.9 10.3 7.9 11.6 9.2 11.0 6.9 8.8 5.6

Time (sec) 9.424 7.588 28.690 7.678 20.401 7.229 35.600 21.830 32.906 16.247 17.872 7.526

Table 12: Features based comparison of recognition accuracy on ISBI2016 dataset. Classifier Original CT



LDA



LSVM



FKNN



WKNN



MLNN



Proposed

     

Performance Metrics Accuracy (%) 74.6 79.5 76.5 80.9 81.67 88.4 73.5 79.3 75.7 80.4 89.7 93.4

FNR (%) 25.4 20.5 23.5 19.1 18.33 11.6 26.5 20.7 24.3 19.6 10.3 6.6

Time (sec) 37.926 18.057 39.742 15.766 27.640 17.025 41.461 11.479 21.097 10.691 21.946 12.917

Table 13: Comparison of proposed recognition accuracy on selected datasets with other techniques Method

Year

Recognition Accuracy

[44]

2019

90.20%

[45]

2017

81.33%

[46]

2019

89.2%

[47]

2019

93.80%

[48]

2019

83.9%

Proposed

2019

94.4%

[49]

2019

91.29

[47]

2018

88.5%

Proposed

2019

93.4%