A deep learning approach for patch-based disease diagnosis from microscopic images

CHAPTER

A deep learning approach for patch-based disease diagnosis from microscopic images

5

Anson Simon, Ravi Vinayakumar, Viswanathan Sowmya, Kutti Padannayil Soman, Ennappadam Anathanarayanan A. Gopalakrishnan Center for Computational Engineering and Networking (CEN), Amrita School of Engineering, Coimbatore, India

5.1 Introduction Microscopy is a technology of using microscopes for disease diagnosis. Objects or areas of interest, which cannot be seen by the naked eye, can be digitally captured with the help of microscopes. These digitally captured images are used as a major source for medical imaging data. New technologies like flow cytometry and molecular biology are also employed for disease diagnosis [1, 2]. Since these technologies are highly expensive, it cannot be afforded in economically backward high diseaseprone areas. Therefore, due to the simple and well-adapted nature of microscopy, diagnostic tasks mostly depend on microscopes [3]. Although microscopes are accessible, due to lack of sufficient skilled lab technicians to operate them, efficient disease diagnosis is not available to many developing countries. This situation, in which disease diagnosis regularly depends upon disease symptoms and clinical signs alone, leads to the misdiagnosis of diseases. This type of misjudgment of diseases can lead to life-threatening conditions, drug resistance, and loss of money to purchase nonessential drugs. Therefore, in order to avoid all these health hazards and to provide a quality diagnosis for better treatment of diseases, it is essential to devise a technological functioning diagnosis system based on modern computer vision methods to process microscopic images. The exceptional performance of computer vision technologies helps to take advantage of enormous medical imaging data for the diagnosis, treatment, and detailed reporting of diseases. Modern computer vision technologies, such as machine learning, image segmentation, pattern classification, etc., provide an efficient way of extracting features like shape, texture, and color of patches from the provided medical images data. This enables more accurate disease diagnosis than is possible with only a laboratory expert’s analysis using their naked eye. Digitally Classification Techniques for Medical Image Analysis and Computer Aided Diagnosis https://doi.org/10.1016/B978-0-12-818004-4.00005-4 # 2019 Elsevier Inc. All rights reserved.

109

110

CHAPTER 5 Patch-based disease diagnosis from microscopic images

captured image by microscope is a two-dimensional tensor consisting of pixel values such that, basic image processing techniques can be easily applied which makes our problem less complicated. Due to the complexity in medical microscopic images, applying computer vision for disease diagnosis is a challenging task. Deep learning [4] technology is a part of machine learning. This deep structured learning has recently revolutionized the sectors like computer vision, natural language processing, speech recognition, machine translation, social network filtering, bioinformatics, and drug design by producing impressive results. In many computer vision applications, deep learning provided results comparable to or even better than human experts. Many previous patch-based medical imaging problems relied upon hand-engineered features, which are task specific. Therefore, there should be a standard approach in medical imaging, which can produce relevant image representations automatically instead of depending on hand-engineered features. After all, deep learning is capable of deriving high-level, complex abstractions as features for the patches present in microscopic images through an iterating learning process. Therefore, in this chapter, we demonstrate the application of deep learning to patchbased disease diagnosis from microscopic images. The “deep” in deep learning points out the multiple layers used in the deep learning model, which are often neural networks. Since computational complexity of the algorithm is related to the number of neurons in a model, deep learning usually results in high computation, which costs time and computational resources. In the present chapter, we proposed a shallow convolutional neural network (CNN) appended with a recurrent layer aimed to scale down the computational complexity by reducing the number of learnable parameters. The main challenges of the present work are to obtain a large database annotated by experts to train the deep learning architecture and hyper-parameters tuning of the network. The main contributions of the chapter are as follows: •

• • •

Three different deep architectures are proposed for the disease diagnosis from microscopic images. The proposed architectures are referred to as: CNN-RNN (recurrent neural network), CNN-LSTM (long short term memory), and CNN-GRU (gated recurrent unit). These deep learning models have been applied on three different disease datasets of microscopic images: tuberculosis (TB), malaria, and intestinal parasites. In all cases, the proposed models produce better performance than state-of-the-art models. The proposed architectures for disease diagnosis use fewer trainable parameters when compared to the existing state-of-the-art deep architecture.

The structure of the chapter is as follows: Section 5.2 provides an outline of the previous works related to our problem analysis. Section 5.3 describes the dataset and network architectures used to detect the patches for the diagnosis of diseases. Comparison and analysis of the results produced by different models on different datasets is provided in Section 5.4. The conclusion regarding the results produced for different datasets is depicted in Section 5.5.

5.2 Related works

5.2 Related works The outlook of computer vision has changed dramatically with the evolution of deep learning methods, producing to a degree of quality that automated object recognition tasks surpasses human proficiency [5]. Patch-based disease diagnosis from microscopic images is a hot research area in the medical field to provide upper class diagnosis and treatment to common people. Progress in the area of microscopy and automated image analysis leads to the orderly analysis of cellular and subcellular phenotypes [6–8]. An important application is the microscopy-based analysis, which includes a study of changes in the subcellular localization or the presence of fluorescent labeled proteins abundantly due to the genetic or environmental perturbation [9–11]. High-throughput (HTP) microscopy provides data required to explain the dynamics of biological systems, which assists in inspecting changes in protein localization in various circumstances. In the recent past, trials have been made for the systematic study of proteome dynamics in yeast and other cells by using computational techniques [12, 13]. In Grys et al. [13], an ensemble of 60 binary support vector machines (SVMs) are used to classify images of single yeast cells from open reading frame-green fluorescent protein (ORF-GFP) collection into 15 particular subcellular localization, where each SVM is trained on manually annotated images of 70,000 cells. This classifier (ensLOC) produced an output with greater than 70% Precision and Recall, which is not possible manually. This method also beats previous techniques based on SVMs for the classification in ORF-GFP collection. Efforts have been made to employ ensLOC classifier to new microscopy datasets. Many hand-engineered computational techniques are used to find numerous pixel intensity statistics and patterns for each cell segmented from the images [11, 14–16]. In addition, feature-reduction techniques are used in order to choose relevant features [17, 18]. An extra load of re-engineering and additional training is required because features cannot be transferred across datasets due to different segmentation and feature-reduction techniques used for specific problems before training. Deep learning technology is very much developed to rise above these limitations associated with the transferability of features across different datasets by learning optimal feature representations straight from the pixel-level data [4]. In many computer vision applications, CNNs have outperformed humans in the modern object recognition tasks and this stimulates the biological medical field to adopt CNN [19]. In recent times, deep learning has been applied to many medical-related tasks like classification of aberrant morphology in breast cancer cells [20], protein localization in yeast [21, 22], and imaging flow cytometry [23]. Previously, deep CNNs have been used for the mitosis [24] and embryo detection on images captured by conventional microscopes [25]. Also a multilayer perceptron network is utilized for the diagnosis of leukemia by analyzing the metaphase chromosomes [26]. Cell counting in microscopic images is also automated with the help of a fully convolutional regression network [27].

111

112

CHAPTER 5 Patch-based disease diagnosis from microscopic images

Most of the malaria diagnosis methods have been analyzed, and progress in accuracy is still needed for better treatment [28]. All of these methods depended on hand-engineered feature extraction and classifiers like k-nearest neighbors (k-NN) algorithm and SVM [29]. Previous TB detection methods also followed the similar pattern of the extraction of features like shape, color, etc., from images followed by SVM or multilayer perceptron networks as classification algorithms [30]. Earlier, multilayer perceptron networks and SVMs were used for automated helminth detection for intestinal parasites infection [31, 32]. Recently, a CNN model was implemented for the patch-based disease diagnosis from the microscopic images captured using a low cost smartphone microscope for diseases like malaria, TB, and intestinal parasites; this produced the best results in this area of research [1]. In this chapter, we analyze the application of deep learning using a CNN appended with different recurrent layers. The proposed architecture produces better results in terms of accuracy and reduced number of learnable parameters than stateof-the-art architecture with the same dataset [1]. Results for the three diagnostic tasks: TB, intestinal parasites, and malaria are analyzed using the proposed deep architecture models.

5.3 Proposed work In this section, a series of steps for the preparation of train/test dataset, proposed deep learning model architectures trained from annotated microscopic images, and detection of pathogens from the test set by the trained deep learning models are described.

5.3.1 Train/test dataset generation The proposed idea demonstrates the diagnosis of three different diseases such as TB, intestinal parasites, and malaria by detecting the patches present in three different types of microscopic image datasets created by Quinn et al. [1]: 1265 sputum images with bounding boxes around bacilli, 1182 thick blood smear images with bounding boxes around parasites, and 1217 stool images with bounding boxes around parasite eggs of hookworm, taenia, and hymenolepsis nana, respectively. Using an annotation software, bounding box annotations are set for patches that can be seen by the naked eye of laboratory experts called pathologists. These annotations are available as an XML file, which is used to generate the ground truth for the dataset [1]. Every image in the dataset was downsampled, and the overlying patches present in the image were split according to the downsampling constant and the patch size, which was determined by the type of pathogen to be detected for each dataset used for different disease diagnosis. Visual surveillance should be carried out for each particular case in order to inspect whether the patch sizes are large enough to include all pathogens clearly visible by the naked eye and no excessive details create additional computations. Positive patches like bacilli, plasmodium, or parasite eggs are located on the center position of the annotations provided by pathologists. These patches are

5.3 Proposed work

extracted with respect to a fixed patch size determined by the type of pathogen to be detected for different datasets. Staining artifacts, blood cells, or impurities, that is, with the absence of any of the mentioned pathogens were spotted from random locations of an image to consider as a negative patch. These negative patches should not crosscut with any of the annotated bounding boxes containing positive patches. For all the three datasets considered, most part of an image does not contain any pathogens, which leads to excessively large number of negative patches compared to the number of positive patches. This type of unbalanced dataset produces a biased model after the training process. Therefore, two measures were adapted in order to make the dataset balanced. First, negative patches were randomly removed from the dataset such that, the number of negative patches was 100 times the number of positive patches. Second, new positive patches were duplicated by augmenting the existing original positive samples by twisting and flipping and thus creating seven extra duplicate positive patches for each original patch. These two methods procreate the dataset, which is balanced to some extent in order to train an unprejudiced model. Half of the images in the dataset are selected randomly for training, and other half is used for testing for all the three datasets. The number of patches used for training and testing the three different diseases are given in Table 5.1.

5.3.2 Proposed deep network architecture Different architectures with a combination of a shallow CNN and different recurrent layers were experimented on for all the three datasets for the diagnosis of TB, intestinal parasites, and malaria. In order to reduce the computational load and additional memory space for the learnable parameters, deep CNN architecture is replaced with a shallow CNN having a convolutional layer and a pooling layer. This shallow CNN is appended with any of the recurrent layers like RNN, LSTM, or GRU, which is connected to the output layer consisting of two neurons that forms a general architecture for all of the three datasets, which consists of microscopic images. Initially, for all of the three datasets, shallow CNN is fixed with the convolutional layer followed by a maxpooling operation. Convolutional layer is the best option Table 5.1 The details of patches used to train and test the detection of three different diseases Training

Disease Tuberculosis Malaria Intestinal parasites

Testing

No. of positive patches

No. of negative patches

No. of positive patches

No. of negative patches

34,000 29,344 592

44,868 31,391 7400

35,344 28,616 696

44,541 31,523 8700

113

114

CHAPTER 5 Patch-based disease diagnosis from microscopic images

followed by the input layer because of the special architecture of CNN, which helps to consider the spatial structure of the images. The number of filters used for the convolution operation is set to seven and filter size of 3  3 is determined experimentally for all the datasets. These parameters are preferred as the best option because of producing the best results for a reduced number of learnable parameters. Since the size of pathogens are small, a filter size of a 3  3 window is apt for learning optimal representation of features in order to detect positive patches. Pooling operation is done followed by convolution operation in order to reduce the size of each feature map into half of the original size by extracting the maximum value corresponding to the prominent features present in the feature maps. This also helps to reduce the computational complexity by reducing the number of learnable parameters. A recurrent layer is utilized after the shallow CNN, instead of making the model deeper by adding more convolution or fully connected layers [33]. Recurrent layers are arguably the efficient neural network, which also have excellent applications in images. Each 2D tensor produced by the convolutional layer as a feature map is passed to the recurrent layer as input. The output feature activation produced by each recurrent unit or at each time step is the stimulation at the peculiar point in the whole input 2D tensor. In the recurrent layer, information of the whole input 2D tensor is passed through lateral connections between the recurrent units, while CNN alone exploits the local information using the filter window. This helps to extract tightly packed feature representation of the input with lateral connections by rejecting repeated features at different positions of input tensor. Also, the model will become capable enough to take care of slight alterations of features across numerous subsequent patches. This initiative to use a single recurrent layer instead of making the network architecture deeper will considerably reduce the number of learnable parameters with overall less computations. Because of the lateral connections in the recurrent layer, model parallelism remains as a limitation for the usage of the recurrent layer. The number of units or cells used in the recurrent layer is fixed to 10 by experimenting with different models in all the sample datasets. This parameter is also determined by comparing the ability of the model to produce best results with a reduced number of learnable parameters. After all, the problem definition is to detect the patches as positive or negative. This shaped the output layer consisting of two neurons with softmax activation function, which provides the probability values. Combining all the inferences from the experiments done in order to get the best model for patch-based disease diagnosis in microscopic images led to the proposal of a general architecture as shown in Fig. 5.1. The details of the proposed architectures to detect three different types of diseases are given in Table 5.2.

5.3.3 Detection of pathogens After the completion of model training process, the proposed model became competent enough to classify the patches in an image into positive or negative by inspecting whether it contains any pathogens or not. Therefore, images used for the testing were

5.4 Experimental results and analysis

FIG. 5.1 Proposed general architecture for patch-based disease diagnosis.

Table 5.2 The details of the proposed architectures to detect three different types of diseases Layer size Disease

Input

Conv

Pool

RNN

Output

Tuberculosis Malaria Intestinal parasites

3  20  20 3  20  20 3  60  60

7  18  18 7  18  18 7  58  58

799 799 7  29  29

7  10 7  10 7  10

2 2 2

also divided into patches in order to spot the pathogens in the whole image. Then, these patches were evaluated separately by the trained network model and labeled the patches with high activation scores as positive. This process alone results in classifying many overlying patches for each original patch present in the test image. This problem developed due to the small stride length used in creating patches, which results in overlapping. In order to solve this problem, a technique called nonmaximum suppression is used with the intention of keeping one activation per patch within the test image. This is done by first finding the patches having a high degree of overlapping and, from those patches, only one patch with high probability score is chosen, while restraining other patches.

5.4 Experimental results and analysis There are three different trained network models; each consists of three different types of recurrent layers appended with a shallow CNN that is tested to all the three

115

116

CHAPTER 5 Patch-based disease diagnosis from microscopic images

datasets. Performance comparison results for three models differed by type of recurrent layers is tabulated separately for each individual dataset in the following sections.

5.4.1 Tuberculosis The proposed CNN-RNN model produced better results with a reduced number of learnable parameters in comparison with the other two proposed models, which can be inferred from Table 5.3. Overfitting ratio is calculated as the ratio of the train loss to the validation loss. This ratio should be nearly equal to 1 to reduce overfitting in order to keep the generality of the model on unseen data. Overfitting is more for CNN-LSTM model, which makes the model difficult to generalize on new data samples. For the proposed CNN-RNN and CNN-GRU model, the overfitting ratio is almost the same and better than the CNN-LSTM model. The CNN-RNN model marginally produces a better performance with less number of learnable parameters. A feature map of the images contains edges, shapes, or other patterns, which is given as input to the recurrent layer. Therefore, there will not be sequential information with long-term dependency between each input to recurrent units or at each time step. This made the CNN-RNN model produces marginally better results than the other two with fewer learnable parameters. Detection results of tubercle bacilli in the patches are shown in Fig. 5.2. By careful examination of this figure, the middle row, which shows false detections (highest scored negative labeled test patches), spotted some tubercle bacilli and pointed out some annotation errors. This shows the dominance of the model over the human diagnosis. Receiver operating characteristics (ROC) and Precision-Recall (P-R) curves for CNN-RNN model are also plotted in Fig. 5.3. A solid and legitimate area

Table 5.3 Classification performance measures for TB detection obtained using the proposed deep models Proposed models

CNN-RNN

CNN-LSTM

CNN-GRU

No. of parameters Accuracy (%) Overfitting ratio ROC: AUC Precision-Recall: AP Precision 0 1 Recall 0 1 F1 score 0 1

1258 96.05 0.965 0.99 0.98 0.97 0.95 0.96 0.96 0.96 0.96

4048 95.94 0.880 0.99 0.99 0.96 0.96 0.97 0.95 0.96 0.95

3098 95.52 0.988 0.99 0.98 0.98 0.93 0.94 0.97 0.95 0.95

FIG. 5.2 Detection results of tuberculosis: correct detections—highest scored positive labeled test patches (top); false detections—highest scored negative labeled test patches (middle); lowest scored test patches (bottom).

CHAPTER 5 Patch-based disease diagnosis from microscopic images

ROC: AUC = 0.99

Precision-Recall: AP = 0.98

1.0

1.0

0.8

0.8 Precision

True positive rate

118

0.6 0.4 0.2

0.6 0.4 0.2

0.0 0.0

0.2

0.4 0.6 False positive rate

0.8

1.0

0.0 0.0

0.2

0.4

0.6

0.8

1.0

Recall

FIG. 5.3 ROC and Precision-Recall curve obtained using the proposed architecture for tuberculosis detection.

1.0

True positive rate False positive rate

0.8

0.6

0.4

0.2

0.0 0.0

0.2

0.4 0.6 Threshold

0.8

1.0

FIG. 5.4 TPR-FPR curve obtained using the proposed architecture for TB detection.

under ROC curve (AUC) value gives the probability that the model produces a higher score to an arbitrarily picked positive case than to an arbitrarily picked negative case. Average precision (AP) outlines P-R curve as the weighted mean of precision produced at each point, with the hike in recall from the previous point utilized as the weight. TPR-FPR curve is also plotted in Fig. 5.4.

5.4 Experimental results and analysis

Table 5.4 Classification performance measures for malaria detection obtained using the proposed deep models Proposed models

CNN-RNN

CNN-LSTM

CNN-GRU

No. of parameters Accuracy (%) Overfitting ratio ROC:AUC P-R:AP Precision 0 1 Recall 0 1 F1 score 0 1

1258 98.86 0.891 1 1 0.99 0.98 0.99 0.99 0.99 0.99

4048 97.48 0.746 1 0.99 0.98 0.97 0.97 0.98 0.98 0.97

3098 97.10 0.816 0.99 0.99 0.97 0.97 0.97 0.97 0.97 0.97

5.4.2 Malaria According to Table 5.4, the proposed CNN-RNN model produced best results for the detection of malaria with reduced number of learnable parameters, similar to TB detection. Detection results for this model for malaria diagnosis is shown in Fig. 5.5. Here also, the middle row, which contains false detections (highest scored negative labeled test patches), contains slightly similar shapes and patterns of pathogens. Therefore, this model helps humans to recheck the conclusion of patches to be positive or not such that appropriate decisions can be made for better diagnosis. Comparing the results, the proposed CNN-RNN model produced AUC and AP value equal to 1 with less number of parameters and better overfitting ratio. ROC and P-R curves for the best model are plotted in Fig. 5.6. TPR-FPR curve is also plotted in Fig. 5.7.

5.4.3 Intestinal parasites In the case of intestinal parasites, the size of pathogen to be detected is large comparing with pathogens present in other two datasets. The proposed CNN-RNN model performs far better than other two models, which is evident from Table 5.5. CNNLSTM and CNN-GRU models produced poor results in detecting positive patches for intestinal parasites. ROC and P-R curves for the best model are plotted in Fig. 5.8. TPR-FPR curve is also plotted in Fig. 5.9. Here, Recall and F1 score are considerably low for CNN-LSTM and CNN-GRU model in detecting positive patches. This is due to the low proportion of positive patches present in the dataset, which leads to a poorly trained model. The proposed CNN-RNN model overcomes this limitation

119

1.000

1.000

1.000

1.000

1.000

1.000

1.000

1.000

1.000

1.000

1.000

1.000

0.998

0.998

0.997

0.997

0.997

0.997

0.997

0.997

0.996

0.996

0.996

0.995

0.000

0.000

0.000

0.000

0.000

0.000

0.000

0.000

0.000

0.000

0.000

0.000

FIG. 5.5 Detection results of malaria: correct detections—highest scored positive labeled test patches (top); false detections—highest scored negative labeled test patches (middle); and lowest scored test patches (bottom).

5.4 Experimental results and analysis

Precision-Recall: AP=1.00 1.0

0.8

0.8 Precision

True positive rate

ROC: AUC = 1.00 1.0

0.6 0.4 0.2

0.6 0.4 0.2

0.0 0.0

0.2

0.4 0.6 0.8 False positive rate

1.0

0.0 0.0

0.2

0.4

0.6 Recall

0.8

1.0

FIG. 5.6 ROC and P-R curve obtained using the proposed architecture for malaria detection.

1.0

True positive rate False positive rate

0.8

0.6

0.4

0.2

0.0 0.0

0.2

0.4 0.6 Threshold

0.8

1.0

FIG. 5.7 TPR-FPR curve obtained using the proposed architecture for malaria detection.

associated with dataset and produced excellent results in comparison with other two models. The comparison of the proposed deep model against the existing state-of-the-art CNN architecture is tabulated in Table 5.6. For all the datasets, the proposed model produced better results with a substantial decrease in the number of trainable parameters, in comparison with the state-of-the-art CNN model [1].

121

CHAPTER 5 Patch-based disease diagnosis from microscopic images

Table 5.5 Classification performance measures for intestinal parasites detection obtained using the proposed deep models Proposed models

CNN-RNN

CNN-LSTM

CNN-GRU

No. of parameters Accuracy (%) Overfitting ratio ROC:AUC P-R:AP Precision 0 1 Recall 0 1 F1 score 0 1

8858 99.53 0.717 1 1 1 0.98 1 0.96 1 0.97

34,448 94.28 0.806 0.96 0.78 0.94 0.96 1 0.24 0.97 0.38

25,898 98.36 1.012 0.99 0.92 0.99 0.92 0.99 0.85 0.99 0.89

ROC: AUC = 1.00

Precision-Recall: AP = 0.98

1.0

1.0

0.8

0.8 Precision

True positive rate

122

0.6 0.4 0.2

0.6 0.4 0.2

0.0 0.0

0.2

0.4

0.6

False positive rate

0.8

1.0

0.0 0.0

0.2

0.4

0.6

0.8

1.0

Recall

FIG. 5.8 ROC and P-R curve obtained using the proposed architecture for intestinal parasites detection.

The comparison of the annotations done by both pathologists (white bounding boxes) and proposed model (black boxes) is shown in Figs. 5.10–5.12. It can be observed that model annotations are more for the detection of pathogens than human annotations for all different datasets examined here.

1.0

True positive rate False positive rate

0.8

0.6

0.4

0.2

0.0 0.0

0.2

0.4 0.6 Threshold

0.8

1.0

FIG. 5.9 TPR-FPR curve obtained using the proposed architecture for intestinal parasites detection.

Table 5.6 Performance comparison of the proposed deep model against the state-of-the-art (SOA): CNN architecture [1] Dataset

Model

Tuberculosis

SOA:CNN Proposed model SOA:CNN Proposed model SOA:CNN Proposed model

Malaria

Intestinal parasites

No. of parameters

ROC: AUC

AP

77,646 1258

1 0.99

0.97 0.98

386,046 1258

0.99 1

0.93 1

296,382 8858

0.99 1

0.93 1

FIG. 5.10 Model annotations (black) versus human annotations (white) for TB detection.

124

CHAPTER 5 Patch-based disease diagnosis from microscopic images

FIG. 5.11 Model annotations (black) versus human annotations (white) for malaria detection.

FIG. 5.12 Model annotations (black) versus human annotations (white) for intestinal parasites detection.

5.5 Conclusion For all of the three disease diagnosis tasks, the proposed CNN-RNN architecture performs better than the other two proposed deep architectures using LSTM and GRU along with CNN. RNN layer is the simplest layer among recurrent layers. Since there is no need of capturing long-term dependent sequential information in the case of images, CNN-RNN model performs better with less overfitting than CNN-LSTM and CNN-GRU model. Also, this is achieved with a reduced number of learnable parameters when compared with the other two models. The performance of the proposed deep models points out that the model can recommend regions likely to contain pathogens, which cannot be observed by the naked eye of pathologists. The limitation of the present work is the scalability of the input patch size. This achievement can lead point of care diagnostics into the next level, which is especially significant in the developing world, where the smart phones and microscopes are more easily accessible than laboratory experts. Even with the presence

References

of laboratory staff, the proposed model can be used as a decision backing tool for detecting possible pathogens in an image, which helps to confirm the final and accurate decision by the technician. This approach can help laboratory staff attain consistency in the diagnosis by concentrating more on the model detected regions of the image, where more chance lies for the occurrence of pathogens. This helps to provide better diagnosis and treatment for the disease. In the future analysis, flexibility of the model should be checked on different microscopic datasets. This will help in creating a generalized model for diverse microscopic image datasets, which helps in detecting patches, microbes, or proteins, etc., which assists the medical field in providing better diagnosis and treatment for people.

References [1] J.A. Quinn, R. Nakasi, P.K.B. Mugagga, P. Byanyima, W. Lubega, A. Andama, Deep convolutional neural networks for microscopy-based point of care diagnostics, in: Machine Learning for Healthcare Conference, 2016, pp. 271–281. [2] K. Lang, D. Wang, S. Fong, et al., A survey of data mining and deep learning in bioinformatics, Appl. Soft Comput. 42 (8) (2018) 139. [3] S. Beagum, N. Dey, A.S. Ashour, Nonparametric de-noising filter optimization using structure-based microscopic image classification, Microsc. Res. Tech. 80 (4) (2017) 419–429. [4] Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature 521 (7553) (2015) 436–444. [5] K. He, X. Zhang, S. Ren, J. Sun, Delving deep into rectifiers: surpassing human-level performance on ImageNet classification, in: Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1026–1034. [6] M.M. Usaj, E.B. Styles, A.J. Verster, H. Friesen, C. Boone, B.J. Andrews, High-content screening for quantitative cell biology, Trends Cell Biol. 26 (8) (2016) 598–611. [7] W. Yu, et al., Mice liver cirrhosis microscopic image analysis using gray level co-occurrence matrix and support vector machines. Frontiers in artificial intelligence and applications, in: International Conference on Information Technology and Intelligent Transportation Systems (ITITS), 2017, pp. 509–515. [8] Y. Wang, et al., Classification of mice hepatic granuloma microscopic images based on a deep convolutional neural network, Appl. Soft Comput. 74 (1) (2018) 40–50. [9] C. Laufer, B. Fischer, M. Billmann, W. Huber, M. Boutros, Mapping genetic interactions in human cancer cells with RNAi and multiparametric phenotyping, Nat. Methods 10 (5) (2013) 427. [10] V. Ljosa, P.D. Caie, R. Ter Horst, K.L. Sokolnicki, E.L. Jenkins, S. Daya, M.E. Roberts, T.R. Jones, S. Singh, A. Genovesio, et al., Comparison of methods for image-based profiling of cellular morphological responses to small-molecule treatment, J. Biomol. Screen. 18 (10) (2013) 1321–1329. [11] Y.T. Chong, J.L.Y. Koh, H. Friesen, S.K. Duffy, M.J. Cox, A. Moses, J. Moffat, C. Boone, B.J. Andrews, Yeast proteome dynamics from single cell imaging and automated analysis, Cell 161 (6) (2015) 1413–1424. [12] M. Breker, M. Schuldiner, The emergence of proteome-wide technologies: systematic analysis of proteins comes of age, Nat. Rev. Mol. Cell Biol. 15 (7) (2014) 453.

125

126

CHAPTER 5 Patch-based disease diagnosis from microscopic images

[13] B.T. Grys, D.S. Lo, N. Sahin, O.Z. Kraus, Q. Morris, C. Boone, B.J. Andrews, Machine learning and computer vision approaches for phenotypic profiling, J. Cell Biol. 216 (1) (2017) 65–71. [14] S.-C. Chen, T. Zhao, G.J. Gordon, R.F. Murphy, Automated image analysis of protein localization in budding yeast, Bioinformatics 23 (13) (2007) i66–i71. [15] N. D
enervaud, J. Becker, R. Delgado-Gonzalo, P. Damay, A.S. Rajkumar, M. Unser, D. Shore, F. Naef, S.J. Maerkl, A chemostat array enables the spatio-temporal analysis of the yeast proteome, Proc. Natl Acad. Sci. USA 110 (39) (2013) 15842–15847. [16] A.X. Lu, A.M. Moses, An unsupervised kNN method to systematically detect changes in protein localization in high-throughput microscopy images, PLoS ONE 11 (7) (2016) e0158712. [17] P. Liberali, B. Snijder, L. Pelkmans, Single-cell and multivariate approaches in genetic perturbation screens, Nat. Rev. Genet. 16 (1) (2015) 18. [18] O.Z. Kraus, B.J. Frey, Computer vision for high content screening, Crit. Rev. Biochem. Mol. Biol. 51 (2) (2016) 102–109. [19] Z. Li, N. Dey, et al., Convolutional neural network based clustering and manifold learning method for diabetic plantar pressure imaging dataset, J. Med. Imaging Health Inf. 7 (3) (2017) 639–652. [20] O. D€urr, B. Sick, Single-cell phenotype classification using deep convolutional neural networks, J. Biomol. Screen. 21 (9) (2016) 998–1003. [21] O.Z. Kraus, J.L. Ba, B.J. Frey, Classifying and segmenting microscopy images with deep multiple instance learning, Bioinformatics 32 (12) (2016) i52–i59. [22] T. P€arnamaa, L. Parts, Accurate classification of protein subcellular localization from high-throughput microscopy images using deep learning, G3 Genes Genomes Genet. 7 (5) (2017) 1385–1392. [23] P. Eulenberg, N. Koehler, T. Blasi, A. Filby, A.E. Carpenter, P. Rees, F.J. Theis, F.A. Wolf, Deep learning for imaging flow cytometry: cell cycle analysis of Jurkat cells, Nat. Commun. (2016), https://doi.org/10.1038/s41467-017-00623-3. [24] D.C. Cireşan, A. Giusti, L.M. Gambardella, J. Schmidhuber, Mitosis detection in breast cancer histology images with deep neural networks, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2013, pp. 411–418. [25] F. Ning, D. Delhomme, Y. LeCun, F. Piano, L. Bottou, P.E. Barbano, Toward automatic phenotyping of developing embryos from videos, IEEE Trans. Image Process. 14 (9) (2005) 1360–1371. [26] Y. Qiu, X. Lu, S. Yan, M. Tan, S. Cheng, S. Li, H. Liu, B. Zheng, Applying deep learning technology to automatically identify metaphase chromosomes using scanning microscopic images: an initial investigation, in: Biophotonics and Immune Responses XI, vol. 9709, International Society for Optics and Photonics, 2016, p. 97090K. [27] W. Xie, J.A. Noble, A. Zisserman, Microscopy cell counting and detection with fully convolutional regression networks, Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 6 (2016) 283–292. [28] L. Rosado, J.M. Correia da Costa, D. Elias, J.S. Cardoso, A review of automatic malaria parasites detection and segmentation in microscopic images, Anti-Infect. Agents 14 (1) (2016) 11–22. [29] S. Beagum, A.S. Ashour, N. Dey, Bag-of-features in microscopic images classification, in: Medical Imaging: Concepts, Methodologies, Tools, and Applications, vol. 1, 2017, pp. 435–456, ISBN-13: 978-1522505716; ISBN-10: 1522505717.

Further reading

[30] M. Rico-Garcia, A. Salazar, C.-A. Madrigal, L.-J. Morantes-Guzman, F.M. CortesMancera, Detection of Mycobacterium tuberculosis in microscopic images of ZiehlNeelsen-stained sputum smears, in: Proc. 6th Latin-American Conference on Networked and Electronic Media (LACNEM 2015), 2015, pp. 1–6. [31] Y.S. Yang, D.K. Park, H.C. Kim, M.-H. Choi, J.-Y. Chai, Automatic identification of human helminth eggs on microscopic fecal specimens using digital image processing and an artificial neural network, IEEE Trans. Biomed. Eng. 48 (6) (2001) 718–730. [32] D. Avci, A. Varol, An expert diagnosis system for classification of human parasite eggs based on multi-class SVM, Expert Syst. Appl. 36 (1) (2009) 43–48. [33] F. Visin, K. Kastner, K. Cho, M. Matteucci, A. Courville, Y. Bengio, Renet: a recurrent neural network based alternative to convolutional networks, Computing Research Repository (CoRR)abs/1505.00393, (2015).

Further reading R. Sachin, V. Sowmya, D. Govind, K.P. Soman, Dependency of various color and intensity planes on CNN based image classification, in: International Symposium on Signal Processing and Intelligent Recognition Systems, Springer, 2017, pp. 167–177. R. Vinayakumar, K.P. Soman, P. Poornachandran, S. Sachin Kumar, Detecting Android malware using long short-term memory (LSTM), J. Intell. Fuzzy Syst. 34 (3) (2018) 1277–1288.

127