Strided fully convolutional neural network for boosting the sensitivity of retinal blood vessels segmentation

Expert Systems With Applications 134 (2019) 36–52

Contents lists available at ScienceDirect

Expert Systems With Applications journal homepage: www.elsevier.com/locate/eswa

Strided fully convolutional neural network for boosting the sensitivity of retinal blood vessels segmentation Toufique Ahmed Soomro a,1,∗, Ahmed J. Afifi b,1, Junbin Gao c, Olaf Hellwich b, Lihong Zheng d, Manoranjan Paul a a

School of Computing and Mathematics,Charles Sturt University, Bathurst, Australia Computer Vision & Remote Sensing, Technische Universität Berlin, Berlin, Germany The University of Sydney Business School, The University of Sydney, Camperdown, Australia d School of Computing and Mathematics,Charles Sturt University, Wagga Wagga, Australia b c

a r t i c l e

i n f o

Article history: Received 8 November 2018 Revised 21 May 2019 Accepted 23 May 2019 Available online 24 May 2019 Keywords: Retina Retinal images Vessels Strided-CNN Pool-CNN

a b s t r a c t In this paper, a deep convolutional neural network (CNN) is proposed for accurate segmentation of retinal blood vessels. This method plays a significant role in observing many eye diseases. A strided-CNN model is proposed for accurate segmentation of retinal vessels, especially the tiny vessels. The model is a fully convolutional model consisting of an encoder part and a decoder part where the pooling layers are replaced with strided convolutional layers. The strided convolutional layer approach was chosen over the pooling layers approach as the former can be trained. The morphological mappings along with the Principal Component Analysis (PCA)- based pre-processing steps are used to generate contrast images for training dataset. Skip connections are implemented to concatenate features from the encoder part and the decoder part to enhance the vessels segmentation especially the tiny vessels and to make the vessel’s edges sharper. We used a class balancing loss function to train and optimize the proposed model to improve vessel image quality. The impact of the proposed segmentation method is evaluated on four databases namely DRIVE, STARE, CHASE-DB1 and HRF. Overall model performance, particularly with respect to tiny vessels, is primarily influenced by sensitivity and accuracy metrics. We demonstrate that our model outperforms other models with a sensitivity of 0.87, 0.808, 0.886 and 0.829 on DRIVE, STARE, CHASE_DB1 and HRF respectively, along with respective accuracies of 0.956, 0.954, 0.976 and 0.962. © 2019 Elsevier Ltd. All rights reserved.

1. Introduction The abnormalities in a retinal image are one of the main observations for many eye diseases that lead to vision loss such as agerelated macular degeneration, glaucoma (Olafsdottir, Hardarson, Gottfredsdottir, Harris, & Stefansson, 2011) and diabetic retinopathy (DR) (Hardarson & Stefansson, 2012). These diseases could be diagnosed by analyzing the changes in retinal blood vessels. The DR is considered as the leading cause of blindness in the developed and developing countries by World Health Organization (WHO) (Diabetic Retinopathy Detection, 2015). It is estimated that DR affects more than 347 million people worldwide. A fast and accurate detection of retinal vessels is critical to observe the changes in the

∗

Corresponding author. E-mail addresses: [email protected] (T.A. Soomro), [email protected], ahmed.afifi@campus.tu-berlin.de (A.J. Afifi), [email protected] (J. Gao), [email protected] (O. Hellwich), [email protected] (L. Zheng), [email protected] (M. Paul). 1 These authors contributed equally to this work. https://doi.org/10.1016/j.eswa.2019.05.029 0957-4174/© 2019 Elsevier Ltd. All rights reserved.

retinal blood vessels, diagnose the severity level of DR, and suggest a proper treatment plan (JJ, G, R, E, & MD, 2007; LD, RJ, WN, LX, & R, 1999; Patton, Aslam, Macgillivray, Pattie, & Deary, 2005). The advanced technologies in image processing and computer vision can assist in accurate segmentation of retinal blood vessels to measure DR severity, and lead to a quick recommendation for an effective treatment. The implementation of computerised methods is necessary to better observe the details of the retinal fundus images. A fundus camera (PJ & ME, 2002) is a specialized camera for acquiring retinal images that gives observations of retina features like vessels, optic disc and abnormalities in the retina. Segmenting vessels from the retinal images is tedious and time consuming process that typically takes two to three days to accurately compile the observations. Retinal blood vessels segmentation is currently a manual process done by an expert for further clinical assessments of the disease and may be prone to errors including user bias (HM, E, SC, MK, & RS, 2005; LD et al., 1999; TY, MD, R, BEK, & SM, 2004). Therefore, a computerised system capable of

T.A. Soomro, A.J. Afifi and J. Gao et al. / Expert Systems With Applications 134 (2019) 36–52

generating the segment images is highly desired to replace manual segmentation. The computerized systems for medical image analysis have their own challenges due to the complexity and heterogeneity of the retinal images (Soomro, Munir, & Choi, 2018). Usually, they require user interaction to diagnose the diseases by searching the most distinguished features from images (Nguyen, Bhuiyan, Park, & Ramamohanarao, 2013). Some data mining approaches have been proposed to extract the most efficient features. Low level features such as morphological features, color features, vessel width, texture, sharpness of vessel boundary as well as the coarseness inside a blood vessel have been incorporated, although some require advanced technology and are not suitable for remote clinics without trained specialists. Middle-level features such as a saliency map extracted from fundus images are applied to train classifiers with the purpose of extracting retinal blood vessels (Xua, Dinga, Abramoff, & Caoa, 2017). Others treat the task as solving an optimal function in terms of minimizing the cost function such as graph-based approach, K-nearest neighbors (KNN), Gaussian mixture model (GMM), and entropy thresholding based models (Narasimhan & Vijayarekha, 2015; Soomro et al., 2018c) among others. Retinal images have vasculature, made of vessels (arteries and veins) that are structured like a tree with roots and branches. Mainly, retinal blood vessels have varying width in different regions and may generally appear as low contrast. Several factors can further compound the challenges in vessel segmentation such as central light reflex, low contrast junctions and crossover of narrow vessels, inclusion of closely parallel vessels, presence of pathologies, varying and low contrast, background noise, and false pixels of vessels segmentation due to optic disc shadow. Most of the issues related to retinal vessels segmentation have been sufficiently addressed by various state-of-the-art methods and validated by their binary results. However, there is still a room for significant improvement to fill some gaps, especially the last three issues. Our main observation from the experiments is that those three cases are interlinked to each other. That is, improving one will result in improving the other two issues. Consequently, we can consider a single point agenda for the research presented in this paper, which is to segment vessels as fine as possible, for which identification is important to diagnose the abnormality. This paper is structured as follows. Section 2 outlines some background related to the problem. The proposed pipeline is elaborated in Section 3. The material and measuring metrics are explained in Section 4. The results and experimental analysis are discussed in Section 5. Section 6 concludes the discussion and future work. 2. Background Many studies in the past were based on the segmentation of retinal blood vessels from retinal fundus images. An extensive review by Soomro et al. (2017b), published in 2017, cited more than 80 works that use image processing techniques, match filtering, morphological operations, vessels tracking and multi-scale techniques. Here, we briefly review the most pertinent work and best performing approaches from that study, compare and discuss their performance with respect to our findings. There are two classes of retinal blood vessels segmentation methods: supervised and unsupervised. Supervised methods require samples of vessels and non-vessels pixels from training databases with help of eye experts. In unsupervised methods, these are not required. Some recent proposed unsupervised methods (Al-Diri, Hunter, & Steel, 2009; Martinez-Perez, AD, & SA, 2007; Martinez-Perez et al., 1999; Mendonca & Campilho, 2006; Soomro & Gao, 2016a; 2016b) are based on multiscale tactics, match filtering techniques and model-based tactics. Unsupervised methods

37

although offer improved performance, the tiny vessels are not properly detected that impacts the overall performance of the segmentation process. The proposed methods in Staal, Abramoff, Niemeijer, Viergever, and van Ginneken (2004), Soares, Roberto, Cesar, Jelinek, and Cree (2006), Lupas, Tegolo, and Trucco (2010), Xinge, Qinmu, Yuan, Yiu-ming, and Jiajia (2011), Marin, Aquino, Gegundez-Arias, and Bravo (2011), Wang, Ji, Lin, and Trucco (2013) and Orlando and Blaschko (2014) are categorized as supervised segmentation methods. During the training, labelled data are used to train a classifier and get the segmented image. Different classifiers are used such as k-nearest neighbors (KNN) (Ricci & Perfetti, 2007; Xinge et al., 2011), Gaussian mixture models (GMM) (Marin et al., 2011; Soares et al., 2006), and support vector machine (SVM). In another approach, Yin, Ng, He, Zhang, and Abbott (2014) proposed a method based on pattern searching and matching i.e. the vessels and nonvessels pixels are determined by matching them with the proposed patterns. The drawback of this method is that it gives false detection on the challenging images. Since the last six years, deep learning methods have become the favorite choice of the researchers for solving computer vision problems. Many deep learning models have been implemented successfully in different applications. In this research paper, we implement the deep convolution neural network (CNN) for accurate detection of vessels, especially the detection of tiny low contrast vessels, which play a vital role in improving the sensitivity of the retinal segmentation procedure. The success of deep CNNs has led many researchers to adopt them to solve different computer vision tasks. Starting from AlexNet (Krizhevsky, Sutskever, & Hinton, 2012) as a base network for object classification, many deeper networks have been proposed to solve different computer vision tasks such as VGGNet (Simonyan & Zisserman, 2015), GoogLeNet (Szegedy et al., 2015), and deep ResNet (He, Zhang, Ren, & Sun, 2016). Moreover, CNNs are applied for non-classification problems, such as object detection and localization, scene segmentation, and depth estimation. They outperform the traditional hand-crafted features methods. Moreover, deep learning-based methods have shown enormous success on image segmentation problem. One of the leading proposed architectures for image segmentation and dense prediction was Fully Convolutional Network (FCN) (Shelhamer, Long, & and, 2017) developed in 2014. As the name suggests, this architecture consists of convolutional layers without using fully-connected layers. This allows the segmented image to be generated from any input image with arbitrary size. Different architectures have been proposed for image segmentation. U-Net (Ronneberger, Fischer, & Brox, 2015) is an encoder-decoder architecture which was proposed to segment medical images. The encoder part extracts the features from the input image and reduces the resolution of the input image through the pooling layers, and the decoder part reconstructs the image and recovers the object details with the help of the skip connections between the encoder and the decoder layers. SegNet (Badrinarayanan, Handa, & Cipolla, 2015) is another architecture proposed for image segmentation. The authors introduced the transferring of max pooling indices instead of the whole values from the encoder to the decoder using skip connections to improve the segmentation resolution and for faster training. As the main goal of our model is to segment the retinal blood vessels images from the preprocessed images, the proposed model should be able to generate an image with the segmented vessels. Our model is a deep encoder-decoder model with skip connections between the encoder part and the decoder part to generate the segmented vessels image. Inspired by VGG-16 Net model (Simonyan & Zisserman, 2015), we designed our encoder part by replacing the pooling layers with strided convolutional layers (Springenberg, Dosovitskiy, Brox, & Riedmiller, 2014). Here,

38

T.A. Soomro, A.J. Afifi and J. Gao et al. / Expert Systems With Applications 134 (2019) 36–52

Fig. 1. A complete pipeline of the proposed method.

the encoder network is a fully convolutional network where each convolutional layer is followed by an activation function. The decoder part is mirrored from the encoder. We added two additional layers to the decoder; the upsampling layers and the concatenation layers. The upsampling layers are used to reconstruct the output images from the encoder part so that the output images have the same resolution as input images. The skip connections and the concatenation layers perform as fusion layers that fuse some feature maps from the encoder part with the feature maps from the upsampling layers. The output from each block in the encoder part is fused with the corresponding upsampled output in the decoder part. This fusion step improves the quality of the output image by making the vessel edges sharper, thereby improving the detection of tiny vessels. Choosing the loss function to optimize and train the model is an important step as it depends on the problem at hand and the nature of the training data and the desired output. The output is an image where each pixel in this image is classified either as vessels pixel or as background. The segmented image typically contains 90% background pixels (black pixels) and 10% segmented vessels (white pixels). It is important that the loss function does not bias towards the background pixels during the training process. To overcome this issue, a class balanced cross-entropy loss function (Xie, Saining, & Tu, 2015) was used to train the model. This loss function makes the training process more stable and the segmented images generated from the model are therefore more accurate. Thorough analysis of above mentioned methods show that the sensitivity is compromised as the tiny vessels are not detected. We propose a method based on fully CNN for accurate detection of vessels and boosting the sensitivity of the segmentation process. Our approach results in improved performance over previous methods in terms of statistical analysis and visualization of tiny vessels. The details are explained in Section 3. 3. The method The main components of the proposed approach have been explained here. These include (1) pre-processing for uniform contrast level of retinal blood vessels against their background, and (2) designing and training a CNN model to a well segmented vessels image as shown in Fig. 1. Both steps are elaborated as follows in details. 3.1. Pre-processing In order to implement a Strided CNN for segmentation, the data need to be pre-processed. We use the pre-processing steps as described by Soomro, Afifi, et al. (2017) to handle the issue of varying-low contrast and background noise. The pre-processing is based on two steps. In the first step, the morphological tactics

Fig. 2. Non-uniform illumination removal process for the Red channel. (a) Input image channel. (b) Estimated background image. (c) Estimated uniform image as a result of subtraction operation.

are used to remove the uneven illumination issue and achieve the uniform contrast image. In the second step, Principle Component Analysis (PCA) is used to transform a colored image into a single gray scale image. Following, we explain each step in details. 3.1.1. Non-uniform illumination removal An acquired fundus colored image Irgb (x, y) consists of three channels; Red, Green, and Blue channels. Channel images show large background intensity variation due to uneven illumination. The morphological operations are utilized for correcting the variations. The background is estimated by the mathematical morphological closing operation. The corrected image Iu (x, y) is obtained using Eq. (1) (as shown in Fig. 2(b)) which is the estimated background. The structuring element chosen for the morphological closing operation is a disk with radius 11. The radius of the disk is selected to be larger than the width of the largest vessel presented in the database. The result of these steps is shown in Fig. 2(c). The process is adopted for all three channels independently. Next step, color-to-gray scale conversion based on PCA is done.

Iurgb (x, y ) = [Irgb (x, y ) − closing(Irgb (x, y ))] + mean(closing(Irgb (x, y ))).

(1)

3.1.2. Color conversion into single gray-scale image The grayscale image is more suitable for our analysis as it gives more detailed feature. In medical images, it is critical to preserve the visual features to detect most important clues. In case of retinal images, the observation of the retinal blood vessels plays an important role in observing the progress of eye diseases. Although the background normalization of the retinal RGB channels gives proper representation for further processing, the conversion of RGB into gray scale shows more promising results. The main objective is to convert the color channels (red, green and blue) into a single channel gray scale image. Unlike previous studies, we use red channel, green channel and blue channel instead of green channel alone to generate a gray scale image. We use PCA in the current

T.A. Soomro, A.J. Afifi and J. Gao et al. / Expert Systems With Applications 134 (2019) 36–52

39

Fig. 3. Block diagram of the proposed PCA-based color-to-gray image conversion (Soomro, Afifi, et al., 2017). (For interpretation of the references to color in the text, the reader is referred to the web version of this article.)

work to achieve the gray scale conversion as shown in Fig. 3. The PCA transformation is used to rotate the axes from intensity values of color space to three orthogonal axes to provide an efficient color to gray conversion. The color-to-gray conversion is used to combine all the three channels that are already treated with their respective non-

uniform removal process and contrast enhancement as well as noise reduction process. The mathematical representation of colorto-gray conversion is elaborated by Soomro, Afifi, et al. (2017) in details. Of note, PCA-based method results in much more discriminated image with respect to vessels, and the histogram is more full than that of uniform background green image. The histogram

40

T.A. Soomro, A.J. Afifi and J. Gao et al. / Expert Systems With Applications 134 (2019) 36–52

Fig. 4. Improved Contrast for PCA based color-to-gray conversion: (a) is the histograms of green channel used largely by previous researchers for vessel extraction process. (b) is the histograms of PCA image.We observe better contrast for PCA-based conversion as its histogram is more rich in representing all different levels of available intensity range. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

of Igray is depicted in Fig. 4(b), which is more spread and represents more levels of intensity as compared to the green image histogram as depicted in Fig. 4(a). 3.2. CNN architecture Designing CNN models depends mostly on the context and on the nature of the data that will be used for training. Simple CNN models, such as AlexNet (Krizhevsky et al., 2012), consist of consecutive blocks of convolutional layers, each followed by an activation function and pooling layers to decrease the features resolution and the computation cost, and some task-specific layers such as fully connected layers (FC) and the output layer that depends on the nature of the task to be solved. These layers perform linear and non-linear operations to extract features that are used to train the network and solve the problem. The proposed CNN model is a deep encoder-decoder model that is trained end-to-end similar to SegNet (Badrinarayanan et al., 2017) as shown in Fig. 5. The encoder comprises five fully consecutive convolutional blocks with the size of 16, 32, 64, 128 and 256 features, respectively. The first two blocks consist of two convolutional layers, and the remaining blocks consist of three convolutional layers. Each convolutional layers has a kernel of size 3 × 3

and a stride of 1, and is followed by leaky Rectified Linear Unit (LReLU) (Xu, Wang, Chen, & Li, 2015) as an activation function. To decrease the resolution of the feature maps after each block, we use strided convolutional layers (Springenberg et al., 2014) instead of the pooling layers as the later ones are mostly used in CNN to decrease the features size. Springenberg et al. (2014) showed that max-pooling layers can be simply replaced by convolutional layers with increased stride. We train and test two models, one uses strided convolutional layers and the other one uses max-pooling layers. Our results show that the strided model performs much better than the model using the max-pooling layer quantitatively and qualitatively. The strided convolutional layers are better because they can be easily reversed and these layers can be trained and tuned rather than fixing them to max or average operations. Here, our strided convolutional layers are of 2 × 2 kernel size with a stride of 2. This decreases the size of the feature maps to a half. The decoder part mirrors the encoder part with some additional layers, which reconstructs the image again and generates the segmented image with the same size of the input image. The size of the features for each convolutional block in the decoder is 256, 128, 64, 32 and 16 respectively. More specifically, we replace the strided convolutional layers in the encoder part by the upsampling (upconvolution) layers of 3 × 3 kernel size in the decoder

Fig. 5. The proposed CNN Architecture is an encoder-decoder model. In the encoder, each conv block consists of 2 or 3 conv layers (in blue), LReLU, and strided-conv (in orange). In the decoder, the upsampling block (in red) is a deconv layer followed by LReLU. Skip connections (the red arrows and the block in green) concatenate and fuse the output from some layers in the encoder with some outputs in the decoder. The last conv block (in purple) consists of one conv layer followed by a sigmoid function to generate the output image in the testing stage. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

T.A. Soomro, A.J. Afifi and J. Gao et al. / Expert Systems With Applications 134 (2019) 36–52

part followed by LReLU as an activation function in each block. After the upsampling layers, there are convolutional layers with the activation function for performing the nonlinearity. To generate better segmented image with sharp vessels’ edges, we concatenate the output of some layers from the encoder part with the corresponding output of the upsampling layers from the decoder part as shown in Fig. 5 (the concatenation links between the encoder part and the decoder part). After the concatenation, we apply a convolutional operation of 1 × 1 kernel size to fuse the concatenated feature maps. We noticed that the concatenation layers (or as they are called the skip connections) preserve the sharp edges of the vessels especially the tiny vessels, which many methods and models failed to extract them that were missed by many other models. The segmented image is generated using a sigmoid layer. During training, the sigmoid layer is removed because the loss function performs the sigmoid operation (see the next section). During testing, the segmented image is generated from the CNN by applying the sigmoid function to the activation of the last convolutional layer. The pixels’ values are near to 1 for the vessels and near to 0 for the background as shown in Fig. 1. 3.3. Loss function Selecting a suitable loss function is a critical step when considering the nature of the problem and the dataset being used to train the CNN. The loss function measures the error between the generated image and the ground-truth image to optimize and update the model weights. Detailed observation of the ground-truth images show that the distribution of vessels/ background pixels is heavily biased. That is, the image is almost black (90% of the pixels are background), and the vessels’ pixels form almost 10% of the whole image pixels. Optimizing the CNN using a straightforward loss function such as L2 norm, used usually in the regression problems, will bias the training process towards the background. Hence, we here use the classbalancing cross entropy loss function (Xie et al., 2015) to train the proposed model and to generate the segmented image. This loss function takes care of the heavily biased class distributions and balances the training process. Formally, we denote the training dataset by S = {(Xn , Yn ), n = 1, . . . , N}, Xn = {xnj , j = 1, . . . , |Xn |} is the input image with |Xn | the number of pixels in the image ; and Yn = {ynj , j = 1, . . . , |Xn |} is the corresponding ground-truth binary segmented image, where ynj ∈ {0, 1}. For simplicity, we drop the subscript n because we consider each image independently. The loss function is defined as:

L(W ) = − β



log p(y j = 1|X ; W )

j∈Y+

− (1 − β )



log p(y j = 0|X ; W ),

(2)

j∈Y−

In Eq. (2), W is the CNN weights to be optimized during the backpropagation. The class-balancing parameter, β , is to balance the training process between the foreground pixels Y+ , and the background pixels Y− . β = |Y− |/|Y |, and (1 − β ) = |Y+ |/|Y |. The function p( · ) is the sigmiod function σ ( · ) which is applied on the activation of the final convolutional layer to generate the segmented images in the testing stage. 4. Experimental design This section contains four parts. The first part gives information about diagnostic problem and data, the second part contains information of CNN implementation details, the third part explains the preparation of the training data; and the last part contains information about measuring metrics. Each part is explained below.

41

4.1. Databases We used four benchmark databases, Digital Retinal Images for Vessel Extraction (DRIVE) (Staal et al., 2004), Structured Analysis of the Retina (STARE) (Soares et al., 2006), The High Resolution Fundus image (HRF) (Budai, Hornegger, & Michelson, 2009) and CHASE_DB1 (Owen et al., 2009). These four different datasets cover a wide content and challenging images. It provides the opportunity to compare the performance of this segmentation method against state-of-the-art methods. The DRIVE database contains 40 colored retinal fundus images that have different progress level of disease. The 3CCD camera is used to capture images, and the anatomical structure occupies the central circular area with a radius approximately 540 pixels. The database is split into two types of images groups namely training set and test set. Both sets contain 20 images taken from different patients. For each image, a manual segmentation of retinal blood vessels as the gold standard is provided. The STARE database contains 20 colored retinal fundus images along with their mask images and ground truth images. Topcon TRV-50 fundus camera was used to acquire STARE database and the image has a resolution of 605 × 700 pixels. 50% of the images in STARE databases contains the pathologies and evaluation of retinal blood vessels segmentation method is challenging on this database. HRF database contains three sets of fundus images namely: healthy retinas, glaucomatous retinas and DR retinas. Each set consists of 15 colored retinal images along with the corresponding manual segmented images and the mask images. HRF database contains 75% pathological images. All images of HRF database were captured with mydriatic fundus camera Canon CF-60 UVi armed with Canon EOS-20D digital camera with the 60-degree field of view (FOV). The HRF image contains the size of 3504 × 2336. CHASE_DB1 database contains 28 retinal fundus images acquired from multiethnic school kids. A hand-held Nidek -NM-200D fundus camera was used to capture these images at 30◦ FOV. The images have a size of 960 × 999 pixels. The CHASE_DB1 database contains manually segmented images as ground truth images. 4.2. CNN implementation details MatConvNet (Vedaldi & Lenc, 2015), a MATLAB toolbox implementing CNNs for computer vision applications, is used to train and evaluate our proposed model. We initialize the weights of the layers using Xavier initialization method (Glorot & Bengio, 2010). The model is trained from scratch using backpropagation. Stochastic Gradient Descent (SGD) is used to optimize the network with the following settings: the momentum is set to 0.9, and the weight decay is set to 10−5 . The learning rate is initialized to 10−3 and is decreased when the validation error does not change. The training process was repeated until the validation accuracy stopped increasing, and the best performing model was selected for validation on the test datasets. 4.3. Training images preparation The training of the proposed network is performed on the training images from each database. DRIVE database has 40 images in total, split into 20 images for training and 20 images for testing. STARE database has 20 images, split into 10 images for training and 10 images for testing. CHASE_DB1 database has 28 images. We followed the split of Fraz et al. (2012), where the first 8 images are used for training and the remaining 20 images are used for testing. For HRF dataset, this dataset contains three classes of images. Each class has 15 images. The first 10 images are used for training and the remaining 5 images are used for testing. Table 1 summarizes the databases and the split sets.

42

T.A. Soomro, A.J. Afifi and J. Gao et al. / Expert Systems With Applications 134 (2019) 36–52 Table 1 Databases information. Databases

Number of images

Testing images

Training images

Normal images

Challenging images

DRIVE STARE HRF CHASE_DB1

40 20 45 28

20 10 15 20

20 10 30 8

20 10 15 0

20 10 30 28

From each image in the training set, we extract patches to increase the number of training images besides the whole images. Each extracted patch has the size half of the original image (half of the width and the height). Then we resize them to the original size. These patches are overlapped and they are extracted from the whole image. After generating the patches, we split the generated images into training and validation sets. The training set contains 90% of randomly selected images, and the remaining 10% is used for validation. No further data augmentation is performed. To summarize, we extracted 871, 1086, 1202, and 1240 patches from each image of the DRIVE, STARE, CHASE_DB1, and HRF databases, respectively. If the patches are extracted from the central regions of the image, the learning process will be biased. In our case, the training process is formed on the basis of complete images and subimages (patches) extracted from pre-processed full images. Each patch is with the half-dimension of the full image. The patches are obtained by randomly selecting its centre in the full image. After extracting the patches, it was clearly found that the patches cover almost all regions of the image and come from the border regions and from the centre. Thus, some of the extracted areas come from border areas and many areas contain vessels and the border region. The network trains on all regions that can be seen in the full image. In addition, the same patches of groundtruth have been extracted so that we have the patches and their groundtruth. To insure that the patch will not exceed the image region and can cover the whole image region, we restrict the center of the patches to be from the area inside the green rectangle, shown in Fig. 6(a). Fig. 6(b), and (c) show the visualization of samples of these patches, which clearly analyses that they come from different regions of the full images.

Here, Se is the sensitivity that is indicating to what extent the classification of vessels and non-vessels pixels. Sp is the specificity, which is the classifier’s ability to identify non-vessels pixels. AC is the accuracy of the segmentation algorithm, reflecting the ratio between the collection of all properly sized pixels (vessels or non-vessels pixels) to all pixels of segmented image. False Positive Rate (FPR) is the fraction of pixels wrongly classified as vessels pixels and is equal to 1 − Sp. A receiver operating characteristic curve (ROC) is plotted with Se against FPR while modifying the threshold on the probability map due to the dependence of the quantitative measure on the threshold. The performance of the methods is also evaluated in the area under the ROC curve (AUC). In the case of an ideal classifier, the AUC must always be equal to 1. The ROC curve is considered as an appropriate measure of accuracy in the classification of objects when the distribution is unbalanced. For the validation and the selection of the optimal epoch, we measure the Dice Similarity Coefficient (DSC) and the Intersection on the Union (IoU). These two metrics are used to determine the segmented image of each epoch with a comparison of the ground truth. The DSC and IoU scores are between (0, 1), with higher values indicating better performance. The main calculation of these two metrics is based on the confusion matrix between the ground truth and the segmented image. These two matrices are calculated according to the Eqs. (6) and (7) (Fu et al., 2018).

4.4. Measuring metrics

5. Results and discussion

Four common criteria are used to evaluate the performance of the proposed method. These metrics are sensitivity (Se), specificity (Sp), accuracy (AC) and ROC AUC. The mathematical representation and calculation of these metrics are,

The experiments and results section is divided into eight parts, each part has been elaborated with details as follows.

Se =

TP . TP + FN

(3)

Sp =

TN . TN + FP

(4)

AC =

TP + TN . TP + FN + TN + FP

(5)

DSC =

2T P . 2T P + F P + F N

(6)

IoU =

TP . TP + FP + FN

(7)

1. 2. 3. 4. 5. 6. 7. 8.

Performance on Pool-CNN Model and Strided-CNN Model. Performance on Databases. Analysis on the effect of Pre-Processing Steps. Performance on different training CNN Epochs. Performance on Challenging images. Analysis of thin vessels and thick vessels. Comparison with Deep Learning Methods. Comparison with Other Retinal Vessels Extraction Methods.

Fig. 6. A complete image and the extracted patches. The center of the patches are selected randomly from the square in (a). (b) Training image patches. (c) Corresponding ground-truth patches.

T.A. Soomro, A.J. Afifi and J. Gao et al. / Expert Systems With Applications 134 (2019) 36–52

Fig. 7. Performance analysis of the Pool-CNN and Strided-CNN on different databases with respect to the measuring metrics.

From now on, we consider the performance over the whole images regions (the background and the retina). 5.1. Performance on pool-CNN model and strided-CNN model We implemented the pool-CNN model and strided-CNN model and validated their performance on DRIVE and STARE database to select the best CNN model. Our results indicate that the StridedCNN model is better than the pool-CNN on basis of performance and visual observation of vessels images. Fig. 7 shows the performance of the pool-CNN model and the strided-CNN model on four databases. The strided-CNN model gives higher sensitivity around 0.80 to 0.886 maintaining higher accuracy around 0.91 to 0.976, while pool-CNN model provides good sensitivity around 0.80 to 0.84 as compared to other existing methods but it gives low accuracy around 0.81 to 0.912. Of note, tiny vessels as well as large vessels are not detected by the pool-CNN model. As a part of further validation, we observed the vessels of these database images of both CNN-model. We randomly selected varied contrast retinal images from databases as shown in Fig. 8. We observed that the Strided-CNN model gives a comparable vessels detection as ground truth images, and as well more vessels than

43

Fig. 8. Performance on randomly selected images between the Stride-CNN Model output images and the Pool-CNN Model output images. The first image in every row is the original color retinal fundus images. The second image in every row is ground truth image. The third image in the every row is the Stride-CNN model output image. The fourth image in every row is pool-CNN model output image. Note: It is observed that there are noisy pixels as well as an optic disc shadow in the Stride-CNN output image. We used image processing tactics based on morphological image reconstruction techniques to remove noisy pixels as well as an optical shadow to allow proper observation of the vessels. The final segmented images of the vessels are shown in the Fig. 10. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

the pool-CNN model vessels images. This suggests that the StridedCNN model gives a much better performance as well as accurate segmented images. The segmented final vessel images are illustrated in Fig. 10, that also show noisy pixels and an optic disc shadow. We used some image processing tactics to remove noise pixels and shadow from the optical disc. Final vessels images are shown in Fig. 10. To justify the use of strided convolution layers, we explain the difference between using pooling layers and strided convolutional layers for the retinal vessels segmentation. The main roles of the pooling layers are to progressively down-sample the features and reduce the spatial size of the outputs. This leads to reduction in the number of the parameters thereby reducing the required computation in the network. Also, it prevents the network from overfitting. In classification problems, if the features have a specific value that is a high activation value after the convolutional layer, its exact location is not as important as the value itself. Overall, the pooling layer reduces the spatial size and keeps the high activation value.

44

T.A. Soomro, A.J. Afifi and J. Gao et al. / Expert Systems With Applications 134 (2019) 36–52

Fig. 9. ROC curves for: DRIVE, STARE, CHASE_DB1 and HRF databases. DRIVE database gives 0.986, STARE database gives 0.988, CHASE_DB1 gives 0.985 and HRF database gives 0.978.

On the other hand, the strided convolutional layers are trainable and they extract useful features that can improve network performance. They play the role of decreasing the spatial resolution of the features. Moreover, the strided convolutional layers preserve the spatial locations in the features and summarize the information without loss in accuracy. Memorizing the spatial locations is important in our case to detect and segment the tiny vessels. 5.2. Performance on databases We have computed and analysed the performance of proposed segmentation method on the DRIVE, STARE, HRF and CHASE_DB1 databases as reported in Table 2 at Epoch 3. Higher sensitivity indicates detection of tiny vessels. The experimental results give a higher sensitivity of 0.87 on DRIVE databas, 0.848 on STARE database, 0.829 on HRF database and 0.886 on CHASE_DB1 database as compared to other existing methods. This demonstrates that the proposed method extracts more tiny vessels closer to ground truth image as shown in Fig. 10. The ROC curves of each database are measured to quantify the proposed segmentation, and it is shown in Fig. 9. The ROC curve is Table 2 Performance model.

of

proposed

strided-CNN

Database

Se

Sp

AC

DRIVE STARE HRF CHASE_DB1

0.87 0.848 0.829 0.886

0.985 0.986 0.961 0.982

0.956 0.968 0.962 0.976

Fig. 10. The manual ground truth images along with their corresponding segmented images. First row contains original retinal color images, second row contains the manual ground truth images which are randomly selected from databases, and third row contains segmented images are achieved through our proposed method. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

T.A. Soomro, A.J. Afifi and J. Gao et al. / Expert Systems With Applications 134 (2019) 36–52 Table 3 Analysis of segmentation model with and without pre-processing. Method

Without pre-processing

Database

Se

Sp

AC

Se

Sp

AC

DRIVE STARE HRF CHASE_DB1

0.678 0.653 0.633 0.644

0.686 0.642 0.641 0.655

0.652 0.621 0.623 0.631

0.87 0.848 0.829 0.886

0.985 0.986 0.962 0.982

0.956 0.968 0.962 0.976

with less noisy pixels than those epochs have low accuracy and high sensitivity, for examples such as performance achieved epoch 1 on DRIVE database and 1 and 5 on STARE database (as shown in Fig. 11). Another case is that some epochs output gives increased accuracy but decreased sensitivity. For example epochs 4, 5 and 6 in DRIVE, and 1, 4 and 5 in STARE database, and 4, 5 and 6 on HRF and CHASE_DB1 databases are missing the tiny vessels (as shown in Fig. 11). We analyse the complex vessels of the segmented image in each epoch, as shown in the Fig. 12. The missing vessels are represented by a red circle and it is clearly observed that the vessels are not so dropped at the optimal Epoch 3. It is validated that Epoch 3 gives more vessels observations and better performance. In addition, we validated the performance of our segmented images in each database by measuring the DSC score and the IoU score. Higher DSC and IoU were achieved at Epoch 3, implying Epoch 3 gives much better image in terms of performance as well as observation of the tiny vessels (shown in Table 4).

With pre-processing

considered as an appropriate accuracy calculation in the classification of objects when the distribution is unbalanced. Consider the AUC measurement of each database, the value achieved for four databases are 0.986 for the DRIVE database, 0.988 for the STARE database, 0.985 for CHASE_DB1, and 0.985 for the HRF database. These values show the validation of a proposed method for the good segmentation of retinal blood vessels. This validates that the proposed method extracts more tiny vessels closer to ground truth image as shown in Fig. 10. Although all the vessels are correctly observed, the image is always blurred because all the images contain anomalies and a central reflex. We used simple image processing techniques as additional postprocessing to validate vessels without noise. It is successfully validated and the output images are shown in Fig. 10.

5.5. Performance on challenging images Various vessels segmentation methods have been proposed in this field. The key issues that still need to be addressed contain: vessels segmentation in the presence of central light reflex, retinal vessels segmentation in presence of abnormalities, and retinal vessels segmentation in presence of low-varying contrast conditions. We carried out in-depth study for each of these cases and observed that retinal vessels segmentation becomes more challenging due to these previously outlined. Fig. 13 illustrates that segmented image areas of our proposed method make it possible to correctly observe the retinal blood vessels under these conditions. The following subsection elaborates the performance of our proposed method on challenging images of different databases.

5.3. Analysis on the effect of pre-processing steps To test the effectiveness of the proposed method, the performance of the CNN segmentation model is evaluated with and without the pre-processing steps. The pre-processing steps play an important role in improving the performance of the segmentation method. Table 3 presents the performances of the proposed method with and without pre-processing. It is clear that the performances improve with the pre-processing. The preprocessing steps remove uneven lighting, reduce noise, and make the image evenly contrasted to produce a well-segmented CNN image.

5.5.1. Analysis on pathological images The DRIVE, STARE, HRF, and HASE_DB1 databases contain many pathological images that were used to test the method performance. The output of performance metrics on pathological images is shown in the Table 5. It is observed that our proposed method gives a good performance, and has the capability of segmenting vessels closer to ground truth images as shown in Fig. 14. We compared the output of our method with three novel methods Nguyen et al. (2013), Hou (2014) and Orlando, Prokofyeva, and Blaschko (2017) as shown in Fig. 14. Our proposed method gives proper vessels observations in output images compared to Nuygen’s, Hou’s output, Orlando unary potentials (UP) ’s method and Orlando fully connected CRFs (FC-CRFs) ’s image. After comparing the segmented images of the proposed method with the novel methods, we analyzed the performances of these methods and compared them with our method (Table 6).

5.4. Analysis on various training CNN epochs We observed the performance of our segmentation model in terms of training epochs to select the best epochs for databases. Our CNN model was trained for 30 epochs, but we focus on the first 6 epochs as they showed remarkable improvements with respect to the generated images. The criteria of selection of best epoch are to achieve the best sensitivity and accuracy as well as visual observation of the tiny vessels. Fig. 11 shows the average performance on databases on each epoch of the stride-CNN. We got a maximum sensitivity of 0.87 on DRIVE, 0.848 on STARE database, 0.829 on HRF, 0.886 on CHASE_DB1, and we got maximum accuracy in all these databases such as 0.956 on DRIVE, 0.968 on STARE, 0.962 on HRF and 0.976 on CHASE_DB1. It is a very interesting analysis because some epochs give an improved sensitivity but decrease the accuracy, and in some epochs, sensitivity decreased but accuracy increased as such analysis shown in Fig. 11. It means that some epochs generate images and segment more tiny vessels

5.6. Observation of tiny vessels While many segmentation methods have been proposed, there are two disadvantages observed visually in all methods: firstly,

Table 4 Analysis of segmentation model on different epochs. Databases

DRIVE

Epochs

DSC

IoU

DSC

IoU

DSC

IoU

DSC

IoU

Epoch Epoch Epoch Epoch Epoch Epoch

0.951 0.949 0.985 0.967 0.957 0.942

0.901 0.911 0.948 0.931 0.935 0.901

0.948 0.938 0.981 0.958 0.948 0.961

0.90 0.941 0.946 0.937 0.929 0.899

0.942 0.942 0.979 0.949 0.951 0.956

0.891 0.889 0.939 0.941 0.885 0.869

0.946 0.941 0.981 0.947 0.953 0.945

0.901 0.892 0.939 0.927 0.931 0.921

1 2 3 4 5 6

45

STARE

HRF

CHASE_DB1

46

T.A. Soomro, A.J. Afifi and J. Gao et al. / Expert Systems With Applications 134 (2019) 36–52

Fig. 11. Analysis of outcome of an each measuring metrics of Strided Fully Convolutional Neural Network on epochs 1 to 6 at different databases.

missing the tiny vessels; and secondly, the decreased sensitivity. Both limitations can only be solved by properly detecting the tiny vessels which contribute largely to good sensitivity. We compared the output of our proposed CNN pineline regarding the issues of tiny vessels with those applications used by Nguyen et al. (2013), Hou (2014), Zhao, Rada, Chen, Harding, and Zheng (2015) and Yan, Yang, and Cheng (2018). Fig. 15 shows the output image of Nguyen (Fig. 15(c)), Hou (Fig. 15(d)), Zhao FR (Fig. 15(e)), and Zhao wavelet-based segmentation method (Fig. 15(f)) and Yan ’s (Fig. 15(g)). It can be clearly observed that the proposed method detects more tiny vessels compared to the methods of Nguyen, Hou, and Zhao. The output of our proposed method is, in fact, comparable to the gold standard image (Fig. 15(b)). It proves ex-

perimentally that our method successfully detects tiny and large vessels properly and give better performance also (As it is clearly seen in Table 6), it provides a good opportunity for eye experts to observe the progression of eye disease. For more validation of tiny vessels detection of the proposed method, we calculate the evaluation metrics of thin and thick vessels separately. We follow the same strategy of detection of thin and thick vessels as it is explained by Yan et al. (2018). Fig. 16 shows the representation of thin and thick vessels. The impact of an optic disc border makes it difficult to detect thin vessels, as shown in Fig. 16(c). Our method, after eliminating the noisy pixels of the vessels has the ability to detect the tiny vessels comparable to the ground truth. Fig. 17 shows the detection of tiny

T.A. Soomro, A.J. Afifi and J. Gao et al. / Expert Systems With Applications 134 (2019) 36–52

47

Table 5 Observation of performance on challenging images. Database

Se

Sp

AC

AUC

DRIVE STARE HRF CHASE_DB1

0.822 0.809 0.786 0.819

0.979 0.974 0.964 0.968

0.951 0.953 0.948 0.951

0.976 0.972 0.965 0.974

Table 6 Comparison of proposed mehtod with Nuygen, Hou, Orlando and Zhao methods. Database

DRIVE

methods

Se

Sp

AC

AUC

Se

STARE Sp

AC

AUC

Nguyen et al. (2013) Hou (2014) Orlando et al. (2017) Zhao et al. (2015) Proposed method

– 73.5 0.789 0.716 0.87

– 96.9 0.968 0.978 0.985

0.940 94.1 – 0.944 0.956

– 96.1 – 0.848 0.986

– 73.4 0.768 0.776 0.848

– 96.5 0.973 0.954 0.986

0.932 93.3 – 0.943 0.968

– 95.7 – 0.865 0.988

Table 7 Results for thin vessels and thick vessels for the DRIVE and STARE databases. Vessels

DRIVE

STARE

Se

Sp

AC

Se

Sp

AC

Thin Thick

0.797 0.944

0.978 0.993

0.937 0.976

0.752 0.943

0.978 0.992

0.945 0.988

Fig. 12. Analysis of retinal vessels from segmented image at different epochs.

vessels with our method compared to the ground truth. The resulting segment image (third column) shows tiny vessels and the performance of our proposed method is enhanced by the detection of these tiny vessels. We calculated the thin and thick vessels of segmented images and their corresponding ground truth images. After that, we calculate the Se, Sp, and AC of thin and thick vessels of DRIVE and STARE databases as shown in Table 7. The statistical analysis based on measuring metrics shows that proper detection of thin vessels improves overall performance. Comparing to the detection of thick vessels, thin vessel is prone to being ignored. Thus, the overall performance of the segmentation decreases.

Fig. 13. Retinal vessels segmentation of challenging images cases. Row 1 shows the retinal images patches effected with central light reflex, low-varying contrast and presence of abnormalities. Row 2 shows the corresponding proposed enhanced images. Row 3 shows manual segmentation of each retinal image patches. Row 4 shows proposed method segmented vessels of retinal image patches.

5.7. Comparison with other learning methods We compared our proposed method with other existing learning methods based on CNN for retinal blood vessels segmentation (Table 8). Our method gives higher accuracy with Fu, Xu, Wong, and Liu (2016) on accuracy in the STARE database but their method has low sensitivity. Overall, our proposed method outperforms other methods in terms of sensitivity and accuracy. It shows

that our proposed method has the capability to detect more tiny vessels. Correspondingly, it improves the sensitivity due to more tiny vessels have been identified. With respect to the patch size, we compare the proposed method with Soomro, Afifi, et al. (2017). The authors extracted random patches with dimensions of one-fourth of the original image for training and then resized them to the original image size. In

48

T.A. Soomro, A.J. Afifi and J. Gao et al. / Expert Systems With Applications 134 (2019) 36–52

Fig. 14. Comparison of vessels observation of challenging images of existing methods with our proposed method. First column shows original color retinal images. Second column shows the ground truth images. Third column shows the output images of Nuygen’s method. Fourth column shows the output images of Hou’s method. Fifth column shows the output images of Orlando unary potentials (UP) ’s method. Sixth column shows the output images of Orlando fully connected CRFs (FC-CRFs) ’s method. last column shows the output images of proposed method. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 15. Comparison of tiny vessels detection image of different methods with our proposed output image. (a) shows original retinal image, (b) shows manual segmented image. (c) shows output image of Nuygen’s method, (d) shows output image of Hou’s method. (e) and (f) Shows output images of Zhao ’s based on different filtering techniques. (g) Shows output image of Yan ’s method, and (h) shows output image of proposed method.

our case, the extracted patches have dimensions half of the original image dimensions. From Table 8, we can clearly notice that training the model with larger patches improve the model performance than the smaller ones. This is because the larger sizes have clearer features and can help in extracting tiny vessels. 5.8. Comparison with other state-of-the-art methods The performance of our proposed method is compared with other existing methods for retinal blood vessels segmentation on the STARE and DRIVE databases. Table 9 shows the comparison results. It is clearly observed that the proposed method outperforms accuracy as well as AUC, and gives the higher sensitivity of is Se = 0.87 than all other existing methods. Our proposed method proves the capability of an accurate detection of retinal blood vessels segmentation. Few researchers used the HRF and CHASE_DB1 database as well. They are new and challenging databases because 50% of the images are abnormal images. Table 10 shows the comparison of the performance of our method with other existing methods on CHASE_DB1 database. Table 11 shows the performance of our method with

Fig. 16. Separation of vessels for quality validation. (a) represents original vessels image, (b) represents thick and thin vessels image, (c) represents optic disc border issues that impact the thin vessels of vessels image and (d) Image analysis with missing vessels as well as tracing thick and thin vessels, green marks indicate missing vessels in the image. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

other existing methods on HRF database. It is clearly observed that our method outperforms the other existing methods on HRF and CHASE_DB1 databases in terms of the sensitivity as well as the accuracy. The sensitivity of 0.886 on CHASE_DB1 and 0.829 on HRF databases are achieved with an accuracy of 0.976 on CHASE_DB1 and 0.962 on HRF database. This achieved performance shows that our method can be used for analysis of retinal blood vessels for observing eye disease progression.

T.A. Soomro, A.J. Afifi and J. Gao et al. / Expert Systems With Applications 134 (2019) 36–52

49

Table 8 Analysis of proposed segmentation methods with CNN-based learning methods. Database

DRIVE

Methods V

Se

Sp

AC

AUC

Se

STARE Sp

AC

AUC

Zhang, CuiWuhan, and Wang (2015) Maji, Santara, Mitra, and Sheet (2016) Liskowski and Krawiec (2016) Fu, Xu, Wong, et al. (2016) Wu, Xu, Gao, Buty, and Mollura (2016) Yao, Zhang, and Xu (2016) Maninis, Pont-Tuset, Arbelaez, and Gool (2016) Fu, Xu, Lin, et al. (2016) Tan, Acharya, Bhandary, Chua, and Sivaprasad (2017) M, D, di Baja G, and L (2017) Song and Lee (2017) Soomro, Afifi, et al. (2017) Brancati, Frucci, Gragnaniello, and Riccio (2018) Yan et al. (2018) Soomro et al. (2018b) Proposed method

– – – 0.760 – 0.773 – 0.729 0.753 0.660 0.750 0.746 0.742 0.765 0.739 0.87

– – – – – 0.960 – – 0.969 0.985 0.979 0.917 0.982 0.981 0.956 0.985

0.940 0.947 0.949 0.952 – 0.936 – 0.947 0.926 0.956 0.949 0.948 0.954 0.954 0.948 0.956

– – 0.973 – 0.97 – 0.822 – – – – 0.831 – 0.975 0.844 0.986

– – – 0.741 – – – 0.714 – – – 0.748 – 0.758 0.748 0.848

– – – – – – – – – – – 0.922 – 0.984 0.962 0.986

– – 0.949 0.958 – – – 0.954 – – – 0.947 – 0.961 0.947 0.968

– – 0.982 – – – 0.831 – – – – 0.835 – 0.981 0.855 0.988

Table 9 Comparison of proposed mehtod with other STATE-OF-ART on DRIVE and STARE databases. Database

DRIVE

methods

Se

Sp

AC

AUC

Se

STARE Sp

AC

AUC

Staal et al. (2004) Soares et al. (2006) Mendonca and Campilho (2006) Martinez-Perez et al. (2007) Al-Diri et al. (2009) Lupas et al. (2010) Palomera-Perez, Martinez-Perez, Benitez-Perez, and Ortega-Arjona (2010) Xinge et al. (2011) Marin et al. (2011) Fraz et al. (2012) Nguyen et al. (2013) Orlando and Blaschko (2014) Yin et al. (2014) Roychowdhury, Koozekanani, and Parhi (2015a) Melinscak, Prentasic, and Loncaric (2015) Annunziata, Garzelli, Ballerini, Mecocci, and Trucco (2016) Li et al. (2016) Zhao et al. (2015) Soomro et al. (2016) Khan et al. (2016) Zhang et al. (2016) Orlando et al. (2017) Ngo and Han (2017) Thangaraj, Periyasamy, and Balaji (2018) Biswal, Pooja, and Subrahmanyam (2018) Soomro et al. (2018c) Soomro et al. (2019) Proposed Method

– – 0.734 0.724 0.728 0.720 0.66 0.741 0.706 0.741 – 0.785 – 0.725 – – 0.756 0.716 0.713 0.734 0.743 0.789 0.746 0.801 0.71 0.752 0.745 0.87

– – 0.976 0.965 0.955 – 0.961 0.975 0.980 0.981 – 0.967 – 0.983 – – 0.981 0.978 0.968 0.967 0.976 0.968 0.984 0.975 0.97 0.976 0.962 0.985

0.946 0.946 0.945 0.934 – 0.959 0.922 0.943 0.945 0.948 0.940 – 0.947 0.952 0.946 – 0.952 0.944 0.941 0.951 0.947 – 0.953 0.961 0.95 0.953 0.948 0.956

– – 0.855 0.845 0.842 – 0.811 0.858 0.843 0.974 – – – 0.962 0.974 – 0.974 0.848 0.841 0.850 0.952 – 0.975 0.888 – – – 0.986

– – 0.699 0.750 0.752 – 0.779 0.726 0.694 0.754 – – – 0.772 – 0.713 0.773 0.776 0.711 0.736 0.767 0.768 – 0.834 0.70 0.786 0.784 0.848

– – 0.973 0.956 0.968 – 0.940 0.975 0.981 0.973 – – – 0.973 – 0.984 0.984 0.954 0.965 0.971 0.976 0.973 – 0.953 0.97 0.982 0.976 0.986

0.951 0.948 0.944 0.941 – – 0.924 0.949 0.952 0.953 0.932 0.951 – 0.951 – 0.956 0.962 0.943 0.942 0.95 0.954 – – 0.944 0.95 0.967 0.951 0.968

– – 0.836 0.853 0.860 – 0.860 0.851 0.838 0.977 – – – 0.969 – 0.965 0.987 0.865 0.838 0.853 0.961 – – 0.894 – – – 0.988

Table 10 Comparison of proposed mehtod with other state-of-the-art methods on CHASE_DB1 database. Methods

Se

Sp

AC

AUC

Fraz et al. (2012) Azzopardia, Strisciuglioa, Ventob, and Petkova (2015) Roychowdhury, Koozekanani, and Parhi (2015b) Strisciuglio, Vento, Azzopardi, and Petkov (2015) Li et al. (2016) Zhang et al. (2016) Li et al. (2016) Fu, Xu, Wong, et al. (2016) Orlando et al. (2017) Biswal et al. (2018) Thangaraj et al. (2018) Fan, Lu, and Li (2017) Yan et al. (2018) Proposed method

0.722 0.759 0.762 0.769 0.811 0.763 0.751 0.713 0.728 0.76 0.629 0.657 0.763 0.886

0.971 0.979 0.958 – 0.971 0.966 0.979 – 0.971 0.97 – – 0.981 0.982

0.947 0.939 0.947 0.931 0.955 0.945 0.958 0.949 – – 0.947 0.951 0.961 0.976

0.971 0.949 0.962 0.949 0.972 0.961 0.972 – – – 0.797 0.815 0.978 0.985

50

T.A. Soomro, A.J. Afifi and J. Gao et al. / Expert Systems With Applications 134 (2019) 36–52 Table 11 Comparison of proposed mehtod with Other STATE-OF-ART on HRF database. Methods

Se

Sp

AC

AUC

Yu et al. (2012) Odstrcilik et al. (2013) Annunziata et al. (2016) Zhang et al. (2016) Orlando et al. (2017) Yan et al. (2018) Proposed method

0.781 0.774 0.713 0.798 0.787 0.808 0.829

0.968 0.967 0.984 0.972 0.958 0.941 0.962

0.951 0.949 0.958 0.956 – 0.929 0.962

– – – 0.961 – – 0.978

of max-pooling layers. These layers are trainable, and they help to extract more features for better vessels segmentation (as seen in the results Section 5). 3. Skip connections and the concatenation layers between the encoder and the decoder are utilized to generate sharper vessels’ edges and more accurate segmentation (especially the tiny vessels). 4. The loss function used for training fulfills the training requirements regarding the nature of the task and the training data. The output is a segmented image where we have to classify each pixel to either a background of a vessel.

Fig. 17. Observation of small vessels. The first image is the retinal color fundus image, the second image is the corresponding ground truth and the third image is the corresponding segmented output of our method. It can clearly see the observation of tiny vessels in a cropped square with the comparison of the ground truth image of small vessels. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

6. Conclusion and future work The segmentation of the retinal blood vessels based on deep learning from the colored retinal fundus images has been a wide interest to many medical specialists and research scientists in recent 5 years. Few methods based on deep learning were implemented but they failed to provide comprehensive solutions. Due to the unresolved issues, accurate vessels detection becomes challenging especially for the tiny vessels. Accurate vessels detection plays an important role in assisting the ophthalmologist for analysing the progress of a disease and recommending quick and timely treatment. In this work, we propose a strided-CNN model along with pre-processing steps to detect and segment retinal vessels accurately especially for the tiny vessels. The pre-processing steps are based on morphological and PCA techniques to achieve well contrast images for better training. The proposed strided-CNN model is a fully convolutional model consisting of the encoder part and the decoder part. Skip connections concatenate some feature maps from the encoder part and the decoder part. This improves vessels segmentation and segment specially the tiny vessels and makes the vessels’ edges sharper. We used a class balancing loss function to train and optimize the proposed model to resolve pixel balance issue. The proposed method was tested on the DRIVE, STARE, CHASE_DB1 and HRF databases, and achieved the higher sensitivity and accuracy. Our proposed method outperforms all other existing methods either image processing based methods or deep learning based methods. In summary, 1. We proposed a pipeline of a preprocessing step and a deep strided-CNN for retinal blood vessels segmentation as shown in the Fig. 1. 2. The proposed CNN is a fully convolutional deep encoderdecoder model which uses strided convolutional layers instead

Many research points are still open and should be considered as future work. In the direction of the model architecture, we would propose and study different architectures including residual convolutional blocks and more simpler models to find out the most important layers that play an important role in improving the performance. A second direction is with respect to the available databases. We will work in the direction of generating synthetic images to enrich the training datasets. The available databases have different resolutions, so studying different input resolutions is an interesting point for studying. Also, applying data augmentation on the training datasets and check the number of the extracted patches and different resolutions will lead to a better understanding of the training process. Another research direction would be with respect to the training process. We will train our CNN model using different loss functions to observe which loss function performs much better on the retinal vessels image. So, these research points may lead to developing a more accurate and efficient model that can potentially be used in real scenarios. Conflicts of interest None. References Al-Diri, B., Hunter, A., & Steel, D. (2009). An active contour model for segmenting and measuring retinal vessels. IEEE Transactions on Medical Imaging, 28(9), 1488–1497. Annunziata, R., Garzelli, A., Ballerini, L., Mecocci, A., & Trucco, E. (2016). Leveraging multiscale hessian-based enhancement with a novel exudate inpainting technique for retinal vessel segmentation. IEEE Journal of Biomedical and Health Informatics, 20(4), 1129–1138. Azzopardia, G., Strisciuglioa, N., Ventob, M., & Petkova, N. (2015). Trainable cosfire filters for vessel delineation with application to retinal images. Medical Image Analysis, 19(1), 46–57. Badrinarayanan, Vijay, Kendall, Alex, Cipolla, & Roberto (2017). Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 99. 1–1 Badrinarayanan, V., Handa, A., & Cipolla, R. (2015). Segnet: A deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling. (pp. 1– 10). arXiv:1505.07293. Biswal, B., Pooja, T., & Subrahmanyam, N. B. (2018). Robust retinal blood vessel segmentation using line detectors with multiple masks. IET Image Processing, 12(3), 389–399. Brancati, N., Frucci, M., Gragnaniello, D., & Riccio, D. (2018). Retinal vessels segmentation based on a convolutional neural network. In Progress in pattern recognition, image analysis, computer vision, and applications. CIARP. In Lecture Notes in Computer Science: 10657 (pp. 119–126). Cham: Springer.

T.A. Soomro, A.J. Afifi and J. Gao et al. / Expert Systems With Applications 134 (2019) 36–52 Budai, A., Hornegger, J., & Michelson, G. (2009). Multiscale approach for blood vessel segmentation on retinal fundus images. Investigative Ophthalmology & Visual Science, 50. Diabetic Retinopathy Detection (2015). https://www.kaggle.com/c/diabeticretinopathy-detection Fan, Z., Lu, J., & Li, W. (2017). Unsupervised blood vessel segmentation of fundus images based on region features and hierarchical growth algorithm., (pp. 1–11). arXiv:1701.00892v2. Fraz, M. M., Remagnino, P., Hoppe, A., Uyyanonvara, B., Rudnicka, A. R., Owen, C. G., & Barman, S. A. (2012). An ensemble classification-based approach applied to retinal blood vessel segmentation. IEEE Transactions on Biomedical Engineering, 59(9), 2538–2548. Fu, H., Xu, Y., Lin, S., Wing, D., Wong, K., & Liu, J. (2016). Deepvessel: Retinal vessel segmentation via deep learning and conditional random field. In International conference on medical image computing and computer-assisted intervention (pp. 132–139). Fu, H., Xu, Y., Wong, D. W. K., & Liu, J. (2016). Retinal vessel segmentation via deep learning network and fully-connected conditional random fields. In International symposium on biomedical imaging (ISIB) (pp. 10–13). Fu, X., Liu, T., Xiong, Z., Smaill, B. H., Stiles, M. K., & Zhao, J. (2018). Segmentation of histological images and fibrosis identification with a convolutional neural network. Computers in Biology and Medicine, 98, 147–158. Glorot, X., & Bengio, Y. (2010). Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics (pp. 249–256.). Hardarson, S., & Stefansson, E. (2012). Retinal oxygen saturation is altered in diabetic retinopathy. British Journal of Ophthalmology, 96, 560–563. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770–778). HM, P., E, F., SC, F., MK, M., & RS, M. (2005). Measuring arteriolar-to- venous ratio in retinal photography of patients with hypertension: Development and application of a new semi-automated method. American Journal of Hypertension, 18, 417–421. Hou, Y. (2014). Automatic segmentation of retinal blood vessels based on improved multiscale line detection. Journal of Computing Science and Engineering, 8(2), 119–128. JJ, W., G, L., R, K., E, R., & MD, K. (2007). Retinal vessel diameter and cardiovascular mortality: pooled data analysis from two older populations. European Heart Journal, 28, 1984–1992. Khan, M. A. U., Soomro, T. A., Khan, T. M., Bailey, D. G., Gao, J., & Mir, N. (2016). Automatic retinal vessel extraction algorithm based on contrast-sensitive schemes. In International conference on image and vision computing New Zealand (IVCNZ) (pp. 1–5). doi:10.1109/IVCNZ.2016.7804441. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Conference on advances in neural information processing systems (pp. 1097–1105). LD, H., RJ, B., WN, K., LX, C., & R, K. (1999). Methods for evaluation of retinal microvascular abnormalities associated with hypertension/sclerosis in the atherosclerosis risk in communities study. Ophthalmology, 106, 2269–2280. Li, Q., Feng, B., Xie, L., Liang, P., Zhang, H., & Wang, T. (2016). A cross-modality learning approach for vessel segmentation in retinal images. IEEE Transactions on Medical Imaging, 35(01), 109–118. Liskowski, P., & Krawiec, K. (2016). Segmenting retinal blood vessels with deep neural networks. IEEE Transactions on Medical Imaging, 35(11), 2369–2380. Lupas, C. A., Tegolo, D., & Trucco, E. (2010). FABC: Retinal vessel segmentation using adaboost. IEEE Transactions on Information Technology in Biomedicine, 14(5), 1267–1274. M, F., D, R., di Baja G, S., & L, S. (2017). Direction-based segmentation of retinal blood vessels. In Progress in pattern recognition, image analysis, computer vision, and applications. CIARP. In Lecture notes in computer science: 10125 (pp. 1–8). Cham: Springer. Maji, D., Santara, A., Mitra, P., & Sheet, D. (2016). Ensemble of deep convolutional neural networks for learning to detect retinal vessels in fundus images. (pp. 1– 4). arXiv:1603.04833v1. Maninis, K.-K., Pont-Tuset, J., Arbelaez, P., & Gool, L. V. (2016). Deep retinal image understanding. In Conference on medical image computing and computer-assisted intervention (pp. 140–148). Marin, D., Aquino, A., Gegundez-Arias, M. E., & Bravo, J. M. (2011). A new supervised method for blood vessel segmentation in retinal images by using gray-level and moment invariants-based features. IEEE Transactions on Medical Imaging, 30(1), 146–158. Martinez-Perez, AD, H., & SA, T. (2007). Segmentation of blood vessels from red-free and fluorescein retinal images. Journal of Medical Image Analysis, 11(1), 47–61. Martinez-Perez, M. E., Hughes, A. D., Stanton, A., Thom, S. A., Anil, A., & Parker, K. H. (1999). Retinal blood vessel segmentation by means of scale-space analysis and region growing. In Conference on medical image computing and computer-assisted intervention: 1 (pp. 90–97.). Melinscak, M., Prentasic, P., & Loncaric, S. (2015). Retinal vessel segmentation using deep neural networks. In International confernence on computer vision theory and application (pp. 1–6). Mendonca, A. M., & Campilho, A. (2006). Segmentation of retinal blood vessels by combining the detection of centerlines and morphological reconstruction. IEEE Transactions on Medical Imaging, 25, 1200–1213. Narasimhan, K., & Vijayarekha, K. (2015). Automatic grading of images based on retinal vessel tortuosity analysis. Indian Journal of Science and Technology, 8, 1–5.

51

Ngo, L., & Han, J. (2017). Multi-level deep neural network for efficient segmentation of blood vessels in fundus images. Electronic Letters, 53(16), 1096– 1098. Nguyen, U. T. V., Bhuiyan, A., Park, L. A. F., & Ramamohanarao, K. (2013). An effective retinal blood vessel segmentation method using multi-scale line detection. Pattern Recognition, 46, 703–715. Odstrcilik, J., Kolar, R., Budai, A., Hornegger, J., Jan, J., Gazarek, J., . . . Angelopoulou, E. (2013). Retinal vessel segmentation by improved matched filtering: evaluation on a new high-resolution fundus image database. IET Image Process, 7(4), 373–383. Olafsdottir, O., Hardarson, S., Gottfredsdottir, M., Harris, A., & Stefansson, E. (2011). Retinal oximetry in primary open-angle glaucoma. Investigative Ophthalmology & Visual Science, 52, 6409–6413. Orlando, J. I., & Blaschko, M. (2014). Learning fully-connected CRFs for blood vessel segmentation in retinal images. In Conference on medical image computing and computer-assisted intervention: 17 (pp. 634–641). Orlando, J. I., Prokofyeva, E., & Blaschko, M. B. (2017). A discriminatively trained fully connected conditional random field model for blood vessel segmentation in fundus images. IEEE Transaction on Biomedical Engineering, 64(1), 16–27. Owen, C. G., Rudnicka, A. R., Mullen, R., Barman, S. A., Monekosso, D., Whincup, P. H., . . . Paterson, C. (2009). Measuring retinal vessel tortuosity in 10-year-old children: validation of the computer-assisted image analysis of the retina (CAIAR) program. Investigative Ophthalmology & Visual Science, 50(5), 2004–2010. Palomera-Perez, M. A., Martinez-Perez, M. E., Benitez-Perez, H., & Ortega-Arjona, J. L. (2010). Parallel multiscale feature extraction and region growing: Application in retinal blood vessel detection. IEEE Transactions on Information Technology in Biomedicine, 14(2), 500–506. Patton, N., Aslam, T., Macgillivray, T., Pattie, A., & Deary, I. (2005). Retinal vascular image analysis as a potential screening tool for cerebrovascular disease: a rationale based on homology between cerebral and retinal microvasculatures. Journal of Anatomy, 206, 319–348. PJ, S., & ME, T. (2002). Ophthalmic photography: Retinal photography, angiography, and electronic imaging (2nd ed.). Boston: Butterworth-Heinemann. Ricci, E., & Perfetti, R. (2007). Retinal blood vessel segmentation using line operators and support vector classification. IEEE Transactions on Medical Imaging, 26(10), 1357–1365. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In International conference on medical image computing and computer-assisted intervention: 1 (pp. 234–241). Cham: Springer. Roychowdhury, S., Koozekanani, D. D., & Parhi, K. K. (2015a). Blood vessel segmentation of fundus images by major vessel extraction and subimage classification. IEEE Journal of Biomedical and Health Informatics, 19(03), 1118–1128. Roychowdhury, S., Koozekanani, D. D., & Parhi, K. K. (2015b). Iterative vessel segmentation of fundus images. IEEE Transactions on Biomedical Engineering, 62(7), 1738–1749. Shelhamer, E., Long, J., & and, T. D. (2017). Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4), 640–651.. Simonyan, K., & Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition. ICLR. Soares, J. V. B., Roberto, J. J. G. L., Cesar, M., Jelinek, J. H. F., & Cree, M. J. (2006). Retinal vessel segmentation using the 2-d gabor wavelet and supervised classification. IEEE Transactions on Medical Imaging, 25(9), 1214–1222. Song, J., & Lee, B. (2017). Development of automatic retinal vessel segmentation method in fundus images via convolutional neural networks. In 39th IEEE annual international conference of the engineering in medicine and biology society (EMBC) (pp. 681–684). doi:10.1109/EMBC.2017.8036916. Soomro, S., Munir, A., & Choi, K. N. (2018). Hybrid two-stage active contour method with region and edge information for intensity inhomogeneous image segmentation. PLoS ONE, 13(1), 1–20. Soomro, T. A., Afifi, A. J., Gao, J., Hellwich, O., Khan, M. U., Paul, M., & Zheng, L. (2017). Boosting sensitivity of a retinal vessel segmentation algorithm with convolutional neural network. In International conference on digital image computing: Techniques and applications (DICTA). Soomro, T. A., Afifi, A. J., Gao, J., Hellwich, O., Paul, M., & Zheng, L. (2018b). Strided u-net model: Retinal vessels segmentation using dice loss. In Digital image computing: Techniques and applications (DICTA) (pp. 1–8). doi:10.1109/DICTA.2018. 8615770. Soomro, T. A., & Gao, J. (2016a). Neural network based denoised methods for retinal fundus images and MRI brain images. In IEEE International joint conference on neural networks (IJCNN): 1 (pp. 1151–1157). Soomro, T. A., & Gao, J. (2016b). Non-invasive contrast normalisation and denosing technique for the retinal fundus image. Annals of Data Science, 3(3), 265–279. Soomro, T. A., Gao, J., Khan, T. M., Hani, A. F. M., Khan, M. A. U., & Paul, M. (2017b). Computerised approaches for the detection of diabetic retinopathy using retinal fundus images: A survey. Journal of Pattern Analysis and Application, 1–35. doi:10.1007/s10044- 017- 0630- y. Soomro, T. A., Gao, J., Lihong, Z., Afifi, A. J., Soomro, S., & Paul, M. (2019). Retinal blood vessels extraction of challenging images. In Data mining. AusDM 2018. Communications in computer and information science: 996 (pp. 1–12). Singapore: Springer. Soomro, T. A., Khan, M. A. U., Gao, J., Khan, T. M., Paul, M., & Mir, N. (2016). Automatic retinal vessel extraction algorithm. In International conference on digital image computing: Techniques and applications (DICTA) (pp. 1–8). doi:10.1109/ DICTA.2016.7797013.

52

T.A. Soomro, A.J. Afifi and J. Gao et al. / Expert Systems With Applications 134 (2019) 36–52

Soomro, T. A., Tariq. M. Khan, Mohammad. A. U. Khan, Gao, J., Paul, M., & Zheng, L. (2018c). Impact of ICA-based image enhancement technique on retinal blood vessels segmentation. IEEE Access. Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014). Striving for simplicity: The all convolutional net. (pp. 1–14). arXiv:1412.6806v3. Staal, J., Abramoff, M. D., Niemeijer, M., Viergever, M. A., & van Ginneken, B. (2004). Ridge based vessel segmentation in color images of the retina. IEEE Transactions on Medical Imaging, 23(4), 501–509. Strisciuglio, N., Vento, M., Azzopardi, G., & Petkov, N. (2015). Unsupervised delineation of the vessel tree in retinal fundus images. Computational Vision and Medical Image Processing, 149–155. Szegedy, C., Liu, W., Jia, Y., Sermanet, Y., Reed, P., Anguelov, S., . . . V. Rabinovich (2015). Going deeper with convolutions. CVPR. Tan, J. H., Acharya, U. R., Bhandary, S. V., Chua, K. C., & Sivaprasad, S. (2017). Segmentation of optic disc fovea and retinal vasculature using a single convolutional neural network. Journal of Computational Science. Thangaraj, S., Periyasamy, V., & Balaji, R. (2018). Retinal vessel segmentation using neural network. IET Image Process, 12(5), 669–678. TY, W., MD, K., R, K., BEK, K., & SM, M. (2004). Computer-assisted measurement of retinal vessel diameters in the beaver dam eye study: methodology, correlation between eyes, and effect of refractive errors. Ophthalmology, 111, 1183–1190. Vedaldi, A., & Lenc, K. (2015). Matconvnet: Convolutional neural networks for matlab. In 23rd ACM international conference on multimedia (pp. 689–692). Wang, Y., Ji, G., Lin, P., & Trucco, E. (2013). Retinal vessel segmentation using multiwavelet kernels and multiscale hierarchical decomposition. Pattern Recognition, 46(8), 2117–2133. Wu, A., Xu, Z., Gao, M., Buty, M., & Mollura, D. J. (2016). Deep vessel tracking: A generalized probabilistic approach via deep learning. In IEEE 13th international symposium on biomedical imaging (ISBI): 1 (pp. 1363–1367). Xie, Saining, & Tu, Z. (2015). Holistically-nested edge detection. In IEEE international conference on computer vision (pp. 1395–1403).

Xinge, Y., Qinmu, P., Yuan, Y., Yiu-ming, C., & Jiajia, L. (2011). Segmentation of retinal blood vessels using the radial projection and semi-supervised approach. Pattern Recognition, 44, 10–11. Xu, B., Wang, N., Chen, T., & Li, M. (2015). Empirical evaluation of rectified activations in convolutional network. (pp. 1–5). arXiv:1505.00853v2. Xua, X., Dinga, W., Abramoff, M. D., & Caoa, R. (2017). An improved arteriovenous classification method for the early diagnostics of various diseases in retinal image. Computer Methods and Programs in Biomedicine, 141, 3–9. Yan, Z., Yang, X., & Cheng, K.-T. T. (2018). Joint segment-level and pixel-wise losses for deep learning based retinal vessel segmentation. IEEE Transactions on Biomedical Engineering, 1–10. doi:10.1109/TBME.2018.2828137. Yao, Z., Zhang, Z., & Xu, L. Q. (2016). Convolutional neural network for retinal blood vessel segmentation. In 9th International symposium on computational intelligence and design (ISCID): 1 (pp. 406–409). Yin, X., Ng, B. W.-H., He, J., Zhang, Y., & Abbott, D. (2014). Accurate image analysis of the retina using hessian matrix and binarisation of thresholded entropy with application of texture mapping. PLoS ONE, 9(4), 1–17. Yu, H., Barriga, S., Agurto, C., Zamora, G., Bauman, W., & Soliz, P. (2012). Fast vessel segmentation in retinal images using multi-scale enhancement and second-order local entropy. Proceedings of SPIE, 8315, 83151B–1–83151B–12. Zhang, J., CuiWuhan, Y., & Wang, J. (2015). Blood vessel segmentation of retinal images based on neural network. In Image and graphics. ICIG 2015. In Lecture Notes in Computer Science: 9218 (pp. 11–17). Zhang, J., Dashtbozorg, B., Bekkers, E., Pluim, J. P. W., Duits, R., & ter Haar Romeny, B. M. (2016). Robust retinal vessel segmentation via locally adaptive derivative frames in orientation scores. IEEE Transactions on Medical Imaging, 35(12), 2631–2642. Zhao, Y., Rada, L., Chen, K., Harding, S. P., & Zheng, Y. (2015). Automated vessel segmentation using infinite perimeter active contour model with hybrid region information with application to retinal images. IEEE Transactions on Medical Imaging, 34(9), 1797–1807.