- Email: [email protected]
Tutorial IV: Leaf disease recognition
CHAPTER 7
Tutorial IV: Leaf disease recognition Contents 7.1 7.2 7.3 7.4
Introduction State of the art Materials and methods Application of LBP in disease recognition of infected plants 7.4.1 Image segmentation 7.4.2 Creation of LBP histogram 7.4.3 One class classification 7.4.4 Generalization of the vines powdery mildew model to different crop diseases 7.5 Segmentation technique 7.6 Classification process and the features extraction process References
241 242 244 245 245 246 248 252 256 257 258
7.1 Introduction Crop infection detection is critical for crop growth monitoring and safeguarding the quality of production. For this purpose, optical recognition can be affectivity utilized by specialists for monitoring possible changes in crop leaves, therefore a highly experienced and specialized background. New techniques handling data mining by AI is able to enhance trustworthy the credibility of a possible infection recognition and thus can be featured into actionable insights to enable interventions. Methods using AI combined with image feature analysis are capable of providing a more precise and effective tool for crop disease detection. Crop infection recognition using visual information is regarded challenging, more specifically, on its automated form. This task is often hard to be tackled, because of the special characteristics in terms of morphology and but also because of lighting conditions. Crop infection evaluation is a critical step for adopting accurate and efficient crop management practices encompassing predictive algorithms and various means of treatment. Intelligent Data Mining and Fusion Systems in Agriculture https://doi.org/10.1016/B978-0-12-814391-9.00007-8
© 2020 Elsevier Inc. All rights reserved.
241
242
Intelligent data mining and fusion systems in agriculture
Fungal diseases are regarded responsible for severe yield degradation, yield low productivity and financial losses, ranging between 5 and 80% depending on the severity of infection, ambient factors related to climate and terrain, and lack of resistance due to susceptible genotype (Sˇroba´rova´ & Kakalı´kova´, 2007). It is crucial to focus on the relation that exists between various unexpected deviations in the characteristics of plant leaf morphology and the variety of the environmental factors that would act as are possible stress factors to a specific crop that demonstrates stress symptoms (Gaunt, 1995).
7.2 State of the art Various techniques are used in the present targeting on identifying plant infections by applying digital image processing. Crop disease identification was achieved in sugarcane leaves by threshold segmentation to isolate the infected leaf location and triangle thresholding to obtain the lesioning area, reaching an average accuracy of 98.60% (Patil & Bodhe, 2011). Texture feature extraction has also been used for plant disease detection. Patil and Kumar (2017) introduced plant disease detection approach base on texture feature extraction including morphological features, uniformity, and relative associations derived by estimating the co-occurrence matrix of the gray image of corn leaves combined with color extraction, giving a trustworthy training feature based set for image enhancement and improved detection. Rothe and Kshirsagar (2015) employed a Back propagation approach for classifying diseased leaf images of cotton. During the training procedure, seven invariant moments were extracted from three leaves images depicting infections from three different diseases respectively. The utilized snake segmentation algorithm achieved a classification of 85.52%, providing an efficient solution for set apart the area of interest (the infected one). However, the proposed method is characterized as time consuming. Zhang, Wu, You, and Zhang (2017) proposed a leaf disease detection procedure in cucumber, by setting apart the infected leaf area through an efficient combination of k-means clustering and color and shape feature extraction, obtaining a score of 85.7% correct recognition. A similar approach has been presented by Guo, Liu, and Li (2014). In this approach texture and color features were combined by a Bayesian approach. The investigated diseases where downy mildew, anthracnose, powdery mildew, and gray mold. The utilized model achieved accuracy rates of 94.0%, 86.7%, 88.8%, and 84.4% respectively. Vianna, Oliveira, and Cunha (2017) proposed a pattern simulation approach for recognizing the globally threatening of late blight in tomato leaves. In this paper, 20 networks were tested, from which the best network
Tutorial IV: Leaf disease recognition
243
produced a prediction of 97.99%. Fiel and Sablatnig (2013) have diversified leaf infection detection by applying Bag of Words with SIFT descriptors in 5 different tree species, attaining an accuracy of 93.6%. Meunkaewjinda, Kumsawat, Attakitmongcol, and Srikaew (2008) presented an automated crop infection diagnosis detection system which utilized various artificial intelligent algorithms in infected vine leaves. Selforganizing feature map and back propagation neural network has been applied in order to detect the colors of vine leaves. Additionally, an altered modified SOM has been applied for segmenting the image and support vector machine is used as a classifier, demonstrating a performance of 86.03%. Self-organizing feature map is also used to detect disease of cotton leaves (Gulhane & Gurjar, 2011). Kebapci, Yanikoglu, and Unal (2011) developed a plant retrieval system taking into consideration the color, shape and texture characteristics by employing the color histogram, the color co-occurrence matrix and a modified Gabor method approach. The experiment has been performed for crop type recognition in domestic plants reaching 73% for successful recognition. Pethybridge and Nelson (2015) have presented an IOS application called ‘Leaf Doctor”, capable of discriminating the possibly infected from the healthy plant tissue areas. In this approach, eight colors denoting the healthy tissue areas form the threshold assisting the recognition of the healthy ones. With respect to the six corresponding infections, the algorithm’s sensitivity was accessed according to targeted range of 10 color corrected images attaining a R2 > 0.79. Nevertheless, some of the main drawbacks of this approach is that for the image processing, a black background is required. The black background has proven to be most of the times responsible for the majority of misclassifications since it can be falsely mixed up with alike colored pixels linked to infection symptoms. Moreover, considering the fact that the samples were destructively taken, makes the method less credible. The employed algorithm was difficult to come to a trustworthy conclusion in variable ambient field conditions and alterations, as a result more stable and trustworthy algorithms are required. The current case study demonstrates a novel application of recognizing four different health status including healthy, downy mildew, powdery mildew and black rot in different tissue samples through One Class Classification. The presented application is able to assess images obtained in real conditions without additional processing and can decide with certainty on the occurrence of infection even at its early stage appearance. The current approach is comprised of a learning procedure with training data relate to each one of the four different crop status. A previously unseen and unlabeled
244
Intelligent data mining and fusion systems in agriculture
feature vector is used as input to a one-class classifiers committee, triggering activations. In the occasion that multiple activations take place, then a conflict occurs since more than one classifiers are regarded as contenders of the new feature vector ownership. The current case study demonstrates high generalization potential regarding the leaf plant disease identification in several crops, where the training set was comprised solely of vine leaf samples.
7.3 Materials and methods The Local Binary Pattern (LBP) is a texture analysis approach, that able to label pixels by defining their proximity limits, forming a binary outcome. The LBP approach takes advantage of its capability of maintaining a stable behavior to the upcoming gray scale level deviations that occur, such as illumination and image processing under complex real-time conditions, proving its computational potential to be implemented in several commercial applications. The LBP operator has been introduced by Li, Wu, Wang, and Zhang (2008) and Llado, Oliver, Freixenet, Marti, and Marti (2009) considering texture features including pattern including the relative pattern’s strength applied with a 3 3 grid, where the center value denotes a threshold. The LBP code is formulated as presented: the thresholded values are scaled by the corresponding weights of the pixels. Since the neighborhood is defined by eight pixels, a set of 28 ¼ 256 discrete labels are produced, corresponding to the relative threshold’s gray levels and remaining pixels of the grid. The contrast indicator (C) is defined as the subtraction of the mean of the pixel levels that lie below the threshold defined from the mean of pixel levels larger or equal to the threshold. In the case that the thresholded pixel levels are equal to zero or one, the contrast value is equal to zero. Fig. 7.1 depicts an illustration of LBP operator function. example
thresholded
6
5
2
1
7
6
1
1
9
8
7
1
Pattern = 11110001
0
1
weights
0
1
0
128
1
64
2
4 8
32
16
LBP = 1 + 16 + 32 + 64 + 128 = 241 C = (6+7+9+8+7)/5 – (5+2+1)/3 = 4.7
Fig. 7.1 Illustration of LBP operator function.
Tutorial IV: Leaf disease recognition
(A)
245
(B)
Fig. 7.2 (A) Hue channel illustration of a vine leaf (Pantazi, Moshou, & Tamouridou, 2019). (B) LBP leaf image of vine crop (Pantazi et al., 2019).
The LBP operator is able to transform an image into a matrix of integer levels that define the detailed local structure of the image (Fig. 7.1). These levels or their derived statistical operators like histogram, are the parameters used for producing image analytics. The LBP operator is mainly applicable to binary images as well as multi-channel images, video streams and multi-dimensional data. The hue channel is derived from image of the leaf as depicted in Fig. 7.2A. The LBP transform application on the Hue channel derived of the segmented image produces an LBP leaf image, as illustrated in Fig. 7.2B.
7.4 Application of LBP in disease recognition of infected plants 7.4.1 Image segmentation As a first step, an image of the diseased leaf that manifests visual symptoms is obtained by a smartphone or tablet (Fig. 7.3A). Afterwards, image segmentation takes places aiming to obtain the leaf area and exclude the image background. Then, a Hue Saturation Value (HSV) transform is imposed to the segmented area (Fig. 7.3B). The GrabCut algorithm operation is summarized in the following steps: 1. The central image points correspond to foreground while the surrounding regions correspond to background. Then, a rectangle is formed including the target region while the internal pixels are recognized as not known and the external ones are denoted as known. 2. An image segmentation is performed by applying Gaussian Mixture Models (GMMs) to the foreground and background utilizing the Orchard-Bouman algorithm to create the clusters.
246
Intelligent data mining and fusion systems in agriculture
Fig. 7.3 (A) Randomly selected image of a vine utilized for the validation of the powdery mildew One Class Classifier (Pantazi et al., 2019). (B) Depiction of the segmented area of interest of the vine leaf illustrated in Fig. 7.1 after HSV transform and the GrabCut algorithm’s operation (Pantazi et al., 2019).
3. 4. 5. 6.
All pixels are classified as foreground or background through the allocation of Gaussian component corresponding to the foreground or the background GMMs. The arising pixel sets yield the learning adaptation, targeting on the creation of new GMMs. A graph is constructed, activating the GrabCut algorithm for assigning a new foreground class. The former steps (4–6) are iterated until all the pixel sets are classified. The GrabCut algorithm creates K components of multivariate GMM corresponding to the background. A similar procedure is followed for K components corresponding to the foreground. The GMM components are labeled according to the color statistics corresponding to each cluster.
7.4.2 Creation of LBP histogram The GrabCut algorithm is applied individually on various single channels of the initial image because of the LBP operator is applied in a single channel. On the initial RGB image two separate transforms options were tested: the first transform obtained in HSV format while the second transform obtained gray scale image. From the HSV transform it has become apparent that the LBP histogram applied on the Hue channel, yielded the highest performance in comparison to the gray scale transform in relation to the detection of disease crop symptoms. The LBP operator manages the labelling of the pixels corresponding to the obtained images based on their mutual distance in the Hue channel, and
Tutorial IV: Leaf disease recognition
247
consequently takes in to account the textural characteristics concerning contrast and Hue variation as illustrated in Fig. 7.3B. The LBP histogram depicts the appearance rate of Hue levels from 0 to 255. For avoiding noisy values in the histogram, the 256 bins has been reduced to 32. An illustration of the reduced histogram is given in Fig. 7.4, where the x and y axes, denote the amount of bins and the allocation of pixels per bin (%) respectively. The inhomogeneity of symptoms in symptomatic leaves induces deviations in texture of a local nature which are evident which appears in the LBP histogram as higher pixel counts in specific bins (Fig. 7.4). As it is shown in Fig. 7.4, the healthy leaf status is associated with subtle alterations in the local texture structure. The pixel count in certain bins in the LBP histograms corresponds to other local deviations that are associated to the collapse of leaf structure produced by the pathogen invasion. Equally, the black rot occurrence provokes high pixel count in the first bin and an additional peak in the sixth bin. In the case of powdery mildew occurrence, lower pixel count is demonstrated in the first and a peak in the ninth bin. This is explained by the contribution of the pixel count to the leaf structure collapse that finally lead to the discrimination of black rot from powdery mildew symptoms.
LBP histogram
1
Healthy Black Rot Powdery mildew
0.9
percentage of pixels
0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 5
10
15
20
25
30
LBP bins
Fig. 7.4 LBP histogram depicting different health conditions (healthy, black rot and powdery mildew infected) in vine crop (Pantazi et al., 2019).
248
Intelligent data mining and fusion systems in agriculture
7.4.3 One class classification The LBP histograms are used for calibrating One Class Classifiers. More exactly, One Class SVMs (OC-SVMs) were employed to classify leaf images according to their health status. For developing these classifiers eight leaf images from vine plants corresponding to each crop health status (healthy, powdery mildew, downy mildew, black rot) constituted sufficient training dataset (Figs. 7.5–7.8). One Class SVMs were assessed by cross validation through application to samples corresponding to health crop status (healthy, powdery mildew, downy mildew, black rot), demonstrating an accurate calibration performance when tested with random images.
Fig. 7.5 The dataset utilized for calibrating the One Class SVM model for the healthy condition (Pantazi et al., 2019).
Fig. 7.6 The dataset utilized for calibrating the One Class SVM model for the powdery mildew infection (Pantazi et al., 2019).
Tutorial IV: Leaf disease recognition
249
Fig. 7.7 The dataset utilized for calibrating the One Class SVM model for the black rot infection (Pantazi et al., 2019).
Fig. 7.8 The dataset utilized for calibrating the One Class SVM model for the downy mildew infection (Pantazi et al., 2019).
7.4.3.1 One class support vector machines (OCSVMs) Classification is performed to discriminate different sample types, whilst regression intends to produce a specific output for available data samples. Unlike the previously mentioned models, One Class Classifiers take advantage of solely target data to express a binary classification argument based on its membership or not to the target dataset. The One class Classification can be also characterized by the term domain description, denoting a set of objects, capable of recognizing those that are similar or dissimilar (labeled as outliers) to the calibration set assisted by novelty detection. There is a plethora of data domain definition or outlier detection techniques in the literature. Tax and Duin (2004) proposed a type of One Class SVM Classification, called Support Vector Data Description (SVDD).
250
Intelligent data mining and fusion systems in agriculture
The SVDD is capable of classifying an unknown object into either target or outlier, based on the hypothesis that target data are bounded in the feature domain. A sphere, with a center a and minimum radius R containing most of N samples {xi; i ¼ 1;::; N}. Slack variables ξi are defined, following the constraints below: ðxi αÞðxi aÞT R2 + ξι ξι 0
(7.1)
The R and the ξi are expressed as an equality by using a weight to slack variables: X F ðR, a, ξi Þ ¼ R2 + C ξι (7.2) i
where C, symbolizes sphere’s volume and it is directly relevant to number of data point exceptions. Consequently, the result is expressed in LaGrange multiplier equation as follows: X X ξi ai R2 + ξi xi 2 + 2αxi + α2 L ðR, a, ai , ξi Þ ¼ R2 + C i X i γ i ξi (7.3) i
where αi 0 and γ i 0 are LaGrange multipliers. By defining the partial derivatives to zero values, three conditions are formed as follows: X αi ¼ 1 ι X ai xi X (7.4) i a¼ X ¼ ai xi ai i i
0 ai C After optimization with respect to αι while satisfying the restrictions expressed in Eq. (7.4) the LaGranzian is expressed as below: X X L¼ ai ðxi xi Þ ai aj xi xj (7.5) i i, j A data sample z belongs to the sphere if its distance from the center is smaller compared to the R (Eq. 7.6): X X ðz aÞðz aÞT ¼ ðz zÞ 2 ai ðz xi Þ + ai aj xi xj R2 (7.6) i i, j
Tutorial IV: Leaf disease recognition
251
For a reduced size dataset, the calibration dataset is defined by the formulation expressed through the Eq.(7.6), in which support vectors are denoted by nonzero αι. In the special case where C < 1, the support vectors equate C. Those data samples originate from the external sphere’s space and consequently they are considered as outliers. The above presented methodology is defined as the Support Vector Domain Description (SVDD). The OCSVM constructs a model by training the classifier using data samples provided by the SVDD. Consequently, the test data are classified by utilizing the variance of the calibration samples (Sch€ olkopf, Platt, Shawe-Taylor, Smola, & Williamson, 2001). 7.4.3.2 One-class support-vector-learning for multi class problems One-Class Classifiers are usually employed to resolve to multi-decision problems. Test data are sampled in random corresponding to various classes. For every one of the individual health condition classes, a classifier is employed and the sphere is constructed aiming to reach a conclusion about for every point emanating from the tested dataset. In the occasion of a newly presented sample, two possibilities arise: 1. In the situation arising when one classifier is assigned a new data sample, it is characterized by the relevant class. 2. Conflict resolution is activated in the case that a couple or more classifiers are assigned for the same data sample. In the occasion that a new sample is presented and not assigned by the classifier, it is considered an outlier. The conflict is handled by employing the Nearest-Support-Vector Strategy. 7.4.3.3 Nearest-support-vector strategy The distance of a new data sample (z) from the support vectors that correspond to each classifier takes its label (Eq. 7.7). By considering that: SV ðtÞ ¼ x^t1 , …, x^t1t , (7.7) The SV denotes the set of support vectors, and i, t their quantity with the classifier t. Aiming to classify z, the decision function f(z) calculate the distances of the data sample to the support vectors in order find the most proximal individually discover in which classifiers it is assigned (Eq. 7.8). f ðzÞ ¼ argminl2C k^ xli zk where C symbolizes each one of the competing classifiers.
(7.8)
252
Intelligent data mining and fusion systems in agriculture
LBP Image segmentation Separation of foreground background GrabCut algorithm
Pixels labelled Derivation of Histogram
OCSVMs Classification into target class and outliers
A classifier for each class Conflict resolution
One Class Classification
Labeling according to proximity
Nearest Support Vector Strategy
Fig. 7.9 Illustration of the followed steps (Pantazi et al., 2019).
All the steps followed in the current case study are presented schematically in Fig. 7.9.
7.4.4 Generalization of the vines powdery mildew model to different crop diseases 7.4.4.1 Validation of the developed disease classification framework A set of eight vine leaf images corresponding to each of the investigated health status were used, including healthy condition, powdery mildew, black rot and downy mildew infected as calibration dataset to the One Class SVM. The calibrated OC-SVM with vine leaves was further employed for disease detection in eighteen different plant species achieved an excellent generalization for all individual cases. Conflict resolution has been proven to be a critical factor, for the health status assessment exceeding 50% of the cases, achieving a 100% in most of the cases. The classifiers assessing the healthy plant status and the three investigated infections (powdery mildew, black rot, downy mildew) were calibrated using solely vine leaf images but the generalization potential is far greater since it can recognize the afore mentioned health conditions in variety of crops. For the detection of powdery mildew occurrence ten samples of each crop were utilized and has proven to be effective in crop health status recognition by One Class Classifier or a multitude of One Class Classifiers, giving rise to a conflict situation. The examined crops were: Vitis vinifera, Brassica oleraceae var capitata, Capsicum annuum, Cucurbita maxima, Cucurbita pepo, Cumis melo, Cucumis sativus, Citrullus lanatus, Lagenaria siceratia, Cucurbita moschata, Fragaria ananassa, Pisum sativum, Solanum melongena, and Malus pumila. Solanum lycopersicum was an exception since it was the only examined crop that has been tested by the proposed method, but wasn’t successfully identified.
Tutorial IV: Leaf disease recognition
253
In the occasion of black rot identification, the classification algorithm was tested on ten samples corresponding to each crop and reached high performance either with or without conflict resolution for the following crops: Vitis vinifera, Brassica oleraceae var italica, Fragaria ananassa, Brassica oleracea var capitate, Malus pumila and Pyrus communis. Regarding downey mildew recognition, ten samples of each of the following crops were tested either with or without the help of conflict resolution: Vitis vinifera, Citrullus lanatus, Cucumis sativus, Lactuca sativa, Brassica oleracea var botrytis. Table 7.1 summarizes the recognition potential of the presented approach regarding the health condition assessment for the investigated crop species. The marked cells represent the all the tested combinations whether it is successful or not. As it is depicted in Table 7.1, the healthy status is correctly identified for all investigated plants. The majority of the investigated plants for powdery
Table 7.1 Catalogue of the crops investigated for health condition assessment
Vitis vinifera Brassica oleraceae var. capitata Brassica oleraceae var. botrytis Brassica oleraceae var. italica Cucurbita pepo Lagernaria siceratia Lactuca sativa Cucumis sativus Cucumis melo Citrullus lanatus Cucurbita maxima Solanum melongena Cucurbita moschata Capsicum annuum Fragaria ananassa Pisum sativum Pyrus communnis Malus pumila Solanum lycopersicum
Healthy status
Powdery mildew
Black rot
Downy mildew
x x
x x
x x
x Fail
x
x
x x x x x x x x x x x x x x x x
x x x x x x x x x x x x x Fail
x x
x
x x x
254
Intelligent data mining and fusion systems in agriculture
mildew infection, have been correctly recognized except the Solanum lycopersicum which was falsely recognized as healthy (annotated as Fail in Table 7.1). Brassica oleraceae var capitate was also the second crop that the model didn’t manage to recognize regarding the downy mildew infection. By observing Table 7.1, 46 combinations were tested, from which the 44 achieved correct classification yielding an overall identification accuracy of 95%. In each discrete condition, the correct classification rate was 100% concerning healthy and black rot infested leaf samples while, 93% and 83.3% regarding powdery mildew and downy mildew infested leaf samples respectively. In the occasion of powdery mildew recognition, there have been fourteen samples correctly classified out of fifteen tested, while for downy mildew, five samples were successfully recognized out of six tested. For estimating the model’s generalization capability, the testing samples were selected from a pool of images that did not belong to the calibration dataset. 7.4.4.2 The generalization potential of one class SVM from vines powdery mildew to other plant diseases The One Class SVM classifier annotates vine leaves as infected by powdery mildew assigns to them the label ‘powderyHSV’ and stores the classifier’s parameters as XML. Fig. 7.10A illustrates the evaluation of the classifiers performance on Cucumis sativus leaves. The generalization ability powderyHSV classifier was confirmed by the correct recognition of the infestation
Fig. 7.10 (A) A randomly selected image of Cucumis sativus leaf utilized for validating the one class classifier corresponding to powdery mildew (Pantazi et al., 2019). (B) Annotation of Cucumis sativus as powdery mildew infected by the One Class Classifier (Pantazi et al., 2019).
Tutorial IV: Leaf disease recognition
255
Fig. 7.11 (A) Conflict emerges between One Class SVM models (powderyHSV Vs BlackrotHSV) in a Cucumis sativus leaf (Pantazi et al., 2019). (B) Conflict resolution between One Class SVM models (powderyHSV Vs BlackrotHSV) reaches to assess correctly the powdery mildew infected leaf. (Pantazi et al., 2019).
on several plant species apart from the one that was used for training (Fig. 7.10B). The PowderyHSV model can recognize powdery mildew infection in various crops achieving accurate generalization in nine species such as Vitis vinifera, Brassica oleracea, Lagenaria siceraria, Cucumis sativus, Cucurbita pepo, Citrullus lanatus, Cucumis melo, Cucurbita moschata and Cucurbita maxima. Regarding the testing of the classifiers, ten samples were selected from each of the species. For guaranteeing the correct classification of the investigated sample images, another model called ‘blackrotHSV’ is activated. The two models are activated both by the observes symptoms simultaneously as shown in Fig. 7.11A. For coming to conclusion with respect to the leaf health status, conflict resolution takes place taking into consideration the distances between the samples and the support vectors. The classifier denoting the sample’s label is successfully assigned as powderyHSV (Fig. 7.11B). In the same way, conflict resolution was consistently achieved for different combinations of the employed OC-SVMs classifiers for powdery, downy mildew and blackrot identification and has consistently attained accurate decisions concerning the leaf health condition of the investigated crops. 7.4.4.3 Generalization of the vines black rot model to different crops diseases The One Class SVM classifier named ‘blackrotHSV’ recognizes correctly black rot infected leaves following the calibration with a dataset consisted
256
Intelligent data mining and fusion systems in agriculture
Fig. 7.12 (A) A randomly selected image of Fragaria ananassa leaf utilized for validating the one class classifier corresponding to black rot (Pantazi et al., 2019). (B) The selected area of interest in Fragaria ananassa image is extracted by the GrabCut algorithm (Pantazi et al., 2019). (C) The output of the OC-SVM classifiers is displayed at the top left of the screen (Pantazi et al., 2019).
of vine leaf images.The parameters of the classifers are stored in XML format for increased portability. Fig. 7.12A–C demonstrate the classifier’s identification performance on Fragaria ananassa leaf images.
7.5 Segmentation technique The current method is able to achieving successfully generalize disease ditection by using for the training of OC-SVMs a small amount of vine leaves samples, (8) arbitrarily chosen, with respect to the four different health conditions. The originality of the of the presented study lays on the GrabCut’s algorithm ability to isolate the leaf area from that belonging to the foreground. In the occasion of multiple leaves present to the image even from other crop species, the GrabCut has proven capable of isolating efficiently the individual sample that is validated. In the current case study, no special conditions were necessary for the image acquisition, because the algorithm can isolate the region of the leaf from any natural background, in field conditions. A significant advantage of the current method is that it is nondestructive, which works efficiently without the need of detaching the leaf from the crop. These characteristics constitue to proof that the presented method is robust and operational, because the functionality of the method is not affected by variations in shading, leaf orientation and ambient lighting. Regarding calibration, no camera settings were needed like focus and shutter settings. The afore mentioned features prove that the methodology is user friendly, highly functional and adjustable to a variety of alternative scenarios in different case studies.
Tutorial IV: Leaf disease recognition
257
The operation of the GrabCut algorithmis robust against variable RBG values due to the use of Gaussian Mixtures.
7.6 Classification process and the features extraction process The current case study achieves an accurate recognition of specific leaf diseases by means of novelty detection. In some cases, the phenology of some diseases appears to be similar in terms of occurrence, meaning that for the same crop there are also detected as outliers. The close distance between the feature vector and the one class classifier spheres explains the operation of the algorithm. The characterization of both infected and healthy crops expressed in HSV features and more specifically the Hue channel of healthy crop images is the cornerstone is that defines the structure the OC-SVM for vine leaves. This behavior is closely related to the feature that has been utilized in the LBP histogram. The LBP is responsible for the robustness against variations to ambient factors including scaling, translation and rotations or illumination fluctuations. The crucial information that is associated to the crop health status is exploited by the LBP histogram as variations in textural characteristics that emanate from the symptom and the visual distribution of the symptoms. The variation of the background is overcome due to ability of the GrabCut to remain unaffected by the possible intrinsic factors that could possibly affect the segmentation process arising from the image background and acquisition factors. The acquisition factors related to orientation and lighting are confronted by the LBP histograms thanks to their variability to these factors. However, various diseases or nutrient stresses can cause similar symptoms that cannot be easily identified by the validated OC-SVMs. Even in this case the presented conflict resolution can act in an independent manner, by exploiting the distance of the feature vector to the support distance so as to determine the class. The proposed technique unites the AI’s assets with progressive image analysis aiming to extract image characteristics that identify the textural alterations correlated to crop health status. The proposed approach exploits the synergistic effect between OC-SVMs reaching high precision in assessing crop health status through conflict resolution arising from the between opposing classifiers. The proposed technique can be exploited towards the determination of crop health status in order to decide the best management option in the framework of precision crop protection.
258
Intelligent data mining and fusion systems in agriculture
References Fiel, S., & Sablatnig, R. (2013, August). Writer identification and writer retrieval using the fisher vector on visual vocabularies. In 2013 12th international conference on document analysis and recognition (pp. 545–549): IEEE. Gaunt, R. E. (1995). New technologies in disease measurement and yield loss appraisal. Canadian Journal of Plant Pathology, 17(2), 185–189. Gulhane, V. A., & Gurjar, A. A. (2011). Detection of diseases on cotton leaves and its possible diagnosis. International Journal of Image Processing (IJIP), 5(5), 590–598. Guo, P., Liu, T., & Li, N. (2014). Design of automatic recognition of cucumber disease image. Information Technology Journal, 13(13), 2129–2136. Kebapci, H., Yanikoglu, B., & Unal, G. (2011). Plant image retrieval using color, shape and texture features. The Computer Journal, 54(9), 1475–1490. Li, J., Wu, W., Wang, T., & Zhang, Y. (2008). One step beyond histograms: Image representation using Markov stationary features. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1–8): . Llado, X., Oliver, A., Freixenet, J., Marti, R., & Marti, J. (2009). A textural approach for mass false positive reduction in mammography. Computerized Medical Imaging and Graphics, 33, 415–422. Meunkaewjinda, A., Kumsawat, P., Attakitmongcol, K., & Srikaew, A. (2008, May). Grape leaf disease detection from color imagery using hybrid intelligent system. In Vol. 1. 2008 5th international conference on electrical engineering/electronics, computer, telecommunications and information technology (pp. 513–516): IEEE. Pantazi, X. E., Moshou, D., & Tamouridou, A. A. (2019). Automated leaf disease detection in different crop species through image features analysis and one class classifiers. Computers and Electronics in Agriculture, 156, 96–104. Patil, S. B., & Bodhe, S. K. (2011). Leaf disease severity measurement using image processing. International Journal of Engineering and Technology, 3(5), 297–301. Patil, J. K., & Kumar, R. (2017). Analysis of content based image retrieval for plant leaf diseases using color, shape and texture features. Engineering in Agriculture, Environment and Food, 10(2), 69–78. Pethybridge, S. J., & Nelson, S. C. (2015). Leaf doctor: A new portable application for quantifying plant disease severity. Plant Disease, 99(10), 1310–1316. Rothe, P. R., & Kshirsagar, R. V. (2015, January). Cotton leaf disease identification using pattern recognition techniques. In 2015 international conference on pervasive computing (ICPC) (pp. 1–6): IEEE. Sch€ olkopf, B., Platt, J. C., Shawe-Taylor, J., Smola, A. J., & Williamson, R. C. (2001). Estimating the support of a high-dimensional distribution. Neural Computation, 13(7), 1443–1471. Sˇroba´rova´, A., & Kakalı´kova´, L. (2007). Fungal disease of grapevines. European Journal of Plant Science and Biotechnology, 1, 84–90. Tax, D., & Duin, R. (2004). Support vector data description. Machine Learning, 54(1), 45–66. Vianna, G. K., Oliveira, G. S., & Cunha, G. V. (2017). A neuro-automata decision support system for the control of late blight in tomato crops. World Academy of Science, Engineering and Technology, International Journal of Computer and Information Engineering, 11(4), 455–462 [Mar 2]. Zhang, S., Wu, X., You, Z., & Zhang, L. (2017). Leaf image based cucumber disease recognition using sparse representation classification. Computers and Electronics in Agriculture, 134, 135–141.
Tutorial IV: Leaf disease recognition Contents 7.1 7.2 7.3 7.4
Introduction State of the art Materials and methods Application of LBP in disease recognition of infected plants 7.4.1 Image segmentation 7.4.2 Creation of LBP histogram 7.4.3 One class classification 7.4.4 Generalization of the vines powdery mildew model to different crop diseases 7.5 Segmentation technique 7.6 Classification process and the features extraction process References
241 242 244 245 245 246 248 252 256 257 258
7.1 Introduction Crop infection detection is critical for crop growth monitoring and safeguarding the quality of production. For this purpose, optical recognition can be affectivity utilized by specialists for monitoring possible changes in crop leaves, therefore a highly experienced and specialized background. New techniques handling data mining by AI is able to enhance trustworthy the credibility of a possible infection recognition and thus can be featured into actionable insights to enable interventions. Methods using AI combined with image feature analysis are capable of providing a more precise and effective tool for crop disease detection. Crop infection recognition using visual information is regarded challenging, more specifically, on its automated form. This task is often hard to be tackled, because of the special characteristics in terms of morphology and but also because of lighting conditions. Crop infection evaluation is a critical step for adopting accurate and efficient crop management practices encompassing predictive algorithms and various means of treatment. Intelligent Data Mining and Fusion Systems in Agriculture https://doi.org/10.1016/B978-0-12-814391-9.00007-8
© 2020 Elsevier Inc. All rights reserved.
241
242
Intelligent data mining and fusion systems in agriculture
Fungal diseases are regarded responsible for severe yield degradation, yield low productivity and financial losses, ranging between 5 and 80% depending on the severity of infection, ambient factors related to climate and terrain, and lack of resistance due to susceptible genotype (Sˇroba´rova´ & Kakalı´kova´, 2007). It is crucial to focus on the relation that exists between various unexpected deviations in the characteristics of plant leaf morphology and the variety of the environmental factors that would act as are possible stress factors to a specific crop that demonstrates stress symptoms (Gaunt, 1995).
7.2 State of the art Various techniques are used in the present targeting on identifying plant infections by applying digital image processing. Crop disease identification was achieved in sugarcane leaves by threshold segmentation to isolate the infected leaf location and triangle thresholding to obtain the lesioning area, reaching an average accuracy of 98.60% (Patil & Bodhe, 2011). Texture feature extraction has also been used for plant disease detection. Patil and Kumar (2017) introduced plant disease detection approach base on texture feature extraction including morphological features, uniformity, and relative associations derived by estimating the co-occurrence matrix of the gray image of corn leaves combined with color extraction, giving a trustworthy training feature based set for image enhancement and improved detection. Rothe and Kshirsagar (2015) employed a Back propagation approach for classifying diseased leaf images of cotton. During the training procedure, seven invariant moments were extracted from three leaves images depicting infections from three different diseases respectively. The utilized snake segmentation algorithm achieved a classification of 85.52%, providing an efficient solution for set apart the area of interest (the infected one). However, the proposed method is characterized as time consuming. Zhang, Wu, You, and Zhang (2017) proposed a leaf disease detection procedure in cucumber, by setting apart the infected leaf area through an efficient combination of k-means clustering and color and shape feature extraction, obtaining a score of 85.7% correct recognition. A similar approach has been presented by Guo, Liu, and Li (2014). In this approach texture and color features were combined by a Bayesian approach. The investigated diseases where downy mildew, anthracnose, powdery mildew, and gray mold. The utilized model achieved accuracy rates of 94.0%, 86.7%, 88.8%, and 84.4% respectively. Vianna, Oliveira, and Cunha (2017) proposed a pattern simulation approach for recognizing the globally threatening of late blight in tomato leaves. In this paper, 20 networks were tested, from which the best network
Tutorial IV: Leaf disease recognition
243
produced a prediction of 97.99%. Fiel and Sablatnig (2013) have diversified leaf infection detection by applying Bag of Words with SIFT descriptors in 5 different tree species, attaining an accuracy of 93.6%. Meunkaewjinda, Kumsawat, Attakitmongcol, and Srikaew (2008) presented an automated crop infection diagnosis detection system which utilized various artificial intelligent algorithms in infected vine leaves. Selforganizing feature map and back propagation neural network has been applied in order to detect the colors of vine leaves. Additionally, an altered modified SOM has been applied for segmenting the image and support vector machine is used as a classifier, demonstrating a performance of 86.03%. Self-organizing feature map is also used to detect disease of cotton leaves (Gulhane & Gurjar, 2011). Kebapci, Yanikoglu, and Unal (2011) developed a plant retrieval system taking into consideration the color, shape and texture characteristics by employing the color histogram, the color co-occurrence matrix and a modified Gabor method approach. The experiment has been performed for crop type recognition in domestic plants reaching 73% for successful recognition. Pethybridge and Nelson (2015) have presented an IOS application called ‘Leaf Doctor”, capable of discriminating the possibly infected from the healthy plant tissue areas. In this approach, eight colors denoting the healthy tissue areas form the threshold assisting the recognition of the healthy ones. With respect to the six corresponding infections, the algorithm’s sensitivity was accessed according to targeted range of 10 color corrected images attaining a R2 > 0.79. Nevertheless, some of the main drawbacks of this approach is that for the image processing, a black background is required. The black background has proven to be most of the times responsible for the majority of misclassifications since it can be falsely mixed up with alike colored pixels linked to infection symptoms. Moreover, considering the fact that the samples were destructively taken, makes the method less credible. The employed algorithm was difficult to come to a trustworthy conclusion in variable ambient field conditions and alterations, as a result more stable and trustworthy algorithms are required. The current case study demonstrates a novel application of recognizing four different health status including healthy, downy mildew, powdery mildew and black rot in different tissue samples through One Class Classification. The presented application is able to assess images obtained in real conditions without additional processing and can decide with certainty on the occurrence of infection even at its early stage appearance. The current approach is comprised of a learning procedure with training data relate to each one of the four different crop status. A previously unseen and unlabeled
244
Intelligent data mining and fusion systems in agriculture
feature vector is used as input to a one-class classifiers committee, triggering activations. In the occasion that multiple activations take place, then a conflict occurs since more than one classifiers are regarded as contenders of the new feature vector ownership. The current case study demonstrates high generalization potential regarding the leaf plant disease identification in several crops, where the training set was comprised solely of vine leaf samples.
7.3 Materials and methods The Local Binary Pattern (LBP) is a texture analysis approach, that able to label pixels by defining their proximity limits, forming a binary outcome. The LBP approach takes advantage of its capability of maintaining a stable behavior to the upcoming gray scale level deviations that occur, such as illumination and image processing under complex real-time conditions, proving its computational potential to be implemented in several commercial applications. The LBP operator has been introduced by Li, Wu, Wang, and Zhang (2008) and Llado, Oliver, Freixenet, Marti, and Marti (2009) considering texture features including pattern including the relative pattern’s strength applied with a 3 3 grid, where the center value denotes a threshold. The LBP code is formulated as presented: the thresholded values are scaled by the corresponding weights of the pixels. Since the neighborhood is defined by eight pixels, a set of 28 ¼ 256 discrete labels are produced, corresponding to the relative threshold’s gray levels and remaining pixels of the grid. The contrast indicator (C) is defined as the subtraction of the mean of the pixel levels that lie below the threshold defined from the mean of pixel levels larger or equal to the threshold. In the case that the thresholded pixel levels are equal to zero or one, the contrast value is equal to zero. Fig. 7.1 depicts an illustration of LBP operator function. example
thresholded
6
5
2
1
7
6
1
1
9
8
7
1
Pattern = 11110001
0
1
weights
0
1
0
128
1
64
2
4 8
32
16
LBP = 1 + 16 + 32 + 64 + 128 = 241 C = (6+7+9+8+7)/5 – (5+2+1)/3 = 4.7
Fig. 7.1 Illustration of LBP operator function.
Tutorial IV: Leaf disease recognition
(A)
245
(B)
Fig. 7.2 (A) Hue channel illustration of a vine leaf (Pantazi, Moshou, & Tamouridou, 2019). (B) LBP leaf image of vine crop (Pantazi et al., 2019).
The LBP operator is able to transform an image into a matrix of integer levels that define the detailed local structure of the image (Fig. 7.1). These levels or their derived statistical operators like histogram, are the parameters used for producing image analytics. The LBP operator is mainly applicable to binary images as well as multi-channel images, video streams and multi-dimensional data. The hue channel is derived from image of the leaf as depicted in Fig. 7.2A. The LBP transform application on the Hue channel derived of the segmented image produces an LBP leaf image, as illustrated in Fig. 7.2B.
7.4 Application of LBP in disease recognition of infected plants 7.4.1 Image segmentation As a first step, an image of the diseased leaf that manifests visual symptoms is obtained by a smartphone or tablet (Fig. 7.3A). Afterwards, image segmentation takes places aiming to obtain the leaf area and exclude the image background. Then, a Hue Saturation Value (HSV) transform is imposed to the segmented area (Fig. 7.3B). The GrabCut algorithm operation is summarized in the following steps: 1. The central image points correspond to foreground while the surrounding regions correspond to background. Then, a rectangle is formed including the target region while the internal pixels are recognized as not known and the external ones are denoted as known. 2. An image segmentation is performed by applying Gaussian Mixture Models (GMMs) to the foreground and background utilizing the Orchard-Bouman algorithm to create the clusters.
246
Intelligent data mining and fusion systems in agriculture
Fig. 7.3 (A) Randomly selected image of a vine utilized for the validation of the powdery mildew One Class Classifier (Pantazi et al., 2019). (B) Depiction of the segmented area of interest of the vine leaf illustrated in Fig. 7.1 after HSV transform and the GrabCut algorithm’s operation (Pantazi et al., 2019).
3. 4. 5. 6.
All pixels are classified as foreground or background through the allocation of Gaussian component corresponding to the foreground or the background GMMs. The arising pixel sets yield the learning adaptation, targeting on the creation of new GMMs. A graph is constructed, activating the GrabCut algorithm for assigning a new foreground class. The former steps (4–6) are iterated until all the pixel sets are classified. The GrabCut algorithm creates K components of multivariate GMM corresponding to the background. A similar procedure is followed for K components corresponding to the foreground. The GMM components are labeled according to the color statistics corresponding to each cluster.
7.4.2 Creation of LBP histogram The GrabCut algorithm is applied individually on various single channels of the initial image because of the LBP operator is applied in a single channel. On the initial RGB image two separate transforms options were tested: the first transform obtained in HSV format while the second transform obtained gray scale image. From the HSV transform it has become apparent that the LBP histogram applied on the Hue channel, yielded the highest performance in comparison to the gray scale transform in relation to the detection of disease crop symptoms. The LBP operator manages the labelling of the pixels corresponding to the obtained images based on their mutual distance in the Hue channel, and
Tutorial IV: Leaf disease recognition
247
consequently takes in to account the textural characteristics concerning contrast and Hue variation as illustrated in Fig. 7.3B. The LBP histogram depicts the appearance rate of Hue levels from 0 to 255. For avoiding noisy values in the histogram, the 256 bins has been reduced to 32. An illustration of the reduced histogram is given in Fig. 7.4, where the x and y axes, denote the amount of bins and the allocation of pixels per bin (%) respectively. The inhomogeneity of symptoms in symptomatic leaves induces deviations in texture of a local nature which are evident which appears in the LBP histogram as higher pixel counts in specific bins (Fig. 7.4). As it is shown in Fig. 7.4, the healthy leaf status is associated with subtle alterations in the local texture structure. The pixel count in certain bins in the LBP histograms corresponds to other local deviations that are associated to the collapse of leaf structure produced by the pathogen invasion. Equally, the black rot occurrence provokes high pixel count in the first bin and an additional peak in the sixth bin. In the case of powdery mildew occurrence, lower pixel count is demonstrated in the first and a peak in the ninth bin. This is explained by the contribution of the pixel count to the leaf structure collapse that finally lead to the discrimination of black rot from powdery mildew symptoms.
LBP histogram
1
Healthy Black Rot Powdery mildew
0.9
percentage of pixels
0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 5
10
15
20
25
30
LBP bins
Fig. 7.4 LBP histogram depicting different health conditions (healthy, black rot and powdery mildew infected) in vine crop (Pantazi et al., 2019).
248
Intelligent data mining and fusion systems in agriculture
7.4.3 One class classification The LBP histograms are used for calibrating One Class Classifiers. More exactly, One Class SVMs (OC-SVMs) were employed to classify leaf images according to their health status. For developing these classifiers eight leaf images from vine plants corresponding to each crop health status (healthy, powdery mildew, downy mildew, black rot) constituted sufficient training dataset (Figs. 7.5–7.8). One Class SVMs were assessed by cross validation through application to samples corresponding to health crop status (healthy, powdery mildew, downy mildew, black rot), demonstrating an accurate calibration performance when tested with random images.
Fig. 7.5 The dataset utilized for calibrating the One Class SVM model for the healthy condition (Pantazi et al., 2019).
Fig. 7.6 The dataset utilized for calibrating the One Class SVM model for the powdery mildew infection (Pantazi et al., 2019).
Tutorial IV: Leaf disease recognition
249
Fig. 7.7 The dataset utilized for calibrating the One Class SVM model for the black rot infection (Pantazi et al., 2019).
Fig. 7.8 The dataset utilized for calibrating the One Class SVM model for the downy mildew infection (Pantazi et al., 2019).
7.4.3.1 One class support vector machines (OCSVMs) Classification is performed to discriminate different sample types, whilst regression intends to produce a specific output for available data samples. Unlike the previously mentioned models, One Class Classifiers take advantage of solely target data to express a binary classification argument based on its membership or not to the target dataset. The One class Classification can be also characterized by the term domain description, denoting a set of objects, capable of recognizing those that are similar or dissimilar (labeled as outliers) to the calibration set assisted by novelty detection. There is a plethora of data domain definition or outlier detection techniques in the literature. Tax and Duin (2004) proposed a type of One Class SVM Classification, called Support Vector Data Description (SVDD).
250
Intelligent data mining and fusion systems in agriculture
The SVDD is capable of classifying an unknown object into either target or outlier, based on the hypothesis that target data are bounded in the feature domain. A sphere, with a center a and minimum radius R containing most of N samples {xi; i ¼ 1;::; N}. Slack variables ξi are defined, following the constraints below: ðxi αÞðxi aÞT R2 + ξι ξι 0
(7.1)
The R and the ξi are expressed as an equality by using a weight to slack variables: X F ðR, a, ξi Þ ¼ R2 + C ξι (7.2) i
where C, symbolizes sphere’s volume and it is directly relevant to number of data point exceptions. Consequently, the result is expressed in LaGrange multiplier equation as follows: X X ξi ai R2 + ξi xi 2 + 2αxi + α2 L ðR, a, ai , ξi Þ ¼ R2 + C i X i γ i ξi (7.3) i
where αi 0 and γ i 0 are LaGrange multipliers. By defining the partial derivatives to zero values, three conditions are formed as follows: X αi ¼ 1 ι X ai xi X (7.4) i a¼ X ¼ ai xi ai i i
0 ai C After optimization with respect to αι while satisfying the restrictions expressed in Eq. (7.4) the LaGranzian is expressed as below: X X L¼ ai ðxi xi Þ ai aj xi xj (7.5) i i, j A data sample z belongs to the sphere if its distance from the center is smaller compared to the R (Eq. 7.6): X X ðz aÞðz aÞT ¼ ðz zÞ 2 ai ðz xi Þ + ai aj xi xj R2 (7.6) i i, j
Tutorial IV: Leaf disease recognition
251
For a reduced size dataset, the calibration dataset is defined by the formulation expressed through the Eq.(7.6), in which support vectors are denoted by nonzero αι. In the special case where C < 1, the support vectors equate C. Those data samples originate from the external sphere’s space and consequently they are considered as outliers. The above presented methodology is defined as the Support Vector Domain Description (SVDD). The OCSVM constructs a model by training the classifier using data samples provided by the SVDD. Consequently, the test data are classified by utilizing the variance of the calibration samples (Sch€ olkopf, Platt, Shawe-Taylor, Smola, & Williamson, 2001). 7.4.3.2 One-class support-vector-learning for multi class problems One-Class Classifiers are usually employed to resolve to multi-decision problems. Test data are sampled in random corresponding to various classes. For every one of the individual health condition classes, a classifier is employed and the sphere is constructed aiming to reach a conclusion about for every point emanating from the tested dataset. In the occasion of a newly presented sample, two possibilities arise: 1. In the situation arising when one classifier is assigned a new data sample, it is characterized by the relevant class. 2. Conflict resolution is activated in the case that a couple or more classifiers are assigned for the same data sample. In the occasion that a new sample is presented and not assigned by the classifier, it is considered an outlier. The conflict is handled by employing the Nearest-Support-Vector Strategy. 7.4.3.3 Nearest-support-vector strategy The distance of a new data sample (z) from the support vectors that correspond to each classifier takes its label (Eq. 7.7). By considering that: SV ðtÞ ¼ x^t1 , …, x^t1t , (7.7) The SV denotes the set of support vectors, and i, t their quantity with the classifier t. Aiming to classify z, the decision function f(z) calculate the distances of the data sample to the support vectors in order find the most proximal individually discover in which classifiers it is assigned (Eq. 7.8). f ðzÞ ¼ argminl2C k^ xli zk where C symbolizes each one of the competing classifiers.
(7.8)
252
Intelligent data mining and fusion systems in agriculture
LBP Image segmentation Separation of foreground background GrabCut algorithm
Pixels labelled Derivation of Histogram
OCSVMs Classification into target class and outliers
A classifier for each class Conflict resolution
One Class Classification
Labeling according to proximity
Nearest Support Vector Strategy
Fig. 7.9 Illustration of the followed steps (Pantazi et al., 2019).
All the steps followed in the current case study are presented schematically in Fig. 7.9.
7.4.4 Generalization of the vines powdery mildew model to different crop diseases 7.4.4.1 Validation of the developed disease classification framework A set of eight vine leaf images corresponding to each of the investigated health status were used, including healthy condition, powdery mildew, black rot and downy mildew infected as calibration dataset to the One Class SVM. The calibrated OC-SVM with vine leaves was further employed for disease detection in eighteen different plant species achieved an excellent generalization for all individual cases. Conflict resolution has been proven to be a critical factor, for the health status assessment exceeding 50% of the cases, achieving a 100% in most of the cases. The classifiers assessing the healthy plant status and the three investigated infections (powdery mildew, black rot, downy mildew) were calibrated using solely vine leaf images but the generalization potential is far greater since it can recognize the afore mentioned health conditions in variety of crops. For the detection of powdery mildew occurrence ten samples of each crop were utilized and has proven to be effective in crop health status recognition by One Class Classifier or a multitude of One Class Classifiers, giving rise to a conflict situation. The examined crops were: Vitis vinifera, Brassica oleraceae var capitata, Capsicum annuum, Cucurbita maxima, Cucurbita pepo, Cumis melo, Cucumis sativus, Citrullus lanatus, Lagenaria siceratia, Cucurbita moschata, Fragaria ananassa, Pisum sativum, Solanum melongena, and Malus pumila. Solanum lycopersicum was an exception since it was the only examined crop that has been tested by the proposed method, but wasn’t successfully identified.
Tutorial IV: Leaf disease recognition
253
In the occasion of black rot identification, the classification algorithm was tested on ten samples corresponding to each crop and reached high performance either with or without conflict resolution for the following crops: Vitis vinifera, Brassica oleraceae var italica, Fragaria ananassa, Brassica oleracea var capitate, Malus pumila and Pyrus communis. Regarding downey mildew recognition, ten samples of each of the following crops were tested either with or without the help of conflict resolution: Vitis vinifera, Citrullus lanatus, Cucumis sativus, Lactuca sativa, Brassica oleracea var botrytis. Table 7.1 summarizes the recognition potential of the presented approach regarding the health condition assessment for the investigated crop species. The marked cells represent the all the tested combinations whether it is successful or not. As it is depicted in Table 7.1, the healthy status is correctly identified for all investigated plants. The majority of the investigated plants for powdery
Table 7.1 Catalogue of the crops investigated for health condition assessment
Vitis vinifera Brassica oleraceae var. capitata Brassica oleraceae var. botrytis Brassica oleraceae var. italica Cucurbita pepo Lagernaria siceratia Lactuca sativa Cucumis sativus Cucumis melo Citrullus lanatus Cucurbita maxima Solanum melongena Cucurbita moschata Capsicum annuum Fragaria ananassa Pisum sativum Pyrus communnis Malus pumila Solanum lycopersicum
Healthy status
Powdery mildew
Black rot
Downy mildew
x x
x x
x x
x Fail
x
x
x x x x x x x x x x x x x x x x
x x x x x x x x x x x x x Fail
x x
x
x x x
254
Intelligent data mining and fusion systems in agriculture
mildew infection, have been correctly recognized except the Solanum lycopersicum which was falsely recognized as healthy (annotated as Fail in Table 7.1). Brassica oleraceae var capitate was also the second crop that the model didn’t manage to recognize regarding the downy mildew infection. By observing Table 7.1, 46 combinations were tested, from which the 44 achieved correct classification yielding an overall identification accuracy of 95%. In each discrete condition, the correct classification rate was 100% concerning healthy and black rot infested leaf samples while, 93% and 83.3% regarding powdery mildew and downy mildew infested leaf samples respectively. In the occasion of powdery mildew recognition, there have been fourteen samples correctly classified out of fifteen tested, while for downy mildew, five samples were successfully recognized out of six tested. For estimating the model’s generalization capability, the testing samples were selected from a pool of images that did not belong to the calibration dataset. 7.4.4.2 The generalization potential of one class SVM from vines powdery mildew to other plant diseases The One Class SVM classifier annotates vine leaves as infected by powdery mildew assigns to them the label ‘powderyHSV’ and stores the classifier’s parameters as XML. Fig. 7.10A illustrates the evaluation of the classifiers performance on Cucumis sativus leaves. The generalization ability powderyHSV classifier was confirmed by the correct recognition of the infestation
Fig. 7.10 (A) A randomly selected image of Cucumis sativus leaf utilized for validating the one class classifier corresponding to powdery mildew (Pantazi et al., 2019). (B) Annotation of Cucumis sativus as powdery mildew infected by the One Class Classifier (Pantazi et al., 2019).
Tutorial IV: Leaf disease recognition
255
Fig. 7.11 (A) Conflict emerges between One Class SVM models (powderyHSV Vs BlackrotHSV) in a Cucumis sativus leaf (Pantazi et al., 2019). (B) Conflict resolution between One Class SVM models (powderyHSV Vs BlackrotHSV) reaches to assess correctly the powdery mildew infected leaf. (Pantazi et al., 2019).
on several plant species apart from the one that was used for training (Fig. 7.10B). The PowderyHSV model can recognize powdery mildew infection in various crops achieving accurate generalization in nine species such as Vitis vinifera, Brassica oleracea, Lagenaria siceraria, Cucumis sativus, Cucurbita pepo, Citrullus lanatus, Cucumis melo, Cucurbita moschata and Cucurbita maxima. Regarding the testing of the classifiers, ten samples were selected from each of the species. For guaranteeing the correct classification of the investigated sample images, another model called ‘blackrotHSV’ is activated. The two models are activated both by the observes symptoms simultaneously as shown in Fig. 7.11A. For coming to conclusion with respect to the leaf health status, conflict resolution takes place taking into consideration the distances between the samples and the support vectors. The classifier denoting the sample’s label is successfully assigned as powderyHSV (Fig. 7.11B). In the same way, conflict resolution was consistently achieved for different combinations of the employed OC-SVMs classifiers for powdery, downy mildew and blackrot identification and has consistently attained accurate decisions concerning the leaf health condition of the investigated crops. 7.4.4.3 Generalization of the vines black rot model to different crops diseases The One Class SVM classifier named ‘blackrotHSV’ recognizes correctly black rot infected leaves following the calibration with a dataset consisted
256
Intelligent data mining and fusion systems in agriculture
Fig. 7.12 (A) A randomly selected image of Fragaria ananassa leaf utilized for validating the one class classifier corresponding to black rot (Pantazi et al., 2019). (B) The selected area of interest in Fragaria ananassa image is extracted by the GrabCut algorithm (Pantazi et al., 2019). (C) The output of the OC-SVM classifiers is displayed at the top left of the screen (Pantazi et al., 2019).
of vine leaf images.The parameters of the classifers are stored in XML format for increased portability. Fig. 7.12A–C demonstrate the classifier’s identification performance on Fragaria ananassa leaf images.
7.5 Segmentation technique The current method is able to achieving successfully generalize disease ditection by using for the training of OC-SVMs a small amount of vine leaves samples, (8) arbitrarily chosen, with respect to the four different health conditions. The originality of the of the presented study lays on the GrabCut’s algorithm ability to isolate the leaf area from that belonging to the foreground. In the occasion of multiple leaves present to the image even from other crop species, the GrabCut has proven capable of isolating efficiently the individual sample that is validated. In the current case study, no special conditions were necessary for the image acquisition, because the algorithm can isolate the region of the leaf from any natural background, in field conditions. A significant advantage of the current method is that it is nondestructive, which works efficiently without the need of detaching the leaf from the crop. These characteristics constitue to proof that the presented method is robust and operational, because the functionality of the method is not affected by variations in shading, leaf orientation and ambient lighting. Regarding calibration, no camera settings were needed like focus and shutter settings. The afore mentioned features prove that the methodology is user friendly, highly functional and adjustable to a variety of alternative scenarios in different case studies.
Tutorial IV: Leaf disease recognition
257
The operation of the GrabCut algorithmis robust against variable RBG values due to the use of Gaussian Mixtures.
7.6 Classification process and the features extraction process The current case study achieves an accurate recognition of specific leaf diseases by means of novelty detection. In some cases, the phenology of some diseases appears to be similar in terms of occurrence, meaning that for the same crop there are also detected as outliers. The close distance between the feature vector and the one class classifier spheres explains the operation of the algorithm. The characterization of both infected and healthy crops expressed in HSV features and more specifically the Hue channel of healthy crop images is the cornerstone is that defines the structure the OC-SVM for vine leaves. This behavior is closely related to the feature that has been utilized in the LBP histogram. The LBP is responsible for the robustness against variations to ambient factors including scaling, translation and rotations or illumination fluctuations. The crucial information that is associated to the crop health status is exploited by the LBP histogram as variations in textural characteristics that emanate from the symptom and the visual distribution of the symptoms. The variation of the background is overcome due to ability of the GrabCut to remain unaffected by the possible intrinsic factors that could possibly affect the segmentation process arising from the image background and acquisition factors. The acquisition factors related to orientation and lighting are confronted by the LBP histograms thanks to their variability to these factors. However, various diseases or nutrient stresses can cause similar symptoms that cannot be easily identified by the validated OC-SVMs. Even in this case the presented conflict resolution can act in an independent manner, by exploiting the distance of the feature vector to the support distance so as to determine the class. The proposed technique unites the AI’s assets with progressive image analysis aiming to extract image characteristics that identify the textural alterations correlated to crop health status. The proposed approach exploits the synergistic effect between OC-SVMs reaching high precision in assessing crop health status through conflict resolution arising from the between opposing classifiers. The proposed technique can be exploited towards the determination of crop health status in order to decide the best management option in the framework of precision crop protection.
258
Intelligent data mining and fusion systems in agriculture
References Fiel, S., & Sablatnig, R. (2013, August). Writer identification and writer retrieval using the fisher vector on visual vocabularies. In 2013 12th international conference on document analysis and recognition (pp. 545–549): IEEE. Gaunt, R. E. (1995). New technologies in disease measurement and yield loss appraisal. Canadian Journal of Plant Pathology, 17(2), 185–189. Gulhane, V. A., & Gurjar, A. A. (2011). Detection of diseases on cotton leaves and its possible diagnosis. International Journal of Image Processing (IJIP), 5(5), 590–598. Guo, P., Liu, T., & Li, N. (2014). Design of automatic recognition of cucumber disease image. Information Technology Journal, 13(13), 2129–2136. Kebapci, H., Yanikoglu, B., & Unal, G. (2011). Plant image retrieval using color, shape and texture features. The Computer Journal, 54(9), 1475–1490. Li, J., Wu, W., Wang, T., & Zhang, Y. (2008). One step beyond histograms: Image representation using Markov stationary features. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1–8): . Llado, X., Oliver, A., Freixenet, J., Marti, R., & Marti, J. (2009). A textural approach for mass false positive reduction in mammography. Computerized Medical Imaging and Graphics, 33, 415–422. Meunkaewjinda, A., Kumsawat, P., Attakitmongcol, K., & Srikaew, A. (2008, May). Grape leaf disease detection from color imagery using hybrid intelligent system. In Vol. 1. 2008 5th international conference on electrical engineering/electronics, computer, telecommunications and information technology (pp. 513–516): IEEE. Pantazi, X. E., Moshou, D., & Tamouridou, A. A. (2019). Automated leaf disease detection in different crop species through image features analysis and one class classifiers. Computers and Electronics in Agriculture, 156, 96–104. Patil, S. B., & Bodhe, S. K. (2011). Leaf disease severity measurement using image processing. International Journal of Engineering and Technology, 3(5), 297–301. Patil, J. K., & Kumar, R. (2017). Analysis of content based image retrieval for plant leaf diseases using color, shape and texture features. Engineering in Agriculture, Environment and Food, 10(2), 69–78. Pethybridge, S. J., & Nelson, S. C. (2015). Leaf doctor: A new portable application for quantifying plant disease severity. Plant Disease, 99(10), 1310–1316. Rothe, P. R., & Kshirsagar, R. V. (2015, January). Cotton leaf disease identification using pattern recognition techniques. In 2015 international conference on pervasive computing (ICPC) (pp. 1–6): IEEE. Sch€ olkopf, B., Platt, J. C., Shawe-Taylor, J., Smola, A. J., & Williamson, R. C. (2001). Estimating the support of a high-dimensional distribution. Neural Computation, 13(7), 1443–1471. Sˇroba´rova´, A., & Kakalı´kova´, L. (2007). Fungal disease of grapevines. European Journal of Plant Science and Biotechnology, 1, 84–90. Tax, D., & Duin, R. (2004). Support vector data description. Machine Learning, 54(1), 45–66. Vianna, G. K., Oliveira, G. S., & Cunha, G. V. (2017). A neuro-automata decision support system for the control of late blight in tomato crops. World Academy of Science, Engineering and Technology, International Journal of Computer and Information Engineering, 11(4), 455–462 [Mar 2]. Zhang, S., Wu, X., You, Z., & Zhang, L. (2017). Leaf image based cucumber disease recognition using sparse representation classification. Computers and Electronics in Agriculture, 134, 135–141.