- Email: [email protected]
Detection of early decay on citrus using hyperspectral transmittance imaging technology coupled with principal component analysis and improved watershed segmentation algorithms
Postharvest Biology and Technology 161 (2020) 111071
Contents lists available at ScienceDirect
Postharvest Biology and Technology journal homepage: www.elsevier.com/locate/postharvbio
Detection of early decay on citrus using hyperspectral transmittance imaging technology coupled with principal component analysis and improved watershed segmentation algorithms
T
Xi Tiana,b,c, Shuxiang Fana,b, Wenqian Huanga,b, Zheli Wanga,b, Jiangbo Lia,b,* a
Beijing Research Center of Intelligent Equipment for Agriculture, Beijing, 100097, China National Research Center of Intelligent EQuipment for Agriculture, Beijing, 100097, China c College of Information and Electrical Engineering, China Agricultural University, Beijing, 100083, China b
A R T I C LE I N FO
A B S T R A C T
Keywords: Hyperspectral transmittance imaging Decay detection Stem-end identification Improved watershed segmentation algorithm Orange
Decay caused by Penicillium spp. fungi is one of the main problems affecting marketing of citrus fruit after harvest because the fungal infection can spread fast from a small number of decayed fruit to the whole consignment. However, the automatic detection of decayed citrus is still a challenge. Early decay of citrus happen on surface peel and present a obvious symptom of water-soaked with cell tissue collapse, which may offer the feasibility of transmittance imaging mode to detect decayed region of citrus. In this study, image processing methods including principal component analysis (PCA), pseudo-color image transformation technology and improved watershed segmentation algorithms (IWSA) were employed to analyze the feasibility of decay detection based on the scanned hyperspectral transmittance images (325−1098 nm) of sound and decayed oranges. The results show that PC3 image is promising for decay segmentation. G components extracted from pseudo-color images of PC3 were selected to enhance image contrasts between decayed and sound tissues, and then decayed regions were segmented perfectly by IWSA whether the defects located on the edge or center position of oranges. However, stem-end tissue had similar features with decayed tissue and therefore were easily misidentified as decayed tissues for those decayed samples with stem-end tissue, and so stem-end identification was carried out. PC2 image and R components extracted from pseudo-color images of PC2 were promising for stem-end identification, then IWSA and morphological parameters were used to extract stem-end region. The stem-end was marked in both operations of decay segmentation and stem-end identification, hence decayed region were further determined for eliminating the misclassification interference of stem-end tissue on decay detection by removing the stem-end region from the operation of decay segmentation. For a validation set including of 84 decayed and 66 sound fruit, the success rates were 93 % and 96 %, respectively, and 94 % for decayed, sound and all fruit. Hyperspectral transmittance imaging offers a novel method for automatic detection of early decayed orange caused by fungus.
1. Introduction Decay of citrus fruit caused by Penicillium spp. fungi, a common pathologic disease that spreads through pores and wounds of skin oil glands, is among the main problems affecting marketing of fruit because the fungal infection can spread fast from a small number of decayed fruit to the whole consignment (Eckert and Eaks, 1989). To prevent the rapid spread of decay and avoid greater economic losses during storage and transportation of citrus fruit, bagging has been used widely, but large-scale use of plastic bags is of environmental concern. Fungicides such as thiophanate-methyl and prochlorazalone, is a low-
⁎
cost and effective solution for inhibiting fungal growth, but long-term use of fungicides may result in food poisoning and environmental pollution (Droby, 2005). With the increasing attention to food safety and environmental friendliness, detection of early decayed citrus, a novel idea proposed from the angle of prevention, would be an alternative to bagging and fungicides. However, detection of early decay has always been a challenging task due to the appearance of decay region is very similar to sound skin. Currently, the detection of decayed citrus in packing lines is performed by workers based on the fluorescence effect induced by ultraviolet (UV) light, However, prolonged exposure to UV is potentially harmful for operators. Hence, it is necessary to develop a
Corresponding author at: Beijing Research Center of Intelligent Equipment for Agriculture, 11# shuguang huayuan middle Rd, Beijing, 100097, China. E-mail address: [email protected] (J. Li).
https://doi.org/10.1016/j.postharvbio.2019.111071 Received 6 September 2019; Received in revised form 12 November 2019; Accepted 14 November 2019 0925-5214/ © 2019 Published by Elsevier B.V.
Postharvest Biology and Technology 161 (2020) 111071
X. Tian, et al.
used for detection of chilling injury in cucumber fruit were mainly selected from transmittance images in short-near infrared region. These reports indicted that hyperspectral transmittance mode could obtain better detection performance than diffused reflectance mode for some defects that were hard to be identified. When pathogenic bacteria infects oranges, large amounts of cell wall-degrading enzymes, mycotoxins and organic acids will be released by the Penicillium, and the wound on surface peel presents the watersoaked symptom with cell tissue collaps. Change of organizational structure and chemical component made the transmittance spectral curve and intensity of decayed tissues different from normal tissues, which offered the feasibility of using transmittance imaging mode to detect decayed region of citrus. Because the water-soaked symptom presents in decayed region of all citrus varieties, decay detection of citrus based on transmittance mode has a wide applicability. The hyperspectral transmittance mode provided the inspiration for detection of early decay in citrus. However, the research on detection of early decay on citrus based on this technology has not been reported as far as we know. The object of this study is to develop a new method for detection of decayed citrus using hyperspectral transmittance imaging. Specific aims are to: (1) investigate the feasibility of using hyperspectral transmittance images with the spectral region of 325–1100 nm to detect the decayed citrus; (2) select the most effective principle component (PC) images for decay detection and stem-end identification by principle component analysis (PCA) method; (3) enhance image contrast between target information (e.g. decayed and stem-end tissues) and sound tissues for improving the decay detection accuracy by pseudo-color image enhancement processing method; (4) segment the decayed region with a novel segmentation algorithm called improved watershed segment algorithm, and further determine the decayed region by eliminating the interference from stem-end tissues.
safer and eco-friendly technology for detection of early decayed citrus. Machine vision system based on color camera has been widely used for detection of the defective citrus since the normal peel and defects (such as canker spot, wind scarring and thrips scarring) usually have different color characteristics (Rong et al., 2017; Li et al., 2009). However, different from common defects, early decay in citrus is a hidden defect with similar peel color characteristic to sound skin tissue. Therefore, detection of early decayed citrus is a huge challenge for traditional color imaging technology. Blacsco et al. (2007) obtained low identification performance (65 %) for detection of decayed oranges affected by green mould by using RGB images. Polymethoxylated flavones, a natural tangeretin in peel of most citrus varieties, can emit orange fluorescence induced by ultraviolet (Swift, 1967; Bosabalidis and Tsekos, 1982). The UV-induced fluorescence can be captured easily by traditional RGB machine vision system. Hence, an on-line inspection system for detection of decayed citrus can be developed based on UVinduced fluorescence imaging technology (Blasco et al., 2007; Kurita et al., 2009). However, this apparent fluorescence phenomenon is not present in all citrus varieties. In addition, some physical defects like peel scratch and insect bite may also cause fluorescence effect under UV light. This phenomenon reduces the performance of using UV-induced fluorescence imaging technology to detect decayed citrus in practical application (Momin et al., 2012; Obenland et al., 2010). Hyperspectral imaging technology integrates spectroscopy and digital imaging information to acquire a grayscale image for each band and a spectrum for each pixel (Wu and Sun, 2013; Jiang et al., 2019), as a result, each hyperspectral image contains a large amount of information in a three-dimensional form called ‘hypercube’ which can be analyzed to characterize the object more reliably than the traditional machine vision or spectroscopy techniques (Liu et al., 2014; Xia et al., 2019). Now, hyperspectral imaging technology has been developed as a scientific and effective tool for detection of visible defects (e.g. scars, insect damage and decay) and non-visible defects (e.g. slight bruise and chilling injuries (ElMasry et al., 2009; Cen et al., 2016). With respect to detection of early fungal infection in citrus fruit, Gómez-Sanchis et al. (2008) evaluated the performance of four feature selection methods and two classification methods for classifying between sound and decay tissues based on hyperspectral imaging technology and geometric factor correction algorithm. Lorente et al. (2013) proved the feasibility of using laser-light backscattering imaging to detect superficial decay in citrus fruit by combining the five independent parameters of Gaussian–Lorentzian distribution function with linear discrimination analysis. Li et al. (2016) achieved the visual classification detection of decayed citrus by combining multispectral imaging with pseudo-color image enhancement processing technology. Therefore, hyperspectral imaging technology can be used as a powerful tool to identify the decayed citrus. Compared with healthy tissues, the optical characteristics of diseased tissues changed with the change of physical and chemical properties of citrus tissues, which provided the possibility for classifying between sound and damaged tissues. The sensing mode of hyperspectral imaging technology could be divided into diffuse reflectance and transmittance modes according to the relative position of sample, illumination and detector. Diffuse reflectance mode has been used widely as an effective technology for superficial defect inspection over the past decade, such as bruises and decay on apples (Xing et al., 2007), freezed damaged in mushroom (Gowen et al., 2009). Unlike diffuse reflectance mode, transmittance mode could get information from deeper tissues, therefore, it maybe a better choice for detection of some specific defects. Lu and Ariana (2013) found that hyperspectral transmittance mode had the better classification result than reflectance mode and the integration of reflectance and transmittance for the detection of fly infestation in pickling cucumbers. Pan et al. (2017) demonstrated that hyperspectral semi-transmittance imaging was more useful as a noninvasive method than reflectance mode for hollowness identification in white radishes. Cen et al. (2016) found that the optimal wavebands
2. Materials and methods 2.1. Citrus samples preparation and decay treatment ‘Gannan’ navel oranges (Citrus sinensis L.) that are sold all over China and even exported to overseas markets have great economical value and was cultivated widely in Jiangxi province, China. However, decay caused by P. digitatum fungi may lead to great economic losses during long-distance transportation. ‘Gannan’ navel oranges free of surface defects or contaminant were purchased from the local fruit supermarket in Beijing during the harvest season in December 2018. All samples have the equator diameter of about 70 mm. Oranges naturally infected with P. digitatum were collected as source of fungi and then placed them in a chamber with relative humidity of 99 % and temperature of 25 ℃ for five days to facilitate the growth of fungi spores. Fungi spores were taken off from infected fruit after proliferation and dissolved in deionized water to make spore suspension by stirring. Next, the fungal spore suspension of about 200 u L was injected into sound oranges with 5 mm deep at the positions of near stem, equator or navel by using a steel needle. Each orange was inoculated one or more areas randomly. In order to accelerate the decay of samples, the inoculated oranges were stored in a plastic box with relative humidity of 99 % and temperature of 25 ℃. Three days later, the decayed areas with the diameter range of 5−15 mm showed at the infected positions. The early decayed areas were still barely visible to the naked eye. A total of 132 sound fruits and 168 infected fruits with spores of P. digitatum fungi were prepared in our study. 66 sound and 84 infected oranges of them were selected randomly as training set for developing the decay detection algorithm, and the remaining samples were used as validation set for verifying the feasibility of algorithm.
2
Postharvest Biology and Technology 161 (2020) 111071
X. Tian, et al.
Fig. 1. Hyperspectral transmittance imaging system.
range of 325–1098 nm. A white board with reflectance of 99 % (Spectraon SRT-99-100, Labsphere Inc., North Sutton, NH, USA) was used to acquire the white image (White) with reflectance mode. For acquiring the white image, the exposure time was set at 3 ms for avoiding signal saturation. The dark image (Dark) was acquired by turning off the lamps and covering the camera lens with a black cap (Li et al., 2014). To reduce the disturbance of dark current from the CCD detector, the raw hyperspectral transmittance images (Raw) were corrected by using the white and dark references according to the following equation:
2.2. Hyperspectral transmittance image acquisition The line-scanning hyperspectral imaging system (Fig. 1) was composed of one 502 × 500 pixels electron magnifying charge-coupled (EMCCD) camera (Andor Luca DL–604 M, Andor Technology plc., Belfast, UK), one imaging spectrograph (ImSpector V10EQE, Spectra Imaging Ltd., Oulu, Finland) connected to a standard 23 mm C-mount zoom lens (OLE23-f/2.4, Spectra Imaging Ltd., Oulu, Finland) covering the spectra range from 325 to 1098 nm at 1.56 nm intervals, one illustration system with one 150 W (W) halogen lamp (JCR, 15 V, 150 W, BAU, Japan) and one convex lens, a motorized linear mobile platform for sample positioning (EZHR17EN, AllMotion, Inc., USA) and a computer. Light source was installed directly beneath the spectrograph for collecting transmittance images. There was a circular hole with diameter of 40 mm on the mobile platform for lighting orange sample from the bottom. The hyperspectral imaging system was operated in a dark chamber for reducing the interference of the external light. In order to acquire high-quality hyperspectral transmittance images of oranges, some crucial parameters were first set before data collection. The distance between mobile platform and the lens was set at 300 mm, the movement speed of sample was set 3.3 mm/s, and the camera was set to run with exposure time of 22 ms and gain of 50. The tested orange was placed on the circular hole of mobile platform and manually oriented the side of decay spot towards the camera but no necessary directly facing the camera, so that the decay spots were randomly distributed on the acquired images for more effectively evaluating the detection performance of proposed algorithms. The imaging spectrograph could scan the tested sample line-by-line as the mobile platform moved the sample through the field of view of the optical system. Each collected spectral image could be called as a three dimensional data cube with two spatial dimensions (x, y) and one spectral dimension (λ). There were 700 × 502 pixels were included in the spatial components (x, y) and 500 bands at 1.56 nm intervals were contained in the spectral
T=
Raw− Dark White− Dark
(1)
The calibrated image T was used for further data processing and analysis. 2.3. Principal component analysis (PCA) Principal component analysis (PCA) is an orthogonal linear transform technology, which could distribute the target components into the new coordinate space according to their information content with the method of linear projection. PCA has been used wildly for defect detection of food and agricultural products based on the hyperspectral image data (Liu et al., 2014; Feng and Sun 2012; ElMasry et al. 2012; Liet al. 2011). PCA can transform the hyperspectral image into sequence of principal component (PC) images through computing a linear projection of the spectral data (Kara and Dirgenali, 2007). The covariance matrix of the images were used to calculate the weighted values, then the linear sum of the original images at individual wavelengths multiplied by the corresponding weighing coefficients (eigenvector) composed the PC image of each band. Additional details about PCA could be found in Howery (1980). In our study, the collected hyperspectral transmittance images were analyzed using PCA method for 3
Postharvest Biology and Technology 161 (2020) 111071
X. Tian, et al.
The Morphological reconstruction could eliminate close textures and noise and restore the sharp contour of the target as well as simplify the gradient image concurrently. Morphological ‘opening and closing’ reconstruction operation is built on the geodesic dilation and erosion (Salembier Clairon and Pardàs Feliu, 1994). The morphological geodesic dilation was defined as follows:
discriminating decayed fruit. 2.4. Pseudo-color image enhancement processing Human physiological vision system is not sensitive to small changes in gray scale, but very sensitive to petty differences in color scale. Pseudo-color image enhancement processing could transform the insensitive gray signal into sensitive color signal for enhancing the target resolution of slight variations in images. This technique is commonly used in the field of image processing due to its advantages of practicality and simplicity. Gray slicing is the most basic and simple method of pseudo-color image processing in recent study. A grayscale image f(x, y) can be regarded as a density function of coordinate (x, y). The grayscale of this image is divided into several levels, which is equivalent to cutting the density function in the intersecting area with some planes that parallel to the x–y plane. Then, the original black-and-white image f(x, y) is transformed into pseudo-color image g(x, y) through gray slicing processing. In our study, the pseudo-color image enhancement processing was employed to transform the 2-D PC gray image obtained from PCA analysis into pseudo-color image for more effectively detecting the decayed fruit.
1 ⎧ Db (∇f (x , y ), r ) = min (∇f (x , y ) ⊕ b, r ) ⎨ Dbi+ 1 (∇f (x , y ), r ) = min (Dbi ⊕ b, r )(i = 1,2, 3, …) ⎩
where ∇[(f(x, y)], r(x, y) and b(x, y) represent gradient image, reference image and structuring element, respectively. The iterative process of morphological geodesic dilation was terminated when the iteration number was equal to preset value or Dbi+ 1 [∇f (x , y ), r ] = Dbi [∇f (x , y ), r ]. According to the above definition, morphological opening reconstruction Ob(rec ) was defined as follows:
Ob(rec ) [∇f (x , y ), r ] = Db(rec ) [(∇f (x , y ) ∘b), r ]
(4)
Similarly, the morphological geodesic erosion was defined as follows: 1 ⎧ Eb (∇f (x , y ), r ) = max ((∇f (x , y ) Θb), r ) ⎨ Ebi+ 1 (∇f (x , y ), r ) = max (Ebi Θb, r )(i = 1,2, 3, …) ⎩
2.5. Improved watershed segmentation method
(5)
Based on erosion operation, morphological closing reconstruction Cb(rec ) was defined as follows:
Image segmentation as a hotspot and difficult question for automatic fruit defect detection is one of the most crucial processing procedures in the field of image analysis and computer vision, (Li et al., 2019). Because watershed transform could obtain continuous and closed target areas by a rapid and precise way, it has been an effective and widely used region segmentation algorithm in the filed of image analysis. The traditional watershed algorithm is difficult to obtain satisfactory results because of over-segmentation caused by irregular gray disturbance and noise in the image. Therefore, an improved watershed segmentation method based on three successive steps including morphological gradient enhancement, morphological reconstruction and marker-controlled watershed segmentation was proposed to segment the decayed region from sound tissues.
Cb(rec ) [∇f (x , y ), r ] = Eb(rec ) [(∇f (x , y ) b), r ]
(6)
where symbols about ‘∘’ and ‘•’ stand for morphological opening and closing operations, respectively. Finally, a morphological ‘opening and closing’ operation OCb(rec ) was defined as follows:
OCb(rec ) [∇f (x , y ), r ] = Ob(rec ) [(∇f (x , y ), r ), r ]
(7)
2.5.3. Marker-controlled watershed segmentation Although morphological reconstruction of gradient image eliminated most of the regional extreme values and noise, some minimum target points that unrelated with the interest object were remained. Therefore, the interest objects were divided into numerous meaningless small areas. The over-segmentation phenomena could be avoided if the interest objects can be obtained before watershed segmentation transformation could suppress the meaningless minimum target points. Here, the target points such as damaged regions were extracted from the morphological reconstruction image by using threshold value method. Specifically, each local minimum region detected from gradient image was judged if it was bigger than the threshold value. Those minimum regions that were larger than the threshold value were marked. Then a binary image was obtained based on the marker result and the gradient image was modified according to the minimum markers. Finally, the watershed transformation was applied to the modified gradient image.
2.5.1. Morphological gradient enhancement Gradient transformation could more effectively reflect the pretty changes among different regions of orange surface compared with original gray images, which can contribute to enhance the image contrast of different tissues. Differential operators of ‘Sobel’, ‘Prewitt’ and ‘Canny’ could be used to obtain the traditional gradient image, while they were sensitive to noise on the orange surface. Therefore, the morphological gradient method was employed to achieve the gradient image in this study. The morphological gradient of image f(x, y) was defined as follows:
∇ [f (x , y )] = (f ⊕ b)(x , y ) − (fΘb)(x , y )
(3)
(2)
where f(x, y) and b(x, y) represent the gray image and disk-shaped structuring element, respectively. The gray gradient in the image increased sharply after morphological gradient processing. The isotropy property of disk-shaped structuring element is helpful for eliminating effectively the morphological gradient relies less on edge direction.
3. Results and discussion 3.1. Overview of spectrum and image Each pixel in hyperspectral image cube has spectral information in the range of 325–1098 nm. In order to obtain the more representative spectra, each spectrum was extracted from a particular region of interest (ROI). In this study, two ROIs (sound_center and sound_edge) from the sound tissues were extracted at the center and edge positions of orange, respectively, for more clearly presenting the spectral features of the whole transmittance images. The average spectra of sound, decayed and stem-end tissues extracted from the same sample in the training set are shown in Fig. 2. As shown in Fig. 2, it could be seen easily that the transmittance spectral intensity and signal-to-noise ratio
2.5.2. Morphological reconstruction Edge information of different tissues on orange surface was enhanced after morphological gradient enhance operation. However, the over-segmentation cannot be avoided if the conventional watershed segmentation is used directly, because some noise and details of the morphological gradient image were also enhanced and sharpened simultaneously. Hence, the gradient image ∇[f (x, y)] was reconstructed based on the morphological ‘opening and closing’ operation for eliminating local extreme values caused by irregular gray disturbance and noise in gradient image and retaining important contour information. 4
Postharvest Biology and Technology 161 (2020) 111071
X. Tian, et al.
Fig. 2. Average spectra of representative ROIs and single-band images at characteristic peaks.
images of different kinds of decayed oranges are shown in Fig. 5. Class Ⅰ represents samples with only one decayed region and this region located on the central position of orange. ClassⅡ also represents samples with only one decayed region, while the decayed region located on the edge position of orange. There are at least two randomly distributed decayed spots on oranges for Class Ⅲ. Class Ⅳ represents those decayed samples with stem-ends. The circles on the RGB images marked the locations of decayed and stem-ends. The first five PC images obtained from the hyperspectral transmittance image in the range of 600−930 nm were shown in Fig. 3. To more clearly show the differences of different PC images, both raw PC images and pseudo-color images are provided. It could be seen that the first four PC images achieved the most valuable information about the original images for decay detection. The PC1 images generally cannot be used for defect detection, because they provide nothing unique features about decayed tissues besides average gray value data of the full wavelengths (Zhang et al., 2014). The more obvious contrast between sound and decayed tissues for all kind s of images can be found in PC2, PC3 and PC4 images, especially the PC3 images.
were very low in the full-wavelength region besides 600−930 nm. This is mainly caused by the strong absorption of fruit tissues, such as water and cellulose. Therefore, only the images including in the spectral range of 600−930 nm were used for further processing. It can be also found from Fig. 2 that different tissues showed similar spectral characteristics and trends, although the spectral intensity values vary greatly. The collapse of epidermic cells and water-soaking of decayed tissues increased the light penetration performance, so intensity of the average spectrum of decayed region was higher than ones of sound and stemend tissues. It also could be found that the difference of transmittance spectra intensity between sound_center and sound_edge was obvious. It primarily attributed to the illumination unit structure of hyperspectral system and geometric construction of citrus. For avoiding the transmittance light leaking from circular hole, the diameter of circular hole used for lighting citrus was only 40 mm that was slightly smaller than equator diameter of orange samples. Thus, the brightness of the edge region of fruit was primarily determined by the scattered light of internal tissue. In addition, because the orange can be considered as elliptic spherical structure in geometric appearance, the non-uniform distribution of light was a common problem affecting images acquisition (Zhang et al., 2018). Two obvious peaks at 710 nm and 820 nm can be observed easily for three different tissues from Fig. 2, and the transmittance spectral intensity of decayed region was higher than other tissues. A single-band image was seemingly promising to be used to classify the decayed oranges. In order to verify this idea, the singleband gray images and pseudo-color images at 710 nm and 820 nm were provided for visualizing the intensity distribution. Although intensity of the decayed region was more brighter than most of sound tissues in the single-band image, some sound regions close to fruit center have the similar intensity values with decayed regions in pseudo-color images at 710 nm and 820 nm, the intertwining of sound and decayed tissues could also be observed from their mean spectra and error bars. Hence, it is difficult to realize the effective detection of rotten region only depending on simple single-wavelength image due to the influence of uneven gray levels on the spherical orange surface.
3.3. Decayed segmentation operation 3.3.1. Pseudo-color image transformation of PC image Pseudo-color image transformation is an effective image processing method to enhance the visual contrast between sound and decayed tissues (Li et al., 2016). For more effectively extracting the target information and improving classification accuracy, PC3 image was transformed into pseudo-color image prior to defection segmentation. The pseudo-color image was transformed into RGB image to choose the most effective component as objective gray image of decay detection in the subsequent steps. By comparing the contrast between sound and decayed tissues of R, G and B components (Fig. 4), it can be found that G component was the most promising to be used for segmentation of the decayed tissues. Comparing with the pseudo-color images of PC3 (Fig. 3), the pseudo-color image of G component presents more obvious contrast between sound and decayed tissues, indicating that pseudocolor image transformation was effective for enhancing the target information (Xia et al., 2019).
3.2. PCA analysis PCA is an effective method for reducing the high spectral dimensionality, enhancing the interest information and removing the noises, therefore, it was employed to detect decayed orange in this study. In order to comprehensively evaluate the detection performance of PCA for early decay detection, all decayed orange samples in training set were divided into four classes according the number of infected regions, the positions of infection and the kinds of tissues. The RGB
3.3.2. IWSA for decay segmentation The proposed IWSA was employed to segment decayed regions on oranges based on the G component image. The critical steps of IWSA including median filter, morphological gradient, gradient image reconstruction and watershed segmentation were shown in Fig. 5, respectively. Comparing with original G component gray image, the 5
Postharvest Biology and Technology 161 (2020) 111071
X. Tian, et al.
Fig. 3. RGB images and the first five PC images (PC1-PC5) obtained from Vis-NIR spectral region (600−930 nm). Images from left to right show eight representative decayed samples of four classes, respectively.
image while retained the main information of the interest target. As a representative example of decay detection for different kinds of samples, PC3 image, G component extracted from pseudo-color image of PC3, pseudo-color image of G component and the resulting images obtained by IWSA are shown in the first to fourth rows of Fig. 6 respectively. It could be seen from inspection results that all decayed regions could be indentified perfectly whether the defects locate on the
morphological gradient operation further enhanced the gray-scale contrast among different tissues and maintained the relatively smooth area meanwhile. Additionally, gradient image reconstruction operation eliminated the local extremum caused by irregular gray scale disturbance and noise in gradient image and remained the most important extremal information about target outline. Operations of both morphological gradient and gradient image reconstruction simplifed the
Fig. 4. Pseudo-color image transformation for PC3 image.
6
Postharvest Biology and Technology 161 (2020) 111071
X. Tian, et al.
Fig. 5. Improved watershed segmentation algorithm.
Fig. 6. Segmentation results of decayed regions on oranges based on improved watershed segmentation algorithm.
stem-end. However, some sound tissues at the edge position are also mistakenly identified as stem-end tissue (the fourth row of Fig. 6), because the two of them have similar intensity feature. Therefore, morphological parameters including area (A) and circularity (C) as new discrimination factors were used for further identification of stem-end based on the segmentation results obtained by IWSA. To be more specific, the segmentation regions obtained by IWSA were filled and labeled firstly, then, A was calculated for each labeled region based on the nonzero pixels (Npixels) and C was calculated according to the following equation:
edge or center positions, showing PCA, pseudo-color image transformation and IWSA were useful for decayed detection based on the hyperspectral transmittance imaging technology. However, unfortunately, the stem-end tissue showed the similar feature with decayed tissues and was misidentified as decayed tissues in Class Ⅳ samples (Fig. 6). Therefore, it is impossible to effectively detect decayed citrus based solely on PC3 image and pseudo-color image. Thus, identification of stem-end tissue was another critical task for improving the detection accuracy of decayed regions. 3.4. Stem-end identification
C=
Checking all PC images again, it can be found that PC2 image could be used for the identification of stem-end tissue since the high contrast between stem-end tissue and decayed tissue. Thus, the pseudo-color image transformation technology was employed to process PC2 image. R, G and B components were extracted from the pseudo-color image of PC2, and R component shows strongest contrast between stem-end and other tissue, as shown in Fig.7. Then, IWSA was used to identify stemend tissue based on R component. The first to fourth rows of Fig. 8 showed the PC2 image, R component extracted from pseudo-color image of PC2, pseudo-color image of R component and the resulting images obtained by IWSA, respectively. It could be found that the stemend tissue could be successfully identified from sound and decay tissues in Class Ⅳ samples, indicating the feasibility of using IWSA to identify
4πA P2
where P is the perimeter of labeled region. The labeled region was identified as stem-end, only if C > 0.65 and 650 > A > 150. Otherwise, the labeled region was considered as sound tissue. The parameter range of A was a statistical result from training samples in this study. However, it could be flexibly set in actual application. Based on the discrimination factors and strategy, the stem-end of class IV can be accurately identified and removed interference from the sound tissue at edge position of image. The results of stem-end identification were shown in the fifth row of Fig. 8
Fig. 7. Pseudo-color image transformation for PC2 image.
7
Postharvest Biology and Technology 161 (2020) 111071
X. Tian, et al.
Fig. 8. Segmentation results of stem-end regions on orange based on improved watershed segmentation algorithm and morphological parameters.
transmittance imaging coupled with PCA and IWSA was evaluated using all samples (84 infected oranges and 66 sound oranges) in the validation set. Six infected samples were not correctly detected and three sound samples were misidentified as infected oranges. The total success rate was 94 % for validation set. More specifically, the success rates for decayed oranges and sound oranges were 93 % and 96 %, respectively. In term of six undetected decayed oranges, the decayed areas were small and coincidentally located on the extreme edge of the images. They were removed in the operation of morphological gradient. In practical application, the detection rate of decayed fruit was crucial for citrus industry, because the fungal infection can spread fast from a small number of decayed fruit to the whole consignment and result in great economic losses. However, this case would not reduce detection efficacy in practical online automatic inspection because multiple images would be acquired for each orange. In the case of the misidentification of sound samples, it mainly attributed to the excessive circularity threshold of irregular stem-end extracted by IWSA. A low rate of false positives could be accepted since there was a greater
3.5. Further decay determination by removing stem-end region It has been mentioned above that the detection accuracy of decayed tissues could be reduced by the stem-end for Class IV samples. By comprehensively analyzing the results of decayed region detection operation and stem-end identification operation, it can be found that the stem-end was segmented well in both operations. Hence, the marked region that located at the same position would be considered as stem-end and removed from the result of decay segmentation operation, and the remained regions of decay segmentation would be decay region. The finally decay determination results after removing the stemend were shown in the last row of Fig. 6. Fig. 9 demonstrates primary procedures for detection of the decayed region on orange with stem-end based on the proposed all methods in this study. 3.6. Verification results The proposed algorithm for decay detection based on hyperspectral
Fig. 9. Flow chart of decay detection on orange based on hyperspectral transmittance imaging coupled with principle component analysis and improved watershed segmentation algorithms. 8
Postharvest Biology and Technology 161 (2020) 111071
X. Tian, et al.
tolerance to reject sound fruit rather than to accept decay fruit for citrus industry (Li et al., 2016).
caused by penicilliumdigitatum in mandarins. J. Food Eng. 89 (1), 80–86. https:// doi.org/10.1016/j.jfoodeng.2008.04.009. Gowen, A.A., Taghizadeh, M., O’Donnell, C.P., 2009. Identification of mushrooms subjected to freeze damage using hyperspectral imaging. J. Food Eng. 93 (1), 7–12. https://doi.org/10.1016/j.jfoodeng.2008.12.021. Jiang, B., He, J., Yang, S., Fu, H., Li, T., Song, H., He, D., 2019. Fusion of machine vision technology and AlexNet-CNNs deep learning network for the detection of postharvest apple pesticide residues. Artif. Intell. Agric. 1, 1–8. https://doi.org/10.1016/j.aiia. 2019.02.001. Kara, S., Dirgenali, F., 2007. A system to diagnose atherosclerosis via wavelet transforms, principal component analysis and artificial neural networks. Expert Syst. Appl. 32 (2), 632–640. https://doi.org/10.1016/j.eswa.2006.01.043. Kurita, M., Kondo, N., Shimizu, H., Ling, P., Falzea, P.D., Shiigi, T., Ninomiya, K., Nishizu, T., Yamamoto, K., 2009. A double image acquisition system with visible and UV LEDs for citrus fruit. J. Robot. Mechatron. 21 (4), 533–540. https://doi.org/10.20965/jrm. 2009.p0533. Li, J., Huang, W., Chen, L., Fan, S., Zhang, B., Guo, Z., Zhao, C., 2014. Variable selection in visible and near-infrared spectral analysis for noninvasive determination of soluble solids content of ‘Ya’ pear. Food Anal. Methods 7 (9), 1891–1902. https://doi.org/10. 1007/s12161-014-9832-8. Li, J., Huang, W., Tian, X., Wang, C., Fan, S., Zhao, C., 2016. Fast detection and visualization of early decay in citrus using Vis-NIR hyperspectral imaging. Comput. Electron. Agric. 127, 582–592. https://doi.org/10.1016/j.compag.2016.07.016. Li, J., Luo, W., Wang, Z., Fan, S., 2019. Early detection of decay on apples using hyperspectral reflectance imaging combining both principal component analysis and improved watershed segmentation method. Postharvest Biol. Technol. 149, 235–246. https://doi.org/10.1016/j.postharvbio.2018.12.007. Li, J., Rao, X., Ying, Y., Ma, B., Guo, J., 2009. Background and external defects segmentation of navel orange based on mask and edge gray value compensation algorithm. Trans. Chin. Soc. Agric. Eng. 25 (12), 133–137. https://doi.org/10.3969/j. issn.1002-6819.2009.12.024. Liu, D., Sun, D.W., Zeng, X.A., 2014. Recent advances in wavelength selection techniques for hyperspectral image processing in the food industry. Food Bioproc. Tech. 7 (2), 307–323. https://doi.org/10.1007/s11947-013-1193-6. Lorente, D., Zude, M., Regen, C., Palou, L., Gómez-Sanchis, J., Blasco, J., 2013. Early decay detection in citrus fruit using laser-light backscattering imaging. Postharvest Biol. Technol. 86, 424–430. https://doi.org/10.1016/j.postharvbio.2013.07.021. Lu, R., Ariana, D.P., 2013. Detection of fruit fly infestation in pickling cucumbers using a hyperspectral reflectance/transmittance imaging system. Postharvest Biol. Technol. 81, 44–50. https://doi.org/10.1117/12.887524. Momin, M.A., Kondo, N., Kuramoto, M., Ogawa, Y., Yamamoto, K., Shiigi, T., 2012. Investigation of excitation wavelength for fluorescence emission of citrus peels based on UV-VIS spectra. Eng. Agric. Environ. Food 5 (4), 126–132. https://doi.org/10. 1016/S1881-8366(12)80008-5. Obenland, D., Margosan, D., Smilanick, J.L., Mackey, B., 2010. Ultraviolet fluorescence to identify navel oranges with poor peel quality and decay. HortTechnology 20 (6), 991–995. https://doi.org/10.1590/S0100-29452010000400045. Pan, L., Sun, Y., Xiao, H., Gu, X., Hu, P., Wei, Y., Tu, K., 2017. Hyperspectral imaging with different illumination patterns for the hollowness classification of white radish. Postharvest Biol. Technol. 126, 40–49. https://doi.org/10.1016/j.postharvbio.2016. 12.006. Rong, D., Rao, X., Ying, Y., 2017. Computer vision detection of surface defect on oranges by means of a sliding comparison window local segmentation algorithm. Comput. Electron. Agric. 137, 59–68. https://doi.org/10.1016/j.compag.2017.02.027. Salembier Clairon, P.J., Pardàs Feliu, M., 1994. Hierarchical morphological segmentation for image sequence coding. Ieee Trans. Image Process. 3 (5), 639–651. https://doi. org/10.1109/83.334980. Swift, L.J., 1967. Thin-layer chromatographic-spectrophotometric analysis for neutral fraction flavones in orange peel juice. J. Agric. Food Chem. 15 (1), 99–101. https:// doi.org/10.1021/jf60149a028. Wu, D., Sun, D.W., 2013. Potential of time series-hyperspectral imaging (TS-HSI) for noninvasive determination of microbial spoilage of salmon flesh. Talanta 111, 39–46. https://doi.org/10.1016/j.talanta.2013.03.041. Xia, Y., Xu, Y., Li, J., Zhang, C., Fan, S., 2019. 2019. Recent advances in emerging techniques for non-destructive detection of seed viability: a review. Atrif Intell. Agric. 1, 35–47. Xing, J., Saeys, W., De Baerdemaeker, J., 2007. Combination of chemometric tools and image processing for bruise detection on apples. Comput. Electron. Agric. 56 (1), 1–13. https://doi.org/10.1016/j.compag.2006.12.002. Zhang, B., Dai, D., Huang, J., Zhou, J., Gui, Q., Dai, F., 2018. Influence of physical and biological variability and solution methods in fruit and vegetable quality nondestructive inspection by using imaging and near-infrared spectroscopy techniques: a review. Crit. Rev. Food Sci. Nutr. 58 (12), 2099–2118. https://doi.org/10.1080/ 10408398.2017.1300789. Zhang, B., Huang, W., Li, J., Zhao, C., Fan, S., Wu, J., Liu, C., 2014. Principles, developments and applications of computer vision for external quality inspection of fruits and vegetables: a review. Food Res. Int. 62, 326–343. https://doi.org/10.1016/j. foodres.2014.03.012.
4. Conclusion Detection of early decayed oranges affected by P. digitatum fungus was observed in this paper for avoiding great economic losses from citrus industry. Hyperspectral transmittance imaging coupled with PCA, pseudo-color image transformation technology and IWSA was employed to analyze the feasibility of decay detection. The results showed that PC3 image was promising for decay segmentation and G component of the pseudo-color image of PC3 could be selected for enhancing image contrast between decayed regions and sound tissues. Further, it was found that all decayed regions could be effectively segmented by IWSA. To eliminate the misclassification of stem-end in the process of decay detection, stem-end identification algorithm was also developed. Study results showed that PC2 image and R component of the pseudo-color image of PC2 was promising for stem-end identification, and stem-end region could be extracted by IWSA and morphological parameters. By performed the combination algorithm of both decay segmentation operation and stem-end identification, the success rates for decayed, sound and total oranges were 93 %, 96 % and 94.00 %, respectively, for 150 samples in the validation set (84 decayed samples and 66 sound samples). Therefore, the hyperspectral transmittance imaging offers a useful reference for automatic detection of early decayed oranges caused by P. digitatum fungus. The other physical defects such as peel scratch and insect bit and their effects on the detection accuracy of early decay will be investigated in the subsequent studies. Declaration of Competing Interest None. Acknowledgements This study was supported by National Natural Science Foundation of China (Grant No. 31772052 and 31972152) and Young Scientist Fund of Beijing Academy of Agriculture and Forestry Sciences (Grant No. QNJJ201818). References Blasco, J., Aleixos, N., Molto, E., 2007. Computer vision detection of peel defects in citrus by means of a region oriented segmentation algorithm. J. Food Eng. 81 (3), 535–543. https://doi.org/10.1016/j.jfoodeng.2006.12.007. Bosabalidis, A., Tsekos, I., 1982. Ultrastructural studies on the secretory cavities of Citrus deliciosa ten. II. Development of the essential oil-accumulating central space of the gland and process of active secretion. Protoplasma 112 (1–2), 63–70. https://doi.org/ 10.1007/BF01280216. Cen, H., Lu, R., Zhu, Q., Mendoza, F., 2016. Nondestructive detection of chilling injury in cucumber fruit using hyperspectral imaging with feature selection and supervised classification. Postharvest Biol. Technol. 111, 352–361. https://doi.org/10.1016/j. postharvbio.2015.09.027. Droby, S., 2005. March. Improving quality and safety of fresh fruits and vegetables after harvest by the use of biocontrol agents and natural materials. International Symposium on Natural Preservatives in Food Systems 709, 45–52. https://doi.org/ 10.17660/ActaHortic.2006.709.5. Eckert, J.W., Eaks, I.L., 1989. Postharvest Disorders and Diseases of Citrus. The Citrus Industry Vol. 5 University California Press, Berkeley, CA, USA. ElMasry, G., Wang, N., Vigneault, C., 2009. Detecting chilling injury in Red Delicious apple using hyperspectral imaging and neural networks. Postharvest Biol. Technol. 52 (1), 1–8. https://doi.org/10.1016/j.postharvbio.2008.11.008. Gómez-Sanchis, J., Gómez-Chova, L., Aleixos, N., Camps-Valls, G., Montesinos-Herrero, C., Moltó, E., Blasco, J., 2008. Hyperspectral system for early detection of rottenness
9
Contents lists available at ScienceDirect
Postharvest Biology and Technology journal homepage: www.elsevier.com/locate/postharvbio
Detection of early decay on citrus using hyperspectral transmittance imaging technology coupled with principal component analysis and improved watershed segmentation algorithms
T
Xi Tiana,b,c, Shuxiang Fana,b, Wenqian Huanga,b, Zheli Wanga,b, Jiangbo Lia,b,* a
Beijing Research Center of Intelligent Equipment for Agriculture, Beijing, 100097, China National Research Center of Intelligent EQuipment for Agriculture, Beijing, 100097, China c College of Information and Electrical Engineering, China Agricultural University, Beijing, 100083, China b
A R T I C LE I N FO
A B S T R A C T
Keywords: Hyperspectral transmittance imaging Decay detection Stem-end identification Improved watershed segmentation algorithm Orange
Decay caused by Penicillium spp. fungi is one of the main problems affecting marketing of citrus fruit after harvest because the fungal infection can spread fast from a small number of decayed fruit to the whole consignment. However, the automatic detection of decayed citrus is still a challenge. Early decay of citrus happen on surface peel and present a obvious symptom of water-soaked with cell tissue collapse, which may offer the feasibility of transmittance imaging mode to detect decayed region of citrus. In this study, image processing methods including principal component analysis (PCA), pseudo-color image transformation technology and improved watershed segmentation algorithms (IWSA) were employed to analyze the feasibility of decay detection based on the scanned hyperspectral transmittance images (325−1098 nm) of sound and decayed oranges. The results show that PC3 image is promising for decay segmentation. G components extracted from pseudo-color images of PC3 were selected to enhance image contrasts between decayed and sound tissues, and then decayed regions were segmented perfectly by IWSA whether the defects located on the edge or center position of oranges. However, stem-end tissue had similar features with decayed tissue and therefore were easily misidentified as decayed tissues for those decayed samples with stem-end tissue, and so stem-end identification was carried out. PC2 image and R components extracted from pseudo-color images of PC2 were promising for stem-end identification, then IWSA and morphological parameters were used to extract stem-end region. The stem-end was marked in both operations of decay segmentation and stem-end identification, hence decayed region were further determined for eliminating the misclassification interference of stem-end tissue on decay detection by removing the stem-end region from the operation of decay segmentation. For a validation set including of 84 decayed and 66 sound fruit, the success rates were 93 % and 96 %, respectively, and 94 % for decayed, sound and all fruit. Hyperspectral transmittance imaging offers a novel method for automatic detection of early decayed orange caused by fungus.
1. Introduction Decay of citrus fruit caused by Penicillium spp. fungi, a common pathologic disease that spreads through pores and wounds of skin oil glands, is among the main problems affecting marketing of fruit because the fungal infection can spread fast from a small number of decayed fruit to the whole consignment (Eckert and Eaks, 1989). To prevent the rapid spread of decay and avoid greater economic losses during storage and transportation of citrus fruit, bagging has been used widely, but large-scale use of plastic bags is of environmental concern. Fungicides such as thiophanate-methyl and prochlorazalone, is a low-
⁎
cost and effective solution for inhibiting fungal growth, but long-term use of fungicides may result in food poisoning and environmental pollution (Droby, 2005). With the increasing attention to food safety and environmental friendliness, detection of early decayed citrus, a novel idea proposed from the angle of prevention, would be an alternative to bagging and fungicides. However, detection of early decay has always been a challenging task due to the appearance of decay region is very similar to sound skin. Currently, the detection of decayed citrus in packing lines is performed by workers based on the fluorescence effect induced by ultraviolet (UV) light, However, prolonged exposure to UV is potentially harmful for operators. Hence, it is necessary to develop a
Corresponding author at: Beijing Research Center of Intelligent Equipment for Agriculture, 11# shuguang huayuan middle Rd, Beijing, 100097, China. E-mail address: [email protected] (J. Li).
https://doi.org/10.1016/j.postharvbio.2019.111071 Received 6 September 2019; Received in revised form 12 November 2019; Accepted 14 November 2019 0925-5214/ © 2019 Published by Elsevier B.V.
Postharvest Biology and Technology 161 (2020) 111071
X. Tian, et al.
used for detection of chilling injury in cucumber fruit were mainly selected from transmittance images in short-near infrared region. These reports indicted that hyperspectral transmittance mode could obtain better detection performance than diffused reflectance mode for some defects that were hard to be identified. When pathogenic bacteria infects oranges, large amounts of cell wall-degrading enzymes, mycotoxins and organic acids will be released by the Penicillium, and the wound on surface peel presents the watersoaked symptom with cell tissue collaps. Change of organizational structure and chemical component made the transmittance spectral curve and intensity of decayed tissues different from normal tissues, which offered the feasibility of using transmittance imaging mode to detect decayed region of citrus. Because the water-soaked symptom presents in decayed region of all citrus varieties, decay detection of citrus based on transmittance mode has a wide applicability. The hyperspectral transmittance mode provided the inspiration for detection of early decay in citrus. However, the research on detection of early decay on citrus based on this technology has not been reported as far as we know. The object of this study is to develop a new method for detection of decayed citrus using hyperspectral transmittance imaging. Specific aims are to: (1) investigate the feasibility of using hyperspectral transmittance images with the spectral region of 325–1100 nm to detect the decayed citrus; (2) select the most effective principle component (PC) images for decay detection and stem-end identification by principle component analysis (PCA) method; (3) enhance image contrast between target information (e.g. decayed and stem-end tissues) and sound tissues for improving the decay detection accuracy by pseudo-color image enhancement processing method; (4) segment the decayed region with a novel segmentation algorithm called improved watershed segment algorithm, and further determine the decayed region by eliminating the interference from stem-end tissues.
safer and eco-friendly technology for detection of early decayed citrus. Machine vision system based on color camera has been widely used for detection of the defective citrus since the normal peel and defects (such as canker spot, wind scarring and thrips scarring) usually have different color characteristics (Rong et al., 2017; Li et al., 2009). However, different from common defects, early decay in citrus is a hidden defect with similar peel color characteristic to sound skin tissue. Therefore, detection of early decayed citrus is a huge challenge for traditional color imaging technology. Blacsco et al. (2007) obtained low identification performance (65 %) for detection of decayed oranges affected by green mould by using RGB images. Polymethoxylated flavones, a natural tangeretin in peel of most citrus varieties, can emit orange fluorescence induced by ultraviolet (Swift, 1967; Bosabalidis and Tsekos, 1982). The UV-induced fluorescence can be captured easily by traditional RGB machine vision system. Hence, an on-line inspection system for detection of decayed citrus can be developed based on UVinduced fluorescence imaging technology (Blasco et al., 2007; Kurita et al., 2009). However, this apparent fluorescence phenomenon is not present in all citrus varieties. In addition, some physical defects like peel scratch and insect bite may also cause fluorescence effect under UV light. This phenomenon reduces the performance of using UV-induced fluorescence imaging technology to detect decayed citrus in practical application (Momin et al., 2012; Obenland et al., 2010). Hyperspectral imaging technology integrates spectroscopy and digital imaging information to acquire a grayscale image for each band and a spectrum for each pixel (Wu and Sun, 2013; Jiang et al., 2019), as a result, each hyperspectral image contains a large amount of information in a three-dimensional form called ‘hypercube’ which can be analyzed to characterize the object more reliably than the traditional machine vision or spectroscopy techniques (Liu et al., 2014; Xia et al., 2019). Now, hyperspectral imaging technology has been developed as a scientific and effective tool for detection of visible defects (e.g. scars, insect damage and decay) and non-visible defects (e.g. slight bruise and chilling injuries (ElMasry et al., 2009; Cen et al., 2016). With respect to detection of early fungal infection in citrus fruit, Gómez-Sanchis et al. (2008) evaluated the performance of four feature selection methods and two classification methods for classifying between sound and decay tissues based on hyperspectral imaging technology and geometric factor correction algorithm. Lorente et al. (2013) proved the feasibility of using laser-light backscattering imaging to detect superficial decay in citrus fruit by combining the five independent parameters of Gaussian–Lorentzian distribution function with linear discrimination analysis. Li et al. (2016) achieved the visual classification detection of decayed citrus by combining multispectral imaging with pseudo-color image enhancement processing technology. Therefore, hyperspectral imaging technology can be used as a powerful tool to identify the decayed citrus. Compared with healthy tissues, the optical characteristics of diseased tissues changed with the change of physical and chemical properties of citrus tissues, which provided the possibility for classifying between sound and damaged tissues. The sensing mode of hyperspectral imaging technology could be divided into diffuse reflectance and transmittance modes according to the relative position of sample, illumination and detector. Diffuse reflectance mode has been used widely as an effective technology for superficial defect inspection over the past decade, such as bruises and decay on apples (Xing et al., 2007), freezed damaged in mushroom (Gowen et al., 2009). Unlike diffuse reflectance mode, transmittance mode could get information from deeper tissues, therefore, it maybe a better choice for detection of some specific defects. Lu and Ariana (2013) found that hyperspectral transmittance mode had the better classification result than reflectance mode and the integration of reflectance and transmittance for the detection of fly infestation in pickling cucumbers. Pan et al. (2017) demonstrated that hyperspectral semi-transmittance imaging was more useful as a noninvasive method than reflectance mode for hollowness identification in white radishes. Cen et al. (2016) found that the optimal wavebands
2. Materials and methods 2.1. Citrus samples preparation and decay treatment ‘Gannan’ navel oranges (Citrus sinensis L.) that are sold all over China and even exported to overseas markets have great economical value and was cultivated widely in Jiangxi province, China. However, decay caused by P. digitatum fungi may lead to great economic losses during long-distance transportation. ‘Gannan’ navel oranges free of surface defects or contaminant were purchased from the local fruit supermarket in Beijing during the harvest season in December 2018. All samples have the equator diameter of about 70 mm. Oranges naturally infected with P. digitatum were collected as source of fungi and then placed them in a chamber with relative humidity of 99 % and temperature of 25 ℃ for five days to facilitate the growth of fungi spores. Fungi spores were taken off from infected fruit after proliferation and dissolved in deionized water to make spore suspension by stirring. Next, the fungal spore suspension of about 200 u L was injected into sound oranges with 5 mm deep at the positions of near stem, equator or navel by using a steel needle. Each orange was inoculated one or more areas randomly. In order to accelerate the decay of samples, the inoculated oranges were stored in a plastic box with relative humidity of 99 % and temperature of 25 ℃. Three days later, the decayed areas with the diameter range of 5−15 mm showed at the infected positions. The early decayed areas were still barely visible to the naked eye. A total of 132 sound fruits and 168 infected fruits with spores of P. digitatum fungi were prepared in our study. 66 sound and 84 infected oranges of them were selected randomly as training set for developing the decay detection algorithm, and the remaining samples were used as validation set for verifying the feasibility of algorithm.
2
Postharvest Biology and Technology 161 (2020) 111071
X. Tian, et al.
Fig. 1. Hyperspectral transmittance imaging system.
range of 325–1098 nm. A white board with reflectance of 99 % (Spectraon SRT-99-100, Labsphere Inc., North Sutton, NH, USA) was used to acquire the white image (White) with reflectance mode. For acquiring the white image, the exposure time was set at 3 ms for avoiding signal saturation. The dark image (Dark) was acquired by turning off the lamps and covering the camera lens with a black cap (Li et al., 2014). To reduce the disturbance of dark current from the CCD detector, the raw hyperspectral transmittance images (Raw) were corrected by using the white and dark references according to the following equation:
2.2. Hyperspectral transmittance image acquisition The line-scanning hyperspectral imaging system (Fig. 1) was composed of one 502 × 500 pixels electron magnifying charge-coupled (EMCCD) camera (Andor Luca DL–604 M, Andor Technology plc., Belfast, UK), one imaging spectrograph (ImSpector V10EQE, Spectra Imaging Ltd., Oulu, Finland) connected to a standard 23 mm C-mount zoom lens (OLE23-f/2.4, Spectra Imaging Ltd., Oulu, Finland) covering the spectra range from 325 to 1098 nm at 1.56 nm intervals, one illustration system with one 150 W (W) halogen lamp (JCR, 15 V, 150 W, BAU, Japan) and one convex lens, a motorized linear mobile platform for sample positioning (EZHR17EN, AllMotion, Inc., USA) and a computer. Light source was installed directly beneath the spectrograph for collecting transmittance images. There was a circular hole with diameter of 40 mm on the mobile platform for lighting orange sample from the bottom. The hyperspectral imaging system was operated in a dark chamber for reducing the interference of the external light. In order to acquire high-quality hyperspectral transmittance images of oranges, some crucial parameters were first set before data collection. The distance between mobile platform and the lens was set at 300 mm, the movement speed of sample was set 3.3 mm/s, and the camera was set to run with exposure time of 22 ms and gain of 50. The tested orange was placed on the circular hole of mobile platform and manually oriented the side of decay spot towards the camera but no necessary directly facing the camera, so that the decay spots were randomly distributed on the acquired images for more effectively evaluating the detection performance of proposed algorithms. The imaging spectrograph could scan the tested sample line-by-line as the mobile platform moved the sample through the field of view of the optical system. Each collected spectral image could be called as a three dimensional data cube with two spatial dimensions (x, y) and one spectral dimension (λ). There were 700 × 502 pixels were included in the spatial components (x, y) and 500 bands at 1.56 nm intervals were contained in the spectral
T=
Raw− Dark White− Dark
(1)
The calibrated image T was used for further data processing and analysis. 2.3. Principal component analysis (PCA) Principal component analysis (PCA) is an orthogonal linear transform technology, which could distribute the target components into the new coordinate space according to their information content with the method of linear projection. PCA has been used wildly for defect detection of food and agricultural products based on the hyperspectral image data (Liu et al., 2014; Feng and Sun 2012; ElMasry et al. 2012; Liet al. 2011). PCA can transform the hyperspectral image into sequence of principal component (PC) images through computing a linear projection of the spectral data (Kara and Dirgenali, 2007). The covariance matrix of the images were used to calculate the weighted values, then the linear sum of the original images at individual wavelengths multiplied by the corresponding weighing coefficients (eigenvector) composed the PC image of each band. Additional details about PCA could be found in Howery (1980). In our study, the collected hyperspectral transmittance images were analyzed using PCA method for 3
Postharvest Biology and Technology 161 (2020) 111071
X. Tian, et al.
The Morphological reconstruction could eliminate close textures and noise and restore the sharp contour of the target as well as simplify the gradient image concurrently. Morphological ‘opening and closing’ reconstruction operation is built on the geodesic dilation and erosion (Salembier Clairon and Pardàs Feliu, 1994). The morphological geodesic dilation was defined as follows:
discriminating decayed fruit. 2.4. Pseudo-color image enhancement processing Human physiological vision system is not sensitive to small changes in gray scale, but very sensitive to petty differences in color scale. Pseudo-color image enhancement processing could transform the insensitive gray signal into sensitive color signal for enhancing the target resolution of slight variations in images. This technique is commonly used in the field of image processing due to its advantages of practicality and simplicity. Gray slicing is the most basic and simple method of pseudo-color image processing in recent study. A grayscale image f(x, y) can be regarded as a density function of coordinate (x, y). The grayscale of this image is divided into several levels, which is equivalent to cutting the density function in the intersecting area with some planes that parallel to the x–y plane. Then, the original black-and-white image f(x, y) is transformed into pseudo-color image g(x, y) through gray slicing processing. In our study, the pseudo-color image enhancement processing was employed to transform the 2-D PC gray image obtained from PCA analysis into pseudo-color image for more effectively detecting the decayed fruit.
1 ⎧ Db (∇f (x , y ), r ) = min (∇f (x , y ) ⊕ b, r ) ⎨ Dbi+ 1 (∇f (x , y ), r ) = min (Dbi ⊕ b, r )(i = 1,2, 3, …) ⎩
where ∇[(f(x, y)], r(x, y) and b(x, y) represent gradient image, reference image and structuring element, respectively. The iterative process of morphological geodesic dilation was terminated when the iteration number was equal to preset value or Dbi+ 1 [∇f (x , y ), r ] = Dbi [∇f (x , y ), r ]. According to the above definition, morphological opening reconstruction Ob(rec ) was defined as follows:
Ob(rec ) [∇f (x , y ), r ] = Db(rec ) [(∇f (x , y ) ∘b), r ]
(4)
Similarly, the morphological geodesic erosion was defined as follows: 1 ⎧ Eb (∇f (x , y ), r ) = max ((∇f (x , y ) Θb), r ) ⎨ Ebi+ 1 (∇f (x , y ), r ) = max (Ebi Θb, r )(i = 1,2, 3, …) ⎩
2.5. Improved watershed segmentation method
(5)
Based on erosion operation, morphological closing reconstruction Cb(rec ) was defined as follows:
Image segmentation as a hotspot and difficult question for automatic fruit defect detection is one of the most crucial processing procedures in the field of image analysis and computer vision, (Li et al., 2019). Because watershed transform could obtain continuous and closed target areas by a rapid and precise way, it has been an effective and widely used region segmentation algorithm in the filed of image analysis. The traditional watershed algorithm is difficult to obtain satisfactory results because of over-segmentation caused by irregular gray disturbance and noise in the image. Therefore, an improved watershed segmentation method based on three successive steps including morphological gradient enhancement, morphological reconstruction and marker-controlled watershed segmentation was proposed to segment the decayed region from sound tissues.
Cb(rec ) [∇f (x , y ), r ] = Eb(rec ) [(∇f (x , y ) b), r ]
(6)
where symbols about ‘∘’ and ‘•’ stand for morphological opening and closing operations, respectively. Finally, a morphological ‘opening and closing’ operation OCb(rec ) was defined as follows:
OCb(rec ) [∇f (x , y ), r ] = Ob(rec ) [(∇f (x , y ), r ), r ]
(7)
2.5.3. Marker-controlled watershed segmentation Although morphological reconstruction of gradient image eliminated most of the regional extreme values and noise, some minimum target points that unrelated with the interest object were remained. Therefore, the interest objects were divided into numerous meaningless small areas. The over-segmentation phenomena could be avoided if the interest objects can be obtained before watershed segmentation transformation could suppress the meaningless minimum target points. Here, the target points such as damaged regions were extracted from the morphological reconstruction image by using threshold value method. Specifically, each local minimum region detected from gradient image was judged if it was bigger than the threshold value. Those minimum regions that were larger than the threshold value were marked. Then a binary image was obtained based on the marker result and the gradient image was modified according to the minimum markers. Finally, the watershed transformation was applied to the modified gradient image.
2.5.1. Morphological gradient enhancement Gradient transformation could more effectively reflect the pretty changes among different regions of orange surface compared with original gray images, which can contribute to enhance the image contrast of different tissues. Differential operators of ‘Sobel’, ‘Prewitt’ and ‘Canny’ could be used to obtain the traditional gradient image, while they were sensitive to noise on the orange surface. Therefore, the morphological gradient method was employed to achieve the gradient image in this study. The morphological gradient of image f(x, y) was defined as follows:
∇ [f (x , y )] = (f ⊕ b)(x , y ) − (fΘb)(x , y )
(3)
(2)
where f(x, y) and b(x, y) represent the gray image and disk-shaped structuring element, respectively. The gray gradient in the image increased sharply after morphological gradient processing. The isotropy property of disk-shaped structuring element is helpful for eliminating effectively the morphological gradient relies less on edge direction.
3. Results and discussion 3.1. Overview of spectrum and image Each pixel in hyperspectral image cube has spectral information in the range of 325–1098 nm. In order to obtain the more representative spectra, each spectrum was extracted from a particular region of interest (ROI). In this study, two ROIs (sound_center and sound_edge) from the sound tissues were extracted at the center and edge positions of orange, respectively, for more clearly presenting the spectral features of the whole transmittance images. The average spectra of sound, decayed and stem-end tissues extracted from the same sample in the training set are shown in Fig. 2. As shown in Fig. 2, it could be seen easily that the transmittance spectral intensity and signal-to-noise ratio
2.5.2. Morphological reconstruction Edge information of different tissues on orange surface was enhanced after morphological gradient enhance operation. However, the over-segmentation cannot be avoided if the conventional watershed segmentation is used directly, because some noise and details of the morphological gradient image were also enhanced and sharpened simultaneously. Hence, the gradient image ∇[f (x, y)] was reconstructed based on the morphological ‘opening and closing’ operation for eliminating local extreme values caused by irregular gray disturbance and noise in gradient image and retaining important contour information. 4
Postharvest Biology and Technology 161 (2020) 111071
X. Tian, et al.
Fig. 2. Average spectra of representative ROIs and single-band images at characteristic peaks.
images of different kinds of decayed oranges are shown in Fig. 5. Class Ⅰ represents samples with only one decayed region and this region located on the central position of orange. ClassⅡ also represents samples with only one decayed region, while the decayed region located on the edge position of orange. There are at least two randomly distributed decayed spots on oranges for Class Ⅲ. Class Ⅳ represents those decayed samples with stem-ends. The circles on the RGB images marked the locations of decayed and stem-ends. The first five PC images obtained from the hyperspectral transmittance image in the range of 600−930 nm were shown in Fig. 3. To more clearly show the differences of different PC images, both raw PC images and pseudo-color images are provided. It could be seen that the first four PC images achieved the most valuable information about the original images for decay detection. The PC1 images generally cannot be used for defect detection, because they provide nothing unique features about decayed tissues besides average gray value data of the full wavelengths (Zhang et al., 2014). The more obvious contrast between sound and decayed tissues for all kind s of images can be found in PC2, PC3 and PC4 images, especially the PC3 images.
were very low in the full-wavelength region besides 600−930 nm. This is mainly caused by the strong absorption of fruit tissues, such as water and cellulose. Therefore, only the images including in the spectral range of 600−930 nm were used for further processing. It can be also found from Fig. 2 that different tissues showed similar spectral characteristics and trends, although the spectral intensity values vary greatly. The collapse of epidermic cells and water-soaking of decayed tissues increased the light penetration performance, so intensity of the average spectrum of decayed region was higher than ones of sound and stemend tissues. It also could be found that the difference of transmittance spectra intensity between sound_center and sound_edge was obvious. It primarily attributed to the illumination unit structure of hyperspectral system and geometric construction of citrus. For avoiding the transmittance light leaking from circular hole, the diameter of circular hole used for lighting citrus was only 40 mm that was slightly smaller than equator diameter of orange samples. Thus, the brightness of the edge region of fruit was primarily determined by the scattered light of internal tissue. In addition, because the orange can be considered as elliptic spherical structure in geometric appearance, the non-uniform distribution of light was a common problem affecting images acquisition (Zhang et al., 2018). Two obvious peaks at 710 nm and 820 nm can be observed easily for three different tissues from Fig. 2, and the transmittance spectral intensity of decayed region was higher than other tissues. A single-band image was seemingly promising to be used to classify the decayed oranges. In order to verify this idea, the singleband gray images and pseudo-color images at 710 nm and 820 nm were provided for visualizing the intensity distribution. Although intensity of the decayed region was more brighter than most of sound tissues in the single-band image, some sound regions close to fruit center have the similar intensity values with decayed regions in pseudo-color images at 710 nm and 820 nm, the intertwining of sound and decayed tissues could also be observed from their mean spectra and error bars. Hence, it is difficult to realize the effective detection of rotten region only depending on simple single-wavelength image due to the influence of uneven gray levels on the spherical orange surface.
3.3. Decayed segmentation operation 3.3.1. Pseudo-color image transformation of PC image Pseudo-color image transformation is an effective image processing method to enhance the visual contrast between sound and decayed tissues (Li et al., 2016). For more effectively extracting the target information and improving classification accuracy, PC3 image was transformed into pseudo-color image prior to defection segmentation. The pseudo-color image was transformed into RGB image to choose the most effective component as objective gray image of decay detection in the subsequent steps. By comparing the contrast between sound and decayed tissues of R, G and B components (Fig. 4), it can be found that G component was the most promising to be used for segmentation of the decayed tissues. Comparing with the pseudo-color images of PC3 (Fig. 3), the pseudo-color image of G component presents more obvious contrast between sound and decayed tissues, indicating that pseudocolor image transformation was effective for enhancing the target information (Xia et al., 2019).
3.2. PCA analysis PCA is an effective method for reducing the high spectral dimensionality, enhancing the interest information and removing the noises, therefore, it was employed to detect decayed orange in this study. In order to comprehensively evaluate the detection performance of PCA for early decay detection, all decayed orange samples in training set were divided into four classes according the number of infected regions, the positions of infection and the kinds of tissues. The RGB
3.3.2. IWSA for decay segmentation The proposed IWSA was employed to segment decayed regions on oranges based on the G component image. The critical steps of IWSA including median filter, morphological gradient, gradient image reconstruction and watershed segmentation were shown in Fig. 5, respectively. Comparing with original G component gray image, the 5
Postharvest Biology and Technology 161 (2020) 111071
X. Tian, et al.
Fig. 3. RGB images and the first five PC images (PC1-PC5) obtained from Vis-NIR spectral region (600−930 nm). Images from left to right show eight representative decayed samples of four classes, respectively.
image while retained the main information of the interest target. As a representative example of decay detection for different kinds of samples, PC3 image, G component extracted from pseudo-color image of PC3, pseudo-color image of G component and the resulting images obtained by IWSA are shown in the first to fourth rows of Fig. 6 respectively. It could be seen from inspection results that all decayed regions could be indentified perfectly whether the defects locate on the
morphological gradient operation further enhanced the gray-scale contrast among different tissues and maintained the relatively smooth area meanwhile. Additionally, gradient image reconstruction operation eliminated the local extremum caused by irregular gray scale disturbance and noise in gradient image and remained the most important extremal information about target outline. Operations of both morphological gradient and gradient image reconstruction simplifed the
Fig. 4. Pseudo-color image transformation for PC3 image.
6
Postharvest Biology and Technology 161 (2020) 111071
X. Tian, et al.
Fig. 5. Improved watershed segmentation algorithm.
Fig. 6. Segmentation results of decayed regions on oranges based on improved watershed segmentation algorithm.
stem-end. However, some sound tissues at the edge position are also mistakenly identified as stem-end tissue (the fourth row of Fig. 6), because the two of them have similar intensity feature. Therefore, morphological parameters including area (A) and circularity (C) as new discrimination factors were used for further identification of stem-end based on the segmentation results obtained by IWSA. To be more specific, the segmentation regions obtained by IWSA were filled and labeled firstly, then, A was calculated for each labeled region based on the nonzero pixels (Npixels) and C was calculated according to the following equation:
edge or center positions, showing PCA, pseudo-color image transformation and IWSA were useful for decayed detection based on the hyperspectral transmittance imaging technology. However, unfortunately, the stem-end tissue showed the similar feature with decayed tissues and was misidentified as decayed tissues in Class Ⅳ samples (Fig. 6). Therefore, it is impossible to effectively detect decayed citrus based solely on PC3 image and pseudo-color image. Thus, identification of stem-end tissue was another critical task for improving the detection accuracy of decayed regions. 3.4. Stem-end identification
C=
Checking all PC images again, it can be found that PC2 image could be used for the identification of stem-end tissue since the high contrast between stem-end tissue and decayed tissue. Thus, the pseudo-color image transformation technology was employed to process PC2 image. R, G and B components were extracted from the pseudo-color image of PC2, and R component shows strongest contrast between stem-end and other tissue, as shown in Fig.7. Then, IWSA was used to identify stemend tissue based on R component. The first to fourth rows of Fig. 8 showed the PC2 image, R component extracted from pseudo-color image of PC2, pseudo-color image of R component and the resulting images obtained by IWSA, respectively. It could be found that the stemend tissue could be successfully identified from sound and decay tissues in Class Ⅳ samples, indicating the feasibility of using IWSA to identify
4πA P2
where P is the perimeter of labeled region. The labeled region was identified as stem-end, only if C > 0.65 and 650 > A > 150. Otherwise, the labeled region was considered as sound tissue. The parameter range of A was a statistical result from training samples in this study. However, it could be flexibly set in actual application. Based on the discrimination factors and strategy, the stem-end of class IV can be accurately identified and removed interference from the sound tissue at edge position of image. The results of stem-end identification were shown in the fifth row of Fig. 8
Fig. 7. Pseudo-color image transformation for PC2 image.
7
Postharvest Biology and Technology 161 (2020) 111071
X. Tian, et al.
Fig. 8. Segmentation results of stem-end regions on orange based on improved watershed segmentation algorithm and morphological parameters.
transmittance imaging coupled with PCA and IWSA was evaluated using all samples (84 infected oranges and 66 sound oranges) in the validation set. Six infected samples were not correctly detected and three sound samples were misidentified as infected oranges. The total success rate was 94 % for validation set. More specifically, the success rates for decayed oranges and sound oranges were 93 % and 96 %, respectively. In term of six undetected decayed oranges, the decayed areas were small and coincidentally located on the extreme edge of the images. They were removed in the operation of morphological gradient. In practical application, the detection rate of decayed fruit was crucial for citrus industry, because the fungal infection can spread fast from a small number of decayed fruit to the whole consignment and result in great economic losses. However, this case would not reduce detection efficacy in practical online automatic inspection because multiple images would be acquired for each orange. In the case of the misidentification of sound samples, it mainly attributed to the excessive circularity threshold of irregular stem-end extracted by IWSA. A low rate of false positives could be accepted since there was a greater
3.5. Further decay determination by removing stem-end region It has been mentioned above that the detection accuracy of decayed tissues could be reduced by the stem-end for Class IV samples. By comprehensively analyzing the results of decayed region detection operation and stem-end identification operation, it can be found that the stem-end was segmented well in both operations. Hence, the marked region that located at the same position would be considered as stem-end and removed from the result of decay segmentation operation, and the remained regions of decay segmentation would be decay region. The finally decay determination results after removing the stemend were shown in the last row of Fig. 6. Fig. 9 demonstrates primary procedures for detection of the decayed region on orange with stem-end based on the proposed all methods in this study. 3.6. Verification results The proposed algorithm for decay detection based on hyperspectral
Fig. 9. Flow chart of decay detection on orange based on hyperspectral transmittance imaging coupled with principle component analysis and improved watershed segmentation algorithms. 8
Postharvest Biology and Technology 161 (2020) 111071
X. Tian, et al.
tolerance to reject sound fruit rather than to accept decay fruit for citrus industry (Li et al., 2016).
caused by penicilliumdigitatum in mandarins. J. Food Eng. 89 (1), 80–86. https:// doi.org/10.1016/j.jfoodeng.2008.04.009. Gowen, A.A., Taghizadeh, M., O’Donnell, C.P., 2009. Identification of mushrooms subjected to freeze damage using hyperspectral imaging. J. Food Eng. 93 (1), 7–12. https://doi.org/10.1016/j.jfoodeng.2008.12.021. Jiang, B., He, J., Yang, S., Fu, H., Li, T., Song, H., He, D., 2019. Fusion of machine vision technology and AlexNet-CNNs deep learning network for the detection of postharvest apple pesticide residues. Artif. Intell. Agric. 1, 1–8. https://doi.org/10.1016/j.aiia. 2019.02.001. Kara, S., Dirgenali, F., 2007. A system to diagnose atherosclerosis via wavelet transforms, principal component analysis and artificial neural networks. Expert Syst. Appl. 32 (2), 632–640. https://doi.org/10.1016/j.eswa.2006.01.043. Kurita, M., Kondo, N., Shimizu, H., Ling, P., Falzea, P.D., Shiigi, T., Ninomiya, K., Nishizu, T., Yamamoto, K., 2009. A double image acquisition system with visible and UV LEDs for citrus fruit. J. Robot. Mechatron. 21 (4), 533–540. https://doi.org/10.20965/jrm. 2009.p0533. Li, J., Huang, W., Chen, L., Fan, S., Zhang, B., Guo, Z., Zhao, C., 2014. Variable selection in visible and near-infrared spectral analysis for noninvasive determination of soluble solids content of ‘Ya’ pear. Food Anal. Methods 7 (9), 1891–1902. https://doi.org/10. 1007/s12161-014-9832-8. Li, J., Huang, W., Tian, X., Wang, C., Fan, S., Zhao, C., 2016. Fast detection and visualization of early decay in citrus using Vis-NIR hyperspectral imaging. Comput. Electron. Agric. 127, 582–592. https://doi.org/10.1016/j.compag.2016.07.016. Li, J., Luo, W., Wang, Z., Fan, S., 2019. Early detection of decay on apples using hyperspectral reflectance imaging combining both principal component analysis and improved watershed segmentation method. Postharvest Biol. Technol. 149, 235–246. https://doi.org/10.1016/j.postharvbio.2018.12.007. Li, J., Rao, X., Ying, Y., Ma, B., Guo, J., 2009. Background and external defects segmentation of navel orange based on mask and edge gray value compensation algorithm. Trans. Chin. Soc. Agric. Eng. 25 (12), 133–137. https://doi.org/10.3969/j. issn.1002-6819.2009.12.024. Liu, D., Sun, D.W., Zeng, X.A., 2014. Recent advances in wavelength selection techniques for hyperspectral image processing in the food industry. Food Bioproc. Tech. 7 (2), 307–323. https://doi.org/10.1007/s11947-013-1193-6. Lorente, D., Zude, M., Regen, C., Palou, L., Gómez-Sanchis, J., Blasco, J., 2013. Early decay detection in citrus fruit using laser-light backscattering imaging. Postharvest Biol. Technol. 86, 424–430. https://doi.org/10.1016/j.postharvbio.2013.07.021. Lu, R., Ariana, D.P., 2013. Detection of fruit fly infestation in pickling cucumbers using a hyperspectral reflectance/transmittance imaging system. Postharvest Biol. Technol. 81, 44–50. https://doi.org/10.1117/12.887524. Momin, M.A., Kondo, N., Kuramoto, M., Ogawa, Y., Yamamoto, K., Shiigi, T., 2012. Investigation of excitation wavelength for fluorescence emission of citrus peels based on UV-VIS spectra. Eng. Agric. Environ. Food 5 (4), 126–132. https://doi.org/10. 1016/S1881-8366(12)80008-5. Obenland, D., Margosan, D., Smilanick, J.L., Mackey, B., 2010. Ultraviolet fluorescence to identify navel oranges with poor peel quality and decay. HortTechnology 20 (6), 991–995. https://doi.org/10.1590/S0100-29452010000400045. Pan, L., Sun, Y., Xiao, H., Gu, X., Hu, P., Wei, Y., Tu, K., 2017. Hyperspectral imaging with different illumination patterns for the hollowness classification of white radish. Postharvest Biol. Technol. 126, 40–49. https://doi.org/10.1016/j.postharvbio.2016. 12.006. Rong, D., Rao, X., Ying, Y., 2017. Computer vision detection of surface defect on oranges by means of a sliding comparison window local segmentation algorithm. Comput. Electron. Agric. 137, 59–68. https://doi.org/10.1016/j.compag.2017.02.027. Salembier Clairon, P.J., Pardàs Feliu, M., 1994. Hierarchical morphological segmentation for image sequence coding. Ieee Trans. Image Process. 3 (5), 639–651. https://doi. org/10.1109/83.334980. Swift, L.J., 1967. Thin-layer chromatographic-spectrophotometric analysis for neutral fraction flavones in orange peel juice. J. Agric. Food Chem. 15 (1), 99–101. https:// doi.org/10.1021/jf60149a028. Wu, D., Sun, D.W., 2013. Potential of time series-hyperspectral imaging (TS-HSI) for noninvasive determination of microbial spoilage of salmon flesh. Talanta 111, 39–46. https://doi.org/10.1016/j.talanta.2013.03.041. Xia, Y., Xu, Y., Li, J., Zhang, C., Fan, S., 2019. 2019. Recent advances in emerging techniques for non-destructive detection of seed viability: a review. Atrif Intell. Agric. 1, 35–47. Xing, J., Saeys, W., De Baerdemaeker, J., 2007. Combination of chemometric tools and image processing for bruise detection on apples. Comput. Electron. Agric. 56 (1), 1–13. https://doi.org/10.1016/j.compag.2006.12.002. Zhang, B., Dai, D., Huang, J., Zhou, J., Gui, Q., Dai, F., 2018. Influence of physical and biological variability and solution methods in fruit and vegetable quality nondestructive inspection by using imaging and near-infrared spectroscopy techniques: a review. Crit. Rev. Food Sci. Nutr. 58 (12), 2099–2118. https://doi.org/10.1080/ 10408398.2017.1300789. Zhang, B., Huang, W., Li, J., Zhao, C., Fan, S., Wu, J., Liu, C., 2014. Principles, developments and applications of computer vision for external quality inspection of fruits and vegetables: a review. Food Res. Int. 62, 326–343. https://doi.org/10.1016/j. foodres.2014.03.012.
4. Conclusion Detection of early decayed oranges affected by P. digitatum fungus was observed in this paper for avoiding great economic losses from citrus industry. Hyperspectral transmittance imaging coupled with PCA, pseudo-color image transformation technology and IWSA was employed to analyze the feasibility of decay detection. The results showed that PC3 image was promising for decay segmentation and G component of the pseudo-color image of PC3 could be selected for enhancing image contrast between decayed regions and sound tissues. Further, it was found that all decayed regions could be effectively segmented by IWSA. To eliminate the misclassification of stem-end in the process of decay detection, stem-end identification algorithm was also developed. Study results showed that PC2 image and R component of the pseudo-color image of PC2 was promising for stem-end identification, and stem-end region could be extracted by IWSA and morphological parameters. By performed the combination algorithm of both decay segmentation operation and stem-end identification, the success rates for decayed, sound and total oranges were 93 %, 96 % and 94.00 %, respectively, for 150 samples in the validation set (84 decayed samples and 66 sound samples). Therefore, the hyperspectral transmittance imaging offers a useful reference for automatic detection of early decayed oranges caused by P. digitatum fungus. The other physical defects such as peel scratch and insect bit and their effects on the detection accuracy of early decay will be investigated in the subsequent studies. Declaration of Competing Interest None. Acknowledgements This study was supported by National Natural Science Foundation of China (Grant No. 31772052 and 31972152) and Young Scientist Fund of Beijing Academy of Agriculture and Forestry Sciences (Grant No. QNJJ201818). References Blasco, J., Aleixos, N., Molto, E., 2007. Computer vision detection of peel defects in citrus by means of a region oriented segmentation algorithm. J. Food Eng. 81 (3), 535–543. https://doi.org/10.1016/j.jfoodeng.2006.12.007. Bosabalidis, A., Tsekos, I., 1982. Ultrastructural studies on the secretory cavities of Citrus deliciosa ten. II. Development of the essential oil-accumulating central space of the gland and process of active secretion. Protoplasma 112 (1–2), 63–70. https://doi.org/ 10.1007/BF01280216. Cen, H., Lu, R., Zhu, Q., Mendoza, F., 2016. Nondestructive detection of chilling injury in cucumber fruit using hyperspectral imaging with feature selection and supervised classification. Postharvest Biol. Technol. 111, 352–361. https://doi.org/10.1016/j. postharvbio.2015.09.027. Droby, S., 2005. March. Improving quality and safety of fresh fruits and vegetables after harvest by the use of biocontrol agents and natural materials. International Symposium on Natural Preservatives in Food Systems 709, 45–52. https://doi.org/ 10.17660/ActaHortic.2006.709.5. Eckert, J.W., Eaks, I.L., 1989. Postharvest Disorders and Diseases of Citrus. The Citrus Industry Vol. 5 University California Press, Berkeley, CA, USA. ElMasry, G., Wang, N., Vigneault, C., 2009. Detecting chilling injury in Red Delicious apple using hyperspectral imaging and neural networks. Postharvest Biol. Technol. 52 (1), 1–8. https://doi.org/10.1016/j.postharvbio.2008.11.008. Gómez-Sanchis, J., Gómez-Chova, L., Aleixos, N., Camps-Valls, G., Montesinos-Herrero, C., Moltó, E., Blasco, J., 2008. Hyperspectral system for early detection of rottenness
9