- Email: [email protected]
Detection of Callosobruchus maculatus (F.) infestation in soybean using soft X-ray and NIR hyperspectral imaging techniques
Journal of Stored Products Research 57 (2014) 43e48
Contents lists available at ScienceDirect
Journal of Stored Products Research journal homepage: www.elsevier.com/locate/jspr
Detection of Callosobruchus maculatus (F.) infestation in soybean using soft X-ray and NIR hyperspectral imaging techniques V. Chelladurai a, K. Karuppiah a, D.S. Jayas a, *, P.G. Fields b, N.D.G. White b a b
Department of Biosystems Engineering, University of Manitoba, Winnipeg, MB R3T 5V6, Canada Cereal Research Centre, Agriculture and Agri-Food Canada, Winnipeg, MB R3T 2M9, Canada
a r t i c l e i n f o
a b s t r a c t
Article history: Accepted 11 December 2013
Soybean (Glycine max L.) is a major oilseed crop grown throughout the world and, total post-harvest losses of soybean are approximately 10%, and 3% of produced soybean is lost during storage. Cowpea weevil (Callosobruchus maculatus (F.)) is the major storage pest which causes extensive storage losses of legumes. Detection of early stages of cowpea weevil infestation could assist farmers and storage facility managers in implementing suitable control practices for insect disinfestations. Soft X-ray and near-infrared (NIR) hyperspectral imaging techniques were used to acquire images of soybeans infested by egg, larval, and pupal stages of C. maculatus along with uninfested and completely damaged (hollowed-out after emergence of adults) soybeans. From soft X-ray images, totally, 33 features (12 histogram and 21 textural features) were extracted and from hyperspectral data 48 features were extracted (30 histogram and 18 spectral features) for analysis. Linear and quadratic discriminant analysis (LDA and QDA) models were developed using these extracted features to classify different stages of infestation. The LDA classifier for soft X-ray images correctly identified more than 86% of uninfested soybeans and 83% of soybeans infested with all developmental stages of C. maculatus except the egg stage. Pair-wise LDA classification models developed from NIR hyperspectral data yielded more than 86 and 87% classification accuracy for uninfested and infested seeds, respectively. The QDA pair-wise classifiers positively differentiated more than 79% uninfested seeds from infested seeds. The principal component analysis of NIR hyperspectral data identified the wavelengths of 960 nm, 1030 nm and 1440 nm being responsible for more than 99% of spectral variability. Combining soft X-ray features with hyperspectral features increased the classification accuracies for egg and larvae compared to either imaging system used alone. Crown Copyright Ó 2013 Published by Elsevier Ltd. All rights reserved.
Keywords: Soybean Soft X-ray imaging NIR hyperspectral imaging Callosobruchus maculatus
1. Introduction Soybean (Glycine max L.) is the most widely grown oilseed crop in the world. The approximate world production of soybean was 265 million tonnes in 2010, and the USA ranked number one in soybean production. India is the fifth largest producer of soybean, and it produced around 15 million tonnes in 2010 (FAOSTAT, 2012). Soybean contains nearly 20% oil and 34e40% protein (Ghosh and Jayas, 2010). Soybeans are normally harvested at a moisture content of 15e18% (wet basis) to avoid shattering losses during harvest and then dried to 14% or less for storage. The total post-harvest losses of soybean are around 8e10%, and during storage nearly 2e3% of produced soybean is lost. Approximately 55 species of insects infest the soybean during storage and bruchid species cause extensive storage losses in soybean (Hagstrum and Subramanyam, 2009; Ghosh and
* Corresponding author. 207, Administration Building, University of Manitoba, Winnipeg, MB R3T 2N2, Canada. Tel.: þ1 204 474 9404; fax: þ1 204 474 7568. E-mail addresses: [email protected], [email protected] (D.S. Jayas).
Jayas, 2010). Cowpea weevil (Callosobruchus maculatus (F.), Coleoptera: Bruchidae) can infest soybean in the field and continue to attack soybean in storage. It causes quantitative as well as qualitative losses in several legumes (Rees, 2004; Hagstrum and Subramanyam, 2009). If the C. maculatus infestation is not identified early enough, it will cause total seed damage (Singh and Jackai, 1985). The conventional insect detection techniques like Berlese funnel and insect traps can identify only adult insects or free living larvae and also these techniques are time consuming. There are several advanced techniques for rapid insect detection and hidden infestations inside cereal and pulse seeds (Neethirajan et al., 2007). Machine vision system equipped with a charge coupled device (CCD) camera can detect adult insects moving inside bulk wheat samples. Over 90% of samples with Rhyzopertha dominica (F.) adults were correctly identified by such a system (Zayas and Flinn, 1998). An automated system with acoustic sensors was developed by Shuman (2003) to detect insect infestation in wheat. It detected the adults of Tribolium castaneum (Herbst), R. dominica and Oryzaephilus surinamensis (L.), but could not detect infestation by larvae or pupae. Near-infrared (NIR) spectroscopy (Maghirang et al.,
0022-474X/$ e see front matter Crown Copyright Ó 2013 Published by Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.jspr.2013.12.005
44
V. Chelladurai et al. / Journal of Stored Products Research 57 (2014) 43e48
2003) was used to detect hidden infestations (live and dead insects) in wheat kernels. A thermal imaging system was tested for identification of insect and fungal damages in stored wheat (Chelladurai et al., 2010; Manickavasagan et al., 2008). Thermal imaging system was also tested for C. maculatus detection in mung bean by Chelladurai et al. (2012), and this system correctly identified more than 80% of early stages of infestation. A soft X-ray imaging method is a non-destructive method and it can quickly detect the internal infestations in grain kernels. Karunakaran et al. (2003, 2004a,b) showed that soft X-ray detected several stored-product insects and achieved an identification accuracy of 84%e98%. While detecting the infestation by Sitophilus oryzae (L.) in wheat kernels, Karunakaran et al. (2003) achieved an accuracy of more than 95% for the samples infested by larval stages and correctly identified more than 99% pupae and adult-infested samples. The classification accuracy of a soft X-ray method to detect the internal infestation of R. dominica and T. castaneum larva in wheat were 98% and 86%, respectively (Karunakaran et al., 2004a,b). The models developed from soft X-ray images accurately detected Sitophilus granarius (L.) eggs and other internal stages in wheat kernels from 5 days after oviposition (Fornal et al., 2007). The soft X-ray method also has the ability of detecting fungal infections in stored wheat kernels (Narvankar et al., 2009). NIR hyperspectral imaging is an advanced technique which combines spectral and spatial information of a sample and gives the data as a hypercube. This technique has been studied for identification of chemical composition of cereals and oilseeds (Delwiche, 1998; Miralbés, 2004), detection of insect infestation and fungal infection (Delwiche, 2003; Singh et al., 2009), and detection of defects in fruits and vegetables (Ariana et al., 2006; Dowell, 2000). Zhou et al. (2010) investigated the use of hyperspectral spectrometry to detect C. maculatus pupae in soybeans, and 88% of infested soybeans were correctly identified by the back propagation neural network (BPNN) model. The present study expands the work done by Zhou et al. (2010) by: (1) assessing the feasibility of using soft X-ray and NIR hyperspectral imaging methods to detect C. maculatus of all life stages in soybean; and (2) determining the classification accuracies of statistical classifiers in identifying uninfested and infested soybeans using soft X-ray and hyperspectral data. 2. Materials and methods 2.1. Sample preparation The soybeans were a mix of varieties cultivated in Manitoba and procured from farmers through SaskPulse Traders Inc., St. Joseph, Manitoba. Callosobuchus maculatus were reared in plastic jars at 30 1 C and 70 5% RH. The cleaned soybeans (7 kg) were spread on a plate, placed in a mesh cage and approximately 100 newly emerged C. maculatus adults were introduced into the cage for 24 h. The soybeans with single eggs on the seed coat were selected using a microscope. The soybeans were divided into 1500 g seed/jar, and held at 30 1 C and 70 10% RH (CONVIRON, Controlled Environments Limited, Winnipeg, MB); for 15 d for seed with larvae, 21 d for seed with pupae and 26 d for totally damaged seed (hollowed-out, adults had exited the seed). For uninfested control samples, 200 undamaged soybeans were randomly selected from the initial sample lot. 2.2. Image acquisition and feature extraction 2.2.1. Soft X-ray method The soft X-ray imaging system used in this study was similar to that used by Karunakaran et al. (2004a,b) and Narvankar et al. (2009). Images of uninfested and infested soybeans were
acquired using a soft X-ray imaging system with a Lixi Fluoroscope (Model: LX-85708, Lixi Inc., Downers Grove, IL). The X-rays produced by the fluoroscope penetrate through the samples (single solybean seed) which was placed on the sample platform (made of Saran wrap) between the X-ray tube and detection system, and images were acquired. From the preliminary experiments, 150 mA tube current and 16.1 kV potential was set for imaging. For each insect life stage 200 random soybeans were selected from the infested seeds. The analog images acquired by the fluoroscope (with 62.5 mm screen resolution) were digitized into 8-bit grayscale images by a capture card (TV@Anywhere Plus, MSI: S36-0000311K45, Taiwan) at a resolution of 60 pixels/mm. The gray scale images were stored in tiff file format and a simple thresholding algorithm developed using MATLAB (Version: 7.13, The Mathworks Inc., Natick, MA) software was used to segment the seeds from the background. The histograms of soybean samples were normalized and grouped into 12 groups for analysis. The gray level co-occurrence matrix method (GLCM) was used to determine the textural properties of the soybeans (Majumdar and Jayas, 2000). A total of 21 textural features extracted using a MATLAB program were: maximum, minimum, mean, median, standard deviation of grey levels and four GLCM features (energy, homogeneity, contrast, correlation) at 0 , 45 , 90 , and 135 orientations and used for further analysis. 2.2.2. NIR hyperspectral imaging system The imaging system used in this study is the same as reported in earlier studies (Mahesh et al., 2008; Singh et al., 2009; Kaliramesh et al., 2013). The hyperspectral imaging system consists of a thermoelectrically cooled Indium Gallium Arsenide (InGaAs) camera (Model No. SU640-1.7RT-D, Sensors Unlimited Inc., Princeton, NJ), for acquiring images and two VariSpec liquid crystal tunable filters (LCTFs) (Model No. MIR06, Cambridge Research and Instrumentation Inc., Woburn, MA) for selecting the wavelengths. The whole imaging unit is fixed on a sample stage, and a light source (two halogentungsten bulbs) was used to illuminate the samples at long wave NIR range (900e1700 nm). An algorithm developed in LabVIEW environment (Version 1, National Instruments, Austin, TX, USA) was used for image acquisition and storage of hyperspectral data. The images of uninfested soybeans and soybeans with various stages of C. maculatus infestation (200 kernels each) were acquired between 960 and 1700 nm with 10 nm intervals. The light source and filters were turned on 30 min prior to imaging to eliminate the effect of thermal drift and the system was aligned at 1300 nm. Reference data were collected at each imaging session using a 99% white reflectance and data of soybeans samples were normalized using this white image data. Five non-touching soybeans seeds were placed on the imaging table for every image and the collected data were analysed using a MATLAB program. Images of soybeans seeds were segmented from background and each seed was labeled using strel (creating morphological structure element) and bwlabel functions, respectively, for data extraction. Principal component analysis was performed for this extracted data to identify the top three wavelength bands for differentiation of uninfested and infested soybeans using MATLAB. Then intensity values of each image at each of these three wavelengths were grouped into 10 histogram groups (totally 30 histogram groups). From the spectral data, six features were extracted (mean, median, maximum, minimum, standard deviation and variance) at each of these three wavelengths (totally 18 spectral features). 2.3. Classification Two types of discriminant analysis models, linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA), using
V. Chelladurai et al. / Journal of Stored Products Research 57 (2014) 43e48
3. Results 3.1. Soft X-ray imaging Soft X-ray images of uninfested soybeans and soybeans (Fig. 1) infested by different life stages of C. maculatus were analysed by MATLAB software. To avoid effects of background and outliers, pixels with gray level value less than 32 (dark) and higher than 225 (bright) were removed and normalized histograms were developed. Uninfested soybeans and soybeans with eggs were darker (more pixels at gray level values 33e48 and 49e64, Fig. 2) than the samples with larvae, pupae or hollowed-out. Seeds with pupae and hollowed-out were brighter (more pixels at gray level values 209e 224) than other seeds. Classification accuracy rose with increases in age and size of the insect using histogram or textural features as well as with both LDA
35000 Uninfested
30000
Egg Larva
25000
Number of pixels
leave-one-out technique were used to classify the infested and uninfested soybeans. Both LDA and QDA models were developed using: (1) histogram features only, (2) textural features only, and (3) combined features (all 33 extracted features: 12 histogram and 21 textural features) for soft X-ray data, and (1) histogram features only, (2) spectral features only, and (3) combined features (all 48 extracted features: 30 histogram and 18 spectral features). Three types of classification models were developed from these combined features: (1) 5-way model (uninfested, egg, larva, pupa and hollowed-out seeds); (2) 3-way (uninfested, larva and pupa in one group, hollowed-out seeds); (3) Pair-wise (uninfested vs. each stage of infestation). All the features (totally 81 features) from soft X-ray images and hyperspectral data were combined and 5-way, 3-way and pair-wise discriminant models were developed. These models were trained by DISCRIM procedure using SAS 9.3 (Statistical Analysis Systems Institute, Inc., Cary, NC, USA).
45
Pupa Hollowed-out
20000
15000
10000
5000
0 32
48
64
80
96 112 128 144 160 176 192 208 224 240
Gray level value Fig. 2. Normalized histograms of soft X-ray images of soybeans uninfested and infested by different life stages of Callosobruchus maculatus.
and QDA classifiers (Table 1). Seeds that were hollowed-out after adults had emerged from the seeds always had the greatest classification accuracy, with accuracies at over 90%. In contrast eggs and uninfested seeds were often misclassified. Histogram features were combined with textural features, and LDA and QDA models developed from these combined features. The QDA classifier yielded 89% and 96% classification accuracies for pupae and hollowed-out samples, respectively in the 5-way
Fig. 1. Soft X-ray images of soybeans uninfested or infested with single Callosobruchus maculatus at different life stages: (a) uninfested, (b) egg, (c) larva, (d) pupa, (e) hollowed-out after adult emergence.
46
V. Chelladurai et al. / Journal of Stored Products Research 57 (2014) 43e48
Table 1 Classification accuracy of uninfested and infested by different developmental stages of Callosobruchus maculatus using histogram and textural features of soft X-ray and histogram and spectral features of NIR hyperspectral imaging data with linear discriminant analysis (LDA) or quadratic discriminant analysis (QDA) classification models. Stage of infestation
Classification accuracy (%) Soft X-rays
Uninfested Egg Larval Pupal Hollowed-out
Table 3 Pair-wise classification of uninfested and infested by different developmental stages of Callosobruchus maculatus using statistical classifiers from soft X-ray, NIR hyperspectral and combined soft X-ray and NIR hyperspectral imaging data with linear discriminant analysis (LDA) or quadratic discriminant analysis (QDA) classification models. Uninfested vs different infestation stages
NIR hyperspectral
Histogram
Textural
Histogram
Spectral
LDA
QDA
LDA
QDA
LDA
QDA
LDA
QDA
65 61 73 86 91
73 59 72 80 89
54 56 83 90 96
52 64 84 92 98
72 52 52 53 73
75 62 41 42 80
72 65 61 51 76
78 63 60 42 79
classification (Table 2). In 3-way classification, both LDA and QDA classifiers correctly classified more than 86% of all samples. Hollowed-out samples had the highest classification accuracy (98% and 99% for LDA and QDA, respectively) in 3-way classification. More than 87% of uninfested soybeans were correctly distinguished from larva and pupa-infested as well as hollowed-out samples by pair-wise classification models (Table 3). The pairwise classification also indicated that there were many misclassifications between uninfested soybeans and soybeans with eggs.
Infestation type
Classification accuracy (%) Soft X-ray NIR hyperspectral Soft X-ray þ NIR hyperspectral LDA QDA LDA
QDA
LDA
QDA
Egg
Uninfested Egg
69 57
64 60
87 91
79 86
83 87
73 82
Larval
Uninfested Larvae
87 83
98 90
92 93
85 93
97 97
86 92
Pupal
Uninfested Pupae
88 93
82 91
86 87
88 87
95 93
93 89
Hollowed-out Uninfested 96 Hollowed-out 95
94 92
98 95
89 96
100 98
96 97
The 5-way LDA and QDA models developed from all the features (30 histogram and 18 spectral) also had more than 60% classification accuracy except for pupal infestation. The QDA classifier had lowest classification accuracy for pupal infestation (40%), and similar to the QDA models from spectral features, 43% of pupainfested seeds were misclassified with larval and egg stage infestations (Table 2). The 3-way LDA and QDA classifiers positively differentiated 90% and 94% uninfested soybeans from infested seeds. This 3-way classification had more than 87% classification accuracy for all the developmental stages of C. maculatus infestation in soybeans. The pair-wise classifiers also correctly classified more than 85% of soybeans infested with different developmental stages of C. maculatus from uninfested seeds (Table 3).
3.2. NIR hyperspectral imaging The average normalized spectral reflectance of uninfested soybeans was lower than that of infested seeds at 960e1350 nm waveband range (Fig. 3). The principal component analysis of hyperspectral data showed that, first and second PC factor loadings explained more than 99% of variability. Three wavelengths (960,1030 and 1440 nm) were identified as critical wavebands for detection of C. maculatus infestation in soybeans using PC factor loading scores. The discriminant analysis models developed from histogram features of hyperspectral images correctly identified 72%e80% of uninfested and completely hollowed-out soybeans (Table 1). As was the case for the soft X-ray data, the QDA classifier generally had higher classification accuracies than LDA classifiers. The LDA and QDA classifiers developed from textural features also positively classified more than 70% of uninfested and hollowed-out soybeans. Classification accuracies of QDA classifiers for initial developmental stages (egg, larva and pupa) were lower than LDA classifiers with spectral features. Pupal infestation had lower classification accuracy than other stages, because 41% pupa-infested samples were misclassified as either larval or egg stage infestations.
3.3. Combining soft X-rays and NIR hyperspectral imaging The 5-way LDA model developed from combined features (33 soft X-ray features and 48 hyperspectral features) correctly classified more than 79% of uninfested and infested soybeans. The QDA classifier had lowest classification accuracy for soybeans infested with C. maculatus eggs (Table 2). The 3-way models from these combined features positively identified more than 83% of infested soybeans from uninfested soybeans. The pair-wise LDA classifiers identified more than 86% of infested soybeans and pair-wise QDA classifiers yielded lower classification accuracies than LDA classifiers for all developmental stages (Table 3). Combining soft X-ray features with hyperspectral features increased the classification accuracies for early developmental stages (eggs and larvae).
Table 2 Classification accuracy of uninfested and infested by different developmental stages of Callosobruchus maculatus using 5- or 3-way models from soft X-ray, NIR hyperspectral and combined soft X-ray and NIR hyperspectral imaging data with linear discriminant analysis (LDA) or quadratic discriminant analysis (QDA). Stage of infestation
Classification accuracy (%) Soft X-rays 5 -way
Uninfested Egg Larval Pupal Hollowed-out a
Soft X-rays þ NIR hyperspectral
NIR hyperspectral 3-way
5-way
3-way
5-way
3-way
LDA
QDA
LDA
QDA
LDA
QDA
LDA
QDA
LDA
QDA
LDA
QDA
49 48 66 67 92
53 55 59 89 96
94 e e 86a 98
93 e e 90a 99
71 68 62 66 77
76 68 61 40 80
90 e e 89a 89
94 e e 87a 90
79 80 83 81 79
71 67 87 70 85
92 e e 92a 94
83 e e 86a 92
Larval and pupal stages combined.
V. Chelladurai et al. / Journal of Stored Products Research 57 (2014) 43e48
47
0.6 Uninfested 0.5
Egg Larva
0.4
Pupa Hollowed-out
0.3
Reflectance
0.2 0.1 0.0 960 -0.1
1060
1160
1260
1360
1460
1560
1660
Wavelength (nm)
-0.2 -0.3 -0.4 -0.5 -0.6
Fig. 3. Reflectance spectra of soybeans uninfested and infested by different life stages of Callosobruchus maculatus.
4. Discussion In all three LDA and QDA classifiers (developed from combined features, histogram features and textural features) from soft X-ray data, misclassification occurred between uninfested soybeans and soybeans with eggs. The classification accuracy of discriminant analysis classifiers were higher for pupal and hollowed-out seeds. The trend of higher histogram values at the brighter region in the infested seeds than that of uninfested seeds was also noticed in R. dominica infestation in wheat kernels (Karunakaran et al., 2004a). When insects develop inside the seeds, they consume germ and endosperm of the seeds, and this amount consumed increases with each larval stage. The empty space caused by feeding allowed for better penetration of soft X-rays through the seed and resulted in brighter regions in the images. The classification accuracies of discriminant models developed from histogram or spectral features from NIR hyperspectral images were low, and QDA classifier yielded lowest classification accuracies for larval and pupal stages of infestation. The 3-way LDA and QDA classification models of combined features of NIR hyperspectral data, in which the egg stage was omitted for analysis yielded higher classification efficiency than other classification models. In most of the classification models from hyperspectral data, pupal infested soybeans had lower classification accuracy than other stages of infestation. Most of the misclassification occurred between pupal and egg stages. One of the top three wavelengths, 1440 nm was responsible for the insect-related moisture content (Maghirang et al., 2003) and may have caused this misclassification between egg and pupal stage infestations. Pair-wise classification models correctly classified more than 80% of infested seeds from uninfested seeds except with the egg stage. Pair-wise classification models yielded higher classification efficiencies, because misclassification between different stages of infestation was eliminated in this analysis. Zhou et al. (2010) tested the NIR hyperspectral imaging to detect the C. maculatus infestation in soybeans, and they classified soybeans into two classes for analysis: healthy (sound and uninfested soybeans) and infected (soybeans with larvae or pupae of C. maculatus). The back propagation neural network (BPNN) model developed from hyperspectral data correctly identified 88% of infested soybeans from uninfested soybeans. But this BPNN model
only classified soybeans into either infested or uninfested. Detection of infestation stage was not achieved by this model and other developmental stages were not studied. They identified wavebands of 780e900, 920e1000, and 1205e1560 nm could be used for detection of C. maculatus pupal infestation in soybeans. Two of these bands contain the critical wavelengths we determined: 1030 and 1440 nm and there low wavelength band is close to our lowest wavelength at 960 nm. Singh et al. (2009) found that the wavelengths 1102 nm and 1305 nm were useful for detection of S. oryzae, R. dominica and Cryptolestes ferrugineus (Stephens) infestation in wheat using NIR hyperspectral imaging system. They also stated that the models developed from combined features (spectral and histogram features) yielded higher classification accuracies. Results from our study also showed that, combining histogram and spectral features from hyperspectral data improved the classification accuracy. The pair-wise discriminant models gave higher classification accuracies than 5-way or 3-way discriminant models. The same trend was noticed by Manickavasagan et al. (2008) for detection of C. ferrugineus in wheat using a thermal imaging system. Combining soft X-ray features and NIR hyperspectral features significantly increased the classification accuracies of LDA models for early developmental stages (eggs and larvae). In this combined model, most of the misclassification happened between uninfested and soybeans with C. maculatus eggs. Combined application of both soft X-ray and NIR hyperspectral imaging systems would increase the cost and time required for detection, so may not be commercially viable. The classification accuracy of LDA and QDA classifiers developed from textural features of soft X-ray images had higher classification accuracy than the LDA and QDA classifiers developed from spectral features of hyperspectral data for larval and pupal infestations in soybeans. The higher penetrating ability of X-rays compared to hyperspectral imaging may be the reason for better distinguishability between the seeds having living insects inside and uninfested seeds. Soft X-ray method is a well established in-line inspection method and has the ability to detect early stages of insect infestation (Schatzki and Fine, 1988; Rex and Mazza, 1989). The noise in the digital images may also cause classification errors and these noises may be eliminated by wavelet transformation. But NIR hyperspectral imaging systems for commercial in-line monitoring
48
V. Chelladurai et al. / Journal of Stored Products Research 57 (2014) 43e48
have been difficult due to the larger size of hyperspectral data produced and time needed for analyzing these data. Especially tunable filter NIR hyperspectral systems, like the one used in this study, takes a significant amount of time for scanning the object in whole wavelength spectrum. Pushbroom-type line scan NIR hyperspectral imaging systems are faster and better adapted to commercial operations (Mehl et al., 2004). Three key wavelengths identified from this study will help to develop multispectral imaging system to detect insect infestations in soybean and will eliminate the drawbacks of longer scanning and data analyzing periods. These multispectral imaging systems can be used as an inline inspection tool for detecting C. maculatus infestations in soybeans in grain handling facilities. Acknowledgments We thank Chandra Bhan Singh, and Mahesh Sivakumar for their help in data analysis, the Natural Sciences and Engineering Research Council of Canada, Ministry of Innovation, Energy and Mines, Province of Manitoba for their financial support for this study. We also thank Canada Foundation for Innovation, Manitoba Research Innovation Fund, and several other partners for creating research infrastructure. References Ariana, D.P., Lu, R., Guyer, D.E., 2006. Near-infrared hyperspectral reflectance imaging for detection of bruises on pickling cucumbers. Comput. Electron. Agric. 53, 60e70. Chelladurai, V., Kaliramesh, S., Jayas, D.S., 2012. Detection of Callosobruchus maculatus (F.) infestation in Mung bean (Vigna Radiata) using thermal imaging technique. Paper No. NABEC/CSBE 12-121. In: NABEC-CSBE/SCGAB 2012 Joint Meeting and Technical Conference, Canada: July 15e18, Orillia, ON, Canada. Chelladurai, V., Jayas, D.S., White, N.D.G., 2010. Thermal imaging for detecting fungal infection in stored wheat. J. Stored Prod. Res. 46, 174e179. Delwiche, S.R., 1998. Protein content of single kernels of wheat by near-infrared reflectance spectroscopy. J. Cereal Sci. 27, 241e254. Delwiche, S.R., 2003. Classification of scab- and other mold-damaged wheat kernels by near-infrared reflectance spectroscopy. Trans. ASAE 46, 731e738. Dowell, F.E., 2000. Differentiating vitreous and nonvitreous durum wheat kernels by using near-infrared spectroscopy. Cereal Chem. 77, 155e158. FAOSTAT, 2012. Crop Production Data. Food and Agricultural Organisation of United Nations, Rome, Italy. http://faostat.fao.org/site/339/default.aspx (accessed 18.11.12). ski, T., Sadowska, J., Grundas, S., Nawrot, J., Niewiada, A., Fornal, J., Jelin Warchalewski, J.R., B1aszczak, W., 2007. Detection of granary weevil Sitophilus granarius (L.) eggs and internal stages in wheat grain using soft X-ray and image analysis. J. Stored Prod. Res. 43, 142e148. Ghosh, P.K., Jayas, D.S., 2010. Storage of soybean. In: Singh, G. (Ed.), The Soybean Botany, Production and Uses. CABI, Wallingford, UK, pp. 247e275.
Hagstrum, D.W., Subramanyam, B., 2009. Stored-product Insect Resource. AACC International Inc., St. Paul. Kaliramesh, S., Chelladurai, V., Jayas, D.S., Alagusundaram, K., White, N.D.G., Fields, P.G., 2013. Detection of infestation by Callosobruchus maculatus in mung bean using near-infrared hyperspectral imaging. J. Stored Prod. Res. 52, 107e 111. Karunakaran, C., Jayas, D.S., White, N.D.G., 2003. Soft X-ray inspection of wheat kernels infested by Sitophilus oryzae. Trans. ASAE 46, 739e745. Karunakaran, C., Jayas, D.S., White, N.D.G., 2004a. Detection of internal wheat seed infestation by Rhyzopertha Dominica using X-ray imaging. J. Stored Prod. Res. 40, 507e516. Karunakaran, C., Jayas, D.S., White, N.D.G., 2004b. Identification of wheat kernels damaged by the red flour beetle using X-ray images. Biosyst. Eng. 87, 267e 274. Maghirang, E.B., Dowell, F.E., Baker, J.E., Throne, J.E., 2003. Automated detection of single wheat kernels containing live or dead insects using near-infrared reflectance spectroscopy. Trans. ASAE 46, 1277e1282. Mahesh, S., Jayas, D.S., Paliwal, J., White, N.D.G., 2008. Protein and Oil Constituents Determination in Wheat Using Near-infrared (NIR) Hyperspectral Imaging. ASABE Paper No. 084895. ASABE, St. Joseph, MI. Majumdar, S., Jayas, D.S., 2000. Classification of cereal grains using machine vision. III. Texture models. Trans. ASAE 4, 1681e1687. Manickavasagan, A., Jayas, D.S., White, N.D.G., 2008. Thermal imaging to detect infestation by Cryptolestes ferrugineus inside wheat kernels. J. Stored Prod. Res. 44, 186e192. Mehl, P.M., Chen, Y.-R., Kim, M.S., Chan, D.E., 2004. Development of hyperspectral imaging technique for the detection of apple surface defects and contaminations. J. Food Eng. 61, 67e81. Miralbés, C., 2004. Quality control in the milling industry using near infrared transmittance spectroscopy. Food Chem. 88, 621e628. Narvankar, D.S., Singh, C.B., Jayas, D.S., White, N.D.G., 2009. Assessment of soft Xray imaging for detection of fungal infection in wheat. Biosyst. Eng. 103, 49e 56. Neethirajan, S., Karunakaran, C., Jayas, D.S., White, N.D.G., 2007. Detection techniques for stored product pests in grain. Food Control 18, 157e162. Rees, D.P., 2004. Insects of Stored Products. CSIRO Publishing, Collingwood. Rex, B.L., Mazza, G., 1989. Cause, control, and detection of hollow heart in potatoes. Am. Potato J. 66, 165e183. Schatzki, T.F., Fine, T.A., 1988. Analysis of radiogram of wheat kernels for quality control. Cereal Chem. 65, 233e239. Shuman, D., 2003. Monitoring system counts insects identifies species. Agric. Res. Magazine 51, 12e13. Singh, S.R., Jackai, L.E.N., 1985. Insect pests of cowpea in Africa: their life cycle, economic importance and potential for control. In: Singh, S.R., Rachiek, O. (Eds.), Cowpea Research, Production and Utilisation. Wiley Interscience Publications, Chichester, pp. 217e231. Singh, C., Jayas, D.S., Paliwal, J., White, N.D.G., 2009. Detection of insect-damaged wheat kernels using near-infrared hyperspectral imaging. J. Stored Prod. Res. 45, 151e158. Zayas, I.Y., Flinn, P.W., 1998. Detection of insects in bulk wheat samples with machine vision. Trans. ASAE 41, 883. Zhou, Z., Zang, Y., Shen, B., Zhou, X., Luo, X., 2010. Detection of cowpea weevil (Callosobruchus maculatus (F.)) in soybean with hyperspectral spectrometry and a backpropagation neural network. In: Yue, S., Wei, H., Wang, L., Song, Y. (Eds.), Proceedings of the 2010 Sixth International Conference on Natural Computation: 10e12 August, Yantai, China. Institute of Electrical and Electronics Engineers Inc., New York, pp. 1223e1227.
Contents lists available at ScienceDirect
Journal of Stored Products Research journal homepage: www.elsevier.com/locate/jspr
Detection of Callosobruchus maculatus (F.) infestation in soybean using soft X-ray and NIR hyperspectral imaging techniques V. Chelladurai a, K. Karuppiah a, D.S. Jayas a, *, P.G. Fields b, N.D.G. White b a b
Department of Biosystems Engineering, University of Manitoba, Winnipeg, MB R3T 5V6, Canada Cereal Research Centre, Agriculture and Agri-Food Canada, Winnipeg, MB R3T 2M9, Canada
a r t i c l e i n f o
a b s t r a c t
Article history: Accepted 11 December 2013
Soybean (Glycine max L.) is a major oilseed crop grown throughout the world and, total post-harvest losses of soybean are approximately 10%, and 3% of produced soybean is lost during storage. Cowpea weevil (Callosobruchus maculatus (F.)) is the major storage pest which causes extensive storage losses of legumes. Detection of early stages of cowpea weevil infestation could assist farmers and storage facility managers in implementing suitable control practices for insect disinfestations. Soft X-ray and near-infrared (NIR) hyperspectral imaging techniques were used to acquire images of soybeans infested by egg, larval, and pupal stages of C. maculatus along with uninfested and completely damaged (hollowed-out after emergence of adults) soybeans. From soft X-ray images, totally, 33 features (12 histogram and 21 textural features) were extracted and from hyperspectral data 48 features were extracted (30 histogram and 18 spectral features) for analysis. Linear and quadratic discriminant analysis (LDA and QDA) models were developed using these extracted features to classify different stages of infestation. The LDA classifier for soft X-ray images correctly identified more than 86% of uninfested soybeans and 83% of soybeans infested with all developmental stages of C. maculatus except the egg stage. Pair-wise LDA classification models developed from NIR hyperspectral data yielded more than 86 and 87% classification accuracy for uninfested and infested seeds, respectively. The QDA pair-wise classifiers positively differentiated more than 79% uninfested seeds from infested seeds. The principal component analysis of NIR hyperspectral data identified the wavelengths of 960 nm, 1030 nm and 1440 nm being responsible for more than 99% of spectral variability. Combining soft X-ray features with hyperspectral features increased the classification accuracies for egg and larvae compared to either imaging system used alone. Crown Copyright Ó 2013 Published by Elsevier Ltd. All rights reserved.
Keywords: Soybean Soft X-ray imaging NIR hyperspectral imaging Callosobruchus maculatus
1. Introduction Soybean (Glycine max L.) is the most widely grown oilseed crop in the world. The approximate world production of soybean was 265 million tonnes in 2010, and the USA ranked number one in soybean production. India is the fifth largest producer of soybean, and it produced around 15 million tonnes in 2010 (FAOSTAT, 2012). Soybean contains nearly 20% oil and 34e40% protein (Ghosh and Jayas, 2010). Soybeans are normally harvested at a moisture content of 15e18% (wet basis) to avoid shattering losses during harvest and then dried to 14% or less for storage. The total post-harvest losses of soybean are around 8e10%, and during storage nearly 2e3% of produced soybean is lost. Approximately 55 species of insects infest the soybean during storage and bruchid species cause extensive storage losses in soybean (Hagstrum and Subramanyam, 2009; Ghosh and
* Corresponding author. 207, Administration Building, University of Manitoba, Winnipeg, MB R3T 2N2, Canada. Tel.: þ1 204 474 9404; fax: þ1 204 474 7568. E-mail addresses: [email protected], [email protected] (D.S. Jayas).
Jayas, 2010). Cowpea weevil (Callosobruchus maculatus (F.), Coleoptera: Bruchidae) can infest soybean in the field and continue to attack soybean in storage. It causes quantitative as well as qualitative losses in several legumes (Rees, 2004; Hagstrum and Subramanyam, 2009). If the C. maculatus infestation is not identified early enough, it will cause total seed damage (Singh and Jackai, 1985). The conventional insect detection techniques like Berlese funnel and insect traps can identify only adult insects or free living larvae and also these techniques are time consuming. There are several advanced techniques for rapid insect detection and hidden infestations inside cereal and pulse seeds (Neethirajan et al., 2007). Machine vision system equipped with a charge coupled device (CCD) camera can detect adult insects moving inside bulk wheat samples. Over 90% of samples with Rhyzopertha dominica (F.) adults were correctly identified by such a system (Zayas and Flinn, 1998). An automated system with acoustic sensors was developed by Shuman (2003) to detect insect infestation in wheat. It detected the adults of Tribolium castaneum (Herbst), R. dominica and Oryzaephilus surinamensis (L.), but could not detect infestation by larvae or pupae. Near-infrared (NIR) spectroscopy (Maghirang et al.,
0022-474X/$ e see front matter Crown Copyright Ó 2013 Published by Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.jspr.2013.12.005
44
V. Chelladurai et al. / Journal of Stored Products Research 57 (2014) 43e48
2003) was used to detect hidden infestations (live and dead insects) in wheat kernels. A thermal imaging system was tested for identification of insect and fungal damages in stored wheat (Chelladurai et al., 2010; Manickavasagan et al., 2008). Thermal imaging system was also tested for C. maculatus detection in mung bean by Chelladurai et al. (2012), and this system correctly identified more than 80% of early stages of infestation. A soft X-ray imaging method is a non-destructive method and it can quickly detect the internal infestations in grain kernels. Karunakaran et al. (2003, 2004a,b) showed that soft X-ray detected several stored-product insects and achieved an identification accuracy of 84%e98%. While detecting the infestation by Sitophilus oryzae (L.) in wheat kernels, Karunakaran et al. (2003) achieved an accuracy of more than 95% for the samples infested by larval stages and correctly identified more than 99% pupae and adult-infested samples. The classification accuracy of a soft X-ray method to detect the internal infestation of R. dominica and T. castaneum larva in wheat were 98% and 86%, respectively (Karunakaran et al., 2004a,b). The models developed from soft X-ray images accurately detected Sitophilus granarius (L.) eggs and other internal stages in wheat kernels from 5 days after oviposition (Fornal et al., 2007). The soft X-ray method also has the ability of detecting fungal infections in stored wheat kernels (Narvankar et al., 2009). NIR hyperspectral imaging is an advanced technique which combines spectral and spatial information of a sample and gives the data as a hypercube. This technique has been studied for identification of chemical composition of cereals and oilseeds (Delwiche, 1998; Miralbés, 2004), detection of insect infestation and fungal infection (Delwiche, 2003; Singh et al., 2009), and detection of defects in fruits and vegetables (Ariana et al., 2006; Dowell, 2000). Zhou et al. (2010) investigated the use of hyperspectral spectrometry to detect C. maculatus pupae in soybeans, and 88% of infested soybeans were correctly identified by the back propagation neural network (BPNN) model. The present study expands the work done by Zhou et al. (2010) by: (1) assessing the feasibility of using soft X-ray and NIR hyperspectral imaging methods to detect C. maculatus of all life stages in soybean; and (2) determining the classification accuracies of statistical classifiers in identifying uninfested and infested soybeans using soft X-ray and hyperspectral data. 2. Materials and methods 2.1. Sample preparation The soybeans were a mix of varieties cultivated in Manitoba and procured from farmers through SaskPulse Traders Inc., St. Joseph, Manitoba. Callosobuchus maculatus were reared in plastic jars at 30 1 C and 70 5% RH. The cleaned soybeans (7 kg) were spread on a plate, placed in a mesh cage and approximately 100 newly emerged C. maculatus adults were introduced into the cage for 24 h. The soybeans with single eggs on the seed coat were selected using a microscope. The soybeans were divided into 1500 g seed/jar, and held at 30 1 C and 70 10% RH (CONVIRON, Controlled Environments Limited, Winnipeg, MB); for 15 d for seed with larvae, 21 d for seed with pupae and 26 d for totally damaged seed (hollowed-out, adults had exited the seed). For uninfested control samples, 200 undamaged soybeans were randomly selected from the initial sample lot. 2.2. Image acquisition and feature extraction 2.2.1. Soft X-ray method The soft X-ray imaging system used in this study was similar to that used by Karunakaran et al. (2004a,b) and Narvankar et al. (2009). Images of uninfested and infested soybeans were
acquired using a soft X-ray imaging system with a Lixi Fluoroscope (Model: LX-85708, Lixi Inc., Downers Grove, IL). The X-rays produced by the fluoroscope penetrate through the samples (single solybean seed) which was placed on the sample platform (made of Saran wrap) between the X-ray tube and detection system, and images were acquired. From the preliminary experiments, 150 mA tube current and 16.1 kV potential was set for imaging. For each insect life stage 200 random soybeans were selected from the infested seeds. The analog images acquired by the fluoroscope (with 62.5 mm screen resolution) were digitized into 8-bit grayscale images by a capture card (TV@Anywhere Plus, MSI: S36-0000311K45, Taiwan) at a resolution of 60 pixels/mm. The gray scale images were stored in tiff file format and a simple thresholding algorithm developed using MATLAB (Version: 7.13, The Mathworks Inc., Natick, MA) software was used to segment the seeds from the background. The histograms of soybean samples were normalized and grouped into 12 groups for analysis. The gray level co-occurrence matrix method (GLCM) was used to determine the textural properties of the soybeans (Majumdar and Jayas, 2000). A total of 21 textural features extracted using a MATLAB program were: maximum, minimum, mean, median, standard deviation of grey levels and four GLCM features (energy, homogeneity, contrast, correlation) at 0 , 45 , 90 , and 135 orientations and used for further analysis. 2.2.2. NIR hyperspectral imaging system The imaging system used in this study is the same as reported in earlier studies (Mahesh et al., 2008; Singh et al., 2009; Kaliramesh et al., 2013). The hyperspectral imaging system consists of a thermoelectrically cooled Indium Gallium Arsenide (InGaAs) camera (Model No. SU640-1.7RT-D, Sensors Unlimited Inc., Princeton, NJ), for acquiring images and two VariSpec liquid crystal tunable filters (LCTFs) (Model No. MIR06, Cambridge Research and Instrumentation Inc., Woburn, MA) for selecting the wavelengths. The whole imaging unit is fixed on a sample stage, and a light source (two halogentungsten bulbs) was used to illuminate the samples at long wave NIR range (900e1700 nm). An algorithm developed in LabVIEW environment (Version 1, National Instruments, Austin, TX, USA) was used for image acquisition and storage of hyperspectral data. The images of uninfested soybeans and soybeans with various stages of C. maculatus infestation (200 kernels each) were acquired between 960 and 1700 nm with 10 nm intervals. The light source and filters were turned on 30 min prior to imaging to eliminate the effect of thermal drift and the system was aligned at 1300 nm. Reference data were collected at each imaging session using a 99% white reflectance and data of soybeans samples were normalized using this white image data. Five non-touching soybeans seeds were placed on the imaging table for every image and the collected data were analysed using a MATLAB program. Images of soybeans seeds were segmented from background and each seed was labeled using strel (creating morphological structure element) and bwlabel functions, respectively, for data extraction. Principal component analysis was performed for this extracted data to identify the top three wavelength bands for differentiation of uninfested and infested soybeans using MATLAB. Then intensity values of each image at each of these three wavelengths were grouped into 10 histogram groups (totally 30 histogram groups). From the spectral data, six features were extracted (mean, median, maximum, minimum, standard deviation and variance) at each of these three wavelengths (totally 18 spectral features). 2.3. Classification Two types of discriminant analysis models, linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA), using
V. Chelladurai et al. / Journal of Stored Products Research 57 (2014) 43e48
3. Results 3.1. Soft X-ray imaging Soft X-ray images of uninfested soybeans and soybeans (Fig. 1) infested by different life stages of C. maculatus were analysed by MATLAB software. To avoid effects of background and outliers, pixels with gray level value less than 32 (dark) and higher than 225 (bright) were removed and normalized histograms were developed. Uninfested soybeans and soybeans with eggs were darker (more pixels at gray level values 33e48 and 49e64, Fig. 2) than the samples with larvae, pupae or hollowed-out. Seeds with pupae and hollowed-out were brighter (more pixels at gray level values 209e 224) than other seeds. Classification accuracy rose with increases in age and size of the insect using histogram or textural features as well as with both LDA
35000 Uninfested
30000
Egg Larva
25000
Number of pixels
leave-one-out technique were used to classify the infested and uninfested soybeans. Both LDA and QDA models were developed using: (1) histogram features only, (2) textural features only, and (3) combined features (all 33 extracted features: 12 histogram and 21 textural features) for soft X-ray data, and (1) histogram features only, (2) spectral features only, and (3) combined features (all 48 extracted features: 30 histogram and 18 spectral features). Three types of classification models were developed from these combined features: (1) 5-way model (uninfested, egg, larva, pupa and hollowed-out seeds); (2) 3-way (uninfested, larva and pupa in one group, hollowed-out seeds); (3) Pair-wise (uninfested vs. each stage of infestation). All the features (totally 81 features) from soft X-ray images and hyperspectral data were combined and 5-way, 3-way and pair-wise discriminant models were developed. These models were trained by DISCRIM procedure using SAS 9.3 (Statistical Analysis Systems Institute, Inc., Cary, NC, USA).
45
Pupa Hollowed-out
20000
15000
10000
5000
0 32
48
64
80
96 112 128 144 160 176 192 208 224 240
Gray level value Fig. 2. Normalized histograms of soft X-ray images of soybeans uninfested and infested by different life stages of Callosobruchus maculatus.
and QDA classifiers (Table 1). Seeds that were hollowed-out after adults had emerged from the seeds always had the greatest classification accuracy, with accuracies at over 90%. In contrast eggs and uninfested seeds were often misclassified. Histogram features were combined with textural features, and LDA and QDA models developed from these combined features. The QDA classifier yielded 89% and 96% classification accuracies for pupae and hollowed-out samples, respectively in the 5-way
Fig. 1. Soft X-ray images of soybeans uninfested or infested with single Callosobruchus maculatus at different life stages: (a) uninfested, (b) egg, (c) larva, (d) pupa, (e) hollowed-out after adult emergence.
46
V. Chelladurai et al. / Journal of Stored Products Research 57 (2014) 43e48
Table 1 Classification accuracy of uninfested and infested by different developmental stages of Callosobruchus maculatus using histogram and textural features of soft X-ray and histogram and spectral features of NIR hyperspectral imaging data with linear discriminant analysis (LDA) or quadratic discriminant analysis (QDA) classification models. Stage of infestation
Classification accuracy (%) Soft X-rays
Uninfested Egg Larval Pupal Hollowed-out
Table 3 Pair-wise classification of uninfested and infested by different developmental stages of Callosobruchus maculatus using statistical classifiers from soft X-ray, NIR hyperspectral and combined soft X-ray and NIR hyperspectral imaging data with linear discriminant analysis (LDA) or quadratic discriminant analysis (QDA) classification models. Uninfested vs different infestation stages
NIR hyperspectral
Histogram
Textural
Histogram
Spectral
LDA
QDA
LDA
QDA
LDA
QDA
LDA
QDA
65 61 73 86 91
73 59 72 80 89
54 56 83 90 96
52 64 84 92 98
72 52 52 53 73
75 62 41 42 80
72 65 61 51 76
78 63 60 42 79
classification (Table 2). In 3-way classification, both LDA and QDA classifiers correctly classified more than 86% of all samples. Hollowed-out samples had the highest classification accuracy (98% and 99% for LDA and QDA, respectively) in 3-way classification. More than 87% of uninfested soybeans were correctly distinguished from larva and pupa-infested as well as hollowed-out samples by pair-wise classification models (Table 3). The pairwise classification also indicated that there were many misclassifications between uninfested soybeans and soybeans with eggs.
Infestation type
Classification accuracy (%) Soft X-ray NIR hyperspectral Soft X-ray þ NIR hyperspectral LDA QDA LDA
QDA
LDA
QDA
Egg
Uninfested Egg
69 57
64 60
87 91
79 86
83 87
73 82
Larval
Uninfested Larvae
87 83
98 90
92 93
85 93
97 97
86 92
Pupal
Uninfested Pupae
88 93
82 91
86 87
88 87
95 93
93 89
Hollowed-out Uninfested 96 Hollowed-out 95
94 92
98 95
89 96
100 98
96 97
The 5-way LDA and QDA models developed from all the features (30 histogram and 18 spectral) also had more than 60% classification accuracy except for pupal infestation. The QDA classifier had lowest classification accuracy for pupal infestation (40%), and similar to the QDA models from spectral features, 43% of pupainfested seeds were misclassified with larval and egg stage infestations (Table 2). The 3-way LDA and QDA classifiers positively differentiated 90% and 94% uninfested soybeans from infested seeds. This 3-way classification had more than 87% classification accuracy for all the developmental stages of C. maculatus infestation in soybeans. The pair-wise classifiers also correctly classified more than 85% of soybeans infested with different developmental stages of C. maculatus from uninfested seeds (Table 3).
3.2. NIR hyperspectral imaging The average normalized spectral reflectance of uninfested soybeans was lower than that of infested seeds at 960e1350 nm waveband range (Fig. 3). The principal component analysis of hyperspectral data showed that, first and second PC factor loadings explained more than 99% of variability. Three wavelengths (960,1030 and 1440 nm) were identified as critical wavebands for detection of C. maculatus infestation in soybeans using PC factor loading scores. The discriminant analysis models developed from histogram features of hyperspectral images correctly identified 72%e80% of uninfested and completely hollowed-out soybeans (Table 1). As was the case for the soft X-ray data, the QDA classifier generally had higher classification accuracies than LDA classifiers. The LDA and QDA classifiers developed from textural features also positively classified more than 70% of uninfested and hollowed-out soybeans. Classification accuracies of QDA classifiers for initial developmental stages (egg, larva and pupa) were lower than LDA classifiers with spectral features. Pupal infestation had lower classification accuracy than other stages, because 41% pupa-infested samples were misclassified as either larval or egg stage infestations.
3.3. Combining soft X-rays and NIR hyperspectral imaging The 5-way LDA model developed from combined features (33 soft X-ray features and 48 hyperspectral features) correctly classified more than 79% of uninfested and infested soybeans. The QDA classifier had lowest classification accuracy for soybeans infested with C. maculatus eggs (Table 2). The 3-way models from these combined features positively identified more than 83% of infested soybeans from uninfested soybeans. The pair-wise LDA classifiers identified more than 86% of infested soybeans and pair-wise QDA classifiers yielded lower classification accuracies than LDA classifiers for all developmental stages (Table 3). Combining soft X-ray features with hyperspectral features increased the classification accuracies for early developmental stages (eggs and larvae).
Table 2 Classification accuracy of uninfested and infested by different developmental stages of Callosobruchus maculatus using 5- or 3-way models from soft X-ray, NIR hyperspectral and combined soft X-ray and NIR hyperspectral imaging data with linear discriminant analysis (LDA) or quadratic discriminant analysis (QDA). Stage of infestation
Classification accuracy (%) Soft X-rays 5 -way
Uninfested Egg Larval Pupal Hollowed-out a
Soft X-rays þ NIR hyperspectral
NIR hyperspectral 3-way
5-way
3-way
5-way
3-way
LDA
QDA
LDA
QDA
LDA
QDA
LDA
QDA
LDA
QDA
LDA
QDA
49 48 66 67 92
53 55 59 89 96
94 e e 86a 98
93 e e 90a 99
71 68 62 66 77
76 68 61 40 80
90 e e 89a 89
94 e e 87a 90
79 80 83 81 79
71 67 87 70 85
92 e e 92a 94
83 e e 86a 92
Larval and pupal stages combined.
V. Chelladurai et al. / Journal of Stored Products Research 57 (2014) 43e48
47
0.6 Uninfested 0.5
Egg Larva
0.4
Pupa Hollowed-out
0.3
Reflectance
0.2 0.1 0.0 960 -0.1
1060
1160
1260
1360
1460
1560
1660
Wavelength (nm)
-0.2 -0.3 -0.4 -0.5 -0.6
Fig. 3. Reflectance spectra of soybeans uninfested and infested by different life stages of Callosobruchus maculatus.
4. Discussion In all three LDA and QDA classifiers (developed from combined features, histogram features and textural features) from soft X-ray data, misclassification occurred between uninfested soybeans and soybeans with eggs. The classification accuracy of discriminant analysis classifiers were higher for pupal and hollowed-out seeds. The trend of higher histogram values at the brighter region in the infested seeds than that of uninfested seeds was also noticed in R. dominica infestation in wheat kernels (Karunakaran et al., 2004a). When insects develop inside the seeds, they consume germ and endosperm of the seeds, and this amount consumed increases with each larval stage. The empty space caused by feeding allowed for better penetration of soft X-rays through the seed and resulted in brighter regions in the images. The classification accuracies of discriminant models developed from histogram or spectral features from NIR hyperspectral images were low, and QDA classifier yielded lowest classification accuracies for larval and pupal stages of infestation. The 3-way LDA and QDA classification models of combined features of NIR hyperspectral data, in which the egg stage was omitted for analysis yielded higher classification efficiency than other classification models. In most of the classification models from hyperspectral data, pupal infested soybeans had lower classification accuracy than other stages of infestation. Most of the misclassification occurred between pupal and egg stages. One of the top three wavelengths, 1440 nm was responsible for the insect-related moisture content (Maghirang et al., 2003) and may have caused this misclassification between egg and pupal stage infestations. Pair-wise classification models correctly classified more than 80% of infested seeds from uninfested seeds except with the egg stage. Pair-wise classification models yielded higher classification efficiencies, because misclassification between different stages of infestation was eliminated in this analysis. Zhou et al. (2010) tested the NIR hyperspectral imaging to detect the C. maculatus infestation in soybeans, and they classified soybeans into two classes for analysis: healthy (sound and uninfested soybeans) and infected (soybeans with larvae or pupae of C. maculatus). The back propagation neural network (BPNN) model developed from hyperspectral data correctly identified 88% of infested soybeans from uninfested soybeans. But this BPNN model
only classified soybeans into either infested or uninfested. Detection of infestation stage was not achieved by this model and other developmental stages were not studied. They identified wavebands of 780e900, 920e1000, and 1205e1560 nm could be used for detection of C. maculatus pupal infestation in soybeans. Two of these bands contain the critical wavelengths we determined: 1030 and 1440 nm and there low wavelength band is close to our lowest wavelength at 960 nm. Singh et al. (2009) found that the wavelengths 1102 nm and 1305 nm were useful for detection of S. oryzae, R. dominica and Cryptolestes ferrugineus (Stephens) infestation in wheat using NIR hyperspectral imaging system. They also stated that the models developed from combined features (spectral and histogram features) yielded higher classification accuracies. Results from our study also showed that, combining histogram and spectral features from hyperspectral data improved the classification accuracy. The pair-wise discriminant models gave higher classification accuracies than 5-way or 3-way discriminant models. The same trend was noticed by Manickavasagan et al. (2008) for detection of C. ferrugineus in wheat using a thermal imaging system. Combining soft X-ray features and NIR hyperspectral features significantly increased the classification accuracies of LDA models for early developmental stages (eggs and larvae). In this combined model, most of the misclassification happened between uninfested and soybeans with C. maculatus eggs. Combined application of both soft X-ray and NIR hyperspectral imaging systems would increase the cost and time required for detection, so may not be commercially viable. The classification accuracy of LDA and QDA classifiers developed from textural features of soft X-ray images had higher classification accuracy than the LDA and QDA classifiers developed from spectral features of hyperspectral data for larval and pupal infestations in soybeans. The higher penetrating ability of X-rays compared to hyperspectral imaging may be the reason for better distinguishability between the seeds having living insects inside and uninfested seeds. Soft X-ray method is a well established in-line inspection method and has the ability to detect early stages of insect infestation (Schatzki and Fine, 1988; Rex and Mazza, 1989). The noise in the digital images may also cause classification errors and these noises may be eliminated by wavelet transformation. But NIR hyperspectral imaging systems for commercial in-line monitoring
48
V. Chelladurai et al. / Journal of Stored Products Research 57 (2014) 43e48
have been difficult due to the larger size of hyperspectral data produced and time needed for analyzing these data. Especially tunable filter NIR hyperspectral systems, like the one used in this study, takes a significant amount of time for scanning the object in whole wavelength spectrum. Pushbroom-type line scan NIR hyperspectral imaging systems are faster and better adapted to commercial operations (Mehl et al., 2004). Three key wavelengths identified from this study will help to develop multispectral imaging system to detect insect infestations in soybean and will eliminate the drawbacks of longer scanning and data analyzing periods. These multispectral imaging systems can be used as an inline inspection tool for detecting C. maculatus infestations in soybeans in grain handling facilities. Acknowledgments We thank Chandra Bhan Singh, and Mahesh Sivakumar for their help in data analysis, the Natural Sciences and Engineering Research Council of Canada, Ministry of Innovation, Energy and Mines, Province of Manitoba for their financial support for this study. We also thank Canada Foundation for Innovation, Manitoba Research Innovation Fund, and several other partners for creating research infrastructure. References Ariana, D.P., Lu, R., Guyer, D.E., 2006. Near-infrared hyperspectral reflectance imaging for detection of bruises on pickling cucumbers. Comput. Electron. Agric. 53, 60e70. Chelladurai, V., Kaliramesh, S., Jayas, D.S., 2012. Detection of Callosobruchus maculatus (F.) infestation in Mung bean (Vigna Radiata) using thermal imaging technique. Paper No. NABEC/CSBE 12-121. In: NABEC-CSBE/SCGAB 2012 Joint Meeting and Technical Conference, Canada: July 15e18, Orillia, ON, Canada. Chelladurai, V., Jayas, D.S., White, N.D.G., 2010. Thermal imaging for detecting fungal infection in stored wheat. J. Stored Prod. Res. 46, 174e179. Delwiche, S.R., 1998. Protein content of single kernels of wheat by near-infrared reflectance spectroscopy. J. Cereal Sci. 27, 241e254. Delwiche, S.R., 2003. Classification of scab- and other mold-damaged wheat kernels by near-infrared reflectance spectroscopy. Trans. ASAE 46, 731e738. Dowell, F.E., 2000. Differentiating vitreous and nonvitreous durum wheat kernels by using near-infrared spectroscopy. Cereal Chem. 77, 155e158. FAOSTAT, 2012. Crop Production Data. Food and Agricultural Organisation of United Nations, Rome, Italy. http://faostat.fao.org/site/339/default.aspx (accessed 18.11.12). ski, T., Sadowska, J., Grundas, S., Nawrot, J., Niewiada, A., Fornal, J., Jelin Warchalewski, J.R., B1aszczak, W., 2007. Detection of granary weevil Sitophilus granarius (L.) eggs and internal stages in wheat grain using soft X-ray and image analysis. J. Stored Prod. Res. 43, 142e148. Ghosh, P.K., Jayas, D.S., 2010. Storage of soybean. In: Singh, G. (Ed.), The Soybean Botany, Production and Uses. CABI, Wallingford, UK, pp. 247e275.
Hagstrum, D.W., Subramanyam, B., 2009. Stored-product Insect Resource. AACC International Inc., St. Paul. Kaliramesh, S., Chelladurai, V., Jayas, D.S., Alagusundaram, K., White, N.D.G., Fields, P.G., 2013. Detection of infestation by Callosobruchus maculatus in mung bean using near-infrared hyperspectral imaging. J. Stored Prod. Res. 52, 107e 111. Karunakaran, C., Jayas, D.S., White, N.D.G., 2003. Soft X-ray inspection of wheat kernels infested by Sitophilus oryzae. Trans. ASAE 46, 739e745. Karunakaran, C., Jayas, D.S., White, N.D.G., 2004a. Detection of internal wheat seed infestation by Rhyzopertha Dominica using X-ray imaging. J. Stored Prod. Res. 40, 507e516. Karunakaran, C., Jayas, D.S., White, N.D.G., 2004b. Identification of wheat kernels damaged by the red flour beetle using X-ray images. Biosyst. Eng. 87, 267e 274. Maghirang, E.B., Dowell, F.E., Baker, J.E., Throne, J.E., 2003. Automated detection of single wheat kernels containing live or dead insects using near-infrared reflectance spectroscopy. Trans. ASAE 46, 1277e1282. Mahesh, S., Jayas, D.S., Paliwal, J., White, N.D.G., 2008. Protein and Oil Constituents Determination in Wheat Using Near-infrared (NIR) Hyperspectral Imaging. ASABE Paper No. 084895. ASABE, St. Joseph, MI. Majumdar, S., Jayas, D.S., 2000. Classification of cereal grains using machine vision. III. Texture models. Trans. ASAE 4, 1681e1687. Manickavasagan, A., Jayas, D.S., White, N.D.G., 2008. Thermal imaging to detect infestation by Cryptolestes ferrugineus inside wheat kernels. J. Stored Prod. Res. 44, 186e192. Mehl, P.M., Chen, Y.-R., Kim, M.S., Chan, D.E., 2004. Development of hyperspectral imaging technique for the detection of apple surface defects and contaminations. J. Food Eng. 61, 67e81. Miralbés, C., 2004. Quality control in the milling industry using near infrared transmittance spectroscopy. Food Chem. 88, 621e628. Narvankar, D.S., Singh, C.B., Jayas, D.S., White, N.D.G., 2009. Assessment of soft Xray imaging for detection of fungal infection in wheat. Biosyst. Eng. 103, 49e 56. Neethirajan, S., Karunakaran, C., Jayas, D.S., White, N.D.G., 2007. Detection techniques for stored product pests in grain. Food Control 18, 157e162. Rees, D.P., 2004. Insects of Stored Products. CSIRO Publishing, Collingwood. Rex, B.L., Mazza, G., 1989. Cause, control, and detection of hollow heart in potatoes. Am. Potato J. 66, 165e183. Schatzki, T.F., Fine, T.A., 1988. Analysis of radiogram of wheat kernels for quality control. Cereal Chem. 65, 233e239. Shuman, D., 2003. Monitoring system counts insects identifies species. Agric. Res. Magazine 51, 12e13. Singh, S.R., Jackai, L.E.N., 1985. Insect pests of cowpea in Africa: their life cycle, economic importance and potential for control. In: Singh, S.R., Rachiek, O. (Eds.), Cowpea Research, Production and Utilisation. Wiley Interscience Publications, Chichester, pp. 217e231. Singh, C., Jayas, D.S., Paliwal, J., White, N.D.G., 2009. Detection of insect-damaged wheat kernels using near-infrared hyperspectral imaging. J. Stored Prod. Res. 45, 151e158. Zayas, I.Y., Flinn, P.W., 1998. Detection of insects in bulk wheat samples with machine vision. Trans. ASAE 41, 883. Zhou, Z., Zang, Y., Shen, B., Zhou, X., Luo, X., 2010. Detection of cowpea weevil (Callosobruchus maculatus (F.)) in soybean with hyperspectral spectrometry and a backpropagation neural network. In: Yue, S., Wei, H., Wang, L., Song, Y. (Eds.), Proceedings of the 2010 Sixth International Conference on Natural Computation: 10e12 August, Yantai, China. Institute of Electrical and Electronics Engineers Inc., New York, pp. 1223e1227.