- Email: [email protected]
Detection of infestation by Callosobruchus maculatus in mung bean using near-infrared hyperspectral imaging
Journal of Stored Products Research 52 (2013) 107e111
Contents lists available at SciVerse ScienceDirect
Journal of Stored Products Research journal homepage: www.elsevier.com/locate/jspr
Detection of infestation by Callosobruchus maculatus in mung bean using near-infrared hyperspectral imaging S. Kaliramesh a, b, V. Chelladurai a, D.S. Jayas a, *, K. Alagusundaram b, N.D.G. White c, P.G. Fields c a
Department of Biosystems Engineering, University of Manitoba, Winnipeg, MB, Canada R3T 2N2 Indian Institute of Crop Processing Technology, Thanjavur, Tamil Nadu, India c Cereal Research Centre, Agriculture and Agri-Food Canada, Winnipeg, MB, Canada R3T 2M9 b
a r t i c l e i n f o
a b s t r a c t
Article history: Accepted 19 December 2012
Mung bean (Vigna radiata (L.) R. Wilczek) is one of the major pulse crops grown in India. Cowpea weevil (Callosobruchus maculates F.) is the major insect that causes qualitative and quantitative losses of mung bean kernels during storage. There is an increasing demand from grain buyers and consumers toward zero-tolerance to contamination by insects in grains and grain products. Uninfested mung bean kernels and kernels infested with different stages of C. maculatus were imaged using a near-infrared (NIR) hyperspectral imaging system within the wavelength region of 1000e1600 nm at 10 nm intervals. The wavelengths corresponding to the highest principal components (PC) factor loadings (1100, 1290 and 1450 nm) were considered to be significant. Six statistical features (maximum, minimum, mean, median, standard deviation, and variance) and ten histogram features from images at the significant wavelengths were extracted and given as input to non-parametric statistical classifiers. Average classification accuracies of more than 85% and 82% were obtained using statistical classifiers for identifying uninfested and infested mung bean kernels, respectively. Mung beans kernels with pupal and adult stages of infestation had higher classification accuracies than the egg and larval stages of infestation using both the classifiers. Crown Copyright Ó 2012 Published by Elsevier Ltd. All rights reserved.
Keywords: Hyperspectral imaging Mung bean Callosobruchus maculatus Non-parametric classifiers
1. Introduction Pulses, which belong to the family Leguminosae, represent an important component of agricultural food crops consumed in developing countries and are considered as a vital crop for achieving food and nutritional security for consumers. India is the largest producer, consumer, and importer of pulses with a production of 17 million tonnes (Mt) in 2010e11 (Ali and Gupta, 2012). Though India contributes 24% to the global pulse production, there was a sharp decline in the availability of pulses from 41 g per capita pet day (gpd) in 1990e91 to 33 gpd in 2009e10, requiring a doubling of imports (from 1.3 to 2.4 Mt). India produces 1.6 Mt mung bean (45% of total world production), with total production area of 3.8 million hectares (Mha) (Ali and Gupta, 2012). In India, 2.4% post-harvest losses occur at the producer level due to improper threshing, winnowing, transportation, and 7.5% losses occur due to improper storage. The storage loss includes the
* Corresponding author. E-mail address: [email protected] (D.S. Jayas).
quantitative and qualitative losses by insect infestation and fungal infection. The major insects causing damage to mung bean are bruchids. Callosobruchus maculatus is the most common bruchid, causing weight loss (56e73% in an individual seed), nutritional quality deterioration, and loss of seed viability (Booker, 1967). Larvae of the insect penetrate into the seed and feed on the endosperm as they grow. The adult chews through the seed coat and emerges from the bean (Beck and Blumer, 2009). Since major part of the life cycle of the insect is completed within the kernel, it is very difficult to detect the infestation without dissecting the grain. The insect detection methods, such as visual inspection, Berlese funnel extraction, carbon dioxide production method, whole grain flotation method, acoustic method, X-ray imaging, thermal imaging, and near-infrared (NIR) spectroscopic method are found to have one or more draw backs in detecting these types of hidden infestations. They are found to be time consuming, sample destructive, less accurate, subjective, and unable to detect the egg and larval stages of infestation within the grain. The three advanced methods: soft X-ray imaging (Karunakaran et al., 2003), thermal imaging (Manickavasagan et al., 2008), and NIR spectroscopy (Maghirang et al., 2003) have shown potential
0022-474X/$ e see front matter Crown Copyright Ó 2012 Published by Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.jspr.2012.12.005
108
S. Kaliramesh et al. / Journal of Stored Products Research 52 (2013) 107e111
for accurate and consistent results in real-time applications. However, the X-ray imaging and thermal imaging detect only the later stages (pupal and adult) of infestation. The NIR spectroscopy which is widely used in the grain industry also had some draw backs, as it gives only spectral information, one cannot locate the exact position of the hidden insect inside the grain, and it requires robust calibration models and sometimes gives inconsistent results. Hyperspectral imaging is an emerging technique that integrates attributes of conventional imaging and spectroscopy to attain both spatial and spectral information from an object (Gowen et al., 2007). Hyperspectral imaging provides a large data set, otherwise called a hypercube, which facilitates a complete and reliable analysis of intrinsic properties and external characteristics of samples. This technology has many advantages such as the method is fairly accurate, non-destructive, and gives consistent results. This technique has been proven to be a reliable method to measure the moisture and oil content in maize (Cogdill et al., 2004), protein and oil content in wheat (Mahesh et al., 2008), detection of fungi in wheat (Singh et al., 2007), and detection of insect damaged wheat kernels (Singh et al., 2009). Though hyperspectral imaging is having many advantages, nobody has tried its usage with regards to beans. Insects are the major deteriorating organisms in beans; after attacking the grains, they leave the grains only on complete deterioration. The particular insect C. maculatus is the major insect which attacks the grain either in the field or during storage, and grows inside the grain (hidden infestation). These types of hidden infestations can be detected either on dissecting the grain or by X-ray imaging or NIR spectroscopy which are having one or more disadvantages. The hyperspectral imaging system has a capacity to detect hidden infestations in quick time without any destruction to the seeds. The hyperspectral imaging technique has not been fully applied for in-line quality assessment of major parameters due to its time consumption for acquiring the image (Mehl et al., 2002). But, Lee et al. (2005) reported that this problem could be overcome by using the optimal wavelength to acquire the image and then the in-line quality assessment could be done by using the multispectral imaging technique. Therefore, objective of this study was to investigate the feasibility of NIR hyperspectral imaging system to detect infestation by C. maculatus in mung bean by selecting the significant wavelengths and developing discriminant algorithms (linear and quadratic).
2. Materials and methods 2.1. Sample preparation Mung beans having 12% moisture content (w.b.) were used in this study. Moisture content was determined by drying 10 g samples, in triplicate, at 130 C for 20 h in a hot air convection oven (Tang and Sokhansanj, 1991). About 150 freshly emerged insects were added to 250 g of sound mung bean and the kernels with a single egg were collected after 24 h by observing the kernels under the microscope. Kernels with a single egg were separated into seven groups (300 kernels in each group) and were incubated at 30 C and 70% r.h. for 4, 8, 11, 15, 22, and 25 days to obtain first, second, third and fourth instar larvae, pupae and adults, respectively (Mookherjee and Chawla, 1962). 2.2. NIR hyperspectral imaging system and image acquisition The imaging system used in this study is the same as reported in the earlier studies (Mahesh et al., 2008; Singh et al., 2009) and is described briefly here for enhanced readability and completeness of this manuscript. The major components of the system include a thermoelectrically cooled Indium Gallium Arsenide (InGaAs) camera (Model No. SU640-1.7RT-D, Sensors Unlimited Inc., Princeton, NJ), with two VariSpec liquid crystal tunable filters (LCTFs) (Model No. MIR06, Cambridge Research and Instrumentation Inc., Woburn, MA), a 25 mm F1.4 C-mount lens (Electrophysics Corp., Fairfield, NJ), a sample stage, and a light source controlled through a Dell Optiplex GX280 Intel(R) (Dell Inc., Round Rock, TX) computer (Fig. 1). The camera could be operated in a room with temperature between 20 and 27 C and the images can be acquired within an NIR region of 900e1700 nm. This system had a spatial resolution of 640 480 pixels with 27 mm pitch and a spectral resolution of 0.01 nm. The electronically tunable, liquid crystal tunable filter (LCTF) is a high quality interference filter which had an aperture of 20 mm and a transmission bandwidth of 10 mm. This filter helps to select a wavelength in the NIR region without any vibration. The sample was illuminated by a pair of 300 W halogen-tungsten bulbs (USHIO Inc., Chiyoda-ku, Tokyo, Japan) emitting light in a wavelength range of 400e2500 nm. The data acquisition board (NI PCI-1422, National Instruments Corp., Austin, TX) was attuned to RS-422 signals generated from the camera system for image acquisition. A control
Fig. 1. Long-wave near-infrared hyperspectral imaging system 1. Mung bean sample, 2. Liquid crystal tunable filter (LCTF), 3. Lens, 4. NIR camera, 5. Copy stand, 6. Illumination (Halogen-tungsten lamp), 7. Data processing system.
S. Kaliramesh et al. / Journal of Stored Products Research 52 (2013) 107e111
program developed in LabVIEW (Version 1, National Instruments, Austin, TX, USA) was used to align the imaging system, acquire images, and store hyperspectral image data in 12-bit binary file. The program stored the imaging system setup, wavelength range, and number of wavelength slices along with the hyperspectral imaging data in a file. The images of uninfested and infested kernels (300 kernels each) at different stages of infestation by C. maculatus were taken after respective periods of incubation as mentioned in the Section 2.1. Prior to image acquisition, the imaging system was stabilized by switching on the sensors for approximately 30 min and was aligned to the central wavelength of 1300 nm in the wavelength region of 1000e1600 nm. At this central wavelength, dark current was acquired at the beginning of each imaging session, by blocking the entrance of the camera. The control program automatically subtracted the dark current from the subsequent acquired images. For every image, five non-touching randomly selected kernels were manually placed in the same stable orientation on a black paper board in the imaging area of the NIR camera (Singh et al., 2009). They were scanned at 61 evenly spaced wavelengths in a range of 1000e1600 nm with 10 nm intervals, because the InGaAs detector has high quantum efficiency (>70%) in this wavelength region. 2.3. Image analysis Program developed using MATLAB (Mathworks Inc. Natick, MA, USA) software was used to extract and analyze the hyperspectral data acquired from NIR hyperspectral imaging system. All 5 nontouching kernels in an image were labeled by bwlabel function and segmented from the background by automatic thresholding method as done by Singh et al. (2007). Then each of the labeled kernels were analyzed by a multivariate image analysis (MVI) program written in MATLAB. Principal component analysis was then applied to the reshaped two-dimensional data set of each kernel. The wavelengths corresponding to the highest factor loadings of the first and second principal components (PCs) were selected as significant wavelengths. Features were extracted from these significant wavelengths. In this study, statistical features namely maximum, minimum, mean, median, standard deviation, and variance from the images corresponding to the significant wavelengths were extracted and used in classifier development. Ten histogram features from the transformed (reflectance) images at each significant wavelength were also extracted by binning the reflectance into 10 equally distributed reflectance groups between 0 and 1 as done by Singh et al. (2009). These features were combined with statistical features and used as input for classification model development.
3. Results and discussion The PCA was performed for each kernel individually and the principal component (PC) loadings of the images were examined for determining relationships between image features of uninfested and insect-infested mung bean kernels. First and second PC factor loadings accounted for nearly 94% and 5% variability, respectively, in the hyperspectral data. The top two wavelengths (1110 and 1290 nm) for the first principal component factor loadings (Fig. 2), and highest factor loading wavelength (1450 nm) for second principal component (Fig. 3) were selected for feature extraction. Six statistical features and 10 histogram features at each significant wavelength were extracted and used in the classification model development. The wavelength region of 1100e 1300 nm corresponds to the carbonehydrogen 1st and 2nd overtones and carbonehydrogen combination band and the significance of wavelengths in this region can be associated with absorption by starch molecules (Osborne, 2006). Singh et al. (2009) reported that wavelengths of 1101.7, 1305.1, and 1447.6 nm were significant in detecting wheat kernels damaged by rice weevil, lesser grain borer, rusty grain beetle, and red flour beetle. Maghirang et al. (2003) reported that, wavelengths 1135 and 1325 nm were significant for insect detection in wheat. The wavelength region of 1202e1300 nm was significant to detect the larval stage of infestation by Sitophilus granarius (L.) in wheat, and insect-infested kernels have less starch compared to uninfested kernels due to consumption of starch by insects during their development (Ridgway and Chambers, 1998). Two types of classification models were developed: (i) using the six statistical features extracted from the significant wavelengths and (ii) using the combined features (statistical features and histogram features). The use of 30 histogram features (10 histogram features at each wavelength) did not classify the infested grain due to the formation of non-positive definite covariance matrices in the discriminant analysis. The cause of this non-positive covariance matrix might be due to the linear dependency of one or more input features on other features. To overcome this problem, statistical image features (6 3 ¼ 18) were combined with histogram image features (10 3 ¼ 30). The same problem was observed when using all the 48 features for classification which indicated that the histogram features have a near perfect linear dependency. Thus, only nine histogram features (groups between 0.1 and 1.0 reflectance) from the images at 1100 nm corresponding to the highest factor loadings of the first PC were considered as histogram features and used in classification.
0.18
0.16
PC Loadings
2.4. Model development The classification algorithms were developed by using nonparametric statistical classifiers (linear and quadratic). In the present study, the classification models were developed by using the linear discriminant analysis (LDA) and the quadratic discriminant analysis (QDA) with leave-one-out cross validation method by using PROC DISCRIM procedure in SAS (version 9.2, SAS Institute Inc., Cary, NC). Four types of analysis were carried out: (i) two-way classification in which seven classes (egg, four larval instars, pupae, and adults) were mixed and treated as infested (one class) vs. uninfested, (ii) five-way classification (uninfested, egg, four larval instars, pupae, and adults), (iii) four-way classification of larval instars, and (iv) each development stage vs. uninfested (pairwise discrimination).
109
0.14 1100 nm
0.12
0.1
0.08
0.06 1000
1290 nm
Uninfested Egg Instar 1 Instar 2 Instar 3 Instar 4 Pupae Adult
1100
1200
1300
1400
1500
1600
Wavelength (nm) Fig. 2. First principal components (PC) mean factor loadings of uninfested and different stages of insect-infested mung bean kernels.
110
S. Kaliramesh et al. / Journal of Stored Products Research 52 (2013) 107e111 0.25
PC Loadings
0.20
0.15
1450 nm Uninfested Egg Instar 1 Instar 2 Instar 3 Instar 4 Pupae Adult
0.10
0.05
0.00 1000
1100
1200
1300
1400
1500
1600
Classification accuracy (%)
100
LDA
QDA
80
60
40
20
Wavelength (nm) Fig. 3. Second principal components (PC) mean factor loadings of uninfested and different stages of insect-infested mung bean kernels.
0 Uninfested
Infested
Class Two-way classification models were developed to differentiate uninfested mung bean kernels from infested mung bean kernels. A total of 2400 kernels (300 uninfested kernels, and 2100 infested kernels by different stages of infestation) were used in the classification. The LDA classifier accurately discriminated 83.0% of uninfested and 80.5% of infested kernels whereas, QDA gave a classification accuracy of 89.6% and 82.0% for uninfested and infested kernels, respectively (Fig. 4) using the statistical features in the model development. With addition of nine histogram features to the classification model the classification accuracy improved to 84.7% and 88.7% using the LDA and QDA classifiers for uninfested and infested kernels, respectively (Fig. 5). Five-way classification models were developed using 2400 kernels (300 uninfested kernels, 300 kernels containing single egg, 1200 kernels with different stages of larval development, 300 kernels with pupae, and 300 kernels with adult stages). The classification accuracies of different classes by using the LDA and QDA classifiers were 80.7% and 90.7% (uninfested), 72.3% and 85.3% (egg), 63.0% and 75.5% (larvae), 84.3% and 94.3% (pupae), and 93.3% and 95.7% (adult), respectively (Table 1) using the statistical features in the classifier. With the addition of nine histogram features to the classification model, the kernels infested with larval stage could be classified with a classification accuracy of 66.3% and 77.3% using the LDA and QDA classifier, respectively (Table 1).
LDA
Classification accuracy (%)
100
QDA
80
Fig. 5. Classification accuracies of uninfested mung bean kernels and infested kernels (all stages combined) by Callosobruchus maculatus using statistical features (1100, 1290, and 1450 nm wavelengths) and histogram features (1100 nm wavelength) by LDA and QDA classifiers.
Four-way classification models were developed using the statistical features, to identify infested kernels with different stages of larval development. A total of 1200 kernels (300 kernels each per instar) were used in the classification. The highest classification accuracy was found in discriminating the kernels with fourth instar larvae with 69.0% and 80.3% accuracy using LDA and QDA classifiers, respectively using the combined features in the model development. The kernels with the other three stages of larval development were classified with classification accuracies of 54.3e 59.3% and 57.0e62.7% using LDA and QDA, respectively (Table 2) using both the features. Pair-wise classification models were developed to differentiate infested kernels by each stage of infestation from uninfested kernels. A total of 600 kernels (300 uninfested and 300 insect infested for each stage of infestation) were used in pair-wise classification. The kernels infested with egg, larval stages of development were correctly identified with 81.3e95.3% accuracy using both LDA and QDA classifiers (Table 3). Among all the stages of infestation, kernels with pupae and adult stages of infestation could be discriminated with the highest classification accuracies of >96.0% using both the statistical and combined features (Table 3). The addition of nine histogram features, improved the classification accuracy of kernels infested with different stages of development.
Table 1 Classification accuracies of the five-way classification model of uninfested mung bean kernels and those infested by different stages of C. maculatus using statistical features (1100, 1290, and 1450 nm wavelengths) and combined features (statistical and histogram features at 1100 nm) by non-parametric classifiers, N ¼ 300.
60
40
Stage
Classification by non-parametric classifiers (%) Classification by using combined features
Classification by using statistical features
20
0 Uninfested
Infested
Class Fig. 4. Two-way classification accuracies of uninfested mung bean kernels and infested kernels (all stages combined) by Callosobruchus maculatus using statistical features (1100, 1290, and 1450 nm wavelengths) by LDA and QDA classifiers.
Uninfested Egg Larvala Pupal Adult a
N ¼ 1200.
LDA
QDA
LDA
QDA
80.7 72.3 63.0 84.3 93.3
90.7 85.3 75.5 94.3 95.7
79.7 69.0 66.3 86.0 90.3
93.7 84.3 77.3 94.3 94.7
S. Kaliramesh et al. / Journal of Stored Products Research 52 (2013) 107e111 Table 2 Classification accuracies of the four-way classification model of mung bean kernels infested by different larval stages of C. maculatus using statistical features (1100, 1290, and 1450 nm wavelengths) and combined features (statistical and histogram features at 1100 nm) by non-parametric classifiers, N ¼ 300. Larval stage
Classification by non-parametric classifiers (%) Classification by using statistical features
Instar Instar Instar Instar
1 2 3 4
Classification by using combined features
Uninfested Egg Uninfested Instar 1 Uninfested Instar 2 Uninfested Instar 3 Uninfested Instar 4 Uninfested Pupae Uninfested Adult
kernels infested by different stages of C. maculatus. Higher classification accuracies were achieved in identifying the kernels with pupae and adult stages of infestation using both the classifiers. The three significant wavelengths used in the present study could be used for future online identification of mung bean kernels infested by C. maculatus by developing multi-spectral imaging systems. Acknowledgments
LDA
QDA
LDA
QDA
59.3 54.3 59.3 67.0
57.0 62.7 61.7 80.7
58.3 57.7 64.7 69.0
65.3 61.3 69.0 80.3
Table 3 Pair-wise classification accuracies of uninfested mung bean kernels and those infested by different stages of C. maculatus using statistical features (1100, 1290, and 1450 nm wavelengths) and combined features (statistical and histogram features at 1100 nm) by non-parametric classifiers, N ¼ 300. Stage
111
Classification by non-parametric classifiers (%) Classification by using statistical features
Classification by using combined features
LDA
QDA
LDA
QDA
89.0 91.0 85.7 84.0 76.0 85.0 88.3 89.3 92.3 81.3 99.7 96.0 99.7 99.7
93.3 88.0 90.7 90.0 88.3 88.3 91.3 93.0 99.3 86.0 100.0 99.0 98.7 99.3
89.7 91.3 85.7 85.3 78.7 84.0 90.0 89.3 91.0 84.3 100.0 96.0 99.7 99.0
96.7 86.7 75.4 89.7 88.0 90.0 90.7 95.3 97.7 89.7 98.7 99.0 99.7 99.0
4. Conclusion The near-infrared hyperspectral imaging can be used to identify the uninfested and the kernels infested with different stages of C. maculatus. In the NIR region, wavelengths of 1100, 1290, and 1450 nm were identified to be significant based on the highest factor loadings in first principal component and second principal component, respectively. The significant wavelengths were used in features extraction and classification. Average classification accuracies of 85% and 88% were obtained for the LDA and QDA, respectively, for identifying the uninfested mung bean samples. LDA and QDA classifier correctly identified more than 82% of
The authors thank Manitoba Innovation, Energy and Mines and Ministry of Food Processing Industries, India for funding this study and Canada Foundation for Innovation, Manitoba Research Innovation Fund and other industry partners for funding the acquisition of research infrastructure. References Ali, M., Gupta, S., 2012. Carrying capacity of Indian agriculture: pulse crops. Current Science 102, 874e881. Beck, C.W., Blumer, L.S., 2009. A Handbook on Bean Beetles, Callosobruchus maculatus. National Science Foundation (18.11.11.). http://www.beanbeetles.org/ handbook.pdf. Booker, R.H., 1967. Observations on three bruchids associated with cowpea in northern Nigeria. Journal of Stored Products Research 3, 1e15. Cogdill, R.P., Hurburgh, C.R., Rippke, G.R., 2004. Single-kernel maize analysis by near-infrared hyperspectral imaging. Transactions of the ASAE 47, 311e320. Gowen, A.A., O’Donnell, C.P., Cullen, P.J., Downey, G., Frias, J.M., 2007. Hyperspectral imaging: an emerging process analytical tool for food quality and safety control. Trends in Food Science and Technology 18, 590e598. Karunakaran, C., Jayas, D.S., White, N.D.G., 2003. X-ray image analysis to detect infestations caused by insects in grain. Cereal Chemistry 80, 553e557. Lee, K.J., Kang, S., Kim, M.S., Noh, S.H., 2005. Hyperspectral Imaging for Detecting Defect on Apples. ASABE Paper No. 053075. ASABE, St. Joseph, MI. Maghirang, E.B., Dowell, F.E., Baker, J.E., Throne, J.E., 2003. Automated detection of single wheat kernels containing live or dead insects using near-infrared reflectance spectroscopy. Transactions of the ASAE 46, 1277e1282. Mahesh, S., Jayas, D.S., Paliwal, J., White, N.D.G., 2008. Protein and Oil Constituents Determination in Wheat Using Near-infrared (NIR) Hyperspectral Imaging. ASABE Paper No. 084895. ASABE, St. Joseph, MI. Manickavasagan, A., Jayas, D.S., White, N.D.G., 2008. Thermal imaging to detect infestation by Cryptolestes ferrugineus inside wheat kernels. Journal of Stored Products Research 44, 186e192. Mehl, P.M., Chen, Y.R., Kim, M.S., Chan, D.E., 2002. Detection of defects on selected apple cultivars using hyperspectral and multispectral image analysis. Applied Engineering in Agriculture 18, 219e226. Mookherjee, P.B., Chawla, M.L., 1962. Effect of temperature and humidity on the development of Callosobruchus maculatus Fab., a serious pest of stored pulses. Indian Journal of Entomology 26, 345e351. Osborne, B.G., 2006. NIR spectroscopy in food analysis. In: Encyclopedia of Analytical Chemistry. John Wiley and Sons Limited, Chichester. Ridgway, C., Chambers, J., 1998. Detection of insects inside wheat kernels by NIR imaging. Journal of Near Infrared Spectroscopy 6, 115e119. Singh, C.B., Jayas, D.S., Paliwal, J., White, N.D.G., 2007. Fungal detection in wheat using near-infrared hyperspectral imaging. Transactions of the ASABE 50 (6), 2171e2176. Singh, C.B., Jayas, D.S., Paliwal, J., White, N.D.G., 2009. Detection of insect-damaged wheat kernels using near-infrared hyperspectral imaging. Journal of Stored Products Research 45, 151e158. Tang, J., Sokhansanj, S., 1991. Determination of moisture content of whole kernel lentil by oven method. Transactions of the ASAE 34, 255e256.
Contents lists available at SciVerse ScienceDirect
Journal of Stored Products Research journal homepage: www.elsevier.com/locate/jspr
Detection of infestation by Callosobruchus maculatus in mung bean using near-infrared hyperspectral imaging S. Kaliramesh a, b, V. Chelladurai a, D.S. Jayas a, *, K. Alagusundaram b, N.D.G. White c, P.G. Fields c a
Department of Biosystems Engineering, University of Manitoba, Winnipeg, MB, Canada R3T 2N2 Indian Institute of Crop Processing Technology, Thanjavur, Tamil Nadu, India c Cereal Research Centre, Agriculture and Agri-Food Canada, Winnipeg, MB, Canada R3T 2M9 b
a r t i c l e i n f o
a b s t r a c t
Article history: Accepted 19 December 2012
Mung bean (Vigna radiata (L.) R. Wilczek) is one of the major pulse crops grown in India. Cowpea weevil (Callosobruchus maculates F.) is the major insect that causes qualitative and quantitative losses of mung bean kernels during storage. There is an increasing demand from grain buyers and consumers toward zero-tolerance to contamination by insects in grains and grain products. Uninfested mung bean kernels and kernels infested with different stages of C. maculatus were imaged using a near-infrared (NIR) hyperspectral imaging system within the wavelength region of 1000e1600 nm at 10 nm intervals. The wavelengths corresponding to the highest principal components (PC) factor loadings (1100, 1290 and 1450 nm) were considered to be significant. Six statistical features (maximum, minimum, mean, median, standard deviation, and variance) and ten histogram features from images at the significant wavelengths were extracted and given as input to non-parametric statistical classifiers. Average classification accuracies of more than 85% and 82% were obtained using statistical classifiers for identifying uninfested and infested mung bean kernels, respectively. Mung beans kernels with pupal and adult stages of infestation had higher classification accuracies than the egg and larval stages of infestation using both the classifiers. Crown Copyright Ó 2012 Published by Elsevier Ltd. All rights reserved.
Keywords: Hyperspectral imaging Mung bean Callosobruchus maculatus Non-parametric classifiers
1. Introduction Pulses, which belong to the family Leguminosae, represent an important component of agricultural food crops consumed in developing countries and are considered as a vital crop for achieving food and nutritional security for consumers. India is the largest producer, consumer, and importer of pulses with a production of 17 million tonnes (Mt) in 2010e11 (Ali and Gupta, 2012). Though India contributes 24% to the global pulse production, there was a sharp decline in the availability of pulses from 41 g per capita pet day (gpd) in 1990e91 to 33 gpd in 2009e10, requiring a doubling of imports (from 1.3 to 2.4 Mt). India produces 1.6 Mt mung bean (45% of total world production), with total production area of 3.8 million hectares (Mha) (Ali and Gupta, 2012). In India, 2.4% post-harvest losses occur at the producer level due to improper threshing, winnowing, transportation, and 7.5% losses occur due to improper storage. The storage loss includes the
* Corresponding author. E-mail address: [email protected] (D.S. Jayas).
quantitative and qualitative losses by insect infestation and fungal infection. The major insects causing damage to mung bean are bruchids. Callosobruchus maculatus is the most common bruchid, causing weight loss (56e73% in an individual seed), nutritional quality deterioration, and loss of seed viability (Booker, 1967). Larvae of the insect penetrate into the seed and feed on the endosperm as they grow. The adult chews through the seed coat and emerges from the bean (Beck and Blumer, 2009). Since major part of the life cycle of the insect is completed within the kernel, it is very difficult to detect the infestation without dissecting the grain. The insect detection methods, such as visual inspection, Berlese funnel extraction, carbon dioxide production method, whole grain flotation method, acoustic method, X-ray imaging, thermal imaging, and near-infrared (NIR) spectroscopic method are found to have one or more draw backs in detecting these types of hidden infestations. They are found to be time consuming, sample destructive, less accurate, subjective, and unable to detect the egg and larval stages of infestation within the grain. The three advanced methods: soft X-ray imaging (Karunakaran et al., 2003), thermal imaging (Manickavasagan et al., 2008), and NIR spectroscopy (Maghirang et al., 2003) have shown potential
0022-474X/$ e see front matter Crown Copyright Ó 2012 Published by Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.jspr.2012.12.005
108
S. Kaliramesh et al. / Journal of Stored Products Research 52 (2013) 107e111
for accurate and consistent results in real-time applications. However, the X-ray imaging and thermal imaging detect only the later stages (pupal and adult) of infestation. The NIR spectroscopy which is widely used in the grain industry also had some draw backs, as it gives only spectral information, one cannot locate the exact position of the hidden insect inside the grain, and it requires robust calibration models and sometimes gives inconsistent results. Hyperspectral imaging is an emerging technique that integrates attributes of conventional imaging and spectroscopy to attain both spatial and spectral information from an object (Gowen et al., 2007). Hyperspectral imaging provides a large data set, otherwise called a hypercube, which facilitates a complete and reliable analysis of intrinsic properties and external characteristics of samples. This technology has many advantages such as the method is fairly accurate, non-destructive, and gives consistent results. This technique has been proven to be a reliable method to measure the moisture and oil content in maize (Cogdill et al., 2004), protein and oil content in wheat (Mahesh et al., 2008), detection of fungi in wheat (Singh et al., 2007), and detection of insect damaged wheat kernels (Singh et al., 2009). Though hyperspectral imaging is having many advantages, nobody has tried its usage with regards to beans. Insects are the major deteriorating organisms in beans; after attacking the grains, they leave the grains only on complete deterioration. The particular insect C. maculatus is the major insect which attacks the grain either in the field or during storage, and grows inside the grain (hidden infestation). These types of hidden infestations can be detected either on dissecting the grain or by X-ray imaging or NIR spectroscopy which are having one or more disadvantages. The hyperspectral imaging system has a capacity to detect hidden infestations in quick time without any destruction to the seeds. The hyperspectral imaging technique has not been fully applied for in-line quality assessment of major parameters due to its time consumption for acquiring the image (Mehl et al., 2002). But, Lee et al. (2005) reported that this problem could be overcome by using the optimal wavelength to acquire the image and then the in-line quality assessment could be done by using the multispectral imaging technique. Therefore, objective of this study was to investigate the feasibility of NIR hyperspectral imaging system to detect infestation by C. maculatus in mung bean by selecting the significant wavelengths and developing discriminant algorithms (linear and quadratic).
2. Materials and methods 2.1. Sample preparation Mung beans having 12% moisture content (w.b.) were used in this study. Moisture content was determined by drying 10 g samples, in triplicate, at 130 C for 20 h in a hot air convection oven (Tang and Sokhansanj, 1991). About 150 freshly emerged insects were added to 250 g of sound mung bean and the kernels with a single egg were collected after 24 h by observing the kernels under the microscope. Kernels with a single egg were separated into seven groups (300 kernels in each group) and were incubated at 30 C and 70% r.h. for 4, 8, 11, 15, 22, and 25 days to obtain first, second, third and fourth instar larvae, pupae and adults, respectively (Mookherjee and Chawla, 1962). 2.2. NIR hyperspectral imaging system and image acquisition The imaging system used in this study is the same as reported in the earlier studies (Mahesh et al., 2008; Singh et al., 2009) and is described briefly here for enhanced readability and completeness of this manuscript. The major components of the system include a thermoelectrically cooled Indium Gallium Arsenide (InGaAs) camera (Model No. SU640-1.7RT-D, Sensors Unlimited Inc., Princeton, NJ), with two VariSpec liquid crystal tunable filters (LCTFs) (Model No. MIR06, Cambridge Research and Instrumentation Inc., Woburn, MA), a 25 mm F1.4 C-mount lens (Electrophysics Corp., Fairfield, NJ), a sample stage, and a light source controlled through a Dell Optiplex GX280 Intel(R) (Dell Inc., Round Rock, TX) computer (Fig. 1). The camera could be operated in a room with temperature between 20 and 27 C and the images can be acquired within an NIR region of 900e1700 nm. This system had a spatial resolution of 640 480 pixels with 27 mm pitch and a spectral resolution of 0.01 nm. The electronically tunable, liquid crystal tunable filter (LCTF) is a high quality interference filter which had an aperture of 20 mm and a transmission bandwidth of 10 mm. This filter helps to select a wavelength in the NIR region without any vibration. The sample was illuminated by a pair of 300 W halogen-tungsten bulbs (USHIO Inc., Chiyoda-ku, Tokyo, Japan) emitting light in a wavelength range of 400e2500 nm. The data acquisition board (NI PCI-1422, National Instruments Corp., Austin, TX) was attuned to RS-422 signals generated from the camera system for image acquisition. A control
Fig. 1. Long-wave near-infrared hyperspectral imaging system 1. Mung bean sample, 2. Liquid crystal tunable filter (LCTF), 3. Lens, 4. NIR camera, 5. Copy stand, 6. Illumination (Halogen-tungsten lamp), 7. Data processing system.
S. Kaliramesh et al. / Journal of Stored Products Research 52 (2013) 107e111
program developed in LabVIEW (Version 1, National Instruments, Austin, TX, USA) was used to align the imaging system, acquire images, and store hyperspectral image data in 12-bit binary file. The program stored the imaging system setup, wavelength range, and number of wavelength slices along with the hyperspectral imaging data in a file. The images of uninfested and infested kernels (300 kernels each) at different stages of infestation by C. maculatus were taken after respective periods of incubation as mentioned in the Section 2.1. Prior to image acquisition, the imaging system was stabilized by switching on the sensors for approximately 30 min and was aligned to the central wavelength of 1300 nm in the wavelength region of 1000e1600 nm. At this central wavelength, dark current was acquired at the beginning of each imaging session, by blocking the entrance of the camera. The control program automatically subtracted the dark current from the subsequent acquired images. For every image, five non-touching randomly selected kernels were manually placed in the same stable orientation on a black paper board in the imaging area of the NIR camera (Singh et al., 2009). They were scanned at 61 evenly spaced wavelengths in a range of 1000e1600 nm with 10 nm intervals, because the InGaAs detector has high quantum efficiency (>70%) in this wavelength region. 2.3. Image analysis Program developed using MATLAB (Mathworks Inc. Natick, MA, USA) software was used to extract and analyze the hyperspectral data acquired from NIR hyperspectral imaging system. All 5 nontouching kernels in an image were labeled by bwlabel function and segmented from the background by automatic thresholding method as done by Singh et al. (2007). Then each of the labeled kernels were analyzed by a multivariate image analysis (MVI) program written in MATLAB. Principal component analysis was then applied to the reshaped two-dimensional data set of each kernel. The wavelengths corresponding to the highest factor loadings of the first and second principal components (PCs) were selected as significant wavelengths. Features were extracted from these significant wavelengths. In this study, statistical features namely maximum, minimum, mean, median, standard deviation, and variance from the images corresponding to the significant wavelengths were extracted and used in classifier development. Ten histogram features from the transformed (reflectance) images at each significant wavelength were also extracted by binning the reflectance into 10 equally distributed reflectance groups between 0 and 1 as done by Singh et al. (2009). These features were combined with statistical features and used as input for classification model development.
3. Results and discussion The PCA was performed for each kernel individually and the principal component (PC) loadings of the images were examined for determining relationships between image features of uninfested and insect-infested mung bean kernels. First and second PC factor loadings accounted for nearly 94% and 5% variability, respectively, in the hyperspectral data. The top two wavelengths (1110 and 1290 nm) for the first principal component factor loadings (Fig. 2), and highest factor loading wavelength (1450 nm) for second principal component (Fig. 3) were selected for feature extraction. Six statistical features and 10 histogram features at each significant wavelength were extracted and used in the classification model development. The wavelength region of 1100e 1300 nm corresponds to the carbonehydrogen 1st and 2nd overtones and carbonehydrogen combination band and the significance of wavelengths in this region can be associated with absorption by starch molecules (Osborne, 2006). Singh et al. (2009) reported that wavelengths of 1101.7, 1305.1, and 1447.6 nm were significant in detecting wheat kernels damaged by rice weevil, lesser grain borer, rusty grain beetle, and red flour beetle. Maghirang et al. (2003) reported that, wavelengths 1135 and 1325 nm were significant for insect detection in wheat. The wavelength region of 1202e1300 nm was significant to detect the larval stage of infestation by Sitophilus granarius (L.) in wheat, and insect-infested kernels have less starch compared to uninfested kernels due to consumption of starch by insects during their development (Ridgway and Chambers, 1998). Two types of classification models were developed: (i) using the six statistical features extracted from the significant wavelengths and (ii) using the combined features (statistical features and histogram features). The use of 30 histogram features (10 histogram features at each wavelength) did not classify the infested grain due to the formation of non-positive definite covariance matrices in the discriminant analysis. The cause of this non-positive covariance matrix might be due to the linear dependency of one or more input features on other features. To overcome this problem, statistical image features (6 3 ¼ 18) were combined with histogram image features (10 3 ¼ 30). The same problem was observed when using all the 48 features for classification which indicated that the histogram features have a near perfect linear dependency. Thus, only nine histogram features (groups between 0.1 and 1.0 reflectance) from the images at 1100 nm corresponding to the highest factor loadings of the first PC were considered as histogram features and used in classification.
0.18
0.16
PC Loadings
2.4. Model development The classification algorithms were developed by using nonparametric statistical classifiers (linear and quadratic). In the present study, the classification models were developed by using the linear discriminant analysis (LDA) and the quadratic discriminant analysis (QDA) with leave-one-out cross validation method by using PROC DISCRIM procedure in SAS (version 9.2, SAS Institute Inc., Cary, NC). Four types of analysis were carried out: (i) two-way classification in which seven classes (egg, four larval instars, pupae, and adults) were mixed and treated as infested (one class) vs. uninfested, (ii) five-way classification (uninfested, egg, four larval instars, pupae, and adults), (iii) four-way classification of larval instars, and (iv) each development stage vs. uninfested (pairwise discrimination).
109
0.14 1100 nm
0.12
0.1
0.08
0.06 1000
1290 nm
Uninfested Egg Instar 1 Instar 2 Instar 3 Instar 4 Pupae Adult
1100
1200
1300
1400
1500
1600
Wavelength (nm) Fig. 2. First principal components (PC) mean factor loadings of uninfested and different stages of insect-infested mung bean kernels.
110
S. Kaliramesh et al. / Journal of Stored Products Research 52 (2013) 107e111 0.25
PC Loadings
0.20
0.15
1450 nm Uninfested Egg Instar 1 Instar 2 Instar 3 Instar 4 Pupae Adult
0.10
0.05
0.00 1000
1100
1200
1300
1400
1500
1600
Classification accuracy (%)
100
LDA
QDA
80
60
40
20
Wavelength (nm) Fig. 3. Second principal components (PC) mean factor loadings of uninfested and different stages of insect-infested mung bean kernels.
0 Uninfested
Infested
Class Two-way classification models were developed to differentiate uninfested mung bean kernels from infested mung bean kernels. A total of 2400 kernels (300 uninfested kernels, and 2100 infested kernels by different stages of infestation) were used in the classification. The LDA classifier accurately discriminated 83.0% of uninfested and 80.5% of infested kernels whereas, QDA gave a classification accuracy of 89.6% and 82.0% for uninfested and infested kernels, respectively (Fig. 4) using the statistical features in the model development. With addition of nine histogram features to the classification model the classification accuracy improved to 84.7% and 88.7% using the LDA and QDA classifiers for uninfested and infested kernels, respectively (Fig. 5). Five-way classification models were developed using 2400 kernels (300 uninfested kernels, 300 kernels containing single egg, 1200 kernels with different stages of larval development, 300 kernels with pupae, and 300 kernels with adult stages). The classification accuracies of different classes by using the LDA and QDA classifiers were 80.7% and 90.7% (uninfested), 72.3% and 85.3% (egg), 63.0% and 75.5% (larvae), 84.3% and 94.3% (pupae), and 93.3% and 95.7% (adult), respectively (Table 1) using the statistical features in the classifier. With the addition of nine histogram features to the classification model, the kernels infested with larval stage could be classified with a classification accuracy of 66.3% and 77.3% using the LDA and QDA classifier, respectively (Table 1).
LDA
Classification accuracy (%)
100
QDA
80
Fig. 5. Classification accuracies of uninfested mung bean kernels and infested kernels (all stages combined) by Callosobruchus maculatus using statistical features (1100, 1290, and 1450 nm wavelengths) and histogram features (1100 nm wavelength) by LDA and QDA classifiers.
Four-way classification models were developed using the statistical features, to identify infested kernels with different stages of larval development. A total of 1200 kernels (300 kernels each per instar) were used in the classification. The highest classification accuracy was found in discriminating the kernels with fourth instar larvae with 69.0% and 80.3% accuracy using LDA and QDA classifiers, respectively using the combined features in the model development. The kernels with the other three stages of larval development were classified with classification accuracies of 54.3e 59.3% and 57.0e62.7% using LDA and QDA, respectively (Table 2) using both the features. Pair-wise classification models were developed to differentiate infested kernels by each stage of infestation from uninfested kernels. A total of 600 kernels (300 uninfested and 300 insect infested for each stage of infestation) were used in pair-wise classification. The kernels infested with egg, larval stages of development were correctly identified with 81.3e95.3% accuracy using both LDA and QDA classifiers (Table 3). Among all the stages of infestation, kernels with pupae and adult stages of infestation could be discriminated with the highest classification accuracies of >96.0% using both the statistical and combined features (Table 3). The addition of nine histogram features, improved the classification accuracy of kernels infested with different stages of development.
Table 1 Classification accuracies of the five-way classification model of uninfested mung bean kernels and those infested by different stages of C. maculatus using statistical features (1100, 1290, and 1450 nm wavelengths) and combined features (statistical and histogram features at 1100 nm) by non-parametric classifiers, N ¼ 300.
60
40
Stage
Classification by non-parametric classifiers (%) Classification by using combined features
Classification by using statistical features
20
0 Uninfested
Infested
Class Fig. 4. Two-way classification accuracies of uninfested mung bean kernels and infested kernels (all stages combined) by Callosobruchus maculatus using statistical features (1100, 1290, and 1450 nm wavelengths) by LDA and QDA classifiers.
Uninfested Egg Larvala Pupal Adult a
N ¼ 1200.
LDA
QDA
LDA
QDA
80.7 72.3 63.0 84.3 93.3
90.7 85.3 75.5 94.3 95.7
79.7 69.0 66.3 86.0 90.3
93.7 84.3 77.3 94.3 94.7
S. Kaliramesh et al. / Journal of Stored Products Research 52 (2013) 107e111 Table 2 Classification accuracies of the four-way classification model of mung bean kernels infested by different larval stages of C. maculatus using statistical features (1100, 1290, and 1450 nm wavelengths) and combined features (statistical and histogram features at 1100 nm) by non-parametric classifiers, N ¼ 300. Larval stage
Classification by non-parametric classifiers (%) Classification by using statistical features
Instar Instar Instar Instar
1 2 3 4
Classification by using combined features
Uninfested Egg Uninfested Instar 1 Uninfested Instar 2 Uninfested Instar 3 Uninfested Instar 4 Uninfested Pupae Uninfested Adult
kernels infested by different stages of C. maculatus. Higher classification accuracies were achieved in identifying the kernels with pupae and adult stages of infestation using both the classifiers. The three significant wavelengths used in the present study could be used for future online identification of mung bean kernels infested by C. maculatus by developing multi-spectral imaging systems. Acknowledgments
LDA
QDA
LDA
QDA
59.3 54.3 59.3 67.0
57.0 62.7 61.7 80.7
58.3 57.7 64.7 69.0
65.3 61.3 69.0 80.3
Table 3 Pair-wise classification accuracies of uninfested mung bean kernels and those infested by different stages of C. maculatus using statistical features (1100, 1290, and 1450 nm wavelengths) and combined features (statistical and histogram features at 1100 nm) by non-parametric classifiers, N ¼ 300. Stage
111
Classification by non-parametric classifiers (%) Classification by using statistical features
Classification by using combined features
LDA
QDA
LDA
QDA
89.0 91.0 85.7 84.0 76.0 85.0 88.3 89.3 92.3 81.3 99.7 96.0 99.7 99.7
93.3 88.0 90.7 90.0 88.3 88.3 91.3 93.0 99.3 86.0 100.0 99.0 98.7 99.3
89.7 91.3 85.7 85.3 78.7 84.0 90.0 89.3 91.0 84.3 100.0 96.0 99.7 99.0
96.7 86.7 75.4 89.7 88.0 90.0 90.7 95.3 97.7 89.7 98.7 99.0 99.7 99.0
4. Conclusion The near-infrared hyperspectral imaging can be used to identify the uninfested and the kernels infested with different stages of C. maculatus. In the NIR region, wavelengths of 1100, 1290, and 1450 nm were identified to be significant based on the highest factor loadings in first principal component and second principal component, respectively. The significant wavelengths were used in features extraction and classification. Average classification accuracies of 85% and 88% were obtained for the LDA and QDA, respectively, for identifying the uninfested mung bean samples. LDA and QDA classifier correctly identified more than 82% of
The authors thank Manitoba Innovation, Energy and Mines and Ministry of Food Processing Industries, India for funding this study and Canada Foundation for Innovation, Manitoba Research Innovation Fund and other industry partners for funding the acquisition of research infrastructure. References Ali, M., Gupta, S., 2012. Carrying capacity of Indian agriculture: pulse crops. Current Science 102, 874e881. Beck, C.W., Blumer, L.S., 2009. A Handbook on Bean Beetles, Callosobruchus maculatus. National Science Foundation (18.11.11.). http://www.beanbeetles.org/ handbook.pdf. Booker, R.H., 1967. Observations on three bruchids associated with cowpea in northern Nigeria. Journal of Stored Products Research 3, 1e15. Cogdill, R.P., Hurburgh, C.R., Rippke, G.R., 2004. Single-kernel maize analysis by near-infrared hyperspectral imaging. Transactions of the ASAE 47, 311e320. Gowen, A.A., O’Donnell, C.P., Cullen, P.J., Downey, G., Frias, J.M., 2007. Hyperspectral imaging: an emerging process analytical tool for food quality and safety control. Trends in Food Science and Technology 18, 590e598. Karunakaran, C., Jayas, D.S., White, N.D.G., 2003. X-ray image analysis to detect infestations caused by insects in grain. Cereal Chemistry 80, 553e557. Lee, K.J., Kang, S., Kim, M.S., Noh, S.H., 2005. Hyperspectral Imaging for Detecting Defect on Apples. ASABE Paper No. 053075. ASABE, St. Joseph, MI. Maghirang, E.B., Dowell, F.E., Baker, J.E., Throne, J.E., 2003. Automated detection of single wheat kernels containing live or dead insects using near-infrared reflectance spectroscopy. Transactions of the ASAE 46, 1277e1282. Mahesh, S., Jayas, D.S., Paliwal, J., White, N.D.G., 2008. Protein and Oil Constituents Determination in Wheat Using Near-infrared (NIR) Hyperspectral Imaging. ASABE Paper No. 084895. ASABE, St. Joseph, MI. Manickavasagan, A., Jayas, D.S., White, N.D.G., 2008. Thermal imaging to detect infestation by Cryptolestes ferrugineus inside wheat kernels. Journal of Stored Products Research 44, 186e192. Mehl, P.M., Chen, Y.R., Kim, M.S., Chan, D.E., 2002. Detection of defects on selected apple cultivars using hyperspectral and multispectral image analysis. Applied Engineering in Agriculture 18, 219e226. Mookherjee, P.B., Chawla, M.L., 1962. Effect of temperature and humidity on the development of Callosobruchus maculatus Fab., a serious pest of stored pulses. Indian Journal of Entomology 26, 345e351. Osborne, B.G., 2006. NIR spectroscopy in food analysis. In: Encyclopedia of Analytical Chemistry. John Wiley and Sons Limited, Chichester. Ridgway, C., Chambers, J., 1998. Detection of insects inside wheat kernels by NIR imaging. Journal of Near Infrared Spectroscopy 6, 115e119. Singh, C.B., Jayas, D.S., Paliwal, J., White, N.D.G., 2007. Fungal detection in wheat using near-infrared hyperspectral imaging. Transactions of the ASABE 50 (6), 2171e2176. Singh, C.B., Jayas, D.S., Paliwal, J., White, N.D.G., 2009. Detection of insect-damaged wheat kernels using near-infrared hyperspectral imaging. Journal of Stored Products Research 45, 151e158. Tang, J., Sokhansanj, S., 1991. Determination of moisture content of whole kernel lentil by oven method. Transactions of the ASAE 34, 255e256.