- Email: [email protected]
SWNIR spectroscopy and spectral scattering techniques
Journal of Food Engineering 125 (2014) 59–68
Contents lists available at ScienceDirect
Journal of Food Engineering journal homepage: www.elsevier.com/locate/jfoodeng
Grading of apples based on firmness and soluble solids content using Vis/SWNIR spectroscopy and spectral scattering techniques Fernando Mendoza a,⇑, Renfu Lu a, Haiyan Cen b a b
USDA/ARS, 524 S. Shaw Lane, Room 105A/224, Michigan State University, East Lansing, MI 48824, United States Department of Biosystems and Agricultural Engineering, Michigan State University, East Lansing, MI 48824, United States
a r t i c l e
i n f o
Article history: Received 28 September 2012 Received in revised form 22 August 2013 Accepted 17 October 2013 Available online 23 October 2013 Keywords: Apples Firmness Soluble solids content Sorting Near-infrared spectroscopy Spectral scattering
a b s t r a c t Sorting of apple fruit based on internal quality can enhance the industry’s competiveness and profitability and assure consumer satisfaction. In this research, visible and shortwave near-infrared (Vis/SWNIR) spectroscopy (460–1100 nm) and spectral scattering (450–1050 nm) were used for sorting three varieties of apple (i.e., ‘Delicious’, ‘Golden Delicious’ and ‘Jonagold’) into two quality grades based on firmness, soluble solids content (SSC), or both firmness and SSC. Vis/SWNIR spectra were obtained in interactance mode under stationery condition, whereas spectral scattering images were acquired online at a conveyor speed of 82 mm/s. A total of 8491 apples for the three varieties harvested from the same orchard in 2009, 2010 and 2011 were used for analysis. First derivative spectra were obtained from the Vis/SWNIR data, while the scattering images were first preprocessed by computing mean reflectance spectra and then performing continuous wavelet transform decompositions. Sorting algorithms were developed using sequential forward selection and linear discriminant analysis, and the classification models were compared in terms of their overall performance and local confusion matrices. A sensitivity analysis was also performed to assess the effect of using different quality threshold criteria on the sorting performance. Overall, consistent and relatively good sorting results for firmness (ranging between 77.9% and 98.2%) and moderate results for SSC (ranging between 62.0% and 91.7%) were obtained using scattering technique. Vis/SWNIR technique showed slightly better sorting results for firmness (ranging between 87.3% and 97.6%) and SSC (ranging between 77.1% and 92.3%). When the classification was performed based on both firmness and SSC, the sorting accuracies generally decreased to between 75.7% and 90.1% for Vis/SWNIR and between 69.7% and 91.5% for spectral scattering. Vis/SWNIR and spectral scattering techniques have potential for online sorting and grading of apples by firmness and SSC. Published by Elsevier Ltd.
1. Introduction Apple is an important agricultural commodity in the global fresh produce market. Consumer choice of apples is driven by the trade-off between price and quality (Harker et al., 2003). Today consumers are demanding better quality, more consistent apples with appropriate taste and texture. Enhancement in fruit quality could thus lead to an increased demand and repeat purchases by the consumer, and increase profit margins for the industry through price differentiations for different quality grades of apples. Hence, appropriate quality control and inspection procedures need to be implemented to sort and grade each individual fruit for internal quality. Currently, apples are sorted, using machine vision systems, mainly by color, shape, and size or weight, but not on internal quality. A study by Harker (2001) found that about 70% US ⇑ Corresponding author. Tel.: +1 517 432 7438; fax: +1 517 432 2892. E-mail address: [email protected] (F. Mendoza). 0260-8774/$ - see front matter Published by Elsevier Ltd. http://dx.doi.org/10.1016/j.jfoodeng.2013.10.022
consumers considered the eating quality to be the most important factor in purchasing apples. Texture and taste define the eating quality of apples, and among them, texture seems to be a primary quality attribute which, together with taste and appearance, should be taken into account when evaluating the overall quality of the fruit (Tu and Baerdemaeker, 1996; Mann et al., 2005). Fruit texture is determined by firmness (force required to bite into the fruit), crispness (amount and pitch of sound generated when the fruit is first bitten with the front teeth), juiciness (amount of juice released from the fruit in the first three chews, when chewing with the back teeth), and mealiness (degree to which the flesh breaks down to a fine lumpy mass) (Mann et al., 2005; Péneau et al., 2006). Apples that are crispy, firm, and juicy are highly favored by consumers (Harker et al., 2008). Those sensory attributes are related to the structure and turgor pressure of fruit tissue (Tong et al., 1999). The taste properties that apply to apples include sweetness, acidity, and astringency. Consumers are capable of distinguishing all these attributes with a high degree of sensitivity (Harker, 2001). Therefore, there is much incentive to
60
F. Mendoza et al. / Journal of Food Engineering 125 (2014) 59–68
sort and grade fruit based on their internal quality for the fresh market. Nondestructive measurement of flesh firmness and sweetness would allow the apple industry to deliver superior, consistent fruit to the marketplace and ensure consumer acceptance and satisfaction (Peng and Lu, 2005). The demand for high-quality fruit calls for reliable and rapid sensing technologies for the nondestructive measurement and sorting of apples. Among many nondestructive sensing techniques that have been developed, optical techniques, especially visible and near-infrared (Vis/SWNIR) spectroscopy and spectral scattering, show great potential for sorting and grading apples for internal quality. Vis/SWNIR measures an aggregate amount of light reflected back from or transmitted through the sample, which is then used to predict certain chemical constituents. The technique, coupled with an appropriate predictive model, has been successfully applied for nondestructive SSC assessment of various fruits (Lu, 2001, 2004; Nicolaï et al., 2007). Spectral scattering technique, however, enhances the measurement of light scattering in the fruit, a physical phenomenon that is dependent on the density, cell structures, and cellular matrices of fruit tissue. With spectral scattering technique, a small continuous-wave light beam is incident on a fruit, generating scattering images at its surface around the incident point as the result of light propagation and backscattering inside the fruit (Lu, 2007). A multi- or hyperspectral imaging system is used to capture backscattering images from the fruit at selected wavelengths or over a specific spectral region. Thereafter, spectral scattering features are extracted from the scattering images for prediction of fruit firmness and SSC using appropriate mathematical models (Mendoza et al., 2011). The objective of this research was to independently assess both visible and shortwave near-infrared (Vis/SWNIR) spectroscopy and spectral scattering techniques for sorting apples into two quality groups by firmness, SSC, or their combination using discriminant analysis. For testing the robustness of the classification models, a sensitivity analysis was performed using different quality threshold criteria for automatic sorting and three varieties of apple (i.e., ‘Jonagold’, ‘Golden Delicious’, and ‘Delicious’) harvested in 2009, 2010 and 2011 with different range of variability for firmness and SSC were evaluated in this study. 2. Materials and methods 2.1. Instrument setup Two nondestructive sensing techniques, i.e., Vis/SWNIR spectroscopy and spectral scattering, for measuring the firmness and SSC of apple fruit were used in this study, and they are described in the following sections. 2.1.1. Visible and shortwave near-infrared (Vis/SWNIR) spectroscopy A miniature Vis/SWNIR spectrometer (USB4000, Ocean Optics, Dunedin, Florida, USA) was used to acquire spectra from apple fruit between 460 nm and 1100 nm in an interactance mode. The DC power supply for a quartz tungsten halogen lamp (Oriel Instruments, Stratford, CT) was regulated with an intensity controller, and the light was delivered to the fruit through a ring guide of 25 mm in diameter. Mounted in the center of the light guide was a beam collimator, which collected the light that has interacted with the fruit tissue and reemerged from an area of 10 mm, and then focused it onto a receiving optic fiber that was connected to the spectrometer. A sufficient buffering zone (about 8 mm) was established between the illumination area and the detection area on the fruit to ensure that the receiving fiber only collected the light that has interacted with the flesh tissue. Based on the preliminary test, the lamp power supply was set to 100 W and the
integration time of the spectrometer was set to 100 ms. This setting allowed acquisition of good spectral signals for all tested apples, while avoiding light saturation due to overexposure. A graphic description of the Vis/SWNIR instrument is given in Mendoza et al. (2012). 2.1.2. Online hyperspectral scattering system (OHSS) A prototype online hyperspectral scattering system (OHSS) was used for acquiring spectral scattering images of apples. This system consisted of a back-illuminated electron-multiplying CCD (EMCCD) camera (Model PhotonMAX: 1024B Air-Cooled, Princeton Instruments, Trenton, NJ, USA) with a 1024 1024 13-lm pixel frametransfer CCD detector, an imaging spectrograph (ImSpector V10E, Spectral Imaging Ltd., Oulu, Finland) covering the spectral region of 400–1000 nm with 2.8 nm spectral resolution, a focusing lens with a near-infrared enhanced lens, and a quartz tungsten halogen light source (Oriel Instruments, Stratford, Conn., USA) with an intensity controller. The incident point light beam was 1.5 mm in size and the line scanning position of the camera was 1.6 mm off from the beam incident center to avoid saturation. The EMCCD camera with dual amplifiers was set to operate at 10 MHz for high-speed imaging using an avalanche gain of 2950, and it was equipped with a deep thermoelectric cooling system operating at 55 °C to suppress system noise. The test apples, which were placed on cups with the stem-calyx axis being horizontal and perpendicular to the moving direction, were transported on a 2.4 m conveyor at a speed of 82 mm/s (1 fruit for 2 s). The OHSS began to capture and save spectral scattering images from the equatorial area of each apple immediately after the two photoelectric beam sensors were triggered by the incoming cup and apple, respectively. The integration time was set at 120 ms for each apple and the acquired images were binned 4 4 to yield the final images of 256 256 pixels with the spatial resolution of 0.20 mm/pixel and the spectral resolution of 4.54 nm/ pixel. A detailed description of the OHSS is given in Mendoza et al. (2011, 2012). 2.2. Experimental procedure ‘Jonagold’ (JG), ‘Golden Delicious’ (GD) and ‘Delicious’ (D) apples were harvested in 2009, 2010 and 2011 from an orchard of Michigan State University’s (MSU) Clarksville Horticultural Experiment Station in Clarksville, MI. In order to have a wider range of physiological condition for the studied varieties, the harvest was carried out over six consecutive weeks for 2009 and 2010, and over four consecutive weeks for 2011. One day after each harvest, a set of 100 apples for each variety were tested, and the rest apples were stored in refrigerated air at 0 °C. One week after the last harvest, tests for the stored apples were started. The number of tested apples for each variety and testing date varied between 60 and 100 fruit, depending on the condition of the fruit (i.e., firmness and SSC) at the time of testing, which was monitored by testing a few samples destructively prior to the experiment. The stored apples for 2009 were tested once a week for the first 6 weeks and then every 2 weeks for up to 8 weeks. The experiment was completed in 5 months for 928, 1176 and 1091 apples of JG, GD and D, respectively. The stored apples for 2010 were tested once a week for the first 4 weeks and then every 2 weeks for up to 6 weeks; the experiment was completed in 4 months for 1037, 1132 and 1102 apples for JG, GD and D, respectively. Similarly, the stored apples for 2011 were tested once a week for the first 2 weeks and then every 2 weeks for up to 4 weeks; the experiment was completed in two and one half months for 640, 825 and 560 apples for JG, GD and D, respectively. After the nondestructive measurements using the two sensing techniques had been completed, the firmness and SSC of the tested
F. Mendoza et al. / Journal of Food Engineering 125 (2014) 59–68
apples were measured, using standard destructive methods, from the same location where Vis/SWNIR spectroscopy and scattering measurements had been carried out. Magness–Taylor (MT) firmness (N) measurements were performed using a Texture Analyzer (model TA.XT2i, Stable Micro Systems, Inc., Surrey, United Kingdom) with a steel probe of 11 mm diameter for a penetration depth of 9 mm at a loading speed of 2 mm/s. Maximum force recorded on the force/displacement curve was used as the measure of fruit firmness. Immediately after the firmness measurements, juice was extracted from the fruit and its SSC (°Brix, expressed as %) was then measured using a digital refractometer (Model PR-101, Atago Co., Tokyo, Japan) with the accuracy of ±0.2%. The test apples had been kept at room temperature (22 °C) for at least 16 h to allow them to reach equilibrium before the optical measurements and firmness and SSC measurements were carried out. The temperature of fruit at the time of testing could affect MT firmness/SSC and optical measurements, which in turn would influence the model prediction of firmness and SSC. Hence by maintaining approximately the same temperature for all test samples, we would minimize the temperature effect on the firmness and SSC prediction. However, under the commercial condition, it may not be easy to maintain all fruit at an equilibrium temperature. Hence it would be necessary to consider the temperature effect in the prediction/classification model, which is beyond the scope of the current study. 2.3. Extracted features Dark and reference corrections were first performed on the Vis/ SWNIR spectral data covering the spectral range of 460–1100 nm at increments of 1 nm, yielding each spectrum of 641 wavelengths. First derivatives were then used to minimize the baseline offset and enhance absorbance peaks. It should be noted that preliminary analysis for the Vis/SWNIRS data showed that better classification results were obtained using first derivatives compared to smoothed relative reflectance (Nicolaï et al., 2007), second derivative and continuous wavelet transform preprocessing methods (Mendoza et al., 2011, 2012). Hence, this paper only reports the results from the first derivative spectral data. Scattering features were extracted from the preprocessed scattering images covering the spectral range of 450–1050 nm at increments of 5 nm, yielding each spectrum of 121 wavelengths and a spatial range of 0–10 mm. Mean reflectance spectra (OHSSmean) were first computed (Lu, 2007), followed by the multi-resolution wavelet transform method based on continuous one-dimensional decomposition using the symlet2 wavelet at 64-scales. The resultant vector of scattering features after applying continuous wavelet transform (OHSScwt) represents the root mean square of the transformed data at each single wavelength. It should be noted that the selected wavelength range (i.e., 450–1050 nm) and increment (i.e., 5 nm) and the scattering distance (i.e., 10 mm) for extracting scattering features were determined after preliminary factorial analysis for selecting the best processing conditions had been performed (not reported here). The details of calculating these spectral features can be found in Mendoza et al. (2011). The extracted features (n) from each set of samples (N) were arranged in the ith row of matrix F: [Fi1, . . . , Fin] that corresponds to a point in the n-dimensional measurement feature space. Prior to the classification analysis, the sets of features, i.e., Vis/SWNIR, OHSSmean, and OHSScwt were normalized, yielding a N n matrix Rnormij whose elements are defined as (Mery et al., 2012):
Rnormij ¼
Rij lj
rj
ð1Þ
61
for i = 1, . . . , N and j = 1, . . . , n, where Rij denotes the jth feature of the ith feature vector, N is the number of samples, and lj and rj are the mean and standard deviation of the jth feature, i.e., the jth column of R. The normalized features have 0 mean and a standard deviation equal to 1. Thus 641 Vis/SWNIR features, 121 OHSSmean and 121 OHSScwt scattering features from each apple were extracted and used for further sorting analyses. 2.4. Quality grading criteria In practical situations, sorting for internal quality would be carried out for those fruit that have met the minimum quality grades for size, weight, and color and free of external defects. Because of a large number of grade combinations for color and size, it is necessary to limit the number of quality grades for firmness and/or SSC so as to lower the packing cost. Two situations may be encountered in the packinghouse with regard to firmness or SSC: first, the packers would like to remove those apples that do not meet the minimum requirement in firmness, SSC or in both; and second, they may want to pack apples with superior quality, so that these apples can be sold at a premium price. For these reasons, this research only considered sorting apples into two quality grades based on firmness, SSC, or both firmness and SSC. The threshold values for firmness and SSC were defined as follows: MT firmness of 60 N and SSC of 12 °Brix for JG, MT firmness of 55 N and SSC of 12 °Brix for GD, and MT firmness of 60 N and SSC of 11 °Brix for D. The selection of the threshold values was based on the recommendations from published studies and also through our interaction with the apple industry in the US Published recommendations on the minimum firmness for apples vary between 49.0 N for ‘Golden Smoothee’ and 62.0 N for ‘Delicious’ apples, and the minimum SSC between 11.6% and 14% for different varieties of apple (Molina et al., 2006; Harker, 2001; Harker et al., 2008). The selected thresholds fall between these ranges for firmness and around the minimum value for SSC. The lower SSC threshold of 11% Brix was selected for D apples so that there were enough samples with the SSC of <11% Brix. It should be pointed out that sorting apples into the two quality grades was intended to assess and demonstrate the capability of the technologies. In practice, packinghouses also need to consider such factors as market demand and fruit inventory when deciding on how to sort fruit or set the sorting criteria. Additionally, a sensitivity analysis to test the robustness of the classification models using different quality threshold criteria for automatic apple sorting was performed for MT firmness and SSC models. Considering that only datasets with a large range of variability could be useful for the test, only apple samples from 2009 to 2010 which had the largest variability were analyzed and compared based on the performance of classification at each tested threshold criteria. The tested quality threshold for MT firmness ranged in steps of 5 N from 50 N to 75 N for JG (2010) and GD (2009), and from 50 N to 70 N for D (2010) apples; and for SSC ranged in steps of 0.5% from 11.5% to 13.5% for JG (2010), from 12% to 14% for GD (2010), and from 11% to 12.5% for D (2009). All sorting tests were performed using the same procedure explained below. 2.5. Classification by linear discriminant analysis (LDA) Classification is the process of putting objects into a class or group according to particular characteristics based on a reduced set of useful information. This means sorting apples by firmness, SSC or their combination using only the best features extracted from the preprocessed Vis/SWNIR and scattering data (OHSSmean or OHSScwt). For this, multivariate analysis was performed using the codes adapted from Balu Matlab Toolbox (Mery, 2011) run in MATLAB 7.5.0 (The MathWorks, Inc., Natick, MA, USA). The best
62
F. Mendoza et al. / Journal of Food Engineering 125 (2014) 59–68
subset of features (that leads to the smallest classification error) was found using the sequential forward selection (SFS) algorithm, and the classification was performed using linear discriminant analysis (LDA). SFS starts with the best single feature and then adds one new feature each time to the features that have been already selected from the previous runs, to maximize the classification performance. The iteration is halted after the model no longer shows improvement with the addition of a new feature (Jain et al., 2000). LDA is a statistical technique for classifying individuals or objects into mutually exclusive groups on the basis of a set of independent variables. LDA involves deriving linear combinations of the independent variables that will discriminate between the a priori defined groups in such a way that the misclassification error rates are minimized. This is accomplished by maximizing the between-group variance relative to the within-group variance. The goal is to find the allocation rule which gives the highest percentage of correct classification (Dillon and Goldstein, 1984). The optimum number of latent variables for the classifier is determined using cross validation, a technique widely used in machine learning problems, to avoid both over-fitting and under-fitting problems. Finally, the performance of the classifier is tested for an independent set of samples. In our experiments, this procedure was repeated four times through rotating the training and test data; each time 75% of the data was used to train and the rest (25%) for test. Only the average training and testing results from the four runs were used in further statistical analysis. This would ensure more consistent evaluation of the classifier, since its performance is influenced, to a certain extent, by the way how training and testing samples are selected. For analysis and comparison, the classification models were independently derived for each sensing technique, variety and harvest season. It was noticed that the number of samples above or below the threshold value of firmness or SSC varied for the three varieties and harvest season (see Fig. 1 and further discussion in Section 3.1). To avoid bias in the model development due to imbalance in the number of samples for each quality grade group, 300 apples were randomly selected for each firmness or SSC group, except for D apples in 2011, where only 250 apples for each group were selected for analysis because of the smaller number of apples for that year. Hence there were a total of 600 apples (or 500 for D in 2011) used in each case for the model development. This sampling procedure was repeated six times; for each of the six sampling data sets, the same calibration and testing procedure, as described earlier, was followed. The average values for latent variables for the models, their overall performance (i.e., accuracy of the classification for the testing set of samples) and local confusion matrices from the six sampling data sets were finally compiled for evaluation of the two sensing techniques for the three sorting scenarios (by firmness, SSC, or their combination) for each variety and harvest season.
3. Results 3.1. Distribution of firmness and SSC measurements by standard destructive methods Heterogeneous, non-normal distributions for MT firmness were observed for the three apple varieties in all harvest seasons (Fig. 1). With the exception of D apples for 2010 and 2011, the MT-firmness distributions appear largely bimodal with a gap between 50 to 70 N, which could have been attributed to faster physiological and microstructural changes in apples when they reached this range of firmness. JG apples from 2009 to 2010 showed the largest range of firmness with the standard deviations of 22.8 and 19.0 N, respectively, followed by GD and D apples whose standard
deviation values were similar for the same harvest seasons, ranging from 15.5 to 16.6 N for GD and from 15.8 to 16.6 N for D. Overall apples for 2011 showed the smallest range of firmness values since the harvesting and testing period of time was shorter than that for 2009 and 2010. The inclusion of 2011 apples in this study was to assess the robustness of the classifier with a smaller range of firmness and SSC values. The standard deviations for JG, GD and D apples in 2011 were of 12.5, 13.7 and 7.1 N, respectively. Firmer apples were found for JG from 2009 and D from 2010. Prediction and classification models are typically built using multivariate statistical methods whose performance generally depends on the characteristics of the data. Recent studies in other areas as soil science (Kuang and Mouazen, 2011) and our own experience in processing apple data (Mendoza et al., 2011, 2012) have shown that the number of samples (size and/or variability) and its density distribution are frequently factors that tend to affect the robustness and accuracy of the calibration models developed. Contrarily, the SSC was fairly normally distributed with the exception of JG and D apples from 2010, whose distributions markedly deviated from normal (Fig. 1). The standard deviation ranged from 1.0% for D apples in 2011 to 1.6% for JG and GD apples in 2010 and 2009, respectively. On average, GD apples were sweeter (ranging from 7.9 to 19.7 °Brix), followed by JG (ranging from 8.3 to 17.6 °Brix), and then D (ranging from 8.9 to 17.0 °Brix). 3.2. Classification results 3.2.1. Classification by MT firmness Table 1 summarizes the average number of variables, overall performance and standard deviation for the testing sets (i.e., from 6 repetitions) using MT firmness as a classification criterion for JG, GD and D apples harvested in the three seasons. Using Vis/SWNIR and OHSS techniques, the best classification was obtained for JG, followed by GD and D, although in most of the cases, the performance for the same varieties varied significantly among the harvest seasons (p < 0.05). Among the three harvest seasons and for both sensing techniques, the model accuracy was more variable for apples harvested in 2010 for all varieties, as indicated by their standard deviations (Table 1). GD and D apples showed larger variability in the model performance for the harvest seasons of 2009 and 2010. The accuracy of the classification models for OHSS was improved by implementing the continuous wavelet transform in the data preprocessing (i.e., OHSScwt). Significant improvements (p < 0.05) were observed for all varieties with 0.8%, 2.1% and 1.2% for JG and 5.5%, 11.9%, and 2.6% for GD for the three harvest seasons, respectively, and 11.4% and 6.9% for D for 2009 and 2010, respectively. On the other hand, comparing both sensing techniques, in most cases, Vis/SWNIR performed significantly better than OHSScwt (p < 0.05). For the model calibration, the number of variables for all varieties and harvest seasons ranged from 3 to 19 when either Vis/SWNIR or OHSScwt was used. In previous studies, scattering technique gave better prediction results than Vis/ SWNIR for predicting apple firmness, because the former, which was also tested under stationary condition, is able to better quantify the light scattering features in the fruit (Lu and Peng, 2005; Nicolaï et al., 2007). The unexpected result can be explained by the fact that scattering technique was operated online, while Vis/ SWNIR technique was in offline mode. Under the online condition, it was more difficult to accurately position each individual apple, which is important for both Vis/SWNIR and scattering techniques. In addition, only a simple mean spectra calculation method was used to extract features from the spectral scattering data in this study, as compared to the more sophisticated spectral and imaging analysis methods used in our previous studies (Mendoza et al., 2011).
63
F. Mendoza et al. / Journal of Food Engineering 125 (2014) 59–68
(a)
(b)
Fig. 1. Distributions of (a) Magness–Taylor (MT) firmness and (b) soluble solids content for ‘Jonagold’ (JG), ‘Golden Delicious’ (GD) and ‘Delicious’ (D) apples harvested in 2009, 2010, and 2011.
Table 1 Overall testing results for classification of ‘Jonagold’, ‘Golden Delicious’ and ‘Delicious’ apples into two grades of firmness for three harvest seasons, using Vis/SWNIR and online spectral scattering techniques.A Jonagold
Golden Delicious
Delicious
No.Bvariables
Accuracy (±std.) (%)
No.Bvariables
Accuracy (±std.) (%)
No.Bvariables
Accuracy (±std.) (%)
17 19 18
97.6 (0.1)a 90.8 (0.7)b 87.3 (0.6)c
19 19 18
89.6 (0.9)a 90.5 (1.5)a 92.0 (0.4)b
19 18 –
89.6 (0.9)a 87.6 (1.3)b –
OHSSmeanD 2009 2010 2011
17 15 12
97.4 (0.3)a 87.3 (1.1)c 92.1 (0.5)d
7 14 12
85.4 (1.5)c 79.5 (1.7)d 89.1 (0.2)e
3 13 –
77.9 (1.2)c 79.5 (1.5)d –
OHSScwtE 2009 2010 2011
13 18 16
98.2 (0.3)e 89.1 (0.6)f 93.2 (0.3)g
19 17 17
90.1 (1.0)a 88.9 (1.1)e 91.4 (0.2)f
18 18 –
86.8 (2.1)b 85.0 (1.4)e –
Vis/SWNIRS 2009 2010 2011
C
Note: In bold are the average classification performance or accuracy of the linear discriminant models over six replicates for the testing set of samples Values in the same columns with different letters are statistically different (p < 0.05). A The firmness threshold values for ‘Jonagold’, ‘Golden Delicious’ and ‘Delicious’ were 60 N, 55 N, and 60 N, respectively. B Average number of features used by the linear discriminant analysis model over six replicates. C Vis/SWNIRS: First derivative of the Vis/SWNIR data for the wavelengths of 460–1100 nm. D OHSSmean: Mean reflectance spectra for the wavelengths of 450–1050 nm. E OHSScwt: Continuous wavelet transform of the spectral scattering data for the wavelengths of 450–1050 nm.
a,b,c,d,e,f,g
64
F. Mendoza et al. / Journal of Food Engineering 125 (2014) 59–68
Table 2 Overall validation or testing results for classification of ‘Jonagold’, ‘Golden Delicious’ and ‘Delicious’ apples into two grades of soluble solids content for three harvest seasons, using Vis/SWNIR and online spectral scattering techniques.A Jonagold
Golden Delicious
Delicious
No.Bvariables
Accuracy (±std.) (%)
No.Bvariables
Accuracy (±std.) (%)
No.Bvariables
Accuracy (±std.) (%)
19 20 18
82.0 (1.3)a 88.7 (0.6)b 83.6 (0.9)c
18 18
– 86.0 (1.6)a 86.0 (0.4)a
18 20 18
77.1 (1.1)a 92.3 (0.7)b 83.9 (1.4)c
OHSSmeanD 2009 2010 2011
16 10 13
78.3 (0.1)d 84.8 (1.6)e 62.9 (0.8)f
– 14 4
– 74.7 (1.6)c 63.8 (1.0)d
3 13 10
68.7 (0.8)d 88.7 (1.1)e 79.4 (0.8)f
OHSScwtE 2009 2010 2011
17 13 8
81.1 (1.4)a 86.0 (1.4)e 62.0 (1.0)f
– 17 13
– 79.3 (0.6)e 64.9 (1.3)d
15 17 16
77.0 (2.0)a 91.7 (1.0)b 81.4 (0.4)g
Vis/SWNIRS 2009 2010 2011
C
Note: In bold are the average classification performance or accuracy of the linear discriminant models over six replicates for the testing set of samples a,b,c,d,e,f,g Values in the same columns with different letters are statistically different (p < 0.05). A The threshold values for soluble solids content for ‘Jonagold’, ‘Golden Delicious’, and ‘Delicious’ were 12 °Brix, 12 °Brix, and 11 °Brix, respectively. B Average number of features used by the linear discriminant analysis model over six replicates. C Vis/SWNIRS : First derivative of the Vis/SWNIRS data for the wavelengths of 460–1100 nm. D OHSSmean: Mean reflectance spectra for the wavelengths of 450–1050 nm. E OHSScwt: Continuous wavelet transform of the spectral scattering data for the wavelengths of 450–1050 nm.
3.2.2. Classification by SSC Likewise, Table 2 summarizes the overall performance for the validation or testing sets using SSC as the classification criterion for JG, GD and D apples harvested in 2009, 2010, and 2011. The correct classification rates in separating apples by SSC were in general lower than those sorted by firmness, with the exception of D apples from 2010. This could be explained by the relatively tighter range of the SSC data in comparison with the MT firmness data (Fig. 1). The accuracy of the classification models for SSC using OHSS was slightly improved by implementing the continuous wavelet transform (i.e., OHSScwt). In this case, significant improvements (p < 0.05) were only found for JG and GD apples at 3.6% for 2009 and 6.2% for 2010, respectively, and for D apples at 12.2%, 3.4%, and 2.5% for 2009, 2010, and 2011, respectively. Vis/SWNIR was significantly better (p < 0.05) than OHSScwt for the SSC classification for almost all seasons and apple varieties. The lowest classification rates of 62.0% and 64.9% for JG and GD, respectively, using OHSScwt were obtained for 2011 (Table 2). However, using Vis/SWNIR for the same varieties and season, the classification rates were improved by 25.8% for JG and by 24.5% for GD apples. These results confirmed that Vis/SWNIR performed better than spectral scattering technique in predicting and grading apple fruit by SSC. 3.2.3. Effect of the quality threshold value on the performance of the classification models Fig. 2 depicts the behavior of the classification models for selected harvest seasons (i.e., datasets with the largest variability) when different quality threshold values were used for analysis. With some exceptions for D apples, overall results showed consistent performance (%) for all cultivars in a wide range of quality threshold values, but they were in most of the cases significantly different among harvest seasons, sensing techniques and/or preprocessing methods. For firmness, JG apples showed the most consistent performance in all tested range (from 50 N to 75 N), followed by GD apples with a consistent performance in the range of 55 N to 70 N. The performance for RD apples, however, showed a tendency to increase when the threshold values are increased; although a steady range was observed in between 55 N and 60 N. Classification performances based on SSC showed that JG and GD apples were more
consistent for quality threshold values higher than 12% Brix onwards in the tested range for each variety. Classification of RD apples was consistent in all the tested range of threshold values (from 11% to 12.5%) using Vis/SWNIR, and from 11% Brix onwards using OHSS. No significant variations in the standard deviations were observed among the tested quality thresholds in all apple varieties. Furthermore, similar behaviors for firmness and SSC were observed for the other harvest seasons do not presented here. 3.2.4. Classification by MT firmness and SSC Since continuous wavelet transform performed consistently better in preprocessing the spectral scattering data for sorting apples by firmness or SSC (Tables 1 and 2; and Fig. 2), only results from the OHSScwt data are presented here. Table 3 summarizes the classification results for the validation sets using firmness and SSC as the grading criterion for JG, GD and D apples harvested in the three seasons. In general, the classification accuracies were lower than those sorted by MT firmness alone, but, in a few cases, were higher than those sorted by SSC. Similar to sorting for single quality parameters, the accuracy of the classification models involving both firmness and SSC was significantly better (p < 0.05) with Vis/SWNIR technique than using the OHSS technique except for JG and GD for 2009. The classification results for the two quality attributes tended to be more variable than that for single attributes. The standard deviation of the accuracy rates from the six runs for all varieties and harvest seasons using Vis/SWNIR was in most cases smaller (ranging from 0.9% to 1.9%) in comparison with that for OHSScwt (ranging from 0.3% to 2.6%), which could be explained by the fact that the scattering technique was performed online, while Vis/SWNIR was performed in stationary condition. On the other hand, the number of variables for the calibration models for the three varieties and harvest seasons was larger for Vis/SWNIR (ranging from 15 to 20) than for OHSScwt (ranging from 13 to 18). It is important to note that the interaction of light with apple tissue for prediction of internal quality is a complicated process, which is affected by many factors including tissue microstructure, physicochemical process, variety, and season (Cen et al., 2009). This may explain why larger numbers of latent variables (up to 20) were chosen in some of the calibration models. When both firmness and sweetness were used for quality classification, the classification algorithm became more
65
F. Mendoza et al. / Journal of Food Engineering 125 (2014) 59–68
(a) Jonagold, 2010
(d) Jonagold, 2010 pMeR
pCWT
NIR
100% 95% 90% 85% 80% 75% 70% 50
55
60
65
70
75
Classification performance
Classification performance
NIR
(b) Golden Delicious, 2009
85% 80% 75% 70% 65
70
75
Quality thresholds for firmness (N)
(c) Delicious,2010 pMeR
12.0
12.5
13.0
13.5
pMeR
pCWT
100% 90% 80% 70% 60% 50% 12.0
12.5
NIR
pCWT
95% 90% 85% 80% 75% 70% 55
50%
13.0
13.5
14.0
(f) Delicious, 2009
100%
50
60%
Quality thresholds for soluble solids content (%)
60
65
70
75
Quality thresholds for firmness (N)
Classification performance
Classification performance
NIR
70%
NIR
90%
60
80%
pCWT
95%
55
90%
(e) Golden Delicious, 2010
100%
50
100%
11.5
Classification performance
Classification performance
pMeR
pCWT
Quality thresholds for soluble solids content (%)
Quality thresholds for firmness (N)
NIR
pMeR
pMeR
pCWT
100% 90% 80% 70% 60% 50% 11.0
11.5
12.0
12.5
13.0
Quality thresholds for soluble solids content (%)
Fig. 2. Classification performance of apples by firmness (a–c) and SSC (d–f) into two quality groups using Vis/SWNIR and online spectral scattering (OHSSmean and OHSScwt) techniques when different quality threshold values are used for analysis.
complex, thus resulting in higher classification error and variability. For further evaluation of the classification performance for both firmness and SSC, Tables 4 and 5 present the average confusion matrices for the validation sets using Vis/SWNIR and spectral scattering techniques. It is noted that in most cases, better classification was achieved for apples ‘Regular’ grade apples than for ‘Premium’ grade apples, when either technique was used. Overall larger numbers of false ‘Premium’ apples were obtained using the OHSScwt technique. Among the false ‘Premium’ apples, the classification algorithm mainly failed to discriminate apples with a high level of SSC; i.e., the misclassified ‘Premium’ grade apples were more likely to be firm but low in SSC. In multivariate analysis with a large set of features, it is of particular interest to know the distribution of the most significant
features involved in the calibration model. The key idea is that a subset of selected features in combination with the selected classifier could perform accurately to solve a specific classification task. Table 6 shows the 10 best selected spectral features using Vis/ SWNIR and OHSScwt techniques for sorting JG, GD and D apples based on firmness and SSC as a selection criterion. These features were determined using the SFS analysis, corresponding to one of the six repetitions for each cultivar at different harvest seasons. In general, the best selected features were distributed along the entire range of wavelengths for both Vis/SWNIR and scattering techniques. It was noted that although some variability in the wavelength selection existed among the six runs for each variety and season, a high degree of consistency for the selected wavelengths was observed. Using Vis/SWNIR technique, no specific patterns of selected wavelengths were evident among the varieties
66
F. Mendoza et al. / Journal of Food Engineering 125 (2014) 59–68
Table 3 Overall validation or testing results for classification of ‘Jonagold’, ‘Golden Delicious’ and ‘Delicious’ apples into ‘Premium’ and ‘Regular’ grades by firmness and soluble solids content for three harvest seasons, using Vis/SWNIRS and spectral scattering techniques.A. Jonagold
Vis/SWNIRS 2009 2010 2011 OHSScwtD 2009 2010 2011
Golden Delicious
Delicious
No.Bvariables
Accuracy (±std.) (%)
No.Bvariables
Accuracy (±std.) (%)
No.Bvariables
Accuracy (±std.) (%)
17 17 19
89.2 (1.6)a 80.9 (1.5)b 85.5 (1.9)c
18 19 18
85.5 (1.2)a 82.1 (1.2)b 90.1 (1.0)c
20 19 15
78.7 (1.8)a 78.4 (1.3)a 75.7 (0.9)b
13 14 17
91.5 (0.7)d 78.4 (1.3)e 74.7 (2.6)f
17 15 15
88.6 (1.7)d 76.5 (2.4)e 86.5 (0.6)a
18 17 13
69.7 (2.1)c 74.5 (1.9)d 75.9 (0.3)b
C
Note: In bold are the average classification performance or accuracy of the linear discriminant models over six replicates for the testing set of samples a,b,c,d,e,f,g Values in the same columns with different letters are statistically different (p < 0.05). E OHSScwt: Continuous wavelet transform of the spectral scattering data for the wavelengths of 450–1050 nm. A Apples were graded as ‘Premium’ if: Magness–Taylor (MT) firmness P 60 N and soluble solids content (SSC) P 12 °Brix for ‘Jonagold’, MT P 55 N and SSC P 12 °Brix for ‘Golden Delicious’, and MT P 60 N and SSC P 11 °Brix for ‘Delicious’; they would be otherwise graded as ‘Regular’. B Average number of features used by the linear discriminant analysis model over six replicates. C Vis/SWNIRS : First derivative of the Vis/SWNIRS data for the wavelengths of 460–1100 nm. D OHSSmean: Mean reflectance spectra for the wavelengths of 450–1050 nm.
Table 4 Average confusion matrices for the classification of ‘Jonagold’, ‘Golden Delicious’, and ‘Delicious’ apples in the validation set by Magness–Taylor (MT) firmness and soluble solids content (SSC) using Vis/SWNIRS data. Confusion matrix (%) Jonagolda Premium
d
Golden Deliciousb d
d
Deliciousc d
Regular
Premium
Regular
Premiumd
Regulard
2009 Premiume Regulare
85.5 14.5
6.0 94.0
84.4 15.6
13.5 86.5
76.7 23.3
19.7 80.3
2010 Premiume Regulare
78.6 21.4
17.8 82.2
80.8 19.2
16.7 83.8
78.7 21.3
22.4 77.6
2011 Premiume Regulare
79.3 20.7
11.8 88.2
89.2 10.8
9.5 90.5
75.9 24.1
25.0 75.0
Note: In bold are the average classification performance or accuracy of the linear discriminant models over six replicates for the testing set of samples a ‘Premium’ Jonagold apples were defined as: MT P 60 N and SSC P 12%; otherwise, they were defined as ‘Regular’. b ‘Premium’ Golden Delicious apples were defined as: MT P 55 N and SSC P 12%; otherwise, they were defined as ‘Regular’. c ‘Premium’ Delicious apples were defined as: MT P 60 N and SSC P 11%; otherwise, they were defined as ‘Regular’. d Actual ‘Premium’ and ‘Regular’ grades. e Classified ‘Premium’ and ‘Regular’ grades.
Table 5 Average confusion matrices for the classification of ‘Jonagold’, ‘Golden Delicious’, and ‘Delicious’ apples in the validation set by Magness–Taylor (MT) firmness and soluble solids content (SSC) using the scattering data preprocessed with continuous wavelet transform (OHSScwt). Confusion matrix (%) Jonagolda Premium
d
Golden Deliciousb d
d
Deliciousc d
Regular
Premium
Regular
Premiumd
Regulard
2009 Premiume Regulare
88.8 11.3
5.7 94.3
86.3 13.8
8.6 91.4
66.7 33.3
28.2 71.8
2010 Premiume Regulare
74.3 25.7
17.4 82.6
73.8 26.2
19.7 80.3
73.4 26.6
25.0 75.0
2011 Premiume Regulare
65.5 34.5
20.0 80.0
89.6 10.4
16.0 84.0
75.9 24.1
23.5 76.5
Note: In bold are the average classification performance or accuracy of the linear discriminant models over six replicates for the testing set of samples a ‘Premium’ Jonagold apples were defined as: MT P 60 N and SSC P 12%; otherwise, they were defined as ‘Regular’. b ‘Premium’ Golden Delicious apples were defined as: MT P 55 N and SSC P 12%; otherwise, they were defined as ‘Regular’. c ‘Premium’ Delicious apples were defined as: MT P 60 N and SSC P 11%; otherwise, they were defined as ‘Regular’. d Actual ‘Premium’ and ‘Regular’ grades. e Classified ‘Premium’ and ‘Regular’ grades.
67
F. Mendoza et al. / Journal of Food Engineering 125 (2014) 59–68
Table 6 Ten best selected features using the sequential forward selection (SFS) method from one repetition for sorting ‘Jonagold’ (JG), ‘Golden Delicious’ (GD) and ‘Delicious’ (D) apples based on the combination of firmness and soluble solids content using Vis/SWNIRS and scattering (OHSScwt) data.
a b
Technique
Cultivara & season
Ten best selected features using SFS methodb
Vis/SWNIR
JG 2009 GD 2011 RD 2010
506 513 474
645 646 655
648 716 754
658 729 820
792 752 872
801 779 944
877 861 997
952 934 1035
1005 951 1066
1046 1085 1091
OHSScwt
JG 2009 GD 2011 RD 2010
595 475 580
650 525 625
655 545 655
660 590 660
675 600 675
700 715 710
730 900 720
825 935 825
900 1025 920
1025 1030 1035
Selected features from one run for apples harvested in 2009, 2010, or 2011. In bold are the spectral features from the short-wave near-infrared region.
and seasons, probably due to the greater number of wavelengths evaluated for each apple (i.e., 641 data points). But using the scattering technique (which consisted of 121 data points), more consistent common features or wavelengths along the spectral range were found among the varieties and seasons (Table 6).
Considering 780 nm as the cutoff between the visible and NIR regions, it was noted that a larger number of features for the Vis/ SWNIR data were selected from the SWNIR region (6 out of 10 for JG and 7 out of 10 for D), with the exception of GD apples (4 out of 10). Contrarily, using the OHSScwt data, more features were selected from the visible region (7 out of 10 for JG and D apples and 6 out of 10 for GD apples). Fig. 3 depicts the classification performance using two of the best features to sort apples into ‘Premium’ and ‘Regular’ grades based on firmness and SSC as the selection criterion. 3.3. Discussion
Fig. 3. Examples of using the two best selected features for spectral scattering technique in sorting apples into ‘Premium’ and ‘Regular’ grades based on the combination of flesh firmness and soluble solids content. (a) ‘Jonagold’ in 2009, (b) ‘Golden Delicious’ in 2011, and (c) ‘Delicious’ in 2010. Dashed lines represent the decision line using the linear discriminant classifier. Note that the validation set of regular apples overlap those for premium quality apples.
Classification accuracies for firmness or SSC were, in most cases, lower than 90%, and were even lower when apples were graded based on both firmness and SSC. This suggests that the techniques need further research and improvement. As mentioned earlier, packinghouses are often faced with two situations, in which they want to either pack superior quality apples for higher prices or just remove inferior quality fruit. In the first situation, the priority is to ensure the quality of each fruit meets the minimum firmness and/ or SSC requirement. This may be achieved by setting the firmness or SSC threshold higher than the minimum level, thus minimizing the error of having soft or low SSC apples in the pack. In the second situation, the priority is to remove as fewer firm or sweet fruit as possible. This may be achieved by lowering the firmness or SSC threshold to below the minimum level. In the statistical terminology, the user can achieve the quality sorting goal by controlling the Type I (false positive) or II (false negative) error. The classification results showed that the two sensors had varying abilities and the optical information measured by them could be complementary. This suggests that better outcomes could be achieved by combining Vis/SWNIRS and scattering techniques based on a data fusion approach. By combining the strengths of both sensors, an ensemble sorting model would have higher accuracy and robustness than a single model. A sensitivity analysis confirmed the potential of Vis/SWNIR and OHSS techniques and robustness of their models in a wide range of quality thresholds for classification of three varieties of apples into two quality grades (i.e., ‘Premium’ and ‘Regular’) based on firmness or SSC. In the experimental conditions of this study, the analysis also revealed that with the exception of GD apples, Vis/SWNIR for firmness measurements performed slightly better than OHSS for JG and RD. Further, Vis/SWNIR showed significant advantages over OHSS in sorting apples by SSC for the three apple varieties (Fig. 2). In addition, the consistency of the sorting procedure is supported by a previous study (Mendoza et al., 2012) using SWNIRS and scattering technique for predicting firmness and soluble solids content based on partial least squares models (PLS). The current classification performances for the same set of apples and cultivars (2009 and 2010), in general, correlated well with the PLS
68
F. Mendoza et al. / Journal of Food Engineering 125 (2014) 59–68
regression models as expressed by the correlation coefficients of prediction (R), standard error of prediction (SEP) and estimated RPD values (ratio of SEP to sample standard deviation) obtained in this previous study for all cultivars. In should be noted, however, that Vis/SWNIR measurements were carried out offline which allowed better control of the samples, while scattering measurements were performed online. Hence, the results from Vis/SWNIR do not necessarily reflect how it would achieve in online applications. It is reasonable to expect that better results for the same sensor would be obtained offline than online. Therefore, further online experiments using Vis/ SWNIR and spectral scattering as well as using different thresholds for the combination of firmness and SSC are needed for more complete evaluation of the potential of each technique for apple quality sorting. In addition, additional challenges in using these sensing techniques for online sorting and grading applications need to be addressed. Among them is orienting fruit in the right orientation for reliable and accurate Vis/SWNIR and spectral scattering measurements. 4. Conclusion In this research, Vis/SWNIR and spectral scattering techniques were used to sort JG, GD and D apples into two quality grades based on firmness, SSC or the combination of both attributes. Consistent and good sorting results for firmness (ranging between 77.9% and 98.2%) and moderate sorting results for SSC (ranging between 62.0% and 91.7%) were obtained using scattering technique. Vis/SWNIR technique, which was operated in stationary condition, showed slightly better sorting results for firmness (ranging between 87.3% and 97.6%) and good results for SSC (ranging between 77.1% and 92.3%). A sensitivity analysis showed that with some exceptions for D apples, classification results by firmness or SSC were consistent for a wide range of quality threshold criteria, but they were in most of the cases significantly different among harvest seasons, sensing techniques and/or preprocessing methods. When the classification was performed based on both firmness and SSC, the sorting accuracies decreased, ranging between 75.7% and 90.1% for Vis/SWNIR and between 69.7% and 91.5% for spectral scattering. Both techniques, coupled with simple discriminant analysis, showed potential for sorting and grading apples by firmness or/and SSC. Further improvements, however, are needed for more accurate sorting and grading of apples by using for example sensor fusion approaches and probably more sophisticated classification techniques such as probabilistic neural network and decision tree. References Cen, H., Lu, R., Dolan, K., 2009. Optimization of inverse algorithm for estimating optical properties of biological materials using spatially-resolved diffuse reflectance. Inverse Problems in Science and Engineering 18, 853–872.
Dillon, W.R., Goldstein, M., 1984. Discriminant analysis: the two group problem. In: Multivariate Analysis: Methods and Applications. John Wiley & Sons, New York, pp. 360–393. Harker, F.R., 2001. Consumer response to apples. In: Proceedings of Washington Tree Fruit Postharvest Conference. (accessed 30.07.12). Harker, F.R., Gunson, F.A., Jaeger, S.R., 2003. The case for fruit quality: an interpretive review of consumer attitudes and preferences for apples. Postharvest Biology and Technology 28 (3), 333–347. Harker, F.R., Kupferman, E.M., Marin, A.B., Gunson, F.A., Triggs, C.M., 2008. Eating quality standards for apples based on consumer preferences. Postharvest Biology and Technology 50 (1), 70–78. Jain, A., Duin, R., Mao, J., 2000. Statistical pattern recognition: a review. IEEE Transactions on Pattern Analysis and Machine Intelligence 22 (1), 4–37. Kuang, B., Mouazen, A.M., 2011. Calibration of a visible and near infrared spectroscopy for soil analysis at field scales across three European farms. European Journal of Soil Science 62 (3), 629–636. Lu, R., 2001. Predicting firmness and sugar content of sweet cherries using nearinfrared diffuse reflectance spectroscopy. Transactions of the ASAE 44 (5), 1265–1271. Lu, R., 2004. Multispectral imaging for predicting firmness and soluble solids content of apple fruit. Postharvest Biology and Technology 31 (2), 147–157. Lu, R., 2007. Nondestructive measurements of firmness and soluble solids content for apple fruit using hyperspectral scattering images. Sensing and Instrumentation for Food Quality and Safety 1 (1), 19–27. Lu, R., Peng, Y., 2005. Comparison of multispectral scattering and visible/NIR spectroscopy for predicting apple fruit firmness. In: Proceedings of 7th Fruit, Nut, and Vegetable Production Engineering International Symposium, Paper No. 62. Mann, H., Bedford, D., Luby, J., Vickers, Z., Tong, C., 2005. Relationship of instrumental and sensory texture measurements of fresh and stored apples to cell number of size. HortScience 40 (6), 1815–1820. Mendoza, F., Lu, R., Ariana, D., Cen, H., Bailey, B., 2011. Integrated spectral and image analysis of hyperspectral scattering data for prediction of apple fruit firmness and soluble solids content. Postharvest Biology and Technology 62 (2), 149– 160. Mendoza, F., Lu, R., Ariana, D., Cen, H., 2012. Comparison and fusion of four nondestructive sensors for predicting apple fruit firmness and soluble solids content. Postharvest Biology and Technology 73, 89–98. Mery, D., 2011. A Toolbox for Computer Vision, Pattern Recognition and Image Processing. http://dmery.ing.puc.cl/index> (accessed 04.09.12). Mery, D., Pedreschi, F., Soto, A., 2012. Automated design of a computer vision system for visual food quality evaluation. Food and Bioprocess Technology. http://dx.doi.org/10.1007/s11947-012-0934-2. Molina, D., Alegre, S., Casero, T., Bonany, J., Carbó, J., Puy, J., Recasens, I., 2006. Quality indexes for ‘Golden Smoothee’ apples in relation to consumer evaluation. Journal of Fruit and Ornamental Plant Research 14 (2), 39–51. Nicolaï, B.M., Beullens, K., Bobelyn, E., Peirs, A., Saeys, W., Theron, K., Lammertyn, J., 2007. Nondestructive measurements of fruit and vegetable quality by means of NIR spectroscopy: a review. Postharvest Biology and Technology 46 (2), 99–118. Péneau, S., Hoehn, E., Roth, H.-R., Escher, F., Nuessli, J., 2006. Importance and consumer perception of freshness of apples. Food Quality and Preference 17 (1– 2), 9–19. Peng, Y., Lu, R., 2005. Modeling multispectral scattering profiles for prediction of apple fruit firmness. Transactions of the ASAE 48 (1), 235–242. Tong, C., Krueger, D., Vickers, Z., Bedford, D., Luby, J., El Shiekh, A., 1999. Comparison of softening-related changes during storage of ‘Honeycrisp’ apple, its parents, and ‘Delicious’. Journal of the American Society for Horticultural Science 124 (4), 407–415. Tu, K., Baerdemaeker, J., 1996. Instrumental measurements to investigate apple mealy texture. International Agrophysics 10 (2), 97–102.
Contents lists available at ScienceDirect
Journal of Food Engineering journal homepage: www.elsevier.com/locate/jfoodeng
Grading of apples based on firmness and soluble solids content using Vis/SWNIR spectroscopy and spectral scattering techniques Fernando Mendoza a,⇑, Renfu Lu a, Haiyan Cen b a b
USDA/ARS, 524 S. Shaw Lane, Room 105A/224, Michigan State University, East Lansing, MI 48824, United States Department of Biosystems and Agricultural Engineering, Michigan State University, East Lansing, MI 48824, United States
a r t i c l e
i n f o
Article history: Received 28 September 2012 Received in revised form 22 August 2013 Accepted 17 October 2013 Available online 23 October 2013 Keywords: Apples Firmness Soluble solids content Sorting Near-infrared spectroscopy Spectral scattering
a b s t r a c t Sorting of apple fruit based on internal quality can enhance the industry’s competiveness and profitability and assure consumer satisfaction. In this research, visible and shortwave near-infrared (Vis/SWNIR) spectroscopy (460–1100 nm) and spectral scattering (450–1050 nm) were used for sorting three varieties of apple (i.e., ‘Delicious’, ‘Golden Delicious’ and ‘Jonagold’) into two quality grades based on firmness, soluble solids content (SSC), or both firmness and SSC. Vis/SWNIR spectra were obtained in interactance mode under stationery condition, whereas spectral scattering images were acquired online at a conveyor speed of 82 mm/s. A total of 8491 apples for the three varieties harvested from the same orchard in 2009, 2010 and 2011 were used for analysis. First derivative spectra were obtained from the Vis/SWNIR data, while the scattering images were first preprocessed by computing mean reflectance spectra and then performing continuous wavelet transform decompositions. Sorting algorithms were developed using sequential forward selection and linear discriminant analysis, and the classification models were compared in terms of their overall performance and local confusion matrices. A sensitivity analysis was also performed to assess the effect of using different quality threshold criteria on the sorting performance. Overall, consistent and relatively good sorting results for firmness (ranging between 77.9% and 98.2%) and moderate results for SSC (ranging between 62.0% and 91.7%) were obtained using scattering technique. Vis/SWNIR technique showed slightly better sorting results for firmness (ranging between 87.3% and 97.6%) and SSC (ranging between 77.1% and 92.3%). When the classification was performed based on both firmness and SSC, the sorting accuracies generally decreased to between 75.7% and 90.1% for Vis/SWNIR and between 69.7% and 91.5% for spectral scattering. Vis/SWNIR and spectral scattering techniques have potential for online sorting and grading of apples by firmness and SSC. Published by Elsevier Ltd.
1. Introduction Apple is an important agricultural commodity in the global fresh produce market. Consumer choice of apples is driven by the trade-off between price and quality (Harker et al., 2003). Today consumers are demanding better quality, more consistent apples with appropriate taste and texture. Enhancement in fruit quality could thus lead to an increased demand and repeat purchases by the consumer, and increase profit margins for the industry through price differentiations for different quality grades of apples. Hence, appropriate quality control and inspection procedures need to be implemented to sort and grade each individual fruit for internal quality. Currently, apples are sorted, using machine vision systems, mainly by color, shape, and size or weight, but not on internal quality. A study by Harker (2001) found that about 70% US ⇑ Corresponding author. Tel.: +1 517 432 7438; fax: +1 517 432 2892. E-mail address: [email protected] (F. Mendoza). 0260-8774/$ - see front matter Published by Elsevier Ltd. http://dx.doi.org/10.1016/j.jfoodeng.2013.10.022
consumers considered the eating quality to be the most important factor in purchasing apples. Texture and taste define the eating quality of apples, and among them, texture seems to be a primary quality attribute which, together with taste and appearance, should be taken into account when evaluating the overall quality of the fruit (Tu and Baerdemaeker, 1996; Mann et al., 2005). Fruit texture is determined by firmness (force required to bite into the fruit), crispness (amount and pitch of sound generated when the fruit is first bitten with the front teeth), juiciness (amount of juice released from the fruit in the first three chews, when chewing with the back teeth), and mealiness (degree to which the flesh breaks down to a fine lumpy mass) (Mann et al., 2005; Péneau et al., 2006). Apples that are crispy, firm, and juicy are highly favored by consumers (Harker et al., 2008). Those sensory attributes are related to the structure and turgor pressure of fruit tissue (Tong et al., 1999). The taste properties that apply to apples include sweetness, acidity, and astringency. Consumers are capable of distinguishing all these attributes with a high degree of sensitivity (Harker, 2001). Therefore, there is much incentive to
60
F. Mendoza et al. / Journal of Food Engineering 125 (2014) 59–68
sort and grade fruit based on their internal quality for the fresh market. Nondestructive measurement of flesh firmness and sweetness would allow the apple industry to deliver superior, consistent fruit to the marketplace and ensure consumer acceptance and satisfaction (Peng and Lu, 2005). The demand for high-quality fruit calls for reliable and rapid sensing technologies for the nondestructive measurement and sorting of apples. Among many nondestructive sensing techniques that have been developed, optical techniques, especially visible and near-infrared (Vis/SWNIR) spectroscopy and spectral scattering, show great potential for sorting and grading apples for internal quality. Vis/SWNIR measures an aggregate amount of light reflected back from or transmitted through the sample, which is then used to predict certain chemical constituents. The technique, coupled with an appropriate predictive model, has been successfully applied for nondestructive SSC assessment of various fruits (Lu, 2001, 2004; Nicolaï et al., 2007). Spectral scattering technique, however, enhances the measurement of light scattering in the fruit, a physical phenomenon that is dependent on the density, cell structures, and cellular matrices of fruit tissue. With spectral scattering technique, a small continuous-wave light beam is incident on a fruit, generating scattering images at its surface around the incident point as the result of light propagation and backscattering inside the fruit (Lu, 2007). A multi- or hyperspectral imaging system is used to capture backscattering images from the fruit at selected wavelengths or over a specific spectral region. Thereafter, spectral scattering features are extracted from the scattering images for prediction of fruit firmness and SSC using appropriate mathematical models (Mendoza et al., 2011). The objective of this research was to independently assess both visible and shortwave near-infrared (Vis/SWNIR) spectroscopy and spectral scattering techniques for sorting apples into two quality groups by firmness, SSC, or their combination using discriminant analysis. For testing the robustness of the classification models, a sensitivity analysis was performed using different quality threshold criteria for automatic sorting and three varieties of apple (i.e., ‘Jonagold’, ‘Golden Delicious’, and ‘Delicious’) harvested in 2009, 2010 and 2011 with different range of variability for firmness and SSC were evaluated in this study. 2. Materials and methods 2.1. Instrument setup Two nondestructive sensing techniques, i.e., Vis/SWNIR spectroscopy and spectral scattering, for measuring the firmness and SSC of apple fruit were used in this study, and they are described in the following sections. 2.1.1. Visible and shortwave near-infrared (Vis/SWNIR) spectroscopy A miniature Vis/SWNIR spectrometer (USB4000, Ocean Optics, Dunedin, Florida, USA) was used to acquire spectra from apple fruit between 460 nm and 1100 nm in an interactance mode. The DC power supply for a quartz tungsten halogen lamp (Oriel Instruments, Stratford, CT) was regulated with an intensity controller, and the light was delivered to the fruit through a ring guide of 25 mm in diameter. Mounted in the center of the light guide was a beam collimator, which collected the light that has interacted with the fruit tissue and reemerged from an area of 10 mm, and then focused it onto a receiving optic fiber that was connected to the spectrometer. A sufficient buffering zone (about 8 mm) was established between the illumination area and the detection area on the fruit to ensure that the receiving fiber only collected the light that has interacted with the flesh tissue. Based on the preliminary test, the lamp power supply was set to 100 W and the
integration time of the spectrometer was set to 100 ms. This setting allowed acquisition of good spectral signals for all tested apples, while avoiding light saturation due to overexposure. A graphic description of the Vis/SWNIR instrument is given in Mendoza et al. (2012). 2.1.2. Online hyperspectral scattering system (OHSS) A prototype online hyperspectral scattering system (OHSS) was used for acquiring spectral scattering images of apples. This system consisted of a back-illuminated electron-multiplying CCD (EMCCD) camera (Model PhotonMAX: 1024B Air-Cooled, Princeton Instruments, Trenton, NJ, USA) with a 1024 1024 13-lm pixel frametransfer CCD detector, an imaging spectrograph (ImSpector V10E, Spectral Imaging Ltd., Oulu, Finland) covering the spectral region of 400–1000 nm with 2.8 nm spectral resolution, a focusing lens with a near-infrared enhanced lens, and a quartz tungsten halogen light source (Oriel Instruments, Stratford, Conn., USA) with an intensity controller. The incident point light beam was 1.5 mm in size and the line scanning position of the camera was 1.6 mm off from the beam incident center to avoid saturation. The EMCCD camera with dual amplifiers was set to operate at 10 MHz for high-speed imaging using an avalanche gain of 2950, and it was equipped with a deep thermoelectric cooling system operating at 55 °C to suppress system noise. The test apples, which were placed on cups with the stem-calyx axis being horizontal and perpendicular to the moving direction, were transported on a 2.4 m conveyor at a speed of 82 mm/s (1 fruit for 2 s). The OHSS began to capture and save spectral scattering images from the equatorial area of each apple immediately after the two photoelectric beam sensors were triggered by the incoming cup and apple, respectively. The integration time was set at 120 ms for each apple and the acquired images were binned 4 4 to yield the final images of 256 256 pixels with the spatial resolution of 0.20 mm/pixel and the spectral resolution of 4.54 nm/ pixel. A detailed description of the OHSS is given in Mendoza et al. (2011, 2012). 2.2. Experimental procedure ‘Jonagold’ (JG), ‘Golden Delicious’ (GD) and ‘Delicious’ (D) apples were harvested in 2009, 2010 and 2011 from an orchard of Michigan State University’s (MSU) Clarksville Horticultural Experiment Station in Clarksville, MI. In order to have a wider range of physiological condition for the studied varieties, the harvest was carried out over six consecutive weeks for 2009 and 2010, and over four consecutive weeks for 2011. One day after each harvest, a set of 100 apples for each variety were tested, and the rest apples were stored in refrigerated air at 0 °C. One week after the last harvest, tests for the stored apples were started. The number of tested apples for each variety and testing date varied between 60 and 100 fruit, depending on the condition of the fruit (i.e., firmness and SSC) at the time of testing, which was monitored by testing a few samples destructively prior to the experiment. The stored apples for 2009 were tested once a week for the first 6 weeks and then every 2 weeks for up to 8 weeks. The experiment was completed in 5 months for 928, 1176 and 1091 apples of JG, GD and D, respectively. The stored apples for 2010 were tested once a week for the first 4 weeks and then every 2 weeks for up to 6 weeks; the experiment was completed in 4 months for 1037, 1132 and 1102 apples for JG, GD and D, respectively. Similarly, the stored apples for 2011 were tested once a week for the first 2 weeks and then every 2 weeks for up to 4 weeks; the experiment was completed in two and one half months for 640, 825 and 560 apples for JG, GD and D, respectively. After the nondestructive measurements using the two sensing techniques had been completed, the firmness and SSC of the tested
F. Mendoza et al. / Journal of Food Engineering 125 (2014) 59–68
apples were measured, using standard destructive methods, from the same location where Vis/SWNIR spectroscopy and scattering measurements had been carried out. Magness–Taylor (MT) firmness (N) measurements were performed using a Texture Analyzer (model TA.XT2i, Stable Micro Systems, Inc., Surrey, United Kingdom) with a steel probe of 11 mm diameter for a penetration depth of 9 mm at a loading speed of 2 mm/s. Maximum force recorded on the force/displacement curve was used as the measure of fruit firmness. Immediately after the firmness measurements, juice was extracted from the fruit and its SSC (°Brix, expressed as %) was then measured using a digital refractometer (Model PR-101, Atago Co., Tokyo, Japan) with the accuracy of ±0.2%. The test apples had been kept at room temperature (22 °C) for at least 16 h to allow them to reach equilibrium before the optical measurements and firmness and SSC measurements were carried out. The temperature of fruit at the time of testing could affect MT firmness/SSC and optical measurements, which in turn would influence the model prediction of firmness and SSC. Hence by maintaining approximately the same temperature for all test samples, we would minimize the temperature effect on the firmness and SSC prediction. However, under the commercial condition, it may not be easy to maintain all fruit at an equilibrium temperature. Hence it would be necessary to consider the temperature effect in the prediction/classification model, which is beyond the scope of the current study. 2.3. Extracted features Dark and reference corrections were first performed on the Vis/ SWNIR spectral data covering the spectral range of 460–1100 nm at increments of 1 nm, yielding each spectrum of 641 wavelengths. First derivatives were then used to minimize the baseline offset and enhance absorbance peaks. It should be noted that preliminary analysis for the Vis/SWNIRS data showed that better classification results were obtained using first derivatives compared to smoothed relative reflectance (Nicolaï et al., 2007), second derivative and continuous wavelet transform preprocessing methods (Mendoza et al., 2011, 2012). Hence, this paper only reports the results from the first derivative spectral data. Scattering features were extracted from the preprocessed scattering images covering the spectral range of 450–1050 nm at increments of 5 nm, yielding each spectrum of 121 wavelengths and a spatial range of 0–10 mm. Mean reflectance spectra (OHSSmean) were first computed (Lu, 2007), followed by the multi-resolution wavelet transform method based on continuous one-dimensional decomposition using the symlet2 wavelet at 64-scales. The resultant vector of scattering features after applying continuous wavelet transform (OHSScwt) represents the root mean square of the transformed data at each single wavelength. It should be noted that the selected wavelength range (i.e., 450–1050 nm) and increment (i.e., 5 nm) and the scattering distance (i.e., 10 mm) for extracting scattering features were determined after preliminary factorial analysis for selecting the best processing conditions had been performed (not reported here). The details of calculating these spectral features can be found in Mendoza et al. (2011). The extracted features (n) from each set of samples (N) were arranged in the ith row of matrix F: [Fi1, . . . , Fin] that corresponds to a point in the n-dimensional measurement feature space. Prior to the classification analysis, the sets of features, i.e., Vis/SWNIR, OHSSmean, and OHSScwt were normalized, yielding a N n matrix Rnormij whose elements are defined as (Mery et al., 2012):
Rnormij ¼
Rij lj
rj
ð1Þ
61
for i = 1, . . . , N and j = 1, . . . , n, where Rij denotes the jth feature of the ith feature vector, N is the number of samples, and lj and rj are the mean and standard deviation of the jth feature, i.e., the jth column of R. The normalized features have 0 mean and a standard deviation equal to 1. Thus 641 Vis/SWNIR features, 121 OHSSmean and 121 OHSScwt scattering features from each apple were extracted and used for further sorting analyses. 2.4. Quality grading criteria In practical situations, sorting for internal quality would be carried out for those fruit that have met the minimum quality grades for size, weight, and color and free of external defects. Because of a large number of grade combinations for color and size, it is necessary to limit the number of quality grades for firmness and/or SSC so as to lower the packing cost. Two situations may be encountered in the packinghouse with regard to firmness or SSC: first, the packers would like to remove those apples that do not meet the minimum requirement in firmness, SSC or in both; and second, they may want to pack apples with superior quality, so that these apples can be sold at a premium price. For these reasons, this research only considered sorting apples into two quality grades based on firmness, SSC, or both firmness and SSC. The threshold values for firmness and SSC were defined as follows: MT firmness of 60 N and SSC of 12 °Brix for JG, MT firmness of 55 N and SSC of 12 °Brix for GD, and MT firmness of 60 N and SSC of 11 °Brix for D. The selection of the threshold values was based on the recommendations from published studies and also through our interaction with the apple industry in the US Published recommendations on the minimum firmness for apples vary between 49.0 N for ‘Golden Smoothee’ and 62.0 N for ‘Delicious’ apples, and the minimum SSC between 11.6% and 14% for different varieties of apple (Molina et al., 2006; Harker, 2001; Harker et al., 2008). The selected thresholds fall between these ranges for firmness and around the minimum value for SSC. The lower SSC threshold of 11% Brix was selected for D apples so that there were enough samples with the SSC of <11% Brix. It should be pointed out that sorting apples into the two quality grades was intended to assess and demonstrate the capability of the technologies. In practice, packinghouses also need to consider such factors as market demand and fruit inventory when deciding on how to sort fruit or set the sorting criteria. Additionally, a sensitivity analysis to test the robustness of the classification models using different quality threshold criteria for automatic apple sorting was performed for MT firmness and SSC models. Considering that only datasets with a large range of variability could be useful for the test, only apple samples from 2009 to 2010 which had the largest variability were analyzed and compared based on the performance of classification at each tested threshold criteria. The tested quality threshold for MT firmness ranged in steps of 5 N from 50 N to 75 N for JG (2010) and GD (2009), and from 50 N to 70 N for D (2010) apples; and for SSC ranged in steps of 0.5% from 11.5% to 13.5% for JG (2010), from 12% to 14% for GD (2010), and from 11% to 12.5% for D (2009). All sorting tests were performed using the same procedure explained below. 2.5. Classification by linear discriminant analysis (LDA) Classification is the process of putting objects into a class or group according to particular characteristics based on a reduced set of useful information. This means sorting apples by firmness, SSC or their combination using only the best features extracted from the preprocessed Vis/SWNIR and scattering data (OHSSmean or OHSScwt). For this, multivariate analysis was performed using the codes adapted from Balu Matlab Toolbox (Mery, 2011) run in MATLAB 7.5.0 (The MathWorks, Inc., Natick, MA, USA). The best
62
F. Mendoza et al. / Journal of Food Engineering 125 (2014) 59–68
subset of features (that leads to the smallest classification error) was found using the sequential forward selection (SFS) algorithm, and the classification was performed using linear discriminant analysis (LDA). SFS starts with the best single feature and then adds one new feature each time to the features that have been already selected from the previous runs, to maximize the classification performance. The iteration is halted after the model no longer shows improvement with the addition of a new feature (Jain et al., 2000). LDA is a statistical technique for classifying individuals or objects into mutually exclusive groups on the basis of a set of independent variables. LDA involves deriving linear combinations of the independent variables that will discriminate between the a priori defined groups in such a way that the misclassification error rates are minimized. This is accomplished by maximizing the between-group variance relative to the within-group variance. The goal is to find the allocation rule which gives the highest percentage of correct classification (Dillon and Goldstein, 1984). The optimum number of latent variables for the classifier is determined using cross validation, a technique widely used in machine learning problems, to avoid both over-fitting and under-fitting problems. Finally, the performance of the classifier is tested for an independent set of samples. In our experiments, this procedure was repeated four times through rotating the training and test data; each time 75% of the data was used to train and the rest (25%) for test. Only the average training and testing results from the four runs were used in further statistical analysis. This would ensure more consistent evaluation of the classifier, since its performance is influenced, to a certain extent, by the way how training and testing samples are selected. For analysis and comparison, the classification models were independently derived for each sensing technique, variety and harvest season. It was noticed that the number of samples above or below the threshold value of firmness or SSC varied for the three varieties and harvest season (see Fig. 1 and further discussion in Section 3.1). To avoid bias in the model development due to imbalance in the number of samples for each quality grade group, 300 apples were randomly selected for each firmness or SSC group, except for D apples in 2011, where only 250 apples for each group were selected for analysis because of the smaller number of apples for that year. Hence there were a total of 600 apples (or 500 for D in 2011) used in each case for the model development. This sampling procedure was repeated six times; for each of the six sampling data sets, the same calibration and testing procedure, as described earlier, was followed. The average values for latent variables for the models, their overall performance (i.e., accuracy of the classification for the testing set of samples) and local confusion matrices from the six sampling data sets were finally compiled for evaluation of the two sensing techniques for the three sorting scenarios (by firmness, SSC, or their combination) for each variety and harvest season.
3. Results 3.1. Distribution of firmness and SSC measurements by standard destructive methods Heterogeneous, non-normal distributions for MT firmness were observed for the three apple varieties in all harvest seasons (Fig. 1). With the exception of D apples for 2010 and 2011, the MT-firmness distributions appear largely bimodal with a gap between 50 to 70 N, which could have been attributed to faster physiological and microstructural changes in apples when they reached this range of firmness. JG apples from 2009 to 2010 showed the largest range of firmness with the standard deviations of 22.8 and 19.0 N, respectively, followed by GD and D apples whose standard
deviation values were similar for the same harvest seasons, ranging from 15.5 to 16.6 N for GD and from 15.8 to 16.6 N for D. Overall apples for 2011 showed the smallest range of firmness values since the harvesting and testing period of time was shorter than that for 2009 and 2010. The inclusion of 2011 apples in this study was to assess the robustness of the classifier with a smaller range of firmness and SSC values. The standard deviations for JG, GD and D apples in 2011 were of 12.5, 13.7 and 7.1 N, respectively. Firmer apples were found for JG from 2009 and D from 2010. Prediction and classification models are typically built using multivariate statistical methods whose performance generally depends on the characteristics of the data. Recent studies in other areas as soil science (Kuang and Mouazen, 2011) and our own experience in processing apple data (Mendoza et al., 2011, 2012) have shown that the number of samples (size and/or variability) and its density distribution are frequently factors that tend to affect the robustness and accuracy of the calibration models developed. Contrarily, the SSC was fairly normally distributed with the exception of JG and D apples from 2010, whose distributions markedly deviated from normal (Fig. 1). The standard deviation ranged from 1.0% for D apples in 2011 to 1.6% for JG and GD apples in 2010 and 2009, respectively. On average, GD apples were sweeter (ranging from 7.9 to 19.7 °Brix), followed by JG (ranging from 8.3 to 17.6 °Brix), and then D (ranging from 8.9 to 17.0 °Brix). 3.2. Classification results 3.2.1. Classification by MT firmness Table 1 summarizes the average number of variables, overall performance and standard deviation for the testing sets (i.e., from 6 repetitions) using MT firmness as a classification criterion for JG, GD and D apples harvested in the three seasons. Using Vis/SWNIR and OHSS techniques, the best classification was obtained for JG, followed by GD and D, although in most of the cases, the performance for the same varieties varied significantly among the harvest seasons (p < 0.05). Among the three harvest seasons and for both sensing techniques, the model accuracy was more variable for apples harvested in 2010 for all varieties, as indicated by their standard deviations (Table 1). GD and D apples showed larger variability in the model performance for the harvest seasons of 2009 and 2010. The accuracy of the classification models for OHSS was improved by implementing the continuous wavelet transform in the data preprocessing (i.e., OHSScwt). Significant improvements (p < 0.05) were observed for all varieties with 0.8%, 2.1% and 1.2% for JG and 5.5%, 11.9%, and 2.6% for GD for the three harvest seasons, respectively, and 11.4% and 6.9% for D for 2009 and 2010, respectively. On the other hand, comparing both sensing techniques, in most cases, Vis/SWNIR performed significantly better than OHSScwt (p < 0.05). For the model calibration, the number of variables for all varieties and harvest seasons ranged from 3 to 19 when either Vis/SWNIR or OHSScwt was used. In previous studies, scattering technique gave better prediction results than Vis/ SWNIR for predicting apple firmness, because the former, which was also tested under stationary condition, is able to better quantify the light scattering features in the fruit (Lu and Peng, 2005; Nicolaï et al., 2007). The unexpected result can be explained by the fact that scattering technique was operated online, while Vis/ SWNIR technique was in offline mode. Under the online condition, it was more difficult to accurately position each individual apple, which is important for both Vis/SWNIR and scattering techniques. In addition, only a simple mean spectra calculation method was used to extract features from the spectral scattering data in this study, as compared to the more sophisticated spectral and imaging analysis methods used in our previous studies (Mendoza et al., 2011).
63
F. Mendoza et al. / Journal of Food Engineering 125 (2014) 59–68
(a)
(b)
Fig. 1. Distributions of (a) Magness–Taylor (MT) firmness and (b) soluble solids content for ‘Jonagold’ (JG), ‘Golden Delicious’ (GD) and ‘Delicious’ (D) apples harvested in 2009, 2010, and 2011.
Table 1 Overall testing results for classification of ‘Jonagold’, ‘Golden Delicious’ and ‘Delicious’ apples into two grades of firmness for three harvest seasons, using Vis/SWNIR and online spectral scattering techniques.A Jonagold
Golden Delicious
Delicious
No.Bvariables
Accuracy (±std.) (%)
No.Bvariables
Accuracy (±std.) (%)
No.Bvariables
Accuracy (±std.) (%)
17 19 18
97.6 (0.1)a 90.8 (0.7)b 87.3 (0.6)c
19 19 18
89.6 (0.9)a 90.5 (1.5)a 92.0 (0.4)b
19 18 –
89.6 (0.9)a 87.6 (1.3)b –
OHSSmeanD 2009 2010 2011
17 15 12
97.4 (0.3)a 87.3 (1.1)c 92.1 (0.5)d
7 14 12
85.4 (1.5)c 79.5 (1.7)d 89.1 (0.2)e
3 13 –
77.9 (1.2)c 79.5 (1.5)d –
OHSScwtE 2009 2010 2011
13 18 16
98.2 (0.3)e 89.1 (0.6)f 93.2 (0.3)g
19 17 17
90.1 (1.0)a 88.9 (1.1)e 91.4 (0.2)f
18 18 –
86.8 (2.1)b 85.0 (1.4)e –
Vis/SWNIRS 2009 2010 2011
C
Note: In bold are the average classification performance or accuracy of the linear discriminant models over six replicates for the testing set of samples Values in the same columns with different letters are statistically different (p < 0.05). A The firmness threshold values for ‘Jonagold’, ‘Golden Delicious’ and ‘Delicious’ were 60 N, 55 N, and 60 N, respectively. B Average number of features used by the linear discriminant analysis model over six replicates. C Vis/SWNIRS: First derivative of the Vis/SWNIR data for the wavelengths of 460–1100 nm. D OHSSmean: Mean reflectance spectra for the wavelengths of 450–1050 nm. E OHSScwt: Continuous wavelet transform of the spectral scattering data for the wavelengths of 450–1050 nm.
a,b,c,d,e,f,g
64
F. Mendoza et al. / Journal of Food Engineering 125 (2014) 59–68
Table 2 Overall validation or testing results for classification of ‘Jonagold’, ‘Golden Delicious’ and ‘Delicious’ apples into two grades of soluble solids content for three harvest seasons, using Vis/SWNIR and online spectral scattering techniques.A Jonagold
Golden Delicious
Delicious
No.Bvariables
Accuracy (±std.) (%)
No.Bvariables
Accuracy (±std.) (%)
No.Bvariables
Accuracy (±std.) (%)
19 20 18
82.0 (1.3)a 88.7 (0.6)b 83.6 (0.9)c
18 18
– 86.0 (1.6)a 86.0 (0.4)a
18 20 18
77.1 (1.1)a 92.3 (0.7)b 83.9 (1.4)c
OHSSmeanD 2009 2010 2011
16 10 13
78.3 (0.1)d 84.8 (1.6)e 62.9 (0.8)f
– 14 4
– 74.7 (1.6)c 63.8 (1.0)d
3 13 10
68.7 (0.8)d 88.7 (1.1)e 79.4 (0.8)f
OHSScwtE 2009 2010 2011
17 13 8
81.1 (1.4)a 86.0 (1.4)e 62.0 (1.0)f
– 17 13
– 79.3 (0.6)e 64.9 (1.3)d
15 17 16
77.0 (2.0)a 91.7 (1.0)b 81.4 (0.4)g
Vis/SWNIRS 2009 2010 2011
C
Note: In bold are the average classification performance or accuracy of the linear discriminant models over six replicates for the testing set of samples a,b,c,d,e,f,g Values in the same columns with different letters are statistically different (p < 0.05). A The threshold values for soluble solids content for ‘Jonagold’, ‘Golden Delicious’, and ‘Delicious’ were 12 °Brix, 12 °Brix, and 11 °Brix, respectively. B Average number of features used by the linear discriminant analysis model over six replicates. C Vis/SWNIRS : First derivative of the Vis/SWNIRS data for the wavelengths of 460–1100 nm. D OHSSmean: Mean reflectance spectra for the wavelengths of 450–1050 nm. E OHSScwt: Continuous wavelet transform of the spectral scattering data for the wavelengths of 450–1050 nm.
3.2.2. Classification by SSC Likewise, Table 2 summarizes the overall performance for the validation or testing sets using SSC as the classification criterion for JG, GD and D apples harvested in 2009, 2010, and 2011. The correct classification rates in separating apples by SSC were in general lower than those sorted by firmness, with the exception of D apples from 2010. This could be explained by the relatively tighter range of the SSC data in comparison with the MT firmness data (Fig. 1). The accuracy of the classification models for SSC using OHSS was slightly improved by implementing the continuous wavelet transform (i.e., OHSScwt). In this case, significant improvements (p < 0.05) were only found for JG and GD apples at 3.6% for 2009 and 6.2% for 2010, respectively, and for D apples at 12.2%, 3.4%, and 2.5% for 2009, 2010, and 2011, respectively. Vis/SWNIR was significantly better (p < 0.05) than OHSScwt for the SSC classification for almost all seasons and apple varieties. The lowest classification rates of 62.0% and 64.9% for JG and GD, respectively, using OHSScwt were obtained for 2011 (Table 2). However, using Vis/SWNIR for the same varieties and season, the classification rates were improved by 25.8% for JG and by 24.5% for GD apples. These results confirmed that Vis/SWNIR performed better than spectral scattering technique in predicting and grading apple fruit by SSC. 3.2.3. Effect of the quality threshold value on the performance of the classification models Fig. 2 depicts the behavior of the classification models for selected harvest seasons (i.e., datasets with the largest variability) when different quality threshold values were used for analysis. With some exceptions for D apples, overall results showed consistent performance (%) for all cultivars in a wide range of quality threshold values, but they were in most of the cases significantly different among harvest seasons, sensing techniques and/or preprocessing methods. For firmness, JG apples showed the most consistent performance in all tested range (from 50 N to 75 N), followed by GD apples with a consistent performance in the range of 55 N to 70 N. The performance for RD apples, however, showed a tendency to increase when the threshold values are increased; although a steady range was observed in between 55 N and 60 N. Classification performances based on SSC showed that JG and GD apples were more
consistent for quality threshold values higher than 12% Brix onwards in the tested range for each variety. Classification of RD apples was consistent in all the tested range of threshold values (from 11% to 12.5%) using Vis/SWNIR, and from 11% Brix onwards using OHSS. No significant variations in the standard deviations were observed among the tested quality thresholds in all apple varieties. Furthermore, similar behaviors for firmness and SSC were observed for the other harvest seasons do not presented here. 3.2.4. Classification by MT firmness and SSC Since continuous wavelet transform performed consistently better in preprocessing the spectral scattering data for sorting apples by firmness or SSC (Tables 1 and 2; and Fig. 2), only results from the OHSScwt data are presented here. Table 3 summarizes the classification results for the validation sets using firmness and SSC as the grading criterion for JG, GD and D apples harvested in the three seasons. In general, the classification accuracies were lower than those sorted by MT firmness alone, but, in a few cases, were higher than those sorted by SSC. Similar to sorting for single quality parameters, the accuracy of the classification models involving both firmness and SSC was significantly better (p < 0.05) with Vis/SWNIR technique than using the OHSS technique except for JG and GD for 2009. The classification results for the two quality attributes tended to be more variable than that for single attributes. The standard deviation of the accuracy rates from the six runs for all varieties and harvest seasons using Vis/SWNIR was in most cases smaller (ranging from 0.9% to 1.9%) in comparison with that for OHSScwt (ranging from 0.3% to 2.6%), which could be explained by the fact that the scattering technique was performed online, while Vis/SWNIR was performed in stationary condition. On the other hand, the number of variables for the calibration models for the three varieties and harvest seasons was larger for Vis/SWNIR (ranging from 15 to 20) than for OHSScwt (ranging from 13 to 18). It is important to note that the interaction of light with apple tissue for prediction of internal quality is a complicated process, which is affected by many factors including tissue microstructure, physicochemical process, variety, and season (Cen et al., 2009). This may explain why larger numbers of latent variables (up to 20) were chosen in some of the calibration models. When both firmness and sweetness were used for quality classification, the classification algorithm became more
65
F. Mendoza et al. / Journal of Food Engineering 125 (2014) 59–68
(a) Jonagold, 2010
(d) Jonagold, 2010 pMeR
pCWT
NIR
100% 95% 90% 85% 80% 75% 70% 50
55
60
65
70
75
Classification performance
Classification performance
NIR
(b) Golden Delicious, 2009
85% 80% 75% 70% 65
70
75
Quality thresholds for firmness (N)
(c) Delicious,2010 pMeR
12.0
12.5
13.0
13.5
pMeR
pCWT
100% 90% 80% 70% 60% 50% 12.0
12.5
NIR
pCWT
95% 90% 85% 80% 75% 70% 55
50%
13.0
13.5
14.0
(f) Delicious, 2009
100%
50
60%
Quality thresholds for soluble solids content (%)
60
65
70
75
Quality thresholds for firmness (N)
Classification performance
Classification performance
NIR
70%
NIR
90%
60
80%
pCWT
95%
55
90%
(e) Golden Delicious, 2010
100%
50
100%
11.5
Classification performance
Classification performance
pMeR
pCWT
Quality thresholds for soluble solids content (%)
Quality thresholds for firmness (N)
NIR
pMeR
pMeR
pCWT
100% 90% 80% 70% 60% 50% 11.0
11.5
12.0
12.5
13.0
Quality thresholds for soluble solids content (%)
Fig. 2. Classification performance of apples by firmness (a–c) and SSC (d–f) into two quality groups using Vis/SWNIR and online spectral scattering (OHSSmean and OHSScwt) techniques when different quality threshold values are used for analysis.
complex, thus resulting in higher classification error and variability. For further evaluation of the classification performance for both firmness and SSC, Tables 4 and 5 present the average confusion matrices for the validation sets using Vis/SWNIR and spectral scattering techniques. It is noted that in most cases, better classification was achieved for apples ‘Regular’ grade apples than for ‘Premium’ grade apples, when either technique was used. Overall larger numbers of false ‘Premium’ apples were obtained using the OHSScwt technique. Among the false ‘Premium’ apples, the classification algorithm mainly failed to discriminate apples with a high level of SSC; i.e., the misclassified ‘Premium’ grade apples were more likely to be firm but low in SSC. In multivariate analysis with a large set of features, it is of particular interest to know the distribution of the most significant
features involved in the calibration model. The key idea is that a subset of selected features in combination with the selected classifier could perform accurately to solve a specific classification task. Table 6 shows the 10 best selected spectral features using Vis/ SWNIR and OHSScwt techniques for sorting JG, GD and D apples based on firmness and SSC as a selection criterion. These features were determined using the SFS analysis, corresponding to one of the six repetitions for each cultivar at different harvest seasons. In general, the best selected features were distributed along the entire range of wavelengths for both Vis/SWNIR and scattering techniques. It was noted that although some variability in the wavelength selection existed among the six runs for each variety and season, a high degree of consistency for the selected wavelengths was observed. Using Vis/SWNIR technique, no specific patterns of selected wavelengths were evident among the varieties
66
F. Mendoza et al. / Journal of Food Engineering 125 (2014) 59–68
Table 3 Overall validation or testing results for classification of ‘Jonagold’, ‘Golden Delicious’ and ‘Delicious’ apples into ‘Premium’ and ‘Regular’ grades by firmness and soluble solids content for three harvest seasons, using Vis/SWNIRS and spectral scattering techniques.A. Jonagold
Vis/SWNIRS 2009 2010 2011 OHSScwtD 2009 2010 2011
Golden Delicious
Delicious
No.Bvariables
Accuracy (±std.) (%)
No.Bvariables
Accuracy (±std.) (%)
No.Bvariables
Accuracy (±std.) (%)
17 17 19
89.2 (1.6)a 80.9 (1.5)b 85.5 (1.9)c
18 19 18
85.5 (1.2)a 82.1 (1.2)b 90.1 (1.0)c
20 19 15
78.7 (1.8)a 78.4 (1.3)a 75.7 (0.9)b
13 14 17
91.5 (0.7)d 78.4 (1.3)e 74.7 (2.6)f
17 15 15
88.6 (1.7)d 76.5 (2.4)e 86.5 (0.6)a
18 17 13
69.7 (2.1)c 74.5 (1.9)d 75.9 (0.3)b
C
Note: In bold are the average classification performance or accuracy of the linear discriminant models over six replicates for the testing set of samples a,b,c,d,e,f,g Values in the same columns with different letters are statistically different (p < 0.05). E OHSScwt: Continuous wavelet transform of the spectral scattering data for the wavelengths of 450–1050 nm. A Apples were graded as ‘Premium’ if: Magness–Taylor (MT) firmness P 60 N and soluble solids content (SSC) P 12 °Brix for ‘Jonagold’, MT P 55 N and SSC P 12 °Brix for ‘Golden Delicious’, and MT P 60 N and SSC P 11 °Brix for ‘Delicious’; they would be otherwise graded as ‘Regular’. B Average number of features used by the linear discriminant analysis model over six replicates. C Vis/SWNIRS : First derivative of the Vis/SWNIRS data for the wavelengths of 460–1100 nm. D OHSSmean: Mean reflectance spectra for the wavelengths of 450–1050 nm.
Table 4 Average confusion matrices for the classification of ‘Jonagold’, ‘Golden Delicious’, and ‘Delicious’ apples in the validation set by Magness–Taylor (MT) firmness and soluble solids content (SSC) using Vis/SWNIRS data. Confusion matrix (%) Jonagolda Premium
d
Golden Deliciousb d
d
Deliciousc d
Regular
Premium
Regular
Premiumd
Regulard
2009 Premiume Regulare
85.5 14.5
6.0 94.0
84.4 15.6
13.5 86.5
76.7 23.3
19.7 80.3
2010 Premiume Regulare
78.6 21.4
17.8 82.2
80.8 19.2
16.7 83.8
78.7 21.3
22.4 77.6
2011 Premiume Regulare
79.3 20.7
11.8 88.2
89.2 10.8
9.5 90.5
75.9 24.1
25.0 75.0
Note: In bold are the average classification performance or accuracy of the linear discriminant models over six replicates for the testing set of samples a ‘Premium’ Jonagold apples were defined as: MT P 60 N and SSC P 12%; otherwise, they were defined as ‘Regular’. b ‘Premium’ Golden Delicious apples were defined as: MT P 55 N and SSC P 12%; otherwise, they were defined as ‘Regular’. c ‘Premium’ Delicious apples were defined as: MT P 60 N and SSC P 11%; otherwise, they were defined as ‘Regular’. d Actual ‘Premium’ and ‘Regular’ grades. e Classified ‘Premium’ and ‘Regular’ grades.
Table 5 Average confusion matrices for the classification of ‘Jonagold’, ‘Golden Delicious’, and ‘Delicious’ apples in the validation set by Magness–Taylor (MT) firmness and soluble solids content (SSC) using the scattering data preprocessed with continuous wavelet transform (OHSScwt). Confusion matrix (%) Jonagolda Premium
d
Golden Deliciousb d
d
Deliciousc d
Regular
Premium
Regular
Premiumd
Regulard
2009 Premiume Regulare
88.8 11.3
5.7 94.3
86.3 13.8
8.6 91.4
66.7 33.3
28.2 71.8
2010 Premiume Regulare
74.3 25.7
17.4 82.6
73.8 26.2
19.7 80.3
73.4 26.6
25.0 75.0
2011 Premiume Regulare
65.5 34.5
20.0 80.0
89.6 10.4
16.0 84.0
75.9 24.1
23.5 76.5
Note: In bold are the average classification performance or accuracy of the linear discriminant models over six replicates for the testing set of samples a ‘Premium’ Jonagold apples were defined as: MT P 60 N and SSC P 12%; otherwise, they were defined as ‘Regular’. b ‘Premium’ Golden Delicious apples were defined as: MT P 55 N and SSC P 12%; otherwise, they were defined as ‘Regular’. c ‘Premium’ Delicious apples were defined as: MT P 60 N and SSC P 11%; otherwise, they were defined as ‘Regular’. d Actual ‘Premium’ and ‘Regular’ grades. e Classified ‘Premium’ and ‘Regular’ grades.
67
F. Mendoza et al. / Journal of Food Engineering 125 (2014) 59–68
Table 6 Ten best selected features using the sequential forward selection (SFS) method from one repetition for sorting ‘Jonagold’ (JG), ‘Golden Delicious’ (GD) and ‘Delicious’ (D) apples based on the combination of firmness and soluble solids content using Vis/SWNIRS and scattering (OHSScwt) data.
a b
Technique
Cultivara & season
Ten best selected features using SFS methodb
Vis/SWNIR
JG 2009 GD 2011 RD 2010
506 513 474
645 646 655
648 716 754
658 729 820
792 752 872
801 779 944
877 861 997
952 934 1035
1005 951 1066
1046 1085 1091
OHSScwt
JG 2009 GD 2011 RD 2010
595 475 580
650 525 625
655 545 655
660 590 660
675 600 675
700 715 710
730 900 720
825 935 825
900 1025 920
1025 1030 1035
Selected features from one run for apples harvested in 2009, 2010, or 2011. In bold are the spectral features from the short-wave near-infrared region.
and seasons, probably due to the greater number of wavelengths evaluated for each apple (i.e., 641 data points). But using the scattering technique (which consisted of 121 data points), more consistent common features or wavelengths along the spectral range were found among the varieties and seasons (Table 6).
Considering 780 nm as the cutoff between the visible and NIR regions, it was noted that a larger number of features for the Vis/ SWNIR data were selected from the SWNIR region (6 out of 10 for JG and 7 out of 10 for D), with the exception of GD apples (4 out of 10). Contrarily, using the OHSScwt data, more features were selected from the visible region (7 out of 10 for JG and D apples and 6 out of 10 for GD apples). Fig. 3 depicts the classification performance using two of the best features to sort apples into ‘Premium’ and ‘Regular’ grades based on firmness and SSC as the selection criterion. 3.3. Discussion
Fig. 3. Examples of using the two best selected features for spectral scattering technique in sorting apples into ‘Premium’ and ‘Regular’ grades based on the combination of flesh firmness and soluble solids content. (a) ‘Jonagold’ in 2009, (b) ‘Golden Delicious’ in 2011, and (c) ‘Delicious’ in 2010. Dashed lines represent the decision line using the linear discriminant classifier. Note that the validation set of regular apples overlap those for premium quality apples.
Classification accuracies for firmness or SSC were, in most cases, lower than 90%, and were even lower when apples were graded based on both firmness and SSC. This suggests that the techniques need further research and improvement. As mentioned earlier, packinghouses are often faced with two situations, in which they want to either pack superior quality apples for higher prices or just remove inferior quality fruit. In the first situation, the priority is to ensure the quality of each fruit meets the minimum firmness and/ or SSC requirement. This may be achieved by setting the firmness or SSC threshold higher than the minimum level, thus minimizing the error of having soft or low SSC apples in the pack. In the second situation, the priority is to remove as fewer firm or sweet fruit as possible. This may be achieved by lowering the firmness or SSC threshold to below the minimum level. In the statistical terminology, the user can achieve the quality sorting goal by controlling the Type I (false positive) or II (false negative) error. The classification results showed that the two sensors had varying abilities and the optical information measured by them could be complementary. This suggests that better outcomes could be achieved by combining Vis/SWNIRS and scattering techniques based on a data fusion approach. By combining the strengths of both sensors, an ensemble sorting model would have higher accuracy and robustness than a single model. A sensitivity analysis confirmed the potential of Vis/SWNIR and OHSS techniques and robustness of their models in a wide range of quality thresholds for classification of three varieties of apples into two quality grades (i.e., ‘Premium’ and ‘Regular’) based on firmness or SSC. In the experimental conditions of this study, the analysis also revealed that with the exception of GD apples, Vis/SWNIR for firmness measurements performed slightly better than OHSS for JG and RD. Further, Vis/SWNIR showed significant advantages over OHSS in sorting apples by SSC for the three apple varieties (Fig. 2). In addition, the consistency of the sorting procedure is supported by a previous study (Mendoza et al., 2012) using SWNIRS and scattering technique for predicting firmness and soluble solids content based on partial least squares models (PLS). The current classification performances for the same set of apples and cultivars (2009 and 2010), in general, correlated well with the PLS
68
F. Mendoza et al. / Journal of Food Engineering 125 (2014) 59–68
regression models as expressed by the correlation coefficients of prediction (R), standard error of prediction (SEP) and estimated RPD values (ratio of SEP to sample standard deviation) obtained in this previous study for all cultivars. In should be noted, however, that Vis/SWNIR measurements were carried out offline which allowed better control of the samples, while scattering measurements were performed online. Hence, the results from Vis/SWNIR do not necessarily reflect how it would achieve in online applications. It is reasonable to expect that better results for the same sensor would be obtained offline than online. Therefore, further online experiments using Vis/ SWNIR and spectral scattering as well as using different thresholds for the combination of firmness and SSC are needed for more complete evaluation of the potential of each technique for apple quality sorting. In addition, additional challenges in using these sensing techniques for online sorting and grading applications need to be addressed. Among them is orienting fruit in the right orientation for reliable and accurate Vis/SWNIR and spectral scattering measurements. 4. Conclusion In this research, Vis/SWNIR and spectral scattering techniques were used to sort JG, GD and D apples into two quality grades based on firmness, SSC or the combination of both attributes. Consistent and good sorting results for firmness (ranging between 77.9% and 98.2%) and moderate sorting results for SSC (ranging between 62.0% and 91.7%) were obtained using scattering technique. Vis/SWNIR technique, which was operated in stationary condition, showed slightly better sorting results for firmness (ranging between 87.3% and 97.6%) and good results for SSC (ranging between 77.1% and 92.3%). A sensitivity analysis showed that with some exceptions for D apples, classification results by firmness or SSC were consistent for a wide range of quality threshold criteria, but they were in most of the cases significantly different among harvest seasons, sensing techniques and/or preprocessing methods. When the classification was performed based on both firmness and SSC, the sorting accuracies decreased, ranging between 75.7% and 90.1% for Vis/SWNIR and between 69.7% and 91.5% for spectral scattering. Both techniques, coupled with simple discriminant analysis, showed potential for sorting and grading apples by firmness or/and SSC. Further improvements, however, are needed for more accurate sorting and grading of apples by using for example sensor fusion approaches and probably more sophisticated classification techniques such as probabilistic neural network and decision tree. References Cen, H., Lu, R., Dolan, K., 2009. Optimization of inverse algorithm for estimating optical properties of biological materials using spatially-resolved diffuse reflectance. Inverse Problems in Science and Engineering 18, 853–872.
Dillon, W.R., Goldstein, M., 1984. Discriminant analysis: the two group problem. In: Multivariate Analysis: Methods and Applications. John Wiley & Sons, New York, pp. 360–393. Harker, F.R., 2001. Consumer response to apples. In: Proceedings of Washington Tree Fruit Postharvest Conference.