- Email: [email protected]
Prediction of Extrudate Properties Using Artificial Neural Networks
PREDICTION OF EXTRUDATE PROPERTIES USING ARTIFICIAL NEURAL NETWORKS T. J. Shankar and S. Bandyopadhyay Agricultural and Food Engineering Department, Indian Institute of Technology, Kharagpur, India
Abstract: A backpropagation artificial neural network (ANN) model was developed to predict the properties of extrudates generated by extrusion cooking of fish muscle-rice flour blend in a single screw extruder. Experimental data obtained in a previous study on extrudate properties of expansion ratio, bulk density and hardness at different combinations of operating variables of barrel temperature, feed content and feed moisture had been analysed using response surface methodology (RSM). A backpropagation neural network model was implemented in MATLAB and was trained for operating variables (inputs) and for each individual measured extrudate properties expansion ratio ER, bulk density BD and harndess H (outputs). The optimized network indicated that one hidden layer with a learning rate of 0.1, steep descent learning rule, 100 000 epochs and a logistic sigmoid transfer function predicted the extrudate properties better than RSM. The agreement of the ANN model with the experimental values, expressed as sum of squared error values, was 9.8 1027 for ER, 5.8 1022 for BD and 3.8 1023 for H. The ANN prediction for the optimized process conditions was superior to the RSM values, with percentage errors of þ6.06% (ER), þ4.08% (BD) and 214.28% (H). Keywords: extrusion cooking; prediction; extrudate properties; artificial neural network (ANN).
INTRODUCTION
Correspondence to: Dr S. Bandyopadhyay, Agricultural and Food Engineering, Department, Indian Institute of Technology, Kharagpur, India. E-mail: [email protected]
DOI: 10.1205/fbp.04205 0960–3085/07/ $30.00 þ 0.00 Food and Bioproducts Processing Trans IChemE, Part C, March 2007 # 2007 Institution of Chemical Engineers
(Harper, 1981). For non-linear problems, artificial neural networks (ANN) are a promising alternative technique. The capability of neural networks to solve such problems suggests that neural networks can become valuable tools for food processing systems involving estimation, prediction and control. A typical neural network consists of a sequence of layers with full connections between successive layers. There are usually two layers; an input layer and an output layer. The layer between the input and output layers is the hidden layer, whose connection weights (Wij and Wjo in Figure 1) are adjustable according to some learning rule. The connection weights determine the mappings between the input and outputs. The hidden unit and output unit in the network act to transfer the internal sum of the inputs to a potential output (Thyagarajan et al., 1998). There are several examples of application of ANN in food processing systems. These are mostly in drying and heat transfer studies such as control of drying systems (Thyagarajan et al., 1998); quality degradation during rice and maize drying (Perrot et al., 1998); modelling of drying process dynamics (Kaminski et al., 1998); optimum prediction of psychometric parameters (Sreekanth et al., 1998); evaluation of surface heat transfer coefficient (Sreekanth et al., 1999); and optimal condition of spray dried whole milk powder processing (Koc et al.,
Extrusion is an important food manufacturing process and extrudate properties are important for both product quality and process efficiency. Parameters such as maximum expansion ratio (ER), minimum bulk density (BD) and minimum hardness (H) can be related to product specific volume and textural attributesm, and can subsequently determine consumer acceptance. Therefore the ability to predict extrudate properties reliably could realize many benefits to food extrusion processing industries. In the literature, statistically designed experiments have been successfully used to systematically collect and correlate process data, and to describe the effects of the changes in operating variables (Harper, 1981). Response surface methodology (RSM) is one of the commonly used statistical data analysis techniques for identifying functional response models. Most RSM models are polynomial in nature, representing a reasonable approximation of the true functional relationships. However it is unlikely that the fitted models can represent the entire space of the independent variables. Rather they work quite well for a relatively small region of the experimental range (Montgomery, 1976). Because of the complexity and variability of the extruder feed ingredients, food extrusion mechanisms become highly non-linear 29
Vol 85 (C1) 29–33
30
SHANKAR and BANDYOPANDHYAY with low fat and bone content), and Indopacific tarpon (bony fish), were used for the experiments. The calculated amounts of fish powder and rice flour were dry mixed, adjusted to the desired level of moisture content by adding water, and then extruded. During extrusion, only the steady state output was taken as the sample product for analysis. The extrudate strands were dried at 60–658C for 2–2.5 h. The dry extrudate, featuring moisture contents of 8–15%, were kept in sealed polyethylene pouches and stored in refrigerator for further study (Giri and Bandyopadhyay, 2000). The extrudate properties, ER, BD and H of the dry products were measured by the methods described previously (Giri and Bandyopadhyay, 2000). The results are shown in Table 1. The 15 extruder runs of the previous work have been expanded into 45 data sets by using the triplicate measurements as individual values. The minimum and maximum SD values of the three extrudate properties from
Table 1. Experimental data for expansion ratio (ER), bulk density (BD) and hardness (H).
Figure 1. Schematic diagram of an artificial neural network.
1999). Other applications include thermal processing (Sablani et al., 1995); prediction of milk shelf life (VallejoCordoba et al., 1995); and dough rheological properties (Ruan et al., 1995). However, we are not aware of reported application of ANN to extrusion cooking. We believe that ANN analysis, being a powerful tool for solutions of nonlinear problems, is well suited to food extrusion processes. In order to demonstrate this, the present authors analysed the data generated by Giri and Bandyopadhyay (2000) during extrusion cooking of a fish muscle-rice flour blend in a single screw extruder by ANN. The specific objective is to develop and evaluate an ANN model to predict quality parameters like expansion ratio, bulk density and hardness individually for the processing conditions like barrel temperature, feed content and feed moisture.
MATERIALS AND METHODS Experimental Data The experimental data on extrusion cooking of fish musclerice flour blend were taken from the previous study by Giri and Bandyopadhyay (2000). The extruder used in their work was a laboratory model single screw, variable length cooking extruder indigenously developed to process animal feed and snack food for research and development purpose (Giri and Bandyopadhyay, 2000). The operating variables were barrel temperature, screw speed, feed moisture content and feed composition, i.e., the percentage of fish powder in the feed. The levels of the variables were fixed at barrel temperature of 100, 120, 1408C, percentage of fish contents of 10, 20 and 30% and feed moisture contents of 30, 40 and 50%. The design variables kept constant during extrusion were barrel length to diameter ratio (14), die diameter (4 mm), and die length at (15 mm). Dried fish mince of two varieties of marine fish, Bombay duck (Harpodon nehereus,
Expt. no. 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45
Barrel temperature 8C
Fish content of feed, %
Feed moisture content, %
ER
BD (kg m23)
H (N)
140 140 140 140 140 140 140 140 140 140 140 100 100 100 100 100 100 100 100 100 100 100 100 153.6 153.6 153.6 86.4 86.4 86.4 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120
30 30 30 30 30 30 10 10 10 10 10 30 30 30 30 30 30 10 10 10 10 10 10 20 20 20 20 20 20 36.8 36.8 36.8 3.2 3.2 3.2 20 20 20 20 20 20 20 20 20
30 30 30 50 50 50 30 30 30 50 50 30 30 30 50 50 50 30 30 30 50 50 50 40 40 40 40 40 40 40 40 40 40 40 40 56.8 56.8 56.8 23.2 23.2 23.2 40 40 40
1.56 1.37 1.75 1.96 1.79 2.13 2.80 2.56 3.04 2.98 3.32 1.20 1.03 1.36 1.26 1.06 1.45 1.50 1.22 1.78 1.50 1.40 1.60 1.90 1.68 2.12 1.22 1.08 1.36 1.44 1.35 1.52 2.64 2.39 2.89 1.82 1.67 1.96 1.26 1.12 1.34 1.56 1.40 1.71
780 744 816 796 763 828 532 505 559 714 729 805 787 822 994 978 1009 771 764 777 1090 1067 1112 823 806 839 918 902 933 791 777 804 767 754 779 856 841 870 703 673 705 848 833 863
17.91 15.39 20.43 35.00 32.45 37.55 8.80 7.60 10.00 17.61 25.59 16.50 15.42 17.58 27.00 22.96 31.04 10.70 9.07 12.33 20.00 18.43 21.57 16.70 9.83 8.57 14.30 11.82 16.78 26.20 23.93 28.47 13.65 10.11 17.19 58.00 52.70 63.30 20.13 17.73 22.53 22.80 20.83 22.47
Source: Giri and Bandyopadhyay (2000).
Trans IChemE, Part C, Food and Bioproducts Processing, 2007, 85(C1): 29–33
PREDICTION USING ARTIFICIAL NEURAL NETWORKS 15 experiments lie in the range of 0.085–0.28, 6.98 –36 and 1.08–6.87 for ER, BD and H, respectively. The previous authors used a five level central composite RSM design, where the factorial boundary is represented by 21 and þ1, the mid point by 0 and the star points by 21.68 and þ1.68. A second order response model was chosen to obtain the regression equations y ¼ bo þ
n X
bi xi þ
i¼1
n X
bii xi2 þ
i¼1
n X n X
bij xi xj þ 1
(1)
i¼1 j¼1
where y is the dependent variable, xi and xj are the coded independent variables, bo, bi, bij are coefficients, n is the number of independent variables and 1 is an unobservable error.
Learning rule A learning rule in which weights and biases are adjusted by error-derivative (delta) vectors and then backpropagated through the network is called the steep descent learning rule.
Learning rate The rate of change of the weights during the backpropagation phase, where the weights and biases are being adjusted to reduce the prediction errors, are controlled by the learning rate. The learning rate used in artificial multi layer feed forward neural networks is a numeric value used during backpropagation to adjust the weights or connections between neurons in adjacent layers. The learning rate is important because if the weights used are too small then it will take a long time to train the neural network and if they are too large, then the neural network may not find the set of weights that can be used reliably for prediction.
Network Parameters
Neural Network Architecture
The network parameters are listed below:
Mean squared error (MSE) MSE ¼ SðYd Yo Þ2 =N
ð2Þ
Sum squared error (SSE) MSE ¼ SðYd Yo Þ2
ð3Þ
Logistic sigmoid transfer function f (u) ¼
31
1 (1 þ eau )
ð4Þ
The 45 (15 3) data sets from Table 1 were considered as follows; 40 were used for training the network, and the remaining five data sets were at the star point (expts. 25, 28, 31, 34 and 37) of the five level central composite design; the data set at optimum process conditions were used for testing the network. Table 2 shows the details of the network parameters details evaluated for the three variables on the MATLAB platform. A single layer network was evaluated with three input nodes to indicate the three independent variables, and a single output node for each of the extrudate properties ER, BD and H (outputs). Three different networks were trained for three different training sessions for each of the extrudate properties ER, BD and H. ANN analysis was performed with MATLAB Version 5.1.0.421 (Math Works, Inc.).
RESULTS AND DISCUSSION where u is the variable and a is the parameter that determines the ‘abruptness’ of the transition, i.e., the slope of the curve near zero. The sigmoid transfer function takes the input, which may have any value between plus and minus infinity, and compresses the output into the range 0–1. This transfer function is commonly used in backpropagation networks, in part because it is differentiable.
The experimental data and the values predicted by RSM have been taken from the previous study. The results showed that the extrudate properties predicted by ANN were much closer to the experimental data than those predicted by RSM. This is reflected in the MSE values of the trained ANN predicted data, which were noticeably smaller than those for the RSM (Table 3). Table 4 indicates that the MSE values of the test cases using ANN were also smaller
Table 2. Artificial neural network details. Extrudate properties Expansion ratio Bulk density Product hardness
tp(1)
tp(2)
tp(3)
tp(4)
No. of nodes in hidden layer
SSE achieved
25 25 25
100 000 100 000 100 000
0.0000009 0.05 0.003
0.1 0.1 0.1
20 25 15
9.8 1027 5.8 1022 3.8 1023
Note 1.Training parameters are: tp(1), epochs between updating display; tp(2), iterations to train; tp(3), sum-squared error goal; and tp(4), learning rate. 2. Three layer feed-forward networks: Transfer function: logistic sigmoid transfer function. Learning rule: steep descent. Network type: train feed-forward network with back propagation. Number of nodes in the input layer: 3. Number of nodes in the output layer: 1. Trans IChemE, Part C, Food and Bioproducts Processing, 2007, 85(C1): 29–33
32
SHANKAR and BANDYOPANDHYAY Table 3. Mean squared error (MSE) values for extrudate properties for the trained data. MSE Extrudate properties Expansion ratio Bulk density Product hardness
ANN
RSM
0.0001 127.4 0.13
0.004 1965.7 0.84
Table 4. Mean squared error (MSE) values for extrudate properties for the tested data. MSE Extrudate properties
ANN
Expansion ratio Bulk density Product hardness
RSM
0.00018 7.20 0.75
0.00020 24.8 1.32
than the RSM, while Table 5 shows the agreement alongside the measured values for the test date. The predictions for the optimal process conditions are compared in Table 6, and the predicted extrudate properties using ANN are again closer to the experimental values than those obtained using RSM. The optimum numbers of nodes in the hidden layer of the network were obtained by trial and error. The logistic sigmoid
transfer function was used, with steep descent learning rule at a learning rate of 0.1, giving SSE values of 9.8 1027, 5.8 1022 and 3.8 1023 for ER, BD and H, respectively. The selection of the number of hidden layers, number of neurons in the hidden layers and the other training parameters like learning rate, learning rule are critical in the design of the back propagation network. It was noticed that with more than 40 nodes in the hidden layer, the SSE values diverged. With learning rates less than 0.05 and greater than 0.2, the network did not reach the goal. The network parameters in Table 2 were optimized by trial and error. It was also observed that the average predicted error improved using one hidden layer rather than two. It is therefore recommended that one hidden layer be the first choice for any practical feedforward network design. More hidden layers may cause over fitting, since the network focuses excessively on the idiosyncrasies of the individual samples. If a large number of hidden layer neurons does not yield a solution, then it may be worth using a second hidden layer and possibly reducing the total number of hidden layer neurons (Masters, 1993). Tables 3– 6 indicate that the ANN model was significantly more accurate than the RSM method. This work demonstrates that ANN can solve real multivariate complex problems like extrusion cooking, where it is difficult to develop a robust model using statistical methods. With the given data, the SSE values of 9.8 1027, 5.8 1022 and 3.8 1023 for ER, BD and H were encouraging.
Table 5. Experimental and predicted extrudate properties of the tested data. Independent variables Extrudate property
Predicted extrudate properties
Temp (8C)
Fish content (%)
Moisture content (%)
Experimental
ANN
Percent error
RSM
Percent error
Expansion ratio
153.6 86.4 120 120 120
20 20 36.8 3.2 20
40 40 40 40 56.8
1.90 1.22 1.44 2.64 1.82
1.82 1.20 1.48 2.67 1.88
þ4.2 þ1.6 þ2.7 21.1 23.2
1.79 1.09 1.59 2.43 2.19
þ5.7 þ10.6 210.4 þ7.9 219.7
Bulk density (kg m23)
153.6 86.4 120 120 120
20 20 36.8 3.2 20
40 40 40 40 56.8
823 918 791 767 856
812 930 777 785 845
þ1.3 21.3 þ1.7 þ2.3 þ1.2
765 969 839 812 894
þ7.0 25.5 26.0 25.8 24.4
Hardness (N)
153.6 86.4 120 120 120
20 20 36.8 3.2 20
40 40 40 40 56.8
16.70 14.30 26.20 13.65 58.60
16.59 13.13 27.17 15.18 61.12
þ0.7 þ8.2 23.7 þ11.2 24.3
17.19 11.17 25.12 17.29 65.29
22.9 þ21.9 þ4.1 226.7 211.4
Table 6. Experimental and predicted extrudate properties at optimum process variables. Optimum process variables (Giri and Bandyopadhyay, 2000) Independent variables Extrudate property Expansion ratio Bulk density (kg m23) Hardness (N)
Predicted extrudate properties
Temp (8C)
Fish content (%)
Moisture content (%)
Experimental
ANN
Percent error
RSM
Percent error
160 160 160
12 12 12
27 27 27
3.3 636 5.6
3.1 610 4.8
þ6.1 þ4.1 214.3
3.9 533 3.0
218.2 þ16.2 246.4
Trans IChemE, Part C, Food and Bioproducts Processing, 2007, 85(C1): 29–33
PREDICTION USING ARTIFICIAL NEURAL NETWORKS CONCLUSION A back propagation neural network was developed to predict the extrudate properties of ER, BD and H of a fish muscle-rice flour blend during extrusion in a single screw cooking extruder. The results showed that the values predicted by ANN were much closer to the experimental data than those predicted by RSM. It was observed that one hidden layer was sufficient for this neural network application; the average predicted error improved using one hidden layer rather than two. A change of learning rate significantly affected the prediction results. The extrudate properties predicted using ANN at optimum process conditions were closer to the experimental values when compared to RSM.
NOMENCLATURE BD ER H SD MSE SSE ANN tp W x y Yd Yo n u
bulk density, kg m23 expansion ratio hardness, N standard deviation mean squared error sum squared error artificial neural networks training parameter weight independent variable dependent variable desired or actual value estimated value number of the data entries variable
Greek symbols a parameter that determines the ‘abruptness’ of the transition near zero, i.e., slope of the curve near zero 1 unobservable error
Subscripts i,j variables
33
REFERENCES Giri, S.K. and Bandyopadhyay, S., 2000, Effect of extrusion variables on extrudate characteristics of fish muscle-rice flour blend in a single screw extruder, J Food Process Preserv, 24: 177– 190. Harper, J.M., 1981, Extrusion of Foods, Vol. 1 (CRC Press, Boca Raton, FL, USA). Kaminski, W., Strumillo, P. and Tomczak, E., 1998, Neurocomputing techniques to modeling of drying process dynamics, Drying Technol, 16(6): 967–991. Koc, A.B., Heinemann, H.P. and Ziegler, G.R., 1999, Determining the optimal conditions for spray dried whole milk powder process using genetic algorithms with neural networks, in An ASAE Meeting Presentation, 18– 21 July, paper no. 993133. Masters, T., 1993, Practical Neural Networks Recipes in Cþþ (Academic Press, Boston, USA). Montgomery, D.C., 1976, Design and Analysis of Experiments (John Wiley, New York, USA). Perrot, N., Bonazzi, C. and Trystram, G, 1998, Application of fuzzy rules-based models to prediction of quality degradation of rice and maize during hot air drying, Drying Technol, 16(8): 1533–1565. Ruan, R., Almaer, S. and Zhang, J., 1995, Prediction of dough rheological properties using neural networks, Cereal Chem, 72(3): 308– 311. Sablani, S.S., Ramaswamy, H.S. and Prasher, S.O., 1995, A neural network approach for thermal processing application, J Food Process Preserv, 19: 283–301. Sreekanth, S., Ramaswamy, H.S. and Sablani, S.S., 1998, Prediction of psychrometric parameters using neural networks, Drying Technol, 16(3–5): 825–837. Sreekanth, S., Ramaswamy, H.S., Sablani, S.S. and Prasher, S.O., 1999, A neural network approach for evaluation of surface heat transfer coefficient, J Food Process Preserv, 23: 329– 348. Thyagarajan, T., Shanmugam, J., Panda, R.C., Ponnavaikko, M. and Rao, P.G., 1998, Artificial neural networks: Principle and application to model based control of drying systems—a review, Drying Technol, 16(6): 931– 966. Vallejo-Cordoba, B., Arteaga, G.E. and Nakai, S., 1995, Predicting milk shelf-life based on artificial neural networks and headspace gas chromatographic data, J Food Science, 60(5): 885– 888. The manuscript was received 2 August 2004 and accepted for publication after revision 6 June 2006.
Trans IChemE, Part C, Food and Bioproducts Processing, 2007, 85(C1): 29–33
Abstract: A backpropagation artificial neural network (ANN) model was developed to predict the properties of extrudates generated by extrusion cooking of fish muscle-rice flour blend in a single screw extruder. Experimental data obtained in a previous study on extrudate properties of expansion ratio, bulk density and hardness at different combinations of operating variables of barrel temperature, feed content and feed moisture had been analysed using response surface methodology (RSM). A backpropagation neural network model was implemented in MATLAB and was trained for operating variables (inputs) and for each individual measured extrudate properties expansion ratio ER, bulk density BD and harndess H (outputs). The optimized network indicated that one hidden layer with a learning rate of 0.1, steep descent learning rule, 100 000 epochs and a logistic sigmoid transfer function predicted the extrudate properties better than RSM. The agreement of the ANN model with the experimental values, expressed as sum of squared error values, was 9.8 1027 for ER, 5.8 1022 for BD and 3.8 1023 for H. The ANN prediction for the optimized process conditions was superior to the RSM values, with percentage errors of þ6.06% (ER), þ4.08% (BD) and 214.28% (H). Keywords: extrusion cooking; prediction; extrudate properties; artificial neural network (ANN).
INTRODUCTION
Correspondence to: Dr S. Bandyopadhyay, Agricultural and Food Engineering, Department, Indian Institute of Technology, Kharagpur, India. E-mail: [email protected]
DOI: 10.1205/fbp.04205 0960–3085/07/ $30.00 þ 0.00 Food and Bioproducts Processing Trans IChemE, Part C, March 2007 # 2007 Institution of Chemical Engineers
(Harper, 1981). For non-linear problems, artificial neural networks (ANN) are a promising alternative technique. The capability of neural networks to solve such problems suggests that neural networks can become valuable tools for food processing systems involving estimation, prediction and control. A typical neural network consists of a sequence of layers with full connections between successive layers. There are usually two layers; an input layer and an output layer. The layer between the input and output layers is the hidden layer, whose connection weights (Wij and Wjo in Figure 1) are adjustable according to some learning rule. The connection weights determine the mappings between the input and outputs. The hidden unit and output unit in the network act to transfer the internal sum of the inputs to a potential output (Thyagarajan et al., 1998). There are several examples of application of ANN in food processing systems. These are mostly in drying and heat transfer studies such as control of drying systems (Thyagarajan et al., 1998); quality degradation during rice and maize drying (Perrot et al., 1998); modelling of drying process dynamics (Kaminski et al., 1998); optimum prediction of psychometric parameters (Sreekanth et al., 1998); evaluation of surface heat transfer coefficient (Sreekanth et al., 1999); and optimal condition of spray dried whole milk powder processing (Koc et al.,
Extrusion is an important food manufacturing process and extrudate properties are important for both product quality and process efficiency. Parameters such as maximum expansion ratio (ER), minimum bulk density (BD) and minimum hardness (H) can be related to product specific volume and textural attributesm, and can subsequently determine consumer acceptance. Therefore the ability to predict extrudate properties reliably could realize many benefits to food extrusion processing industries. In the literature, statistically designed experiments have been successfully used to systematically collect and correlate process data, and to describe the effects of the changes in operating variables (Harper, 1981). Response surface methodology (RSM) is one of the commonly used statistical data analysis techniques for identifying functional response models. Most RSM models are polynomial in nature, representing a reasonable approximation of the true functional relationships. However it is unlikely that the fitted models can represent the entire space of the independent variables. Rather they work quite well for a relatively small region of the experimental range (Montgomery, 1976). Because of the complexity and variability of the extruder feed ingredients, food extrusion mechanisms become highly non-linear 29
Vol 85 (C1) 29–33
30
SHANKAR and BANDYOPANDHYAY with low fat and bone content), and Indopacific tarpon (bony fish), were used for the experiments. The calculated amounts of fish powder and rice flour were dry mixed, adjusted to the desired level of moisture content by adding water, and then extruded. During extrusion, only the steady state output was taken as the sample product for analysis. The extrudate strands were dried at 60–658C for 2–2.5 h. The dry extrudate, featuring moisture contents of 8–15%, were kept in sealed polyethylene pouches and stored in refrigerator for further study (Giri and Bandyopadhyay, 2000). The extrudate properties, ER, BD and H of the dry products were measured by the methods described previously (Giri and Bandyopadhyay, 2000). The results are shown in Table 1. The 15 extruder runs of the previous work have been expanded into 45 data sets by using the triplicate measurements as individual values. The minimum and maximum SD values of the three extrudate properties from
Table 1. Experimental data for expansion ratio (ER), bulk density (BD) and hardness (H).
Figure 1. Schematic diagram of an artificial neural network.
1999). Other applications include thermal processing (Sablani et al., 1995); prediction of milk shelf life (VallejoCordoba et al., 1995); and dough rheological properties (Ruan et al., 1995). However, we are not aware of reported application of ANN to extrusion cooking. We believe that ANN analysis, being a powerful tool for solutions of nonlinear problems, is well suited to food extrusion processes. In order to demonstrate this, the present authors analysed the data generated by Giri and Bandyopadhyay (2000) during extrusion cooking of a fish muscle-rice flour blend in a single screw extruder by ANN. The specific objective is to develop and evaluate an ANN model to predict quality parameters like expansion ratio, bulk density and hardness individually for the processing conditions like barrel temperature, feed content and feed moisture.
MATERIALS AND METHODS Experimental Data The experimental data on extrusion cooking of fish musclerice flour blend were taken from the previous study by Giri and Bandyopadhyay (2000). The extruder used in their work was a laboratory model single screw, variable length cooking extruder indigenously developed to process animal feed and snack food for research and development purpose (Giri and Bandyopadhyay, 2000). The operating variables were barrel temperature, screw speed, feed moisture content and feed composition, i.e., the percentage of fish powder in the feed. The levels of the variables were fixed at barrel temperature of 100, 120, 1408C, percentage of fish contents of 10, 20 and 30% and feed moisture contents of 30, 40 and 50%. The design variables kept constant during extrusion were barrel length to diameter ratio (14), die diameter (4 mm), and die length at (15 mm). Dried fish mince of two varieties of marine fish, Bombay duck (Harpodon nehereus,
Expt. no. 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45
Barrel temperature 8C
Fish content of feed, %
Feed moisture content, %
ER
BD (kg m23)
H (N)
140 140 140 140 140 140 140 140 140 140 140 100 100 100 100 100 100 100 100 100 100 100 100 153.6 153.6 153.6 86.4 86.4 86.4 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120
30 30 30 30 30 30 10 10 10 10 10 30 30 30 30 30 30 10 10 10 10 10 10 20 20 20 20 20 20 36.8 36.8 36.8 3.2 3.2 3.2 20 20 20 20 20 20 20 20 20
30 30 30 50 50 50 30 30 30 50 50 30 30 30 50 50 50 30 30 30 50 50 50 40 40 40 40 40 40 40 40 40 40 40 40 56.8 56.8 56.8 23.2 23.2 23.2 40 40 40
1.56 1.37 1.75 1.96 1.79 2.13 2.80 2.56 3.04 2.98 3.32 1.20 1.03 1.36 1.26 1.06 1.45 1.50 1.22 1.78 1.50 1.40 1.60 1.90 1.68 2.12 1.22 1.08 1.36 1.44 1.35 1.52 2.64 2.39 2.89 1.82 1.67 1.96 1.26 1.12 1.34 1.56 1.40 1.71
780 744 816 796 763 828 532 505 559 714 729 805 787 822 994 978 1009 771 764 777 1090 1067 1112 823 806 839 918 902 933 791 777 804 767 754 779 856 841 870 703 673 705 848 833 863
17.91 15.39 20.43 35.00 32.45 37.55 8.80 7.60 10.00 17.61 25.59 16.50 15.42 17.58 27.00 22.96 31.04 10.70 9.07 12.33 20.00 18.43 21.57 16.70 9.83 8.57 14.30 11.82 16.78 26.20 23.93 28.47 13.65 10.11 17.19 58.00 52.70 63.30 20.13 17.73 22.53 22.80 20.83 22.47
Source: Giri and Bandyopadhyay (2000).
Trans IChemE, Part C, Food and Bioproducts Processing, 2007, 85(C1): 29–33
PREDICTION USING ARTIFICIAL NEURAL NETWORKS 15 experiments lie in the range of 0.085–0.28, 6.98 –36 and 1.08–6.87 for ER, BD and H, respectively. The previous authors used a five level central composite RSM design, where the factorial boundary is represented by 21 and þ1, the mid point by 0 and the star points by 21.68 and þ1.68. A second order response model was chosen to obtain the regression equations y ¼ bo þ
n X
bi xi þ
i¼1
n X
bii xi2 þ
i¼1
n X n X
bij xi xj þ 1
(1)
i¼1 j¼1
where y is the dependent variable, xi and xj are the coded independent variables, bo, bi, bij are coefficients, n is the number of independent variables and 1 is an unobservable error.
Learning rule A learning rule in which weights and biases are adjusted by error-derivative (delta) vectors and then backpropagated through the network is called the steep descent learning rule.
Learning rate The rate of change of the weights during the backpropagation phase, where the weights and biases are being adjusted to reduce the prediction errors, are controlled by the learning rate. The learning rate used in artificial multi layer feed forward neural networks is a numeric value used during backpropagation to adjust the weights or connections between neurons in adjacent layers. The learning rate is important because if the weights used are too small then it will take a long time to train the neural network and if they are too large, then the neural network may not find the set of weights that can be used reliably for prediction.
Network Parameters
Neural Network Architecture
The network parameters are listed below:
Mean squared error (MSE) MSE ¼ SðYd Yo Þ2 =N
ð2Þ
Sum squared error (SSE) MSE ¼ SðYd Yo Þ2
ð3Þ
Logistic sigmoid transfer function f (u) ¼
31
1 (1 þ eau )
ð4Þ
The 45 (15 3) data sets from Table 1 were considered as follows; 40 were used for training the network, and the remaining five data sets were at the star point (expts. 25, 28, 31, 34 and 37) of the five level central composite design; the data set at optimum process conditions were used for testing the network. Table 2 shows the details of the network parameters details evaluated for the three variables on the MATLAB platform. A single layer network was evaluated with three input nodes to indicate the three independent variables, and a single output node for each of the extrudate properties ER, BD and H (outputs). Three different networks were trained for three different training sessions for each of the extrudate properties ER, BD and H. ANN analysis was performed with MATLAB Version 5.1.0.421 (Math Works, Inc.).
RESULTS AND DISCUSSION where u is the variable and a is the parameter that determines the ‘abruptness’ of the transition, i.e., the slope of the curve near zero. The sigmoid transfer function takes the input, which may have any value between plus and minus infinity, and compresses the output into the range 0–1. This transfer function is commonly used in backpropagation networks, in part because it is differentiable.
The experimental data and the values predicted by RSM have been taken from the previous study. The results showed that the extrudate properties predicted by ANN were much closer to the experimental data than those predicted by RSM. This is reflected in the MSE values of the trained ANN predicted data, which were noticeably smaller than those for the RSM (Table 3). Table 4 indicates that the MSE values of the test cases using ANN were also smaller
Table 2. Artificial neural network details. Extrudate properties Expansion ratio Bulk density Product hardness
tp(1)
tp(2)
tp(3)
tp(4)
No. of nodes in hidden layer
SSE achieved
25 25 25
100 000 100 000 100 000
0.0000009 0.05 0.003
0.1 0.1 0.1
20 25 15
9.8 1027 5.8 1022 3.8 1023
Note 1.Training parameters are: tp(1), epochs between updating display; tp(2), iterations to train; tp(3), sum-squared error goal; and tp(4), learning rate. 2. Three layer feed-forward networks: Transfer function: logistic sigmoid transfer function. Learning rule: steep descent. Network type: train feed-forward network with back propagation. Number of nodes in the input layer: 3. Number of nodes in the output layer: 1. Trans IChemE, Part C, Food and Bioproducts Processing, 2007, 85(C1): 29–33
32
SHANKAR and BANDYOPANDHYAY Table 3. Mean squared error (MSE) values for extrudate properties for the trained data. MSE Extrudate properties Expansion ratio Bulk density Product hardness
ANN
RSM
0.0001 127.4 0.13
0.004 1965.7 0.84
Table 4. Mean squared error (MSE) values for extrudate properties for the tested data. MSE Extrudate properties
ANN
Expansion ratio Bulk density Product hardness
RSM
0.00018 7.20 0.75
0.00020 24.8 1.32
than the RSM, while Table 5 shows the agreement alongside the measured values for the test date. The predictions for the optimal process conditions are compared in Table 6, and the predicted extrudate properties using ANN are again closer to the experimental values than those obtained using RSM. The optimum numbers of nodes in the hidden layer of the network were obtained by trial and error. The logistic sigmoid
transfer function was used, with steep descent learning rule at a learning rate of 0.1, giving SSE values of 9.8 1027, 5.8 1022 and 3.8 1023 for ER, BD and H, respectively. The selection of the number of hidden layers, number of neurons in the hidden layers and the other training parameters like learning rate, learning rule are critical in the design of the back propagation network. It was noticed that with more than 40 nodes in the hidden layer, the SSE values diverged. With learning rates less than 0.05 and greater than 0.2, the network did not reach the goal. The network parameters in Table 2 were optimized by trial and error. It was also observed that the average predicted error improved using one hidden layer rather than two. It is therefore recommended that one hidden layer be the first choice for any practical feedforward network design. More hidden layers may cause over fitting, since the network focuses excessively on the idiosyncrasies of the individual samples. If a large number of hidden layer neurons does not yield a solution, then it may be worth using a second hidden layer and possibly reducing the total number of hidden layer neurons (Masters, 1993). Tables 3– 6 indicate that the ANN model was significantly more accurate than the RSM method. This work demonstrates that ANN can solve real multivariate complex problems like extrusion cooking, where it is difficult to develop a robust model using statistical methods. With the given data, the SSE values of 9.8 1027, 5.8 1022 and 3.8 1023 for ER, BD and H were encouraging.
Table 5. Experimental and predicted extrudate properties of the tested data. Independent variables Extrudate property
Predicted extrudate properties
Temp (8C)
Fish content (%)
Moisture content (%)
Experimental
ANN
Percent error
RSM
Percent error
Expansion ratio
153.6 86.4 120 120 120
20 20 36.8 3.2 20
40 40 40 40 56.8
1.90 1.22 1.44 2.64 1.82
1.82 1.20 1.48 2.67 1.88
þ4.2 þ1.6 þ2.7 21.1 23.2
1.79 1.09 1.59 2.43 2.19
þ5.7 þ10.6 210.4 þ7.9 219.7
Bulk density (kg m23)
153.6 86.4 120 120 120
20 20 36.8 3.2 20
40 40 40 40 56.8
823 918 791 767 856
812 930 777 785 845
þ1.3 21.3 þ1.7 þ2.3 þ1.2
765 969 839 812 894
þ7.0 25.5 26.0 25.8 24.4
Hardness (N)
153.6 86.4 120 120 120
20 20 36.8 3.2 20
40 40 40 40 56.8
16.70 14.30 26.20 13.65 58.60
16.59 13.13 27.17 15.18 61.12
þ0.7 þ8.2 23.7 þ11.2 24.3
17.19 11.17 25.12 17.29 65.29
22.9 þ21.9 þ4.1 226.7 211.4
Table 6. Experimental and predicted extrudate properties at optimum process variables. Optimum process variables (Giri and Bandyopadhyay, 2000) Independent variables Extrudate property Expansion ratio Bulk density (kg m23) Hardness (N)
Predicted extrudate properties
Temp (8C)
Fish content (%)
Moisture content (%)
Experimental
ANN
Percent error
RSM
Percent error
160 160 160
12 12 12
27 27 27
3.3 636 5.6
3.1 610 4.8
þ6.1 þ4.1 214.3
3.9 533 3.0
218.2 þ16.2 246.4
Trans IChemE, Part C, Food and Bioproducts Processing, 2007, 85(C1): 29–33
PREDICTION USING ARTIFICIAL NEURAL NETWORKS CONCLUSION A back propagation neural network was developed to predict the extrudate properties of ER, BD and H of a fish muscle-rice flour blend during extrusion in a single screw cooking extruder. The results showed that the values predicted by ANN were much closer to the experimental data than those predicted by RSM. It was observed that one hidden layer was sufficient for this neural network application; the average predicted error improved using one hidden layer rather than two. A change of learning rate significantly affected the prediction results. The extrudate properties predicted using ANN at optimum process conditions were closer to the experimental values when compared to RSM.
NOMENCLATURE BD ER H SD MSE SSE ANN tp W x y Yd Yo n u
bulk density, kg m23 expansion ratio hardness, N standard deviation mean squared error sum squared error artificial neural networks training parameter weight independent variable dependent variable desired or actual value estimated value number of the data entries variable
Greek symbols a parameter that determines the ‘abruptness’ of the transition near zero, i.e., slope of the curve near zero 1 unobservable error
Subscripts i,j variables
33
REFERENCES Giri, S.K. and Bandyopadhyay, S., 2000, Effect of extrusion variables on extrudate characteristics of fish muscle-rice flour blend in a single screw extruder, J Food Process Preserv, 24: 177– 190. Harper, J.M., 1981, Extrusion of Foods, Vol. 1 (CRC Press, Boca Raton, FL, USA). Kaminski, W., Strumillo, P. and Tomczak, E., 1998, Neurocomputing techniques to modeling of drying process dynamics, Drying Technol, 16(6): 967–991. Koc, A.B., Heinemann, H.P. and Ziegler, G.R., 1999, Determining the optimal conditions for spray dried whole milk powder process using genetic algorithms with neural networks, in An ASAE Meeting Presentation, 18– 21 July, paper no. 993133. Masters, T., 1993, Practical Neural Networks Recipes in Cþþ (Academic Press, Boston, USA). Montgomery, D.C., 1976, Design and Analysis of Experiments (John Wiley, New York, USA). Perrot, N., Bonazzi, C. and Trystram, G, 1998, Application of fuzzy rules-based models to prediction of quality degradation of rice and maize during hot air drying, Drying Technol, 16(8): 1533–1565. Ruan, R., Almaer, S. and Zhang, J., 1995, Prediction of dough rheological properties using neural networks, Cereal Chem, 72(3): 308– 311. Sablani, S.S., Ramaswamy, H.S. and Prasher, S.O., 1995, A neural network approach for thermal processing application, J Food Process Preserv, 19: 283–301. Sreekanth, S., Ramaswamy, H.S. and Sablani, S.S., 1998, Prediction of psychrometric parameters using neural networks, Drying Technol, 16(3–5): 825–837. Sreekanth, S., Ramaswamy, H.S., Sablani, S.S. and Prasher, S.O., 1999, A neural network approach for evaluation of surface heat transfer coefficient, J Food Process Preserv, 23: 329– 348. Thyagarajan, T., Shanmugam, J., Panda, R.C., Ponnavaikko, M. and Rao, P.G., 1998, Artificial neural networks: Principle and application to model based control of drying systems—a review, Drying Technol, 16(6): 931– 966. Vallejo-Cordoba, B., Arteaga, G.E. and Nakai, S., 1995, Predicting milk shelf-life based on artificial neural networks and headspace gas chromatographic data, J Food Science, 60(5): 885– 888. The manuscript was received 2 August 2004 and accepted for publication after revision 6 June 2006.
Trans IChemE, Part C, Food and Bioproducts Processing, 2007, 85(C1): 29–33