- Email: [email protected]
Preemptive identification of optimum fermentation time for black tea using electronic nose
Available online at www.sciencedirect.com
Sensors and Actuators B 131 (2008) 110–116
Preemptive identification of optimum fermentation time for black tea using electronic nose Nabarun Bhattacharya a,∗ , Bipan Tudu b , Arun Jana a , Devdulal Ghosh a , Rajib Bandhopadhyaya b , Manabendra Bhuyan c b
a Centre for Development of Advance Computing (C-DAC), Kolkata 700091, India Department of Instrumentation and Electronics Engineering, Jadavpur University, Kolkata, India c Tezpur University, Assam, India
Available online 23 December 2007
Abstract During black tea manufacturing, tealeaves pass through the fermentation process, when the grassy smell is transformed into a floral smell. Optimum fermentation is extremely crucial in deciding the final quality of finished tea and it is very important to terminate the fermentation process at the right time. Present day industry practice for monitoring of fermentation is purely subjective and is carried out by experienced personnel. In this paper, a study has been made on real-time smell monitoring of black tea during the fermentation process using electronic nose as well as prediction of the correct fermentation time. The study has been implemented in two steps. First, for prediction of optimum fermentation time, five different time-delay neural networks (TDNNs), named as multiple-time-delay neural networks (m-TDNN), have been used. During the second study, we have investigated the possibility of existence of different smell stages during the fermentation runs of black tea processing using self-organizing map (SOM), and then used three TDNNs for different smell stages. The results show excellent promise for the instrument to be used by the industry. © 2007 Elsevier B.V. All rights reserved. Keywords: Electronic nose; Black tea; Fermentation; Sensors; Self-organizing map (SOM); Time-delay neural network (TDNN)
1. Introduction Black tea is produced from the plant called Camellia sinensis. After the tealeaves are plucked from the C. sinensis plant, a number of processing stages, viz., withering, pre-conditioning, cut-tear-curl operation (CTC), fermentation and drying are involved in producing finished black tea. Out of these stages, the fermentation process is extremely crucial [1], where the residence time of tealeaves on the fermentation floor, trough or conveyor plays the pivotal role in deciding final quality of finished black tea. In this process, tealeaves change colour from green to coppery brown or black and grassy smell of leaves transforms into floral smell. The duration of the fermentation process varies due to temperature, humidity, and location of tea factory and month of the year. The optimum fermentation time for black tea man-
∗ Corresponding author at: Centre for Development of Advanced Computing (C-DAC), E-2/1, Block-GP, Sector-V, Salt Lake, Kolkata 700091, West Bengal, India. E-mail address: [email protected] (N. Bhattacharya).
0925-4005/$ – see front matter © 2007 Elsevier B.V. All rights reserved. doi:10.1016/j.snb.2007.12.032
ufacturing process may vary from 40 to 120 min in Indian conditions. To ensure optimum fermentation, the entire fermentation process should be carefully monitored and the process should be terminated once the optimum fermentation point is reached. To this end, it is extremely beneficial to predict the optimum fermentation time at an earlier point in time using an instrumental method. It is also observed that during the fermentation process, smell intensity is different at different instant of time and a temporal smell pattern occurs. Electronic nose has been applied successfully for several applications related to quality of food and agro products. On black tea, pioneering work has been done by Dutta et al. [2] for classification of black tea aroma. Correlation of “tea-tasters” marks with the electronic nose signature has been successfully derived in Ref. [3]. In the previous studies by the authors [4,5], electronic nose-based monitoring of volatile emission patterns during black tea fermentation process and detection of the optimum fermentation time on the basis of peaks in the sensor outputs have been successfully achieved. Various methodologies such as singular value decomposition (SVD) and Mahalanobis distance computation
N. Bhattacharya et al. / Sensors and Actuators B 131 (2008) 110–116
have been made use of to evolve aroma index at any particular sniffing cycle. The computed aroma indices have been plotted against time to draw the fermentation aroma profile. In this paper, we present the methodology and results of online prediction of optimum fermentation time. Because of the inherent nature of temporal pattern in the electronic nose data during black tea fermentation process, time-delay neural network (TDNN) has been selected for the on-line prediction [6]. Our approach is somewhat similar to [7], where multiple classifier system has been applied on olfactory signals, but we have used only TDNN models for the prediction. Multiple-TDNN architectures, named as m-TDNN, have been constructed and on-line switching between these networks is done as the fermentation process progresses. Each TDNN comprises of different training sets of data. For example, the first (TDNN 1) model is trained by first 20 min of data collected from electronic nose during the process of fermentation, second one is trained by first 25 min of data and so on. In the second step of our study, unsupervised method of clustering, the self-organizing map (SOM) has been applied for investigating the existence of smell stages. Four distinct smell clusters have been obtained on the data for all the fermentation runs, out of which the last stage represents over-fermentation. The first three clusters have been modeled by three independent TDNN algorithms for the prediction of the optimum time of fermentation. The novelty of the method is that this is the first attempt towards on-line prediction of optimum fermentation time during black tea processing using an electronic nose and is expected to be extremely beneficial for the user industry.
111
2. Customized electronic nose for black tea fermentation An electronic nose uses an array of non-specific broadly tuned sensors to discriminate odours by analyzing sensor array data with pattern recognition methods [8]. A customized electronic nose set-up has been developed such that the same can be used in production floor of tea processing units for monitoring of volatile emission pattern during the fermentation process. The electronic nose consists of (a) sensor array, (b) micro-pump with programmable sequence control, (c) PC-based data acquisition and (d) olfaction software as illustrated in Fig. 1. For black tea, an array of metal oxide semiconductor (MOS) sensors has been used for assessment of volatiles in the set-up. A series of experiments were carried out using a number of commercially available MOS sensors. From the response sensitivity of individual sensors, a set of eight gas sensors from Figaro, Japan (TGS-832, TGS-823, TGS-831, TGS-816, TGS-2600, TGS-2610, TGS-2611 and TGS-2620) has been selected for odour capture in fermentation process of black tea. The outputs of the sensors are acquired in the PC through PCI data acquisition cards. The MOS sensors are conductometric in nature, and their resistance decreases when subjected to the odour vapour molecules. The change in resistance with respect to their original values (R/R) is converted into voltage and then taken to the PC through analog to digital converter cards for subsequent analysis in the computational models. The experimental sniffing cycle consists of automated sequence of internal operations: (i) headspace generation, (ii) sampling, (iii) purging and (iv) dormancy before the start of the
Fig. 1. Customized electronic nose set-up.
112
N. Bhattacharya et al. / Sensors and Actuators B 131 (2008) 110–116
next sniffing cycle. Headspace generation ensures adequate concentration of volatiles released by tea within the sample holder by blowing regulated flow of air on the sample. During sampling, the sensor array is exposed to a constant flow of volatiles through pipelines inside the electronic nose system. During purging operation, sensor heads are cleared with blow of fresh air so that the sensors can go back to their baseline values. The programmable time dormancy cycle is the suspended mode of the electronic nose between two consecutive sniffing cycles. The PC-based data acquisition and automated operation of all these cycles are controlled by the specially designed software, called olfaction software. The software has got features like programmable sequence control, dynamic fermentation profile display, data logging, alarm annunciation, data archival, etc. The software has been developed in LabVIEW® of National Instruments. Experimental conditions during the investigation of the different stages of the fermentation process of black tea are as follows: • • • • • •
amount of fermenting tea sample = 50 g, temperature = ambient, headspace generation time = 30 s, data collection time = 50 s, purging time = 100 s, airflow rate = 5 ml/s.
3. Experimental 3.1. Place and time of experiments Experiments were carried out in three locations in different tea gardens in north India as mentioned in Table 1. Both orthodox and CTC types of black tea samples have been used in the experiments while investigating the dynamic appearance of “smell bands” during black tea fermentation process. The experiments at the Glenburn Tea Estate, Darjeeling, West Bengal, India were based on fermenting cycles of orthodox and flavoury tea. The experiments at Tocklai Experimental Station, Jorhat, Assam, India was with CTC type of black tea on a miniature manufacturing facility with controlled environment using fast fermenting clones. The Dalmia Tea Plantations, West Bengal, India is basically a “bought leaf factory” producing huge quantities of CTC type black tea daily. Longest trials were done at this factory. Details of fermentation trials in respective tea industries carried out under the present study are listed in Table 1.
Fig. 2. Data matrix formed out of sensor array output.
3.2. Data analysis and experimental results In the present electronic nose-based study on black tea fermentation process, each sniffing cycle produces a huge amount of eight-dimensional data. In our customized system, each sniffing cycle consists of headspace generation (30 s), sampling (50 s) and purging (100 s) operations. Computer acquires sensor data during headspace generation and sampling cycles only in an automated sequence of operation. Clearly, the sensor outputs will be at their baseline during the headspace generation and significant variation in sensor outputs will be observed during the sampling cycle. Therefore, the data matrix stored in the computer in each sniffing cycle will consist of a mixture of both baseline as well as actual sensor responses as shown in Fig. 2. In the above data matrix, the segment b11 –bh8 represents baseline responses of the sensors during headspace generation and S11 –Sm8 represents the sensor responses when exposed to tea odour during sampling. In our study, maximum value of Sij for each column has been found out and a vector M is formed with these maximum values. So M = [ Si1 max
· · · Si8 max ]
(1)
For each fermentation run, the vectors comprising of the maximum values associated with each sniffing cycle are considered for data analysis. In the previous studies by the authors for monitoring of black tea fermentation, it was observed that the smell intensity, as measured by electronic nose, passes through two distinct peaks during fermentation, and the second peak denotes the optimum fermentation point. This point was validated by expert human
Table 1 Experiment details Experiment serial no.
Location
Type of tea
Time
Number of fermentation cycles
1.
Glenburn Tea Estate, Darjeeling, West Bengal, India
Orthodox flavoury tea
August 2005
10
2.
Tocklai Experimental Station, Jorhat, Assam, India
CTC
September 2005
10
3
Dalmia Tea Plantations, West Bengal, India
CTC
October 2005 November 2005 December 2005
22 15 24
N. Bhattacharya et al. / Sensors and Actuators B 131 (2008) 110–116
113
Fig. 3. Architecture of time-delay neural network.
panel as well as by chemical tests like the colorimetric tests. From the on-line aroma profiling using electronic nose, optimum fermentation point is known only when it has already crossed, resulting in over-fermentation and degradation of final tea quality, however small. This section presents an on-line prediction algorithm for optimum fermentation time. The smell profile during black tea fermentation shows a temporal nature and time-delay neural network is selected because of its inherent capability to handle time varying patterns [6]. 3.2.1. The time-delay neural network TDNN is a dynamic neural network that is constructed by embedding local memory in both input and hidden layers of a multi-layer feed–forward neural network as shown in Fig. 3. This algorithm has been successfully applied for prediction of Carbonation Tower’s Temperature [9]. We have used the temporal back propagation used to train the model. 3.2.1.1. The training algorithm. The temporal back propagation algorithm is a supervised learning algorithm. The output of TDNN is compared with the desired response and the error is calculated and by temporal back propagation algorithm and this error is minimized until the desired tolerance level is obtained. The weights connecting the two layers of a TDNN are updated by the following equation: ¯ ji (n + 1) = w ¯ ji (n) + ηδj (n)¯xi (n) w
(2)
¯ ji = [wji (0), wji (1), . . . , wji (p)] and wji (l) denotes the where w weight connecting the output of neuron j to input of neuron i and l indicates the order of tapped delay and n indicates the number of iterations. η indicates the learning coefficient and is 0 < η < 1. Here x¯ i (n) = [xi (n), xi (n − 1), . . . xi (n − p)] and xi (n − p) is the input signal at ith layer at pth delay node. δj is the local gradient for neuron j and δj = ej (n)f (vj (n))
(3)
Eq. (2) is applicable if neuron j is in output layer. Here ej (n) is the error signal of jth neuron at nth iteration, vj (n) is the induced local field and f (·) is the derivative of the activation function f(·). If neuron j is in hidden layer then Eq. (3) is applied. ¯ rj δj (n) = f (vj (n)) Tr (n)w (4) r∈A
where A is the set of all neurons whose input is fed by neuron j in a forward manner and neuron r belongs to the set A and r (n) = [δr (n), δr (n + 1), . . . , δr (n + p)]T
(5)
3.2.2. Prediction of optimum fermentation time using multiple-TDNN Electronic nose data has been collected from 81 fermentation runs as mentioned in Table 1. For all the experiments, the maximum values of the sensor responses are integrated to form a large data matrix for m-TDNN-based analysis. Data obtained from randomly chosen 51 runs have been used for training of the m-TDNN and data from rest 30 runs have been used for evaluation of performance of the prediction models. In the experiments carried out, the sniffing cycle was repeated every 5 min and in each cycle, the maximum response from each sensor was recorded in a matrix. While the number of columns of this matrix would be equal to the number of sensors in the sensor array (which is eight in our case), the number of rows will depend on the duration of the fermentation process. In the experiments, the fermentation completion time varied from 40 to 120 min and the prediction algorithm is initiated after 20 min from the beginning of the fermentation cycle. Prior to that, the TDNN model has been observed to predict unreasonable results because of very few input data. We have trained five different time-delay neural network (m-TDNN) architectures with number of delays in input nodes (i) from 3 to 7 keeping fermentation completion time as target as shown in Fig. 3. The first TDNN model (TDNN 1 in Fig. 3) is trained by first 20 min of fermented data, second one (TDNN 2 in Fig. 3) is trained
114
N. Bhattacharya et al. / Sensors and Actuators B 131 (2008) 110–116
Fig. 4. Block diagram of the proposed prediction model.
by first 25 min of fermented data and the remaining models are trained by first 30, 35 and 40 min of fermented data, respectively. Comparing the number of on-line values available for prediction of fermentation completion time with the input delay nodes of the trained models, the fermentation completion time is obtained activating one model at one time. Figs. 4 and 5 show the main structure of the prediction model based on five TDNN algorithms. On-line responses from the sensor array are normalized in the preprocessing block in Fig. 4 and these normalized values lie between 0 and 1. The selector block directs the on-line test data towards the corresponding TDNN. For example, if the online test data were associated with 20 min data then TDNN 1 model is activated. 3.2.3. Results with multiple-TDNN In total, 30 fermentation cycles were experimented with this proposed model. Accuracies of different TDNN models are shown in Table 2. From the results, it is observed that with less data, uncertainty of prediction is more. But with the progress of time, as more data are available, accuracy increases. In practice, the requirement is to terminate the fermentation process at right time and prediction of the correct fermentation time few min-
Table 2 Summary of results Model
Accuracy
TDNN 1 TDNN 2 TDNN 3 TDNN 4 TDNN 5
0.9625 0.8568 0.9756 0.9932 0.9989
Table 3 Smell stage duration Smell stage
Characteristics
Smell stage 1 (T0–T1) Smell stage 2 (T1–T2) Smell stage 3 (T2–T3) Smell stage 4 (T3–T4)
Grassy smell stage Under fermentation stage Optimum fermentation stage Over-fermentation stage
utes ahead suffices to maintain the quality of the finished product. The method described above employing multiple-TDNN models thus promises to satisfy the objective. But the method requires a large number of networks, and in the subsequent sections an attempt has been made to reduce the number of such networks using self-organizing map. 3.2.4. SOM-based smell stage partition The self-organizing map (SOM), developed by Kohonen [10] is an unsupervised learning algorithm and the principal goal of it is to transform an incoming signal pattern of arbitrary dimension into a two-dimensional discrete map and to transform this adaptively in a topologically ordered fashion [11]. So neurons in this neural network are arranged in a twodimensional grid and there happens a competition among these neurons to represent the input pattern. After this, the winning neurons and the similar pattern neurons, i.e. the neighboring
Fig. 5. Activation of different TDNN models with time.
Fig. 6. Stages during the fermentation process.
N. Bhattacharya et al. / Sensors and Actuators B 131 (2008) 110–116
115
Fig. 7. SOM-based smell partitions.
neurons are placed in contiguous locations in output space. This section explores the existence of distinguishable smell stages during the fermentation process by electronic nose and then presents a prediction algorithm for the optimum fermentation time. The same dataset, which has been used in the multiple-TDNN model as explained in the previous section, is considered for unsupervised clustering using SOM to identify the distinct smell stages during the fermentation process. Out of the entire data of 81 fermentation runs, randomly chosen data from 51 runs have been used for clustering using SOM and data from rest 30 runs have been used for evaluation of performance of the smell stage-based prediction model. 3.2.4.1. Results with SOM and discussion. As already explained, the training dataset, consisting of electronic nose data for 51 fermentation runs, has been fed to the SOM model. Four distinct smell clusters have been obtained as shown in Fig. 7. This establishes the fact that, irrespective of fermentation duration, the transformation of smell from grassy to floral aroma traverses through four defined stages. These smell stages appear in cascade and the width of individual smell bands depend on the overall fermentation time, which is dependent on type of leaves, location of its production and season of processing. In consultation with the tea industries, the four identified stages of black tea fermentation are mentioned in Table 3. The thematic smell bands identified by electronic nose have been shown in Fig. 6 and the cluster formation for each stage by SOM is shown in Fig. 7. A closer look at the smell bands reveals the following facts:
3.2.5. Smell stage-based TDNN model for prediction of optimum fermentation time The self-organizing map as shown in Fig. 7 shows distinct clusters for the four stages of the fermentation process. Since the last stage is for the over-fermented tea, it does not contribute for the prediction model and three stages prior the optimum point of fermentation is considered for the computation. These three TDNNs are named as (a) time-delay neural network trained with the data of grassy smell stage (GSS-TDNN), (b) time-delay neural network trained with the data of under fermentation stage (UFS-TDNN) and (c) time-delay neural network trained with the data of optimum fermentation stage (OFS-TDNN). The advantage of this method is that the number of delay taps is much less in these TDNN models as compared to the models of the previous method. This SOM and TDNN model is simpler to implement and computational complexity is much less and have shown to be adequate to serve the purpose of on-line prediction of optimum fermentation time. 4. Conclusion It may be concluded from the above study that during black tea fermentation process, volatile emission follows fixed patterns of smell signatures with time, which is manifested in the form of distinct smell stages as detected by electronic nose. Two methods for the prediction of optimum fermentation time have been discussed and both the methods appear to be suitable for the purpose. However, the combined SOM- and TDNN-based prediction algorithm proves to be the better alternative as the computational complexity is relatively less. References
(a) The time span of the optimum fermentation band is very short compared to other stages of smell. (b) Grassy smell duration is more than the optimum fermentation duration but less than the under fermentation duration. (c) The under fermentation phase persists for considerable length of time.
[1] H. Co, G.W. Sanderson, Biochemistry of tea fermentation: conversion of amino acids to black tea aroma constituents, J. Food Sci. 35 (1970) 160–164. [2] R. Dutta, E.L. Hines, J.W. Gardner, K.R. Kashwan, M. Bhuyan, Tea quality prediction using a tin oxide-based electronic nose: an artificial intelligence approach, Sens. Actuators B: Chem. 94 (2003) 228–237.
116
N. Bhattacharya et al. / Sensors and Actuators B 131 (2008) 110–116
[3] N. Bhattacharyya, R. Bandyopadhyay, M. Bhuyan, A. Ghosh, R.K. Mudi, Correlation of multi-sensor array data with “tasters” panel evaluation for objective assessment of black tea flavour, in: International Proceedings of the ISOEN-2005, Barcelona, Spain, April 13–15, 2005. [4] N. Bhattacharyya, S. Seth, B. Tudu, P. Tamuly, A. Jana, D. Ghosh, R. Bandyopadhyay, M. Bhuyan, Monitoring of black tea fermentation process using electronic nose, J. Food Eng. 80 (2007) 1146–1156. [5] N. Bhattacharyya, S. Seth, B. Tudu, P. Tamuly, A. Jana, D. Ghosh, R. Bandyopadhyay, M. Bhuyan, S. Sabhapandit, Detection of optimum fermentation time for black tea manufacturing using electronic nose, Sens. Actuators B: Chem. 122 (2007) 627–634. [6] A. Waibel, T. Hanazawa, G. Hinton, K. Shikano, K.J. Langa, Phoneme recognition using time delay neural networks, IEEE Trans. Acoust. Speech Signal Process. 37 (1989) 328–339. [7] E. Phaisangittisagul, H.T. Nagle, Enhancing multiple classifier system performance for machine olfaction using odor-type signatures, Sens. Actuators B: Chem. 125 (2007) 246–253. [8] T.C. Pearce, S.S. Schiffman, H.T. Nagle, J.W. Gardner (Eds.), A Handbook of Machine Olfaction, Wiley–VCH, 2003. [9] Dan Shi, Hongjian Zhang, Liming Yang, Time-delay neural network for the prediction of Carbonation Tower’s Temperature, IEEE Trans. Inst. Met. 52 (2003) 1125–1128. [10] T. Kohonen, Self Organizing Maps, 3rd ed., Springer, 2001. [11] S. Haykin, Neural Networks, A Comprehensive Foundation, 2nd ed., Pearson Education, Asia, 1998.
Biographies Nabarun Bhattacharyya is additional director in Centre for Development of Advanced Computing (C-DAC), Kolkata, India, which is a premier R&D Institute under Department of Information Technology, Government of India. He received his BE degree in electronics and telecommunication engineering from
Jadavpur University, Kolkata, India in the year 1984. His research areas are agri-electronics, machine olfaction, soft computing and pattern recognition. Bipan Tudu received his MTech degree in instrumentation and electronics engineering in 2004 from the Jadavpur University, Kolkata, India. He is currently a lecturer in the Department of Instrumentation and Electronics Engineering. His main research interest includes pattern recognition, artificial intelligence and machine olfaction. Arun Jana is presently a project engineer in Centre for Development of Advanced Computing (C-DAC), Kolkata, India, which is a premier R&D Institute under Department of Information Technology, Government of India. He received his master in computer applications (MCA) from the Utkal University, India in the year 2004. His research interests are machine learning and soft computing. Devdulal Ghosh is presently a project engineer in Centre for Development of Advanced Computing (C-DAC), Kolkata, India, which is a premier R&D Institute under Department of Information Technology, Government of India. He received his BE degree in computer science from the Burdwan University, India in the year 2004. His research interests are virtual instrumentation, software engineering and object oriented programming. Rajib Bandopadhyay received his PhD degree in the year 2001 and currently a professor in the Department of Instrumentation Engineering, Jadavpur University, Kolkata, India. His research interests are in the fields of machine olfaction, intelligent control and NQR-based instrumentation systems. He is a fellow of IETE (India). Manabendra Bhuyan is presently a professor in the Department of Electronics, Tezpur University, India. He received his PhD degree in the year 1998. His research interests are in the fields of electronic instrumentation, machine olfaction and machine vision.
Sensors and Actuators B 131 (2008) 110–116
Preemptive identification of optimum fermentation time for black tea using electronic nose Nabarun Bhattacharya a,∗ , Bipan Tudu b , Arun Jana a , Devdulal Ghosh a , Rajib Bandhopadhyaya b , Manabendra Bhuyan c b
a Centre for Development of Advance Computing (C-DAC), Kolkata 700091, India Department of Instrumentation and Electronics Engineering, Jadavpur University, Kolkata, India c Tezpur University, Assam, India
Available online 23 December 2007
Abstract During black tea manufacturing, tealeaves pass through the fermentation process, when the grassy smell is transformed into a floral smell. Optimum fermentation is extremely crucial in deciding the final quality of finished tea and it is very important to terminate the fermentation process at the right time. Present day industry practice for monitoring of fermentation is purely subjective and is carried out by experienced personnel. In this paper, a study has been made on real-time smell monitoring of black tea during the fermentation process using electronic nose as well as prediction of the correct fermentation time. The study has been implemented in two steps. First, for prediction of optimum fermentation time, five different time-delay neural networks (TDNNs), named as multiple-time-delay neural networks (m-TDNN), have been used. During the second study, we have investigated the possibility of existence of different smell stages during the fermentation runs of black tea processing using self-organizing map (SOM), and then used three TDNNs for different smell stages. The results show excellent promise for the instrument to be used by the industry. © 2007 Elsevier B.V. All rights reserved. Keywords: Electronic nose; Black tea; Fermentation; Sensors; Self-organizing map (SOM); Time-delay neural network (TDNN)
1. Introduction Black tea is produced from the plant called Camellia sinensis. After the tealeaves are plucked from the C. sinensis plant, a number of processing stages, viz., withering, pre-conditioning, cut-tear-curl operation (CTC), fermentation and drying are involved in producing finished black tea. Out of these stages, the fermentation process is extremely crucial [1], where the residence time of tealeaves on the fermentation floor, trough or conveyor plays the pivotal role in deciding final quality of finished black tea. In this process, tealeaves change colour from green to coppery brown or black and grassy smell of leaves transforms into floral smell. The duration of the fermentation process varies due to temperature, humidity, and location of tea factory and month of the year. The optimum fermentation time for black tea man-
∗ Corresponding author at: Centre for Development of Advanced Computing (C-DAC), E-2/1, Block-GP, Sector-V, Salt Lake, Kolkata 700091, West Bengal, India. E-mail address: [email protected] (N. Bhattacharya).
0925-4005/$ – see front matter © 2007 Elsevier B.V. All rights reserved. doi:10.1016/j.snb.2007.12.032
ufacturing process may vary from 40 to 120 min in Indian conditions. To ensure optimum fermentation, the entire fermentation process should be carefully monitored and the process should be terminated once the optimum fermentation point is reached. To this end, it is extremely beneficial to predict the optimum fermentation time at an earlier point in time using an instrumental method. It is also observed that during the fermentation process, smell intensity is different at different instant of time and a temporal smell pattern occurs. Electronic nose has been applied successfully for several applications related to quality of food and agro products. On black tea, pioneering work has been done by Dutta et al. [2] for classification of black tea aroma. Correlation of “tea-tasters” marks with the electronic nose signature has been successfully derived in Ref. [3]. In the previous studies by the authors [4,5], electronic nose-based monitoring of volatile emission patterns during black tea fermentation process and detection of the optimum fermentation time on the basis of peaks in the sensor outputs have been successfully achieved. Various methodologies such as singular value decomposition (SVD) and Mahalanobis distance computation
N. Bhattacharya et al. / Sensors and Actuators B 131 (2008) 110–116
have been made use of to evolve aroma index at any particular sniffing cycle. The computed aroma indices have been plotted against time to draw the fermentation aroma profile. In this paper, we present the methodology and results of online prediction of optimum fermentation time. Because of the inherent nature of temporal pattern in the electronic nose data during black tea fermentation process, time-delay neural network (TDNN) has been selected for the on-line prediction [6]. Our approach is somewhat similar to [7], where multiple classifier system has been applied on olfactory signals, but we have used only TDNN models for the prediction. Multiple-TDNN architectures, named as m-TDNN, have been constructed and on-line switching between these networks is done as the fermentation process progresses. Each TDNN comprises of different training sets of data. For example, the first (TDNN 1) model is trained by first 20 min of data collected from electronic nose during the process of fermentation, second one is trained by first 25 min of data and so on. In the second step of our study, unsupervised method of clustering, the self-organizing map (SOM) has been applied for investigating the existence of smell stages. Four distinct smell clusters have been obtained on the data for all the fermentation runs, out of which the last stage represents over-fermentation. The first three clusters have been modeled by three independent TDNN algorithms for the prediction of the optimum time of fermentation. The novelty of the method is that this is the first attempt towards on-line prediction of optimum fermentation time during black tea processing using an electronic nose and is expected to be extremely beneficial for the user industry.
111
2. Customized electronic nose for black tea fermentation An electronic nose uses an array of non-specific broadly tuned sensors to discriminate odours by analyzing sensor array data with pattern recognition methods [8]. A customized electronic nose set-up has been developed such that the same can be used in production floor of tea processing units for monitoring of volatile emission pattern during the fermentation process. The electronic nose consists of (a) sensor array, (b) micro-pump with programmable sequence control, (c) PC-based data acquisition and (d) olfaction software as illustrated in Fig. 1. For black tea, an array of metal oxide semiconductor (MOS) sensors has been used for assessment of volatiles in the set-up. A series of experiments were carried out using a number of commercially available MOS sensors. From the response sensitivity of individual sensors, a set of eight gas sensors from Figaro, Japan (TGS-832, TGS-823, TGS-831, TGS-816, TGS-2600, TGS-2610, TGS-2611 and TGS-2620) has been selected for odour capture in fermentation process of black tea. The outputs of the sensors are acquired in the PC through PCI data acquisition cards. The MOS sensors are conductometric in nature, and their resistance decreases when subjected to the odour vapour molecules. The change in resistance with respect to their original values (R/R) is converted into voltage and then taken to the PC through analog to digital converter cards for subsequent analysis in the computational models. The experimental sniffing cycle consists of automated sequence of internal operations: (i) headspace generation, (ii) sampling, (iii) purging and (iv) dormancy before the start of the
Fig. 1. Customized electronic nose set-up.
112
N. Bhattacharya et al. / Sensors and Actuators B 131 (2008) 110–116
next sniffing cycle. Headspace generation ensures adequate concentration of volatiles released by tea within the sample holder by blowing regulated flow of air on the sample. During sampling, the sensor array is exposed to a constant flow of volatiles through pipelines inside the electronic nose system. During purging operation, sensor heads are cleared with blow of fresh air so that the sensors can go back to their baseline values. The programmable time dormancy cycle is the suspended mode of the electronic nose between two consecutive sniffing cycles. The PC-based data acquisition and automated operation of all these cycles are controlled by the specially designed software, called olfaction software. The software has got features like programmable sequence control, dynamic fermentation profile display, data logging, alarm annunciation, data archival, etc. The software has been developed in LabVIEW® of National Instruments. Experimental conditions during the investigation of the different stages of the fermentation process of black tea are as follows: • • • • • •
amount of fermenting tea sample = 50 g, temperature = ambient, headspace generation time = 30 s, data collection time = 50 s, purging time = 100 s, airflow rate = 5 ml/s.
3. Experimental 3.1. Place and time of experiments Experiments were carried out in three locations in different tea gardens in north India as mentioned in Table 1. Both orthodox and CTC types of black tea samples have been used in the experiments while investigating the dynamic appearance of “smell bands” during black tea fermentation process. The experiments at the Glenburn Tea Estate, Darjeeling, West Bengal, India were based on fermenting cycles of orthodox and flavoury tea. The experiments at Tocklai Experimental Station, Jorhat, Assam, India was with CTC type of black tea on a miniature manufacturing facility with controlled environment using fast fermenting clones. The Dalmia Tea Plantations, West Bengal, India is basically a “bought leaf factory” producing huge quantities of CTC type black tea daily. Longest trials were done at this factory. Details of fermentation trials in respective tea industries carried out under the present study are listed in Table 1.
Fig. 2. Data matrix formed out of sensor array output.
3.2. Data analysis and experimental results In the present electronic nose-based study on black tea fermentation process, each sniffing cycle produces a huge amount of eight-dimensional data. In our customized system, each sniffing cycle consists of headspace generation (30 s), sampling (50 s) and purging (100 s) operations. Computer acquires sensor data during headspace generation and sampling cycles only in an automated sequence of operation. Clearly, the sensor outputs will be at their baseline during the headspace generation and significant variation in sensor outputs will be observed during the sampling cycle. Therefore, the data matrix stored in the computer in each sniffing cycle will consist of a mixture of both baseline as well as actual sensor responses as shown in Fig. 2. In the above data matrix, the segment b11 –bh8 represents baseline responses of the sensors during headspace generation and S11 –Sm8 represents the sensor responses when exposed to tea odour during sampling. In our study, maximum value of Sij for each column has been found out and a vector M is formed with these maximum values. So M = [ Si1 max
· · · Si8 max ]
(1)
For each fermentation run, the vectors comprising of the maximum values associated with each sniffing cycle are considered for data analysis. In the previous studies by the authors for monitoring of black tea fermentation, it was observed that the smell intensity, as measured by electronic nose, passes through two distinct peaks during fermentation, and the second peak denotes the optimum fermentation point. This point was validated by expert human
Table 1 Experiment details Experiment serial no.
Location
Type of tea
Time
Number of fermentation cycles
1.
Glenburn Tea Estate, Darjeeling, West Bengal, India
Orthodox flavoury tea
August 2005
10
2.
Tocklai Experimental Station, Jorhat, Assam, India
CTC
September 2005
10
3
Dalmia Tea Plantations, West Bengal, India
CTC
October 2005 November 2005 December 2005
22 15 24
N. Bhattacharya et al. / Sensors and Actuators B 131 (2008) 110–116
113
Fig. 3. Architecture of time-delay neural network.
panel as well as by chemical tests like the colorimetric tests. From the on-line aroma profiling using electronic nose, optimum fermentation point is known only when it has already crossed, resulting in over-fermentation and degradation of final tea quality, however small. This section presents an on-line prediction algorithm for optimum fermentation time. The smell profile during black tea fermentation shows a temporal nature and time-delay neural network is selected because of its inherent capability to handle time varying patterns [6]. 3.2.1. The time-delay neural network TDNN is a dynamic neural network that is constructed by embedding local memory in both input and hidden layers of a multi-layer feed–forward neural network as shown in Fig. 3. This algorithm has been successfully applied for prediction of Carbonation Tower’s Temperature [9]. We have used the temporal back propagation used to train the model. 3.2.1.1. The training algorithm. The temporal back propagation algorithm is a supervised learning algorithm. The output of TDNN is compared with the desired response and the error is calculated and by temporal back propagation algorithm and this error is minimized until the desired tolerance level is obtained. The weights connecting the two layers of a TDNN are updated by the following equation: ¯ ji (n + 1) = w ¯ ji (n) + ηδj (n)¯xi (n) w
(2)
¯ ji = [wji (0), wji (1), . . . , wji (p)] and wji (l) denotes the where w weight connecting the output of neuron j to input of neuron i and l indicates the order of tapped delay and n indicates the number of iterations. η indicates the learning coefficient and is 0 < η < 1. Here x¯ i (n) = [xi (n), xi (n − 1), . . . xi (n − p)] and xi (n − p) is the input signal at ith layer at pth delay node. δj is the local gradient for neuron j and δj = ej (n)f (vj (n))
(3)
Eq. (2) is applicable if neuron j is in output layer. Here ej (n) is the error signal of jth neuron at nth iteration, vj (n) is the induced local field and f (·) is the derivative of the activation function f(·). If neuron j is in hidden layer then Eq. (3) is applied. ¯ rj δj (n) = f (vj (n)) Tr (n)w (4) r∈A
where A is the set of all neurons whose input is fed by neuron j in a forward manner and neuron r belongs to the set A and r (n) = [δr (n), δr (n + 1), . . . , δr (n + p)]T
(5)
3.2.2. Prediction of optimum fermentation time using multiple-TDNN Electronic nose data has been collected from 81 fermentation runs as mentioned in Table 1. For all the experiments, the maximum values of the sensor responses are integrated to form a large data matrix for m-TDNN-based analysis. Data obtained from randomly chosen 51 runs have been used for training of the m-TDNN and data from rest 30 runs have been used for evaluation of performance of the prediction models. In the experiments carried out, the sniffing cycle was repeated every 5 min and in each cycle, the maximum response from each sensor was recorded in a matrix. While the number of columns of this matrix would be equal to the number of sensors in the sensor array (which is eight in our case), the number of rows will depend on the duration of the fermentation process. In the experiments, the fermentation completion time varied from 40 to 120 min and the prediction algorithm is initiated after 20 min from the beginning of the fermentation cycle. Prior to that, the TDNN model has been observed to predict unreasonable results because of very few input data. We have trained five different time-delay neural network (m-TDNN) architectures with number of delays in input nodes (i) from 3 to 7 keeping fermentation completion time as target as shown in Fig. 3. The first TDNN model (TDNN 1 in Fig. 3) is trained by first 20 min of fermented data, second one (TDNN 2 in Fig. 3) is trained
114
N. Bhattacharya et al. / Sensors and Actuators B 131 (2008) 110–116
Fig. 4. Block diagram of the proposed prediction model.
by first 25 min of fermented data and the remaining models are trained by first 30, 35 and 40 min of fermented data, respectively. Comparing the number of on-line values available for prediction of fermentation completion time with the input delay nodes of the trained models, the fermentation completion time is obtained activating one model at one time. Figs. 4 and 5 show the main structure of the prediction model based on five TDNN algorithms. On-line responses from the sensor array are normalized in the preprocessing block in Fig. 4 and these normalized values lie between 0 and 1. The selector block directs the on-line test data towards the corresponding TDNN. For example, if the online test data were associated with 20 min data then TDNN 1 model is activated. 3.2.3. Results with multiple-TDNN In total, 30 fermentation cycles were experimented with this proposed model. Accuracies of different TDNN models are shown in Table 2. From the results, it is observed that with less data, uncertainty of prediction is more. But with the progress of time, as more data are available, accuracy increases. In practice, the requirement is to terminate the fermentation process at right time and prediction of the correct fermentation time few min-
Table 2 Summary of results Model
Accuracy
TDNN 1 TDNN 2 TDNN 3 TDNN 4 TDNN 5
0.9625 0.8568 0.9756 0.9932 0.9989
Table 3 Smell stage duration Smell stage
Characteristics
Smell stage 1 (T0–T1) Smell stage 2 (T1–T2) Smell stage 3 (T2–T3) Smell stage 4 (T3–T4)
Grassy smell stage Under fermentation stage Optimum fermentation stage Over-fermentation stage
utes ahead suffices to maintain the quality of the finished product. The method described above employing multiple-TDNN models thus promises to satisfy the objective. But the method requires a large number of networks, and in the subsequent sections an attempt has been made to reduce the number of such networks using self-organizing map. 3.2.4. SOM-based smell stage partition The self-organizing map (SOM), developed by Kohonen [10] is an unsupervised learning algorithm and the principal goal of it is to transform an incoming signal pattern of arbitrary dimension into a two-dimensional discrete map and to transform this adaptively in a topologically ordered fashion [11]. So neurons in this neural network are arranged in a twodimensional grid and there happens a competition among these neurons to represent the input pattern. After this, the winning neurons and the similar pattern neurons, i.e. the neighboring
Fig. 5. Activation of different TDNN models with time.
Fig. 6. Stages during the fermentation process.
N. Bhattacharya et al. / Sensors and Actuators B 131 (2008) 110–116
115
Fig. 7. SOM-based smell partitions.
neurons are placed in contiguous locations in output space. This section explores the existence of distinguishable smell stages during the fermentation process by electronic nose and then presents a prediction algorithm for the optimum fermentation time. The same dataset, which has been used in the multiple-TDNN model as explained in the previous section, is considered for unsupervised clustering using SOM to identify the distinct smell stages during the fermentation process. Out of the entire data of 81 fermentation runs, randomly chosen data from 51 runs have been used for clustering using SOM and data from rest 30 runs have been used for evaluation of performance of the smell stage-based prediction model. 3.2.4.1. Results with SOM and discussion. As already explained, the training dataset, consisting of electronic nose data for 51 fermentation runs, has been fed to the SOM model. Four distinct smell clusters have been obtained as shown in Fig. 7. This establishes the fact that, irrespective of fermentation duration, the transformation of smell from grassy to floral aroma traverses through four defined stages. These smell stages appear in cascade and the width of individual smell bands depend on the overall fermentation time, which is dependent on type of leaves, location of its production and season of processing. In consultation with the tea industries, the four identified stages of black tea fermentation are mentioned in Table 3. The thematic smell bands identified by electronic nose have been shown in Fig. 6 and the cluster formation for each stage by SOM is shown in Fig. 7. A closer look at the smell bands reveals the following facts:
3.2.5. Smell stage-based TDNN model for prediction of optimum fermentation time The self-organizing map as shown in Fig. 7 shows distinct clusters for the four stages of the fermentation process. Since the last stage is for the over-fermented tea, it does not contribute for the prediction model and three stages prior the optimum point of fermentation is considered for the computation. These three TDNNs are named as (a) time-delay neural network trained with the data of grassy smell stage (GSS-TDNN), (b) time-delay neural network trained with the data of under fermentation stage (UFS-TDNN) and (c) time-delay neural network trained with the data of optimum fermentation stage (OFS-TDNN). The advantage of this method is that the number of delay taps is much less in these TDNN models as compared to the models of the previous method. This SOM and TDNN model is simpler to implement and computational complexity is much less and have shown to be adequate to serve the purpose of on-line prediction of optimum fermentation time. 4. Conclusion It may be concluded from the above study that during black tea fermentation process, volatile emission follows fixed patterns of smell signatures with time, which is manifested in the form of distinct smell stages as detected by electronic nose. Two methods for the prediction of optimum fermentation time have been discussed and both the methods appear to be suitable for the purpose. However, the combined SOM- and TDNN-based prediction algorithm proves to be the better alternative as the computational complexity is relatively less. References
(a) The time span of the optimum fermentation band is very short compared to other stages of smell. (b) Grassy smell duration is more than the optimum fermentation duration but less than the under fermentation duration. (c) The under fermentation phase persists for considerable length of time.
[1] H. Co, G.W. Sanderson, Biochemistry of tea fermentation: conversion of amino acids to black tea aroma constituents, J. Food Sci. 35 (1970) 160–164. [2] R. Dutta, E.L. Hines, J.W. Gardner, K.R. Kashwan, M. Bhuyan, Tea quality prediction using a tin oxide-based electronic nose: an artificial intelligence approach, Sens. Actuators B: Chem. 94 (2003) 228–237.
116
N. Bhattacharya et al. / Sensors and Actuators B 131 (2008) 110–116
[3] N. Bhattacharyya, R. Bandyopadhyay, M. Bhuyan, A. Ghosh, R.K. Mudi, Correlation of multi-sensor array data with “tasters” panel evaluation for objective assessment of black tea flavour, in: International Proceedings of the ISOEN-2005, Barcelona, Spain, April 13–15, 2005. [4] N. Bhattacharyya, S. Seth, B. Tudu, P. Tamuly, A. Jana, D. Ghosh, R. Bandyopadhyay, M. Bhuyan, Monitoring of black tea fermentation process using electronic nose, J. Food Eng. 80 (2007) 1146–1156. [5] N. Bhattacharyya, S. Seth, B. Tudu, P. Tamuly, A. Jana, D. Ghosh, R. Bandyopadhyay, M. Bhuyan, S. Sabhapandit, Detection of optimum fermentation time for black tea manufacturing using electronic nose, Sens. Actuators B: Chem. 122 (2007) 627–634. [6] A. Waibel, T. Hanazawa, G. Hinton, K. Shikano, K.J. Langa, Phoneme recognition using time delay neural networks, IEEE Trans. Acoust. Speech Signal Process. 37 (1989) 328–339. [7] E. Phaisangittisagul, H.T. Nagle, Enhancing multiple classifier system performance for machine olfaction using odor-type signatures, Sens. Actuators B: Chem. 125 (2007) 246–253. [8] T.C. Pearce, S.S. Schiffman, H.T. Nagle, J.W. Gardner (Eds.), A Handbook of Machine Olfaction, Wiley–VCH, 2003. [9] Dan Shi, Hongjian Zhang, Liming Yang, Time-delay neural network for the prediction of Carbonation Tower’s Temperature, IEEE Trans. Inst. Met. 52 (2003) 1125–1128. [10] T. Kohonen, Self Organizing Maps, 3rd ed., Springer, 2001. [11] S. Haykin, Neural Networks, A Comprehensive Foundation, 2nd ed., Pearson Education, Asia, 1998.
Biographies Nabarun Bhattacharyya is additional director in Centre for Development of Advanced Computing (C-DAC), Kolkata, India, which is a premier R&D Institute under Department of Information Technology, Government of India. He received his BE degree in electronics and telecommunication engineering from
Jadavpur University, Kolkata, India in the year 1984. His research areas are agri-electronics, machine olfaction, soft computing and pattern recognition. Bipan Tudu received his MTech degree in instrumentation and electronics engineering in 2004 from the Jadavpur University, Kolkata, India. He is currently a lecturer in the Department of Instrumentation and Electronics Engineering. His main research interest includes pattern recognition, artificial intelligence and machine olfaction. Arun Jana is presently a project engineer in Centre for Development of Advanced Computing (C-DAC), Kolkata, India, which is a premier R&D Institute under Department of Information Technology, Government of India. He received his master in computer applications (MCA) from the Utkal University, India in the year 2004. His research interests are machine learning and soft computing. Devdulal Ghosh is presently a project engineer in Centre for Development of Advanced Computing (C-DAC), Kolkata, India, which is a premier R&D Institute under Department of Information Technology, Government of India. He received his BE degree in computer science from the Burdwan University, India in the year 2004. His research interests are virtual instrumentation, software engineering and object oriented programming. Rajib Bandopadhyay received his PhD degree in the year 2001 and currently a professor in the Department of Instrumentation Engineering, Jadavpur University, Kolkata, India. His research interests are in the fields of machine olfaction, intelligent control and NQR-based instrumentation systems. He is a fellow of IETE (India). Manabendra Bhuyan is presently a professor in the Department of Electronics, Tezpur University, India. He received his PhD degree in the year 1998. His research interests are in the fields of electronic instrumentation, machine olfaction and machine vision.