- Email: [email protected]
Two-level PLS model for quality prediction of multiphase batch processes
Two-level PLS model for quality prediction of multiphase batch processes Zhiqiang Ge, Zhihuan Song, Luping Zhao, Furong Gao PII: DOI: Reference:
S0169-7439(13)00170-6 doi: 10.1016/j.chemolab.2013.09.008 CHEMOM 2708
To appear in:
Chemometrics and Intelligent Laboratory Systems
Received date: Revised date: Accepted date:
6 March 2013 14 September 2013 16 September 2013
Please cite this article as: Zhiqiang Ge, Zhihuan Song, Luping Zhao, Furong Gao, Twolevel PLS model for quality prediction of multiphase batch processes, Chemometrics and Intelligent Laboratory Systems (2013), doi: 10.1016/j.chemolab.2013.09.008
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT Two-level PLS model for quality prediction of multiphase batch processes
T
Zhiqiang Gea, Zhihuan Songa, Luping Zhaob, Furong Gaob a
IP
State Key Laboratory of Industrial Control Technology, Institute of Industrial Process Control, Department of Control Science and Engineering, Zhejiang University, Hangzhou, 310027, P. R. China
b
SC R
Department of Chemical and Biomolecular engineering, The Hong Kong University of Science and Technology, Hong Kong,
NU
Abstract
Statistical quality prediction methods for multiphase batch processes have gained much attention in
MA
recent years. While most methods are focused on the data information inside each phase, the relationships
D
among different phases have rarely been explored and used for quality prediction, although it may have
TE
significant impacts on prediction of the final quality. In this paper, a two-level partial least squares (PLS) model is proposed, in which the relationships among different single phases are modeled and incorporated
CE P
for quality prediction. In the first level of this method, a representative intraphase-PLS model is built for each single phase, while in the second level, a series of interphase-PLS models are constructed to capture
AC
the relationships among different phases. With the incorporation of the additional interphase information, the multiphase quality prediction performance can be improved, which is evaluated through an industrial case study.
Keywords:
Multiphase batch process; Quality prediction; Partial least squares; Two-level modeling
method; Interphase relationship.
Corresponding author: Tel.:+86-87951442, E-mail address: [email protected]
-1-
ACCEPTED MANUSCRIPT 1. Introduction Since batch and semi-batch processes play more and more important roles in the modern industry,
IP
T
quality control of the batch process has become a significant research topic in recent years. However, due
SC R
to the measurement difficulty or high costs of the instrumentation, batch processes often lack online measurements of the quality variables, which are critical for quality control. Therefore, significant efforts have been made in the part years developing quality prediction methods. Compared to the first-principle
NU
model-based method, the data-based multivariate statistical analysis method can derive the model directly
MA
from historical process data, without knowing any prior knowledge of the process. Besides, the multivariate statistical analysis method is particularly useful to handle high dimensional and cross-correlated variables,
D
which are commonly observed cases in the batch process.
TE
Among the widely used multivariate statistical analysis methods such as principal component analysis
CE P
(PCA), partial least squares (PLS), canonical correlation analysis (CCA), PLS may be the most popular one, which has been extended to the multiway counterpart for batch process modeling [1]. Along last several
AC
decades, the multiway PLS (MPLS) method has been well acknowledged for monitoring and quality prediction in batch processes [2-12]. However, it has recently been explored that the traditional MPLS model cannot efficiently reveal the multiphase data behavior, which is very common in many batch processes. In those batch processes, different phases may have different data characteristics. For example, a typical fermentation process contains the preculture phase and the production phase. The data characteristic and the variable relationship in these two phases are both different from each other. Therefore, a batch process is desired to be divided into several phases, in order to improve the modeling performance. So far, a lot of methods have been developed for multiphase batch process modeling. According to the recent survey paper on the topic of multiphase batch process, all of those developed methods can be
-2-
ACCEPTED MANUSCRIPT roughly summarized into two different categories: multiblock methods and phase-separated methods [13]. By grouping the batch process into several blocks, the multiblock method characterizes the multiphase
IP
T
batch process through a single model structure. Compared to the MPLS model, both of the data
SC R
interpretation and the process understanding have been enhanced. The idea of the phase-separated method is to build a separated model for each phase in the batch process. In the past years, various multiphase modeling approaches have been developed, including Undey and Cinar, Lu et al., Muthuswamy and
NU
Srinivasan, Camacho et al., Zhao et al., Ge et al., etc [14-20].
MA
However, while the phase-separated models can efficiently capture different data characteristics in multiple phases of the batch process, the interphase relationships have rarely been explored and used for
D
quality prediction. Nevertheless, the ignored interphase relationship may have significant impacts on
TE
prediction of the final product quality. In other words, the quality prediction performance might be
modeling.
CE P
improved if the effective interphase relationships can be revealed and incorporated for multiphase
AC
In this paper, a two-level PLS modeling approach is proposed for quality prediction of multiphase batch processes. In the first level, a separated intraphase-PLS model is developed in each phase of the batch process. Based on the extracted latent variable information in different phases, a series of interphase-PLS models are constructed in the second level. For example, for prediction of the final product quality in the second phase of the batch process, the interphase relationship between the first and the second phases should be built; for quality prediction in the third phase, the interphase relationships among the first three phases should be built; the rest can be deduced by analogy until the last phase, in which the interphase relationships among all phases should be built for quality prediction in the last phase. The remainder of this paper is organized as follows. In section 2, detailed methodology description of
-3-
ACCEPTED MANUSCRIPT the proposed two-level PLS model based multiphase quality prediction method is provided, including the data unfolding and phase division step, development of the two-level PLS model, and the online multiphase
IP
T
quality prediction algorithm. An industrial application study of the injection molding process is used for
SC R
performance evaluation of the proposed method, which is provided in Section 3. Finally, conclusions are made in the last section.
NU
2. Multiphase quality prediction based on two-level PLS model
MA
In this section, detailed methodology descriptions of the two-level PLS model is demonstrated. First, the three-dimensional batch process dataset is unfolded into two-dimension, the mean trajectory is removed
D
for data scaling, and the whole batch process is divided into different phases. Second, a separated
TE
intraphase-PLS model is constructed inside each phase of the batch process. Third, a series of
CE P
interphase-PLS models are developed in the second level, which are used to capture the interphase relationships among different phases. Based on the developed two-level PLS model, an online multiphase
AC
quality prediction algorithm is then formulated.
2.1. Data unfolding and phase division The batch process data matrix X is typically collected in a three-way manner, which is
X( I J K ) , where I is the batch number, J is the number of measured variables, and K represents the batch duration. For multivariate statistical modeling, the three-way dataset is always unfolded into a two-dimensional dataset X( I JK ) through the batch direction. Then, the data scaling is carried out before the multivariate statistical modeling step, thus the mean and deviation values of the process variables along each time slice are removed from the original variables. For multiphase batch process modeling, an important issue is how to divide the batch process into
-4-
ACCEPTED MANUSCRIPT different phases, which is referred as the phase division problem. Generally, there are three major ways for phase division in the batch process [7]. The first way is based on the expert knowledge, which is
IP
T
straightforward, and has been used in many early research works [21, 22]. The second way is based on the
SC R
process analysis techniques. Similar to the expert knowledge-based method, it is also under the assumption that some certain required process features are known beforehand, such as phase change points, process behaviors in different phases, etc [23, 24]. Later, some purely data-driven methods have been developed for
NU
phase division, which are more useful in practice. Two typical methods are the sub-PCA based method
MA
which is based on the change of the variable correlations, and the multiphase algorithm which detects the phase division points based on the prediction ability of the PCA model [15, 16]. More recently, with the
D
consideration of the phase transition behavior, a new phase division approach has been proposed, which is
TE
based on the repeatability degree of different batches during each time interval [25]. In the present paper,
CE P
we assumed that the batch process has already been divided into different phases. The data unfolding the phase division of the batch process are illustrated in Figure 1.
AC
[Figure 1 about here]
2.2. Two-level PLS model development After the batch process has been divided into different phases, the two-level PLS model can be developed, including the intraphase PLS model built in each phase, and a series of interphase PLS models constructed for connections of different phases. Suppose the multiphase batch process dataset is represented as
X( I JK ) [X1 ( I JK1 ) X2 ( I JK2 )
Xs ( I JK s )
XS ( I JK S )] , where S is the
number of phases. Detailed descriptions of these PLS models are demonstrated in the following two subsections. 2.2.1. Intraphase PLS models
-5-
ACCEPTED MANUSCRIPT To build the representative PLS model inside each phase of the batch process, the PLS prediction model for each time slice inside the phase should be developed in the first step, given as follows
s 1 k
R W (P W ) Q s k
sT k
sT k
(1)
SC R
s k
IP
Y Tks Q ks T Fks
T
X ks ( I J ) Tks PksT E ks
where Y is the quality data matrix of the batch process which are measured at the end of each batch,
s 1,2,
, S , k 1, 2,
, K s is the number index of time interval inside each phase. Pks and Q ks
NU
are the loading matrices, Tks is the score matrix, E ks and Fks are the residual matrices of each PLS
MA
model. Wks is the weighting matrix of each PLS model, and R ks represents each regression matrix. Next, the intraphase-PLS model can be developed, which is actually the mean representative of the
D
time slice PLS model in the phase. The regression matrix R *s , latent variable matrix Ts* and its *
TE
regression matrix Vs can be determined as follows
1 R s K
CE P
* s
AC
Ts*
where s 1, 2,
1 Ks
1 V s K * s
Ks
R k 1
s k
Ks
T k 1
s k
(2)
Ks
W (P k 1
s k
sT k
Wks ) 1
, S is the phase number in the batch process. For each single phase, we have assumed
that the data behavior keeps similar during the phase, thus, the representative PLS model is able to capture the general relationship among process variables [16]. That is to say, we have ignored the dynamic information in each phase of the batch process. 2.2.2. Interphase PLS models After the intraphase-PLS model has been developed in each phase, the interphase PLS models can then be constructed to connect the relationships among different phases. In the second phase, the representative latent matrices of the first and second phases are augmented together, given as -6-
ACCEPTED MANUSCRIPT L12 [T1* T2* ]
(3)
quality information of the batch process Y , which is described as follows
R1 2 W1 2 (P
T 1 2
where P1 2 and
(4)
SC R
Y T1 2Q1T 2 F12
IP
L1 2 T1 2 P1T2 E12
T
Then the first interphase-PLS model is build to capture the relationships between L1 2 and the final
1
T 1 2
W1 2 ) Q
Q1 2 are the loading matrices of this interphase PLS model, T1 2 is the score
NU
matrix, E1 2 and F1 2 are the residual matrices, W1 2 is the weighting matrix, and R1 2 is the
MA
regression matrix.
With the ongoing of the batch process, the relationships among more phases can be built. Generally,
D
the relationships among the first s phases can be built as follows
L1s [T1* T2*
TE
Ts* ]
(5)
L1 s T1 s P1T s E1 s Y T1 s Q1T s F1 s
CE P
(6)
R1 s W1 s (P1T s W1 s ) 1 Q1T s
, S . Therefore, a total of S 1 interphase-PLS models can be constructed for
AC
where s 1, 2,
connecting different phases.
2.3. Online quality prediction algorithm Based on the developed two-level PLS models, an online quality prediction algorithm can be new
formulated. For a new measured process data sample x kc
in a specific phase sc , the online prediction
of the final quality variables can be made by the following steps. First, the latent variable of this new data sample is calculated by the representative intraphase-PLS model in the sc -th phase, given as new * t new kc x kc Vs
(7)
Suppose the latent matrix of the data samples which starts from the first data sample of the sc -th phase
-7-
ACCEPTED MANUSCRIPT new
until the latest one before the current data sample is represented as Tsc ,kc , the mean latent variable vector inside the sc -th phase at the kc -th time interval can be calculated as follows
T
* new new t new t kc ]} sc , kc mean{[Tsc , kc
(8)
SC R
IP
where mean{} is to calculate the mean value of the latent variable vector inside the sc -th phase. To calculate the quality variable in the second level, the mean latent variable vectors of the previous phases
new*
as t s
, sc should also be incorporated. Denote the mean latent variable vector of each previous phase
(s 1, 2,
, sc 1) , the integrated latent variable information of the new batch can be represented
NU
s 1, 2,
MA
as follows *T l scnew,kc [t1new*T t new 2
*T *T T t new t new sc 1 sc , kc ]
(9)
TE
calculated as follows
D
Next, the final quality variable can be determined by the (sc 1) -th interphase-PLS model, which is
(10)
CE P
yˆ new,kc R1T sc l scnew,kc
2.4. Implementation procedures
AC
In summary, the offline modeling and online quality prediction procedures of the proposed method are listed as follows.
Offline modeling stage Step 1: Collect the historical dataset for the batch process; Step 2: Unfold the three-way dataset into a two-dimensional dataset for scale, and divided the whole batch process into different phases; Step 3: Build intraphase-PLS models in different single phases (First level); Step 4: Based on the extracted latent variable information in different phases, construct a series of interphase-PLS models (Second level);
-8-
ACCEPTED MANUSCRIPT Step 5: Save both of the intraphase-PLS and interphase-PLS models. Online monitoring stage
IP
T
Step 1: Acquire the new data sample from online batch process;
SC R
Step 2: Data pre-treatment of the new data sample by the same step used in the modeling stage; Step 3: Calculate the latent variable information of the new data sample with the intraphase-PLS model in the corresponding phase;
NU
Step 4: Ensemble the latent variable information in all available phases, calculate the quality prediction
MA
results by using the corresponding interphase-PLS model in the second level. Step 5: Performance evaluation and process analysis.
TE
and Figure 3, respectively.
D
Systematic illustrations of the offline modeling and online monitoring scheme are shown in Figure 2
CE P
[Figure 2-3 about here]
AC
3. Case study of injection molding process Injection molding process is a typical multiphase batch process, which usually contains injection phase, packing-holding phase, plastication phase, cooling phase, etc. A simplified flowchart of a reciprocating-screw injection molding process is shown in Figure 4. For process analysis and quality control purposes, some important variables have been measured in this process, such as temperatures, pressures, and the screw velocity. In the present study, the weight of the final product is selected as the quality variable, for prediction of which a total of 11 key variables have been selected and listed in Table 1. Detailed description and experimental design of the injection molding process can be found in Lu et al [16]. [Figure 4 about here]
-9-
ACCEPTED MANUSCRIPT [Table 1 about here] For batch process modeling and testing, a dataset containing 150 batches has been generated, among
IP
T
which 100 batches are used for model training and the remaining 50 batches are for testing. For simplicity,
SC R
the same duration of all batches is assumed, which is 635 sampling intervals. Based on the phase division method, the whole injection molding process can be divided into seven different phases. Therefore, the training dataset is divided into seven parts, each of which is used for development of the intraphase-PLS
NU
model. Then number of each PLS model has been determined through the cross-validation approach. As a
MA
result, the numbers of identified components in each phase are 4, 3, 5, 6, 3, 4 and 5. After the latent variable information has been extracted in each phase of the batch process, the interphase-PLS models can then be
D
constructed in the second level. Since there are seven phases in this process, a total of six interphase-PLS
TE
models should be built. For performance evaluation of the quality prediction method in each time interval,
CE P
the root mean square error (RMSE) criterion is used, which is defined as follows
where kc 1, 2, batches, yˆ i
y i 1
i
yˆ ikc
2
(11)
nte
AC
RMSE (kc)
nte
kc
, K represents the time interval of the batch process, nte is the number of testing
is the predicted quality values made in the kc-th time interval, y i is the corresponding
measured quality values, and i is the number index of the testing batches. Similarly, for performance evaluation of a new batch inside each phase and during the whole batch, the corresponding RSME indices can be defined as follows Ks
RMSE ( s )
y yˆ
kc 1
Ks
- 10 -
s 2 kc
(12)
ACCEPTED MANUSCRIPT K
RMSEBatch
kc 1
2 kc
(13)
K
T
, K s is the predicted quality values obtained in each phase, and yˆ kc is the
IP
s where yˆ kc , kc 1, 2,
y yˆ
SC R
predicted value in each time interval, y is the corresponding measured quality values at the end of the batch.
For comparison purpose, the quality prediction results of the phase-based sub-PLS models are also
NU
presented here, which is adopted from reference [16]. After quality prediction of the 50 testing batches, the
MA
overall results of the RMSE index are shown in Figure 5. As can be seen, the values of the RMSE index generated by the two-level PLS model are smaller than those provided by the sub-PLS model in most time
D
intervals along the whole batch duration, which means that the quality prediction performance of the testing
TE
batches has been generally improved by the two-level PLS model. Figure 6 (a-g) shows the RMSE results
CE P
of each testing batch in different phases. In the first phase, the same results have been obtained by the sub-PLS model and the two-level PLS model. This is because no interphase-PLS model has been
AC
incorporated in the first phase. Starting from the second phase, the RMSE values are different for the two methods. Generally, in each phase, most of the star points which represent for the results of two-level PLS model are lower than the circle points, denoting the results of sub-PLS model. Therefore, the quality prediction performance of the testing batches has been improved in each of the six phases. Similarly, the RMSE value through the whole batch direction for each of the 50 testing batches is given in Figure 6 (h), through which one can also find that the quality prediction results of most batches have been improved by the two-level PLS model. [Figure 5-6 about here] Particularly, detailed prediction results of two testing batches are further illustrated in Figure 7 (a), and
- 11 -
ACCEPTED MANUSCRIPT Figure 7 (b), respectively. From both two figures, we can find the prediction values of the two-level PLS model are much closer to the actual weights of the final product. By examining the RMSE values of these
IP
T
two testing batches in each of the seven phases, similar conclusions can be explored. Detailed RMSE
SC R
values of the two testing batches in different phases are tabulated in Table 2. As been shown in this table, the prediction performance has been greatly improved in the last two phases for both of the two testing batches. In the last row of Table 2, the results of the RMSE values along the whole batch duration for the
NU
two testing batches are also provided. Again, the superiority of the two-level PLS model has been evaluated
MA
in both two batches.
[Figure 7 about here]
CE P
4. Conclusions
TE
D
[Table 2 about here]
In the present paper, a two-level PLS model has been developed for quality prediction of multiphase
AC
batch processes. In the first level of this method, a representative intraphase-PLS model was built in each phase of the batch process, based on which the latent variable data information is extracted. Then, a series of interphase-PLS models were constructed in the second level, which were used to model the relationships among different phases. Based on these interphase-PLS models, different phases of the batch process can be connected. With the additional relationship information extracted among different phases, the quality prediction performance has been improved, which has been evaluated and confirmed by an industrial injection molding process. Compared to the conventional sub-PLS model, the weight prediction of the final product becomes more accurate with the incorporation of the interphase-PLS models. It is worth to note that with the duration of the batch process, more and more data information have
- 12 -
ACCEPTED MANUSCRIPT been incorporated for interphase modeling. Therefore, a linear model may be insufficient to capture the interphase relationship among different phases. In this case, the linear PLS model can be extended to more
IP
T
complex cases, such as kernel approach, local modeling approach, mixture models, etc [26-28]. While the
SC R
prediction accuracy may be improved, as a compromise, the computational burden will be increased for both offline modeling and online utilization. Another important issue is how to guarantee the modeling reliability, particularly for the interphase model. This is because with the increase of the number of latent
NU
variables incorporated for interphase modeling, we may need more training batches in order to guarantee
MA
the modeling reliability. However, it is difficult to set a ratio of the number of modeling data to the number of latent variables, since different processes may need different numbers of effective modeling data. In
D
practice, what we can do is to collect the training data in as much wide operation region as possible. Finally,
TE
the proposed two-level modeling method can be considered a compromise between the traditional multiway
CE P
modeling method which connects all time slices into a single model and the phase-based modeling method
AC
which only models variable relationships inside each single phase.
Acknowledgement This work was supported in part by the National Natural Science Foundation of China (NSFC) (61370029), Project National 973 (2012CB720500), and the Fundamental Research Funds for the Central Universities (2013QNA5016).
References [1] P. Nomikos, J. F. MacGregor, Multi-way partial least square in monitoring batch processes. Chem. Intel. Lab. Syst. 30 (1995) 97-108.
- 13 -
ACCEPTED MANUSCRIPT [2] Z. Q. Ge, Z. H. Song, F. R. Gao, Review of Recent Research on Data-Based Process Monitoring, Ind. Eng. Chem. Res. 52 (2013) 3543-3562.
IP
T
[3] E. N. M. Sprang, H. J. Ramaker, J. A. Westerhuis, S. P. Guiden, A. K. Smilde, Critical evaluation of
SC R
approaches for on-line batch process monitoring. Chem. Eng. Sci. 57 (2008) 3979–3991. [4] J. H. Chen, C. M. Song, T. Y. Hsu, Online monitoring of batch processes using IOHMM based MPLS. Ind. Eng. Chem. Res. 49 (2010) 2800–2811.
MA
space. J. Process Contr. 20 (2010) 716-725.
NU
[5] C. R. Alvarez, A. Brandolin, M. C. Sanchez, Batch process monitoring in the original measurement’s
[6] T. Chen, J. Zhang, On-line multivariate statistic monitoring of batch processes using Gaussian mixture
D
model. Comput. Chem. Eng. 34 (2010) 500–507.
TE
[7] S. W. Andersen, G. C. Runger, Partitioned partial least squares regression with application to a batch
CE P
fermentation process. J. Chemometrics 25 (2011) 159-168. [8] Q. P. He, J. Wang, Statistics pattern analysis: A new process monitoring framework and its application
AC
to semiconductor batch processes. AIChE J. 57 (2011) 107-121. [9] J. Yu, Nonlinear bioprocess monitoring using multiway kernel localized fisher discriminant analysis. Ind. Eng. Chem. Res. 50 (2011) 3390-3402. [10] Y. W. Zhang, Z. Y. Hu, On-line batch process monitoring using hierarchical kernel partial least squares. Chem. Eng. Res. Des. 89 (2011) 2078-2084. [11] Y. Q. Chang, Y. S. Lu, F. L. Wang, S. Wang, S. M. Feng, Sub-stage PCA modeling and monitoring method for uneven-length batch processes. Candian J. Chem. Eng. 90 (2012) 144-152. [12] M. J. Anzanello, S. L. Albin, W. A. Chaovalitwongse, Multicriteria variable selection for classification of production batches. European J. Operat. Res. 218 (2012) 97-105.
- 14 -
ACCEPTED MANUSCRIPT [13] Y. Yao, F. R. Gao, A survey on multistage/multiphase statistical modeling methods for batch processes. Annual Reviews in Control 33 (2009) 172-183.
IP
T
[14] C. Undey, A. Cinar, Statistical monitoring of multistage, multiphase batch processes. IEEE Contr.
SC R
Syst. Mag. 22 (2002) 53-63.
[15] N. Y. Lu, F. R. Gao, F. L. Wang, A sub-PCA modeling and online monitoring strategy for batch processes. AIChE J. 50 (2004) 255-259.
NU
[16] N. Y. Lu, F. R. Gao, Stage-based process analysis and quality prediction for batch processes. Ind. Eng.
MA
Chem. Res. 44 (2005) 3547-3555.
[17] K. Muthuswamy, R. Srinivasan, Phase-based supervisory control for fermentation process
D
development. J. Process Control 13 (2003) 367-382.
TE
[18] J. Camacho, J. Pico, A. Ferrer, Multi-phase analysis framework for handling batch process data. J
CE P
Chemometrics 22 (2008) 632-643.
[19] C. H. Zhao, F. R. Gao, F. L. Wang, Phase-based joint modeling and spectroscopy analysis for batch
AC
process monitoring. Ind. Eng. Chem. Res. 49 (2010) 669-681. [20] Z. Q. Ge, Z. H. Song, F. R. Gao, Nonlinear quality prediction for multiphase batch processes, AIChE J. 58 (2012) 1778-1787. [21] J. A. Westerhuis, T. Kourti, J. F. MacGregor, Analysis of multiblock and hierarchical PCA and PLS models. J. Chemometrics 12 (1998) 301-321. [22] S. P. Reinikainen, A. Hoskuldsson, Multivariate statistical analysis of a multi-step industrial process, Anal. Chinica Acta 595 (2007) 248-256. [23] P. Facco, M. Olivi, C. Rebuscini, F. Bezzo, M. Barolo, Multivariate statistical estimation of product quality in the industrial batch production of a resin. Dycops 2 (2007) 93-98.
- 15 -
ACCEPTED MANUSCRIPT [24] X. T. Doan, R. Srinivasan, Online monitoring of multi-phase batch processes using phase-based multivariate statistical process control. Comput. Chem. Eng. 32 (2008) 230-243.
IP
T
[25] Z. Ge, L. Zhao, Y. Yao, Z. Song, F. Gao, Utilizing transition information in online quality prediction
SC R
of multiphase batch processes. J. Process Contr. 22 (2012) 599-611.
[26] Z. Q. Ge, Z. H. Song, Online monitoring of nonlinear multiple mode processes based on adaptive local model approach, Contr. Eng. Pract. 16 (2008) 1427-1437.
NU
[27] Z. Q. Ge, C. J. Yang, Z. H. Song, Improved kernel PCA-based monitoring approach for nonlinear
MA
processes, Chem. Eng. Sci. 64 (2009) 2245-2255.
[28] Z. Ge, F. Gao, Z. Song. Mixture probabilistic PCR model for soft sensing of multimode processes.
AC
CE P
TE
D
Chem. Intel. Lab. Syst., 105 (2011), 91-105.
- 16 -
ACCEPTED MANUSCRIPT Figure Captions Figure 1: Data unfolding and phase division of the multiphase batch process
SC R
Figure 3: Online quality prediction scheme of the two-level PLS model
IP
T
Figure 2: Offline modeling of the two-level PLS method
Figure 4: Simplified schematic flowchart of the injection molding machine Figure 5: RMSE values of the 50 testing batch in each time interval
NU
Figure 6: RMES values of the 50 testing batches, (a) First phase; (b) Second phase; (c) Third phase; (d)
MA
Fourth phase; (e) Fifth phase; (f) Sixth phase; (g) Seventh phase; (h) Whole batch duration
AC
CE P
TE
D
Figure 7: Quality prediction results of the two methods, (a) First batch; (b) Second batch
- 17 -
ACCEPTED MANUSCRIPT Table Captions Table 1: Variables selected for quality prediction
AC
CE P
TE
D
MA
NU
SC R
IP
T
Table 2: RMSE values of the two testing batches in different phases and during the whole batch
- 18 -
ACCEPTED MANUSCRIPT
Table 1: Variables selected for quality prediction Variables
Unit
No.
Variables
1 2 3 4 5 6
Valve 1 opening Valve 2 opening Screw stroke Screw velocity Ejector stroke Mold stroke
% % mm mm/s mm mm
7 8 9 10 11
Mold velocity Injection press Temperature 3 Temperature 2 Temperature 1
mm/s Bar ℃ ℃ ℃
IP
SC R
NU MA D TE CE P AC
- 19 -
Unit
T
No.
ACCEPTED MANUSCRIPT
Table 2: RMSE values of the two testing batches in different phases and during the whole batch First batch
sub-PLS
Two-level PLS
0.0108 0.0108 0.0108 0.0108 0.0111 0.0112 0.0129 0.0112
0.0108 0.0092 0.0109 0.0109 0.0113 0.0059 0.0025 0.0088
0.0076 0.0117 0.0157 0.0113 0.0036 0.0115 0.0097 0.0113
0.0076 0.0060 0.0036 0.0050 0.0044 0.0037 0.0028 0.0048
SC R
IP
Two-level PLS
AC
CE P
TE
D
MA
Batch direction
sub-PLS
NU
Phase #1 Phase #2 Phase #3 Phase #4 Phase #5 Phase #6 Phase #7
Second batch
T
Phases
- 20 -
ACCEPTED MANUSCRIPT
X1
X2
…
XK2
……
XK
…
X K1
Second phase
MA
First phase
X K1 +1
……
NU
X1
SC R
IP
T
X
X KS-1 +1 …
XK
S-th phase
AC
CE P
TE
D
Figure 1: Data unfolding and phase division of the multiphase batch process.
- 21 -
ACCEPTED MANUSCRIPT
First phase
Second phase
S-th phase
…
X K1
X K1 +1
…
XK2
X KS-1 +1 …
……
XK
SC R
IP
X1
T
Y
…… ……
NU
Intraphase-PLS model #2
Intraphase-PLS model #S
MA
Intraphase-PLS model #1
D
……
Interphase-PLS model #S-1
TE
Interphase-PLS model #1
AC
CE P
Figure 2: Offline modeling of the two-level PLS method
- 22 -
ACCEPTED MANUSCRIPT
First phase
…
……
Latent variables #1
Latent variables #2
T
…
S-th phase
…
IP
New batch
Second phase
SC R
Intraphase-PLS models
Latent variables #S
NU
……
MA
Interphase-PLS models
D
Final qualtiy prediction results
AC
CE P
TE
Figure 3: Online quality prediction scheme of the two-level PLS model.
- 23 -
SC R
IP
T
ACCEPTED MANUSCRIPT
P: pressure sensor T: temperature sensor
NU
IT: inferred temperature sensor LDT&LVT: displacement & velocity sensor
MA
SV1&SV2: servo-valves
AC
CE P
TE
D
Figure 4: Simplified schematic flowchart of the injection molding machine.
- 24 -
MA
NU
RMSE
SC R
IP
T
ACCEPTED MANUSCRIPT
TE
D
Sampling intervals
AC
CE P
Figure 5: RMSE values of the 50 testing batch in each time interval
- 25 -
SC R
IP
T
RMSE
RMSE
ACCEPTED MANUSCRIPT
Testing batches
Testing batches
(b)
RMSE
(d)
RMSE
CE P AC
RMSE
(c)
Testing batches
TE
Testing batches
D
RMSE
MA
NU
(a)
Testing batches
Testing batches
(e)
(f)
- 26 -
SC R
IP
T
RMSE
RMSE
ACCEPTED MANUSCRIPT
Testing batches
Testing batches
(g)
(h)
NU
Figure 6: RMES values of the 50 testing batches, (a) First phase; (b) Second phase; (c) Third phase; (d)
AC
CE P
TE
D
MA
Fourth phase; (e) Fifth phase; (f) Sixth phase; (g) Seventh phase; (h) Whole batch duration
- 27 -
SC R
IP
T
Weight (g)
ACCEPTED MANUSCRIPT
NU
Sampling intervals
D TE CE P AC
Weight (g)
MA
(a)
Sampling intervals (b)
Figure 7: Quality prediction results of the two methods, (a) First batch; (b) Second batch.
- 28 -
ACCEPTED MANUSCRIPT Highlights
IP
T
> A two-level PLS modeling structure is proposed for quality prediction. > Intraphase PLS
SC R
model is developed inside each phase of the batch process. > Interphase PLS models are developed to capture relationships among different phases. > An industrial injection molding
AC
CE P
TE
D
MA
NU
process is used for performance evaluation. >
- 29 -
S0169-7439(13)00170-6 doi: 10.1016/j.chemolab.2013.09.008 CHEMOM 2708
To appear in:
Chemometrics and Intelligent Laboratory Systems
Received date: Revised date: Accepted date:
6 March 2013 14 September 2013 16 September 2013
Please cite this article as: Zhiqiang Ge, Zhihuan Song, Luping Zhao, Furong Gao, Twolevel PLS model for quality prediction of multiphase batch processes, Chemometrics and Intelligent Laboratory Systems (2013), doi: 10.1016/j.chemolab.2013.09.008
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT Two-level PLS model for quality prediction of multiphase batch processes
T
Zhiqiang Gea, Zhihuan Songa, Luping Zhaob, Furong Gaob a
IP
State Key Laboratory of Industrial Control Technology, Institute of Industrial Process Control, Department of Control Science and Engineering, Zhejiang University, Hangzhou, 310027, P. R. China
b
SC R
Department of Chemical and Biomolecular engineering, The Hong Kong University of Science and Technology, Hong Kong,
NU
Abstract
Statistical quality prediction methods for multiphase batch processes have gained much attention in
MA
recent years. While most methods are focused on the data information inside each phase, the relationships
D
among different phases have rarely been explored and used for quality prediction, although it may have
TE
significant impacts on prediction of the final quality. In this paper, a two-level partial least squares (PLS) model is proposed, in which the relationships among different single phases are modeled and incorporated
CE P
for quality prediction. In the first level of this method, a representative intraphase-PLS model is built for each single phase, while in the second level, a series of interphase-PLS models are constructed to capture
AC
the relationships among different phases. With the incorporation of the additional interphase information, the multiphase quality prediction performance can be improved, which is evaluated through an industrial case study.
Keywords:
Multiphase batch process; Quality prediction; Partial least squares; Two-level modeling
method; Interphase relationship.
Corresponding author: Tel.:+86-87951442, E-mail address: [email protected]
-1-
ACCEPTED MANUSCRIPT 1. Introduction Since batch and semi-batch processes play more and more important roles in the modern industry,
IP
T
quality control of the batch process has become a significant research topic in recent years. However, due
SC R
to the measurement difficulty or high costs of the instrumentation, batch processes often lack online measurements of the quality variables, which are critical for quality control. Therefore, significant efforts have been made in the part years developing quality prediction methods. Compared to the first-principle
NU
model-based method, the data-based multivariate statistical analysis method can derive the model directly
MA
from historical process data, without knowing any prior knowledge of the process. Besides, the multivariate statistical analysis method is particularly useful to handle high dimensional and cross-correlated variables,
D
which are commonly observed cases in the batch process.
TE
Among the widely used multivariate statistical analysis methods such as principal component analysis
CE P
(PCA), partial least squares (PLS), canonical correlation analysis (CCA), PLS may be the most popular one, which has been extended to the multiway counterpart for batch process modeling [1]. Along last several
AC
decades, the multiway PLS (MPLS) method has been well acknowledged for monitoring and quality prediction in batch processes [2-12]. However, it has recently been explored that the traditional MPLS model cannot efficiently reveal the multiphase data behavior, which is very common in many batch processes. In those batch processes, different phases may have different data characteristics. For example, a typical fermentation process contains the preculture phase and the production phase. The data characteristic and the variable relationship in these two phases are both different from each other. Therefore, a batch process is desired to be divided into several phases, in order to improve the modeling performance. So far, a lot of methods have been developed for multiphase batch process modeling. According to the recent survey paper on the topic of multiphase batch process, all of those developed methods can be
-2-
ACCEPTED MANUSCRIPT roughly summarized into two different categories: multiblock methods and phase-separated methods [13]. By grouping the batch process into several blocks, the multiblock method characterizes the multiphase
IP
T
batch process through a single model structure. Compared to the MPLS model, both of the data
SC R
interpretation and the process understanding have been enhanced. The idea of the phase-separated method is to build a separated model for each phase in the batch process. In the past years, various multiphase modeling approaches have been developed, including Undey and Cinar, Lu et al., Muthuswamy and
NU
Srinivasan, Camacho et al., Zhao et al., Ge et al., etc [14-20].
MA
However, while the phase-separated models can efficiently capture different data characteristics in multiple phases of the batch process, the interphase relationships have rarely been explored and used for
D
quality prediction. Nevertheless, the ignored interphase relationship may have significant impacts on
TE
prediction of the final product quality. In other words, the quality prediction performance might be
modeling.
CE P
improved if the effective interphase relationships can be revealed and incorporated for multiphase
AC
In this paper, a two-level PLS modeling approach is proposed for quality prediction of multiphase batch processes. In the first level, a separated intraphase-PLS model is developed in each phase of the batch process. Based on the extracted latent variable information in different phases, a series of interphase-PLS models are constructed in the second level. For example, for prediction of the final product quality in the second phase of the batch process, the interphase relationship between the first and the second phases should be built; for quality prediction in the third phase, the interphase relationships among the first three phases should be built; the rest can be deduced by analogy until the last phase, in which the interphase relationships among all phases should be built for quality prediction in the last phase. The remainder of this paper is organized as follows. In section 2, detailed methodology description of
-3-
ACCEPTED MANUSCRIPT the proposed two-level PLS model based multiphase quality prediction method is provided, including the data unfolding and phase division step, development of the two-level PLS model, and the online multiphase
IP
T
quality prediction algorithm. An industrial application study of the injection molding process is used for
SC R
performance evaluation of the proposed method, which is provided in Section 3. Finally, conclusions are made in the last section.
NU
2. Multiphase quality prediction based on two-level PLS model
MA
In this section, detailed methodology descriptions of the two-level PLS model is demonstrated. First, the three-dimensional batch process dataset is unfolded into two-dimension, the mean trajectory is removed
D
for data scaling, and the whole batch process is divided into different phases. Second, a separated
TE
intraphase-PLS model is constructed inside each phase of the batch process. Third, a series of
CE P
interphase-PLS models are developed in the second level, which are used to capture the interphase relationships among different phases. Based on the developed two-level PLS model, an online multiphase
AC
quality prediction algorithm is then formulated.
2.1. Data unfolding and phase division The batch process data matrix X is typically collected in a three-way manner, which is
X( I J K ) , where I is the batch number, J is the number of measured variables, and K represents the batch duration. For multivariate statistical modeling, the three-way dataset is always unfolded into a two-dimensional dataset X( I JK ) through the batch direction. Then, the data scaling is carried out before the multivariate statistical modeling step, thus the mean and deviation values of the process variables along each time slice are removed from the original variables. For multiphase batch process modeling, an important issue is how to divide the batch process into
-4-
ACCEPTED MANUSCRIPT different phases, which is referred as the phase division problem. Generally, there are three major ways for phase division in the batch process [7]. The first way is based on the expert knowledge, which is
IP
T
straightforward, and has been used in many early research works [21, 22]. The second way is based on the
SC R
process analysis techniques. Similar to the expert knowledge-based method, it is also under the assumption that some certain required process features are known beforehand, such as phase change points, process behaviors in different phases, etc [23, 24]. Later, some purely data-driven methods have been developed for
NU
phase division, which are more useful in practice. Two typical methods are the sub-PCA based method
MA
which is based on the change of the variable correlations, and the multiphase algorithm which detects the phase division points based on the prediction ability of the PCA model [15, 16]. More recently, with the
D
consideration of the phase transition behavior, a new phase division approach has been proposed, which is
TE
based on the repeatability degree of different batches during each time interval [25]. In the present paper,
CE P
we assumed that the batch process has already been divided into different phases. The data unfolding the phase division of the batch process are illustrated in Figure 1.
AC
[Figure 1 about here]
2.2. Two-level PLS model development After the batch process has been divided into different phases, the two-level PLS model can be developed, including the intraphase PLS model built in each phase, and a series of interphase PLS models constructed for connections of different phases. Suppose the multiphase batch process dataset is represented as
X( I JK ) [X1 ( I JK1 ) X2 ( I JK2 )
Xs ( I JK s )
XS ( I JK S )] , where S is the
number of phases. Detailed descriptions of these PLS models are demonstrated in the following two subsections. 2.2.1. Intraphase PLS models
-5-
ACCEPTED MANUSCRIPT To build the representative PLS model inside each phase of the batch process, the PLS prediction model for each time slice inside the phase should be developed in the first step, given as follows
s 1 k
R W (P W ) Q s k
sT k
sT k
(1)
SC R
s k
IP
Y Tks Q ks T Fks
T
X ks ( I J ) Tks PksT E ks
where Y is the quality data matrix of the batch process which are measured at the end of each batch,
s 1,2,
, S , k 1, 2,
, K s is the number index of time interval inside each phase. Pks and Q ks
NU
are the loading matrices, Tks is the score matrix, E ks and Fks are the residual matrices of each PLS
MA
model. Wks is the weighting matrix of each PLS model, and R ks represents each regression matrix. Next, the intraphase-PLS model can be developed, which is actually the mean representative of the
D
time slice PLS model in the phase. The regression matrix R *s , latent variable matrix Ts* and its *
TE
regression matrix Vs can be determined as follows
1 R s K
CE P
* s
AC
Ts*
where s 1, 2,
1 Ks
1 V s K * s
Ks
R k 1
s k
Ks
T k 1
s k
(2)
Ks
W (P k 1
s k
sT k
Wks ) 1
, S is the phase number in the batch process. For each single phase, we have assumed
that the data behavior keeps similar during the phase, thus, the representative PLS model is able to capture the general relationship among process variables [16]. That is to say, we have ignored the dynamic information in each phase of the batch process. 2.2.2. Interphase PLS models After the intraphase-PLS model has been developed in each phase, the interphase PLS models can then be constructed to connect the relationships among different phases. In the second phase, the representative latent matrices of the first and second phases are augmented together, given as -6-
ACCEPTED MANUSCRIPT L12 [T1* T2* ]
(3)
quality information of the batch process Y , which is described as follows
R1 2 W1 2 (P
T 1 2
where P1 2 and
(4)
SC R
Y T1 2Q1T 2 F12
IP
L1 2 T1 2 P1T2 E12
T
Then the first interphase-PLS model is build to capture the relationships between L1 2 and the final
1
T 1 2
W1 2 ) Q
Q1 2 are the loading matrices of this interphase PLS model, T1 2 is the score
NU
matrix, E1 2 and F1 2 are the residual matrices, W1 2 is the weighting matrix, and R1 2 is the
MA
regression matrix.
With the ongoing of the batch process, the relationships among more phases can be built. Generally,
D
the relationships among the first s phases can be built as follows
L1s [T1* T2*
TE
Ts* ]
(5)
L1 s T1 s P1T s E1 s Y T1 s Q1T s F1 s
CE P
(6)
R1 s W1 s (P1T s W1 s ) 1 Q1T s
, S . Therefore, a total of S 1 interphase-PLS models can be constructed for
AC
where s 1, 2,
connecting different phases.
2.3. Online quality prediction algorithm Based on the developed two-level PLS models, an online quality prediction algorithm can be new
formulated. For a new measured process data sample x kc
in a specific phase sc , the online prediction
of the final quality variables can be made by the following steps. First, the latent variable of this new data sample is calculated by the representative intraphase-PLS model in the sc -th phase, given as new * t new kc x kc Vs
(7)
Suppose the latent matrix of the data samples which starts from the first data sample of the sc -th phase
-7-
ACCEPTED MANUSCRIPT new
until the latest one before the current data sample is represented as Tsc ,kc , the mean latent variable vector inside the sc -th phase at the kc -th time interval can be calculated as follows
T
* new new t new t kc ]} sc , kc mean{[Tsc , kc
(8)
SC R
IP
where mean{} is to calculate the mean value of the latent variable vector inside the sc -th phase. To calculate the quality variable in the second level, the mean latent variable vectors of the previous phases
new*
as t s
, sc should also be incorporated. Denote the mean latent variable vector of each previous phase
(s 1, 2,
, sc 1) , the integrated latent variable information of the new batch can be represented
NU
s 1, 2,
MA
as follows *T l scnew,kc [t1new*T t new 2
*T *T T t new t new sc 1 sc , kc ]
(9)
TE
calculated as follows
D
Next, the final quality variable can be determined by the (sc 1) -th interphase-PLS model, which is
(10)
CE P
yˆ new,kc R1T sc l scnew,kc
2.4. Implementation procedures
AC
In summary, the offline modeling and online quality prediction procedures of the proposed method are listed as follows.
Offline modeling stage Step 1: Collect the historical dataset for the batch process; Step 2: Unfold the three-way dataset into a two-dimensional dataset for scale, and divided the whole batch process into different phases; Step 3: Build intraphase-PLS models in different single phases (First level); Step 4: Based on the extracted latent variable information in different phases, construct a series of interphase-PLS models (Second level);
-8-
ACCEPTED MANUSCRIPT Step 5: Save both of the intraphase-PLS and interphase-PLS models. Online monitoring stage
IP
T
Step 1: Acquire the new data sample from online batch process;
SC R
Step 2: Data pre-treatment of the new data sample by the same step used in the modeling stage; Step 3: Calculate the latent variable information of the new data sample with the intraphase-PLS model in the corresponding phase;
NU
Step 4: Ensemble the latent variable information in all available phases, calculate the quality prediction
MA
results by using the corresponding interphase-PLS model in the second level. Step 5: Performance evaluation and process analysis.
TE
and Figure 3, respectively.
D
Systematic illustrations of the offline modeling and online monitoring scheme are shown in Figure 2
CE P
[Figure 2-3 about here]
AC
3. Case study of injection molding process Injection molding process is a typical multiphase batch process, which usually contains injection phase, packing-holding phase, plastication phase, cooling phase, etc. A simplified flowchart of a reciprocating-screw injection molding process is shown in Figure 4. For process analysis and quality control purposes, some important variables have been measured in this process, such as temperatures, pressures, and the screw velocity. In the present study, the weight of the final product is selected as the quality variable, for prediction of which a total of 11 key variables have been selected and listed in Table 1. Detailed description and experimental design of the injection molding process can be found in Lu et al [16]. [Figure 4 about here]
-9-
ACCEPTED MANUSCRIPT [Table 1 about here] For batch process modeling and testing, a dataset containing 150 batches has been generated, among
IP
T
which 100 batches are used for model training and the remaining 50 batches are for testing. For simplicity,
SC R
the same duration of all batches is assumed, which is 635 sampling intervals. Based on the phase division method, the whole injection molding process can be divided into seven different phases. Therefore, the training dataset is divided into seven parts, each of which is used for development of the intraphase-PLS
NU
model. Then number of each PLS model has been determined through the cross-validation approach. As a
MA
result, the numbers of identified components in each phase are 4, 3, 5, 6, 3, 4 and 5. After the latent variable information has been extracted in each phase of the batch process, the interphase-PLS models can then be
D
constructed in the second level. Since there are seven phases in this process, a total of six interphase-PLS
TE
models should be built. For performance evaluation of the quality prediction method in each time interval,
CE P
the root mean square error (RMSE) criterion is used, which is defined as follows
where kc 1, 2, batches, yˆ i
y i 1
i
yˆ ikc
2
(11)
nte
AC
RMSE (kc)
nte
kc
, K represents the time interval of the batch process, nte is the number of testing
is the predicted quality values made in the kc-th time interval, y i is the corresponding
measured quality values, and i is the number index of the testing batches. Similarly, for performance evaluation of a new batch inside each phase and during the whole batch, the corresponding RSME indices can be defined as follows Ks
RMSE ( s )
y yˆ
kc 1
Ks
- 10 -
s 2 kc
(12)
ACCEPTED MANUSCRIPT K
RMSEBatch
kc 1
2 kc
(13)
K
T
, K s is the predicted quality values obtained in each phase, and yˆ kc is the
IP
s where yˆ kc , kc 1, 2,
y yˆ
SC R
predicted value in each time interval, y is the corresponding measured quality values at the end of the batch.
For comparison purpose, the quality prediction results of the phase-based sub-PLS models are also
NU
presented here, which is adopted from reference [16]. After quality prediction of the 50 testing batches, the
MA
overall results of the RMSE index are shown in Figure 5. As can be seen, the values of the RMSE index generated by the two-level PLS model are smaller than those provided by the sub-PLS model in most time
D
intervals along the whole batch duration, which means that the quality prediction performance of the testing
TE
batches has been generally improved by the two-level PLS model. Figure 6 (a-g) shows the RMSE results
CE P
of each testing batch in different phases. In the first phase, the same results have been obtained by the sub-PLS model and the two-level PLS model. This is because no interphase-PLS model has been
AC
incorporated in the first phase. Starting from the second phase, the RMSE values are different for the two methods. Generally, in each phase, most of the star points which represent for the results of two-level PLS model are lower than the circle points, denoting the results of sub-PLS model. Therefore, the quality prediction performance of the testing batches has been improved in each of the six phases. Similarly, the RMSE value through the whole batch direction for each of the 50 testing batches is given in Figure 6 (h), through which one can also find that the quality prediction results of most batches have been improved by the two-level PLS model. [Figure 5-6 about here] Particularly, detailed prediction results of two testing batches are further illustrated in Figure 7 (a), and
- 11 -
ACCEPTED MANUSCRIPT Figure 7 (b), respectively. From both two figures, we can find the prediction values of the two-level PLS model are much closer to the actual weights of the final product. By examining the RMSE values of these
IP
T
two testing batches in each of the seven phases, similar conclusions can be explored. Detailed RMSE
SC R
values of the two testing batches in different phases are tabulated in Table 2. As been shown in this table, the prediction performance has been greatly improved in the last two phases for both of the two testing batches. In the last row of Table 2, the results of the RMSE values along the whole batch duration for the
NU
two testing batches are also provided. Again, the superiority of the two-level PLS model has been evaluated
MA
in both two batches.
[Figure 7 about here]
CE P
4. Conclusions
TE
D
[Table 2 about here]
In the present paper, a two-level PLS model has been developed for quality prediction of multiphase
AC
batch processes. In the first level of this method, a representative intraphase-PLS model was built in each phase of the batch process, based on which the latent variable data information is extracted. Then, a series of interphase-PLS models were constructed in the second level, which were used to model the relationships among different phases. Based on these interphase-PLS models, different phases of the batch process can be connected. With the additional relationship information extracted among different phases, the quality prediction performance has been improved, which has been evaluated and confirmed by an industrial injection molding process. Compared to the conventional sub-PLS model, the weight prediction of the final product becomes more accurate with the incorporation of the interphase-PLS models. It is worth to note that with the duration of the batch process, more and more data information have
- 12 -
ACCEPTED MANUSCRIPT been incorporated for interphase modeling. Therefore, a linear model may be insufficient to capture the interphase relationship among different phases. In this case, the linear PLS model can be extended to more
IP
T
complex cases, such as kernel approach, local modeling approach, mixture models, etc [26-28]. While the
SC R
prediction accuracy may be improved, as a compromise, the computational burden will be increased for both offline modeling and online utilization. Another important issue is how to guarantee the modeling reliability, particularly for the interphase model. This is because with the increase of the number of latent
NU
variables incorporated for interphase modeling, we may need more training batches in order to guarantee
MA
the modeling reliability. However, it is difficult to set a ratio of the number of modeling data to the number of latent variables, since different processes may need different numbers of effective modeling data. In
D
practice, what we can do is to collect the training data in as much wide operation region as possible. Finally,
TE
the proposed two-level modeling method can be considered a compromise between the traditional multiway
CE P
modeling method which connects all time slices into a single model and the phase-based modeling method
AC
which only models variable relationships inside each single phase.
Acknowledgement This work was supported in part by the National Natural Science Foundation of China (NSFC) (61370029), Project National 973 (2012CB720500), and the Fundamental Research Funds for the Central Universities (2013QNA5016).
References [1] P. Nomikos, J. F. MacGregor, Multi-way partial least square in monitoring batch processes. Chem. Intel. Lab. Syst. 30 (1995) 97-108.
- 13 -
ACCEPTED MANUSCRIPT [2] Z. Q. Ge, Z. H. Song, F. R. Gao, Review of Recent Research on Data-Based Process Monitoring, Ind. Eng. Chem. Res. 52 (2013) 3543-3562.
IP
T
[3] E. N. M. Sprang, H. J. Ramaker, J. A. Westerhuis, S. P. Guiden, A. K. Smilde, Critical evaluation of
SC R
approaches for on-line batch process monitoring. Chem. Eng. Sci. 57 (2008) 3979–3991. [4] J. H. Chen, C. M. Song, T. Y. Hsu, Online monitoring of batch processes using IOHMM based MPLS. Ind. Eng. Chem. Res. 49 (2010) 2800–2811.
MA
space. J. Process Contr. 20 (2010) 716-725.
NU
[5] C. R. Alvarez, A. Brandolin, M. C. Sanchez, Batch process monitoring in the original measurement’s
[6] T. Chen, J. Zhang, On-line multivariate statistic monitoring of batch processes using Gaussian mixture
D
model. Comput. Chem. Eng. 34 (2010) 500–507.
TE
[7] S. W. Andersen, G. C. Runger, Partitioned partial least squares regression with application to a batch
CE P
fermentation process. J. Chemometrics 25 (2011) 159-168. [8] Q. P. He, J. Wang, Statistics pattern analysis: A new process monitoring framework and its application
AC
to semiconductor batch processes. AIChE J. 57 (2011) 107-121. [9] J. Yu, Nonlinear bioprocess monitoring using multiway kernel localized fisher discriminant analysis. Ind. Eng. Chem. Res. 50 (2011) 3390-3402. [10] Y. W. Zhang, Z. Y. Hu, On-line batch process monitoring using hierarchical kernel partial least squares. Chem. Eng. Res. Des. 89 (2011) 2078-2084. [11] Y. Q. Chang, Y. S. Lu, F. L. Wang, S. Wang, S. M. Feng, Sub-stage PCA modeling and monitoring method for uneven-length batch processes. Candian J. Chem. Eng. 90 (2012) 144-152. [12] M. J. Anzanello, S. L. Albin, W. A. Chaovalitwongse, Multicriteria variable selection for classification of production batches. European J. Operat. Res. 218 (2012) 97-105.
- 14 -
ACCEPTED MANUSCRIPT [13] Y. Yao, F. R. Gao, A survey on multistage/multiphase statistical modeling methods for batch processes. Annual Reviews in Control 33 (2009) 172-183.
IP
T
[14] C. Undey, A. Cinar, Statistical monitoring of multistage, multiphase batch processes. IEEE Contr.
SC R
Syst. Mag. 22 (2002) 53-63.
[15] N. Y. Lu, F. R. Gao, F. L. Wang, A sub-PCA modeling and online monitoring strategy for batch processes. AIChE J. 50 (2004) 255-259.
NU
[16] N. Y. Lu, F. R. Gao, Stage-based process analysis and quality prediction for batch processes. Ind. Eng.
MA
Chem. Res. 44 (2005) 3547-3555.
[17] K. Muthuswamy, R. Srinivasan, Phase-based supervisory control for fermentation process
D
development. J. Process Control 13 (2003) 367-382.
TE
[18] J. Camacho, J. Pico, A. Ferrer, Multi-phase analysis framework for handling batch process data. J
CE P
Chemometrics 22 (2008) 632-643.
[19] C. H. Zhao, F. R. Gao, F. L. Wang, Phase-based joint modeling and spectroscopy analysis for batch
AC
process monitoring. Ind. Eng. Chem. Res. 49 (2010) 669-681. [20] Z. Q. Ge, Z. H. Song, F. R. Gao, Nonlinear quality prediction for multiphase batch processes, AIChE J. 58 (2012) 1778-1787. [21] J. A. Westerhuis, T. Kourti, J. F. MacGregor, Analysis of multiblock and hierarchical PCA and PLS models. J. Chemometrics 12 (1998) 301-321. [22] S. P. Reinikainen, A. Hoskuldsson, Multivariate statistical analysis of a multi-step industrial process, Anal. Chinica Acta 595 (2007) 248-256. [23] P. Facco, M. Olivi, C. Rebuscini, F. Bezzo, M. Barolo, Multivariate statistical estimation of product quality in the industrial batch production of a resin. Dycops 2 (2007) 93-98.
- 15 -
ACCEPTED MANUSCRIPT [24] X. T. Doan, R. Srinivasan, Online monitoring of multi-phase batch processes using phase-based multivariate statistical process control. Comput. Chem. Eng. 32 (2008) 230-243.
IP
T
[25] Z. Ge, L. Zhao, Y. Yao, Z. Song, F. Gao, Utilizing transition information in online quality prediction
SC R
of multiphase batch processes. J. Process Contr. 22 (2012) 599-611.
[26] Z. Q. Ge, Z. H. Song, Online monitoring of nonlinear multiple mode processes based on adaptive local model approach, Contr. Eng. Pract. 16 (2008) 1427-1437.
NU
[27] Z. Q. Ge, C. J. Yang, Z. H. Song, Improved kernel PCA-based monitoring approach for nonlinear
MA
processes, Chem. Eng. Sci. 64 (2009) 2245-2255.
[28] Z. Ge, F. Gao, Z. Song. Mixture probabilistic PCR model for soft sensing of multimode processes.
AC
CE P
TE
D
Chem. Intel. Lab. Syst., 105 (2011), 91-105.
- 16 -
ACCEPTED MANUSCRIPT Figure Captions Figure 1: Data unfolding and phase division of the multiphase batch process
SC R
Figure 3: Online quality prediction scheme of the two-level PLS model
IP
T
Figure 2: Offline modeling of the two-level PLS method
Figure 4: Simplified schematic flowchart of the injection molding machine Figure 5: RMSE values of the 50 testing batch in each time interval
NU
Figure 6: RMES values of the 50 testing batches, (a) First phase; (b) Second phase; (c) Third phase; (d)
MA
Fourth phase; (e) Fifth phase; (f) Sixth phase; (g) Seventh phase; (h) Whole batch duration
AC
CE P
TE
D
Figure 7: Quality prediction results of the two methods, (a) First batch; (b) Second batch
- 17 -
ACCEPTED MANUSCRIPT Table Captions Table 1: Variables selected for quality prediction
AC
CE P
TE
D
MA
NU
SC R
IP
T
Table 2: RMSE values of the two testing batches in different phases and during the whole batch
- 18 -
ACCEPTED MANUSCRIPT
Table 1: Variables selected for quality prediction Variables
Unit
No.
Variables
1 2 3 4 5 6
Valve 1 opening Valve 2 opening Screw stroke Screw velocity Ejector stroke Mold stroke
% % mm mm/s mm mm
7 8 9 10 11
Mold velocity Injection press Temperature 3 Temperature 2 Temperature 1
mm/s Bar ℃ ℃ ℃
IP
SC R
NU MA D TE CE P AC
- 19 -
Unit
T
No.
ACCEPTED MANUSCRIPT
Table 2: RMSE values of the two testing batches in different phases and during the whole batch First batch
sub-PLS
Two-level PLS
0.0108 0.0108 0.0108 0.0108 0.0111 0.0112 0.0129 0.0112
0.0108 0.0092 0.0109 0.0109 0.0113 0.0059 0.0025 0.0088
0.0076 0.0117 0.0157 0.0113 0.0036 0.0115 0.0097 0.0113
0.0076 0.0060 0.0036 0.0050 0.0044 0.0037 0.0028 0.0048
SC R
IP
Two-level PLS
AC
CE P
TE
D
MA
Batch direction
sub-PLS
NU
Phase #1 Phase #2 Phase #3 Phase #4 Phase #5 Phase #6 Phase #7
Second batch
T
Phases
- 20 -
ACCEPTED MANUSCRIPT
X1
X2
…
XK2
……
XK
…
X K1
Second phase
MA
First phase
X K1 +1
……
NU
X1
SC R
IP
T
X
X KS-1 +1 …
XK
S-th phase
AC
CE P
TE
D
Figure 1: Data unfolding and phase division of the multiphase batch process.
- 21 -
ACCEPTED MANUSCRIPT
First phase
Second phase
S-th phase
…
X K1
X K1 +1
…
XK2
X KS-1 +1 …
……
XK
SC R
IP
X1
T
Y
…… ……
NU
Intraphase-PLS model #2
Intraphase-PLS model #S
MA
Intraphase-PLS model #1
D
……
Interphase-PLS model #S-1
TE
Interphase-PLS model #1
AC
CE P
Figure 2: Offline modeling of the two-level PLS method
- 22 -
ACCEPTED MANUSCRIPT
First phase
…
……
Latent variables #1
Latent variables #2
T
…
S-th phase
…
IP
New batch
Second phase
SC R
Intraphase-PLS models
Latent variables #S
NU
……
MA
Interphase-PLS models
D
Final qualtiy prediction results
AC
CE P
TE
Figure 3: Online quality prediction scheme of the two-level PLS model.
- 23 -
SC R
IP
T
ACCEPTED MANUSCRIPT
P: pressure sensor T: temperature sensor
NU
IT: inferred temperature sensor LDT&LVT: displacement & velocity sensor
MA
SV1&SV2: servo-valves
AC
CE P
TE
D
Figure 4: Simplified schematic flowchart of the injection molding machine.
- 24 -
MA
NU
RMSE
SC R
IP
T
ACCEPTED MANUSCRIPT
TE
D
Sampling intervals
AC
CE P
Figure 5: RMSE values of the 50 testing batch in each time interval
- 25 -
SC R
IP
T
RMSE
RMSE
ACCEPTED MANUSCRIPT
Testing batches
Testing batches
(b)
RMSE
(d)
RMSE
CE P AC
RMSE
(c)
Testing batches
TE
Testing batches
D
RMSE
MA
NU
(a)
Testing batches
Testing batches
(e)
(f)
- 26 -
SC R
IP
T
RMSE
RMSE
ACCEPTED MANUSCRIPT
Testing batches
Testing batches
(g)
(h)
NU
Figure 6: RMES values of the 50 testing batches, (a) First phase; (b) Second phase; (c) Third phase; (d)
AC
CE P
TE
D
MA
Fourth phase; (e) Fifth phase; (f) Sixth phase; (g) Seventh phase; (h) Whole batch duration
- 27 -
SC R
IP
T
Weight (g)
ACCEPTED MANUSCRIPT
NU
Sampling intervals
D TE CE P AC
Weight (g)
MA
(a)
Sampling intervals (b)
Figure 7: Quality prediction results of the two methods, (a) First batch; (b) Second batch.
- 28 -
ACCEPTED MANUSCRIPT Highlights
IP
T
> A two-level PLS modeling structure is proposed for quality prediction. > Intraphase PLS
SC R
model is developed inside each phase of the batch process. > Interphase PLS models are developed to capture relationships among different phases. > An industrial injection molding
AC
CE P
TE
D
MA
NU
process is used for performance evaluation. >
- 29 -