A Novel Approach to T-S Fuzzy Modeling of Nonlinear Dynamic Systems with Uncertainties using Symbolic Interval-Valued Outputs

17th 17th IFAC IFAC Symposium Symposium on on System System Identification Identification 17th Symposium on Identification Beijing International Convention 17th IFAC IFAC Symposium on System SystemCenter Identification Beijing International Convention Center 17th IFAC Symposium on System Identification Beijing International Center October 19-21, 2015. 2015. Convention Beijing, China China Available online at www.sciencedirect.com Beijing International Convention Center October 19-21, Beijing, Beijing International Convention Center October 19-21, 2015. Beijing, China October October 19-21, 19-21, 2015. 2015. Beijing, Beijing, China China

ScienceDirect

A A A

IFAC-PapersOnLine 48-28 (2015) 1196–1201

Novel Approach to T-S Fuzzy Modeling Novel Approach to Fuzzy Novel Approach to T-S T-S Systems Fuzzy Modeling Modeling of Nonlinear Dynamic with of Nonlinear Dynamic Systems with of Nonlinear Dynamic Systems Uncertainties using Symbolicwith Uncertainties using Symbolic Uncertainties usingOutputs Symbolic Interval-Valued Interval-Valued Interval-Valued Outputs Outputs

Salman Zaidi Zaidi and and Andreas Andreas Kroll Kroll Salman Salman Zaidi and Andreas Kroll Salman Zaidi and Andreas Kroll Measurement Measurement and and Control Control Department, Department, Faculty Faculty of of Mechanical Mechanical Measurement and Control Department, Faculty of Mechanical Engineering, University of Kassel, Germany. Engineering, University of Kassel, Germany. Measurement and Control Department, Faculty of Mechanical Engineering, University of Kassel, Germany. e-mail: {salman.zaidi, andreas.kroll}@mrt.uni-kassel.de e-mail: {salman.zaidi, andreas.kroll}@mrt.uni-kassel.de Engineering, University of Kassel, Germany. e-mail: {salman.zaidi, andreas.kroll}@mrt.uni-kassel.de e-mail: {salman.zaidi, andreas.kroll}@mrt.uni-kassel.de Abstract: A A novel novel approach approach to to Takagi-Sugeno Takagi-Sugeno (T-S) (T-S) fuzzy fuzzy modeling modeling of of aa class class of of nonlinear nonlinear Abstract: Abstract: A novel approach to Takagi-Sugeno (T-S)for fuzzy a classError of nonlinear dynamic systems having variability in their their outputs outputs the modeling Nonlinearof Output (NOE) dynamic systems having variability in the Nonlinear (NOE) Abstract: A novel approach to Takagi-Sugeno (T-S)for fuzzy modeling ofOutput a classError of nonlinear dynamic systems having in theirinput-output outputs for datasets the Nonlinear Output Error (NOE) case is in article. were by case is addressed addressed in this this variability article. Multiple Multiple were obtained obtained by repeating repeating dynamic systems having variability in theirinput-output outputs for datasets the Nonlinear Output Error (NOE) case is addressed in this article. Multiple input-output datasets were obtained by repeating the identification experiment. The variability in the output time series is captured by defining the TheMultiple variability in the output time series capturedbybyrepeating defining caseidentification is addressed experiment. in this article. input-output datasets were isobtained the identification experiment. Thetime variability the output time series isprovide captured defining the envelops of of response response at each each instant.in These envelops actually the by confidence the envelops at instant. envelops actually the confidence identification experiment. Thetime variability inThese the output time series isprovide captured by defining the envelops of response at each time instant. These envelops actually provide the confidence interval basedofupper upper and lower lower bounds of the output output time series using using theprovide extended Chebyshev’s interval based and bounds the series the extended the envelops response at each time of instant. Thesetime envelops actually theChebyshev’s confidence interval based upper and bounds ofapproach, the output series the extended Chebyshev’s Inequality. Different from the in which two independent T-S models Inequality. Different fromlower the previous previous intime which twousing independent T-S fuzzy fuzzy models interval based upper and lower bounds ofapproach, the output time series using the extended Chebyshev’s Inequality. Different from the previous approach, in which two independent T-S fuzzy were used for identifying each bound, a single T-S fuzzy model is identified in this work, which were used for identifying single T-S in fuzzy model identified inT-S thisfuzzy work,models which Inequality. Different fromeach the bound, previousa approach, which two isindependent models were used for identifying each bound, a single T-S fuzzy model isvariables. identifiedThis in this work, which resulted in interval parameters for the antecedent and consequent variables. This is accomplished resulted in interval parameters for the antecedent and consequent is accomplished were used for identifying each bound, a single T-S fuzzy model is identified in this work, which resulted in interval parameters forinto the the antecedent consequent variables. accomplished by first transforming transforming the bounds bounds symbolicand interval-valued data and andThis thenis using this data data by first the symbolic interval-valued data then this resulted in interval parameters forinto the the antecedent and consequent variables. This isusing accomplished by first transforming the bounds into the symbolic interval-valued data and then using this data for identification. In order to get the expected value of the response, the estimated and for identification. In the order to getinto thethe expected value of the response, lower and by first transforming bounds symbolic interval-valued data the andestimated then usinglower this data for identification. In order to get the expected value of the response, the estimated lower and upper bound time series of the identified T-S fuzzy model were averaged out at each time instant, upper bound time In series of the identified T-S fuzzy model wereresponse, averagedthe out estimated at each time instant, for identification. order to get the expected value of the lower and upper bound time series of the identified T-S fuzzy model were averaged out at each time instant, as permitted by the the extended Chebyshev’s Inequality. The proposed approach is demonstrated demonstrated as permitted by extended Chebyshev’s The proposed approach is upper bound time series of the identified T-SInequality. fuzzy model were averaged out at each time instant, as permitted by the extended electro-mechanical Chebyshev’s Inequality. Thevalve. proposed approach is demonstrated on an industrial industrial diesel-engine throttle on an diesel-engine throttle as permitted by the extended electro-mechanical Chebyshev’s Inequality. Thevalve. proposed approach is demonstrated on an industrial diesel-engine electro-mechanical throttle valve. on an industrial diesel-engine electro-mechanical throttle valve. © 2015, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Keywords: Nonlinear Nonlinear system system identification, identification, fuzzy fuzzy modeling, modeling, uncertainty uncertainty modeling, modeling, symbolic symbolic Keywords: Keywords: system identification, fuzzy modeling, interval-valued clustering, interval-valued linear regression interval-valued clustering, interval-valued Keywords: Nonlinear Nonlinear system identification,linear fuzzyregression modeling, uncertainty uncertainty modeling, modeling, symbolic symbolic interval-valued interval-valued clustering, clustering, interval-valued interval-valued linear linear regression regression 1. INTRODUCTION INTRODUCTION Logic System System (PFLS) (PFLS) (Liu (Liu and and Li (2005)) was proposed. 1. Logic Li (2005)) was proposed. 1. Logic (Liu and Li was proposed. Different from (PFLS) the T2 FLS, FLS, the PFLS uses the the probability Different from the T2 uses 1. INTRODUCTION INTRODUCTION Logic System System (PFLS) (Liu the andPFLS Li (2005)) (2005)) wasprobability proposed. Different from the T2 FLS, the PFLS uses the probability density function (PDF) in its secondary membership funcdensity function (PDF) in its secondary membership funcDifferent from the T2 FLS, the PFLS uses the probability Takagi-Sugeno (TS) (TS) fuzzy fuzzy models models have have been been widely widely tion. density function (PDF) in its secondary membership func-aa Takagi-Sugeno The output of the PFLS is a random variable with The output(PDF) of the in PFLS is a random variable with density function its secondary membership funcTakagi-Sugeno (TS) fuzzy models have been because widely tion. adopted in the field of system identification because tion. The output of the PFLS is a random variable with a adopted in the field of system identification Takagi-Sugeno (TS) fuzzy models have been widely certain certain PDF which actually provides measure of stochasstochasPDF which provides aa measure of tion. The output ofactually the PFLS is a random variable with a adopted in the field of system identification because of their universal function approximation ability (Ying of their universal function approximation abilitybecause (Ying tic adopted in the field of system identification certain PDF which actually provides a measure of stochastic uncertainty associated with the However, the uncertainty associated the output. output. However, the certain PDF which actuallywith provides a measure of stochasof universal function approximation (Ying (1998)). In particular, TS fuzzy fuzzy models with withability affine conseconse(1998)). particular, TS models affine of their their In universal function approximation ability (Ying procedure tic uncertainty associated with the output. However, procedure of translating translating data uncertainty into the PDF PDFthe of of data uncertainty into the of tic uncertainty associated with the output. However, the (1998)). In particular, TS fuzzy models with affine consequents have found their theirTS extensive application the of procedure of translating data uncertainty into the PDF of quents found extensive application in the field field of Probabilistic (1998)).have In particular, fuzzy models with in affine conseProbabilistic Fuzzy Sets (PFs) and to interpret the output Fuzzy Sets (PFs) and to interpret thePDF output of translating data uncertainty into the of quentsmodeling have found their extensive application in the (1996); field of procedure fuzzy and control, for instance instance see Kroll Kroll Probabilistic Fuzzy Sets (PFs) and to interpret the output fuzzy and control, for see quentsmodeling have found their extensive application in the (1996); field of PDF PDF accordingly has not(PFs) been and developed yet. Motivating Motivating accordingly has not been developed yet. Probabilistic Fuzzy Sets to interpret the output fuzzy modeling and control, for instance see Kroll (1996); Nelles (2001); Babuˇ s ka (1998). Assuming a fixed model Nelles (2001); Babuˇ ska (1998). Assuming fixed (1996); model from fuzzy modeling and control, for instance seea Kroll PDF accordingly not developed yet. from the fact that that has the model model response should beMotivating associated fact the response should be associated PDF the accordingly has not been been developed yet. Motivating Nelles Babuˇ sska (1998). aa determining fixed structure, the TS TS fuzzy modeling consists of of determining structure, the fuzzy consists Nelles (2001); (2001); Babuˇ ka modeling (1998). Assuming Assuming fixed model model with from the fact that the model response should be associated with a quality tag or confidence measure, the prediction a quality tag or confidence measure, the prediction from the fact that the model response should be associated structure, the TS fuzzy modeling consists of determining validity regions andfuzzy the corresponding corresponding localofaffine affine models, interval with a quality tag or confidence measure, the prediction validity regions and the local models, structure, the TS modeling consists determining based T2 FLS was proposed by Khosravi et al. al. based T2 was proposed by Khosravi et with a quality tag FLS or confidence measure, the prediction validity regions andnonlinear the corresponding affine models, so that the the overall behavior of oflocal the system system can be be interval interval based T2 FLS was proposed by Khosravi et al. so that overall behavior the can validity regions andnonlinear the corresponding local affine models, (2012). The approach used was to construct the prediction (2012). The approach used was to construct the prediction interval based T2 FLS was proposed by Khosravi et al. so nonlinear behavior of can appropriately described by fuzzy fuzzy merging of system these models. appropriately described by merging of these models. so that that the the overall overall nonlinear behavior of the the system can be be interval (2012). approach was construct the prediction interval based on the the used additive noise term and and the model based on additive noise term model (2012). The The approach used was to to construct the the prediction appropriately described by fuzzy merging of these models. appropriately described by fuzzy merging of these models. interval based on the additive noise term and the variance. The noise was assumed to be independent and Conventional fuzzy fuzzy modeling modeling does does not not have have the the ability ability to to variance. The noise assumed to term be independent and interval based on thewas additive noise and the model model Conventional variance. The noise was(i.i.d.) assumed toGaussian. be independent and identically distributed (i.i.d.) and Gaussian. The model Conventional fuzzy modeling does not have the ability to handle uncertainties that originate during system modelidentically distributed and The model The noise was assumed to be independent and handle uncertainties that originate during modelConventional fuzzy modeling does not havesystem the ability to variance. identically distributed (i.i.d.) and Gaussian. The model has, however, no to the inherent varihandle uncertainties that during system modeling. It typical typical uses ordinary ordinary or so-called so-called type-1 fuzzy sets has, however, no ability ability(i.i.d.) to incorporate incorporate the inherent variidentically distributed and Gaussian. The model ing. It uses or type-1 fuzzy sets handle uncertainties that originate originate during system modeling. It typical uses(2001)) ordinaryand or so-called type-1 rule fuzzybase. sets ability has, however, no ability to incorporate the inherent variability in the system. (T1 FSs)(Mendel (2001)) and a deterministic rule base. in the system. has, however, no ability to incorporate the inherent vari(T1 FSs)(Mendel a deterministic ing. It typical uses ordinary or so-called type-1 fuzzy sets ability in the system. (T1 FSs)(Mendel (2001)) and a deterministic rule base. In order to improve improve(2001)) the uncertainty uncertainty handling capability capability of The ability in the system. In to the handling of (T1order FSs)(Mendel and a deterministic rule base. aforementioned fuzzy fuzzy system data The aforementioned system uses uses input-output input-output data In to the capability of fuzzy systems, in Mendel Mendel (2001), it it handling was suggested suggested to use use fuzzy systems, in (2001), was to In order order to improve improve the uncertainty uncertainty handling capability of in The aforementioned fuzzy uses data in which each data point point is system described byinput-output real single-valued single-valued which each data is described by real The aforementioned fuzzy system uses input-output data fuzzy systems, in Mendel (2001), it was suggested to use Type-2 Fuzzy Logic Logic Systems (T2 it FLSs) which use to TypeType-2 Fuzzy Systems (T2 FLSs) use Typefuzzy systems, in Mendel (2001), was which suggested use crisp in which each data point is described by real single-valued crisp number. However, in many complex situations, the number. However, in many complex situations, the in which each data point is described by real single-valued Type-2 Fuzzy Logic Systems (T2 FLSs) which use TypeFuzzyFuzzy Sets (T2 (T2 FSs). The (T2 fuzzyFLSs) systems based on T2 data crisp cannot number. However, in many complex situations, the 22Type-2 Fuzzy Sets FSs). The fuzzy systems based T2 Logic Systems which use on Typecannot be pinned down to single numbers due to the data be pinned down to single numbers due to the crisp number. However, in many complex situations, 2 Fuzzy Sets (T2 FSs). The fuzzy systems based on T2 FSs haveSets shown superior performance in the thebased presence of presence data cannot be pinned down to single numbers due to the FSs have shown performance in presence of 2 Fuzzy (T2superior FSs). The fuzzy systems on T2 of uncertainties uncertainties or the the naturenumbers of data. data. due Suchtodata data presence of or nature of Such data cannot be pinned down to single the FSs shown performance in the of large uncertainties (Mendel (2001). is large uncertainties (Mendel (2001). Their Their downside is the the FSs have have shown superior superior performance in downside the presence presence of can presence of uncertainties nature of Such data can be better better represented or as the a collection collection ofdata. intervals, possibe represented as a of intervals, possipresence of uncertainties or the nature of data. Such data large uncertainties (Mendel (2001). Their downside is the increased computational complexity –– especially increased computational complexity especially during large uncertainties (Mendel (2001). Their downsideduring is the bly can be as aa collection of intervals, possiassociated with weights weights or probability density function with probability function can associated be better better represented represented asor collection ofdensity intervals, possiincreased computational complexity –lack especially during bly the step of type reduction – and the lack of systematic systematic bly associated with2000). weights orthe probability density function the step of type reduction – and the of increased computational complexity – especially during (Bock (Bock and Diday 2000). For the ease of computation, the and Diday For ease of computation, the bly associated with weights or probability density function the step of type reduction – and the lack of systematic approach to type incorporate uncertainties in their secondary use approach incorporate uncertainties secondary the step ofto reduction – and the in lacktheir of systematic (Bock and Diday 2000). For the ease of computation, the of symbolic symbolic interval data is quite quite common. Only very very use of interval data is common. Only (Bock and Diday 2000). For the ease of computation, the approach incorporate in fuzzy sets.to Inspired by the theuncertainties T2 FLS, FLS, the the Probabilistic Probabilistic Fuzzy fuzzy sets. Inspired by T2 Fuzzy approach to incorporate uncertainties in their their secondary secondary use of of symbolic symbolic interval interval data data is is quite quite common. common. Only Only very very fuzzy sets. Inspired by the T2 FLS, the Probabilistic Fuzzy use fuzzy sets. Inspired by the T2 FLS, the Probabilistic Fuzzy Copyright © 2015, IFAC 1196 2405-8963 © IFAC (International Federation of Automatic Control) Copyright IFAC 2015 2015 1196Hosting by Elsevier Ltd. All rights reserved. Copyright © IFAC 2015 1196 Copyright © IFAC 2015 1196 Peer review under responsibility of International Federation of Automatic Copyright © IFAC 2015 1196Control. 10.1016/j.ifacol.2015.12.294

2015 IFAC SYSID October 19-21, 2015. Beijing, China

Salman Zaidi et al. / IFAC-PapersOnLine 48-28 (2015) 1196–1201

few articles discusses the fuzzy modeling using interval data. One of the early attempt include the Interval Fuzzy ˇ Model (INFUMO) (Skrjanc et al. (2005)). The nonlinear function to be approximated is first described by the upper and lower bounds which contain all the measurements. These bounds were then approximated independently using the technique of linear programming. In INFUMO, the upper and lower set of consequent parameters were obtained, whereas the partitioning is performed using crisp antecedent which resulted in single-valued antecedent parameters. This limitation was overcome by the model proposed by Xu and Sun (Xu and Sun (2009)). They proposed an interval TS fuzzy model, in which they used interval arithmetic to obtain interval consequent parameters. For partitioning the input space, they performed clustering separately for center and half range values of the antecedent variables and later combined them using T-norm operator. They derived the TS-model only for the case of one step ahead prediction (Nonlinear Autoregressive eXogenous (NARX) model). The interval response was obtained by assuming interval parameters in the system dynamics and simulating all possible combinations to get extreme values of the response. However, in system identification, these interval parameters are not known before hand and the model is often required to be derived for the case of recursive evaluation (Nonlinear Output-Error (NOE) model); for instance, in simulation or model predictive control. In Zaidi and Kroll (2013), a method was proposed to identify the nonlinear dynamic systems having variability in output by identifying independently the max-min envelops of the response in NOE case using two independent T-S fuzzy models. Later, in Zaidi and Kroll (2014b), the approach was extended using the probability theory. In that method, the expected value of the response at each time instant as well as the envelops based on the (1 − α)100% Confidence Intervals (CI) using the extended Chebyshev inequality (Chebyshev (1867), Kab´ an (2012)) were identified using two independent T-S fuzzy models for the NOE case. In this article, an identification approach is presented to model a nonlinear dynamic system having variability in output by advancing the approach taken in Zaidi and Kroll (2014b). Instead of using two separate models for modeling the CI based lower and upper bounds of output time series, the proposed approach uses interval-valued data directly in T-S modeling to come up with a single T-S model that uses the interval data directly in the modeling process. For demonstrating the effectiveness of the proposed approach, an electro-mechanical throttle was chosen as a case study (a benchmark problem, see Zaidi and Kroll (2014a)). The presented approach begins by pre-processing the data to generate the interval data representing (1 − α)100% confidence bounds obtained by the extended Chebyshev’s Inequality. The procedure then extends the classical TS fuzzy modeling to the case of symbolic-interval valued data. First, the partitioning of antecedent space is carried out using Fuzzy C-Means clustering algorithm for the symbolic-interval valued data (de Carvalho (2007)). The resulting interval prototypes are then used in the multidimensional antecedent fuzzy sets. Second, For determining the parameters of the local modal, the center and range method suitable for the symbolic interval data (Lima Neto

1197

and de Carvalho (2008)) is used in conjunction with the weights determined previously by the multidimensional fuzzy sets. The model estimated this way is valid for the NARX case. Lastly, a nonlinear optimization technique is used to get the parameters of the proposed model for the NOE case wherein the initial values for the optimization are taken from the NARX model. The paper is organized as follows. In Section 2, the problem statement is formulated. In Section 3, The proposed identification approach is described. Section 4 contains the experimental results on the electro-mechanical throttle. Finally, a brief conclusion is drawn in the last section. 2. PROBLEM STATEMENT The following nomenclature has been adopted in this article. Scalars, vectors and matrices are represented as the normal lower case (e.g. x), bold lower case (e.g. x) and normal upper case (e.g. X) letters, respectively. The roman (e.g. z) and san-serif letters (e.g. z) are used to distinguish, respectively, deterministic and random variables. Consider the following Single-Input-Single-Output (SISO) stochastic dynamic system. The case can be extended to the Multiple-Input-Single-Output (MISO) and MultipleInput-Multiple-Output (MIMO) case. Assume that the input signal has no randomness; therefore, treat it as a deterministic signal. However, these assumptions should not be restrictive to the developed method. The system can be described mathematically as follows yt = F (xt ) + ζt

(1)

where, – t in the subscript denotes the time index, t = 1, · · · , n – yt is the stochastic scalar dependent signal (output/regressent) – xt is the stochastic vector independent signal (regressor) consisting of lagged values of input (deterministic) and output (stochastic) signals, i.e., xt = [yt−1 , . . . , yt−ny , ut−τ −1 , . . . , ut−τ −nu ]⊺ , where ny , τ and nu represent the number of lagged output samples, discrete dead time and number of lagged input samples, respectively. – F (·) is the stochastic function – ζt is the additive i.i.d Gaussian noise with zero mean and finite variance The expected Value of Eq. (1) is given by y(t) = E(yt ) = E(F (xt )) = f (t)

(2)

The variance of Eq. (1) can be calculated by the variance sum law 2 2 σy2t = σF (xt ) + σζt

(3)

where F(xt ) and ζt are assumed to be independent of each other. σζt can be estimated experimentally by repeated measurements while holding the inputs constant. If σζt ≪ σF (xt ) , then σyt ≅ σF (xt ) . The task is to estimate a model that describes not only the expected response of the considered stochastic dynamics (f (t) in (2)), but also the model should also be able to

1197

2015 IFAC SYSID 1198 October 19-21, 2015. Beijing, China

Salman Zaidi et al. / IFAC-PapersOnLine 48-28 (2015) 1196–1201

provide the spread or deviation of the response around the expected response (σy2t in (3)). 3. IDENTIFICATION APPROACH The entire modeling procedure for the chosen case study of the identification of the electro-mechanical throttle can be summarized as follows. 3.1 Experiment design and data generation The model quality heavily depends upon the data used for identification. The input signal used for identification should be designed in such a way that it is persistently exciting and able to excite all the amplitude and frequency modes of interest (Ljung (1999)). In order to adequately capture the stochastic modes of the system, the identification experiment has to be repeated for a suitably enough number of times.

Chebyshev’s Inequality assumes that the true values of population mean and standard deviation are known. However, in many practical scenarios, this is not the case. These population parameters have to be estimated from the sample mean and standard deviation provided that the sample size is big enough. In terms of sample parameters, the approximated Chebyshev’s Inequality (Kab´an (2012)) is given by: � � 1 m−1 Pr(|yt − myt | ≥ k syt ) ≤ � + 1 , k2 m(m + 1) (6) t = 1, . . . n, where myt and syt are the sample mean and standard deviation, respectively, given as follows: m

my t = m

sy t =

3.2 Determination of CI based upper and lower bounds Let n and m denote the length of one experiment and the total number of experiments, respectively. The dataset obtained can be lumped into a matrix Z = [u, Y ] ∈ Rn×(m+1) , where u ∈ Rn×1 is the input vector and Y ∈ Rn×m is the output matrix. The output matrix can be represented as follows:  1 2  y1 y1 . . . y1m−1 y1m  y 1 y 2 . . . y m−1 y m  2  2  2 2 Y = . . . . ..  ,  .. .. . . .. . 

yn1 yn2 . . . ynm−1 ynm At any given time instant t, the corresponding row of the output matrix can be seen as m realizations of the stochastic output variable yt following a certain probability distribution function conditioned by past lagged output and input values, i.e. Pr(yt |yt−1 , . . . , yt−ny , ut−τ −1 , . . . , ut−τ −nu ), (4)

In order to capture the scattering of yt at each time instant, one of the possible approaches can be to use the maximum and minimum value. The approach is, however, considered as least robust statistics due to its sensitivity to outliers in data and the dependence on the given sample. An alternative and more reliable approach is to use the probability theory and statistics and calculate the upper and lower confidence bounds on yt based on the standard deviation or a multiple of it. Having the knowledge of exact distribution, these bounds can be calculated using some known coverage factor k, e.g., for normal distribution and 95% confidence level, the value of k = 1.96. In general, the distribution can be of any arbitrarily shape. In order to use the coverage factor which is applicable to the distribution of any kind, Chebyshev’s Inequality (Chebyshev (1867)) can be used. It guarantees that no more than 1/k 2 distribution’s values of a random variable yt (of any arbitrary distribution) can be more than k standard deviations (σt ) away from the mean (µt ), where k ≥ 1, mathematically, 1 Pr(|yt − µt | ≥ k σt ) ≤ 2 , t = 1, . . . n. (5) k

1 � j y , m j=1 t

1 � j (y − myt )2 , m − 1 j=1 t

t = 1, . . . , n

(7)

t = 1, . . . , n

(8)

From (6), the (1 − α)100% CI bounds of yt are determined in the form of the interval [ytl , ytu ], which are approximated from (6) as follows:

ytl ytu

= my t − sy t

�

= my t + sy t

�

m−1 � , α m(m + 1) m−1 � , α m(m + 1)

(9a) (9b)

t = 1, . . . , n.

Using (9), the expected response can be approximated by averaging ytl and ytu in (9) ytm := myt =

ytl + ytu , 2

t = 1, . . . , n.

(10)

Let yt = [ytl , ytu ] ∈ ℑ = {[a, b] : a, b ∈ R, a ≤ b} (t = 1, . . . , n) represents the interval-valued output at each time instant t. The input-output data pairs are collected in T = {(ut , yt )}t=1,...,n which will be used for the identification of the T-S fuzzy model using symbolic intervalued data. The input-output data pairs for the expected values are stored in T m = {(ut , ytm )}t=1,...,n , which will be used for evaluating the modeling performance for the expected response. The developed model should be able to provide good modeling performance for both the spread (in T ) and the expected response (in T m ) for the NOE case.

3.3 Model structure The i-th fuzzy rule of a classical (T1) TS fuzzy model with c rules having antecedents defined by multidimensional reference fuzzy sets Kroll (1996) and consequents by affine functions for the SISO case is given by:

1198

Ri :

IF z IS vi THEN yˆi = fi (x)

(11)

2015 IFAC SYSID October 19-21, 2015. Beijing, China

where: Ri : z: vi : yˆi : fi : x:

Salman Zaidi et al. / IFAC-PapersOnLine 48-28 (2015) 1196–1201

this research is given next, see de Carvalho (2007) and the references therein for more details.

i-th fuzzy rule, antecedent variable, z ∈ Rp , i-th cluster prototype, vi ∈ Rp , crisp output of the i-th rule, yˆi ∈ R, ⊺ affine conclusion function, fi (x) = a⊺i · [1 x⊺ ] , q consequent variable, x ∈ R .

In this classical T-S fuzzy model, the cluster prototypes vi and the consequent parameters ai are defined as crisp numbers. The final crisp output of the model is given by weighted average of the yi in (11) of all c rules

yˆ(x, z) =

c �

µi (z)ˆ yi (x).

Let the symbolic interval-valued data to be clustered be given by z(k) = (z1 (k), . . . , zp (k)), k = 1, . . . , n; where zj (k) = [ajk , bjk ] ∈ ℑ = {[a, b] : a, b ∈ R, a ≤ b}, j = 1, . . . , p. Let each prototype gi of cluster Pi be represented as a vector of intervals, i.e. gi = (g1i , . . . , gpi ), i = 1, . . . , c; where gji = [αji , βij ] ∈ ℑ = {[a, b] : a, b ∈ R, a ≤ b}, j = 1, . . . , p. The IFCM minimizes the adequacy criterion based on the squared Euclidean distances between vectors of intervals as follows

(12) W =

i=1

Where µi (z) is the membership of the scheduling variable z defined by the orthogonal membership function of fuzzy c-means clustering (FCM) [Bezdek (1981)] −1  2 � ν−1 c � � || ||z − v i  , µi (z) =  ||z − v || j j=1

1199

(13)

3.4 Antecedent structure determination - clustering of interval data using IFCM The antecedent variable z(k) is clustered using IFCM which actually contains both the point-valued data and the symbolic interval-valued data. Since a point-valued variable can be seen as a special case of interval-valued data, we will deal with z(k) using the theory of symbolic interval-valued data. The IFCM clustering furnishes the fuzzy partitioning of the space of z(k) and forms interval prototypes. The brief description of the method used in

p � � � (ajk − αji )2 + (bjk − βij )2 . j=1

(14)

As in the standard fuzzy C-Means algorithm, the algorithm starts with the random initialization of either the partition matrix or the cluster prototypes, and then it iterates between the representation and the allocation steps. In the representation step, the clustering prototypes are updated as follows:

i=1

On account of using symbolic interval-valued output in this research for dynamic system identification, x, z, and yˆi all become intervals in (11), which cannot be modeled using the classical T-S fuzzy model since that uses crisp data. In the proposed modeling approach, the interval data is first clustered using the Fuzzy C-Means clustering for the symbolic interval-valued data (IFCM) (de Carvalho (2007)) and then parameters of the local model were estimated using the center and range method suitable for symbolic interval-valued data (Lima Neto and de Carvalho (2008)). Having the NARX model estimated, a non-linear optimization technique was applied to derive an optimal model for the NOE case. The details of these steps are given in the sequel.

µi (z(k))ν

i=1 k=1

and ν is the fuzzy index, ν > 1. Since the membership c � µi (z) = 1 holds. functions in (13) are orthogonal,

In case of NARX nonlinear dynamic systems, z and x are chosen as the vectors of lagged inputs and lagged measured outputs. Taking τ = 0, the consequent variable x can be written as: x(k) = [u(k − 1), . . . , u(k − nuc ), y(k − 1), . . . , y(k − nyc )]⊺ with q = nuc + nyc . In general, the components of the antecedent variable z can differ from x, or this can even be functions of them.

n c � �

αji =

n �

µi (z(k))ν ajk

k=1 n �

µi

(z(k))ν

k=1

and βij =

n �

µi (z(k))ν bjk

k=1 n �

(15) µi (z(k))ν

k=1

In the allocation step, the values of membership function are updated according to:  � �p � 1 −1 c � [(ajk − αji )2 + (bjk − βij )2 ] ν−1 j=1  µi (z(k)) =  �p j j 2 j j 2 [(a − α ) + (b − β ) ] k h k h j=1 h=1 (16)

3.5 Consequent parameters determination - weighted linear regression using interval data The Centre and Range method was used for this purpose [Lima Neto and de Carvalho (2008)]. The method applies the weighted linear regression on mid-points and ranges of the interval valued consequent variable x(k), which can later be used for determining the lower and upper bounds of the output yˆ(k). Let the consequent variable x(k) be written as x(k) = (x1 (k), . . . , xq (k)), where xl (k) = [clk , dlk ] ∈ ℑ = {[c, d] : c, d ∈ R, c ≤ d}, l = 1, . . . , q and k = 1, . . . , n. Furthermore, assume that xcen (k) and xhr (k) represent the centre and the half range values of x(k). The center values cen xcen (k) = (xcen 1 (k), . . . , xq (k)) and the half range values hr hr hr x (k) = (x1 (k), . . . , xq (k)) are calculated as follows: xcen l (k) =

clk + dlk dlk − clk , xhr (k) = l 2 2 l = 1, . . . , q, k = 1, . . . , n

Let the consequent parameters of the i-th rule (the parameters of the i-th local affine model) be represented as ai =

1199

2015 IFAC SYSID 1200 October 19-21, 2015. Beijing, China

Salman Zaidi et al. / IFAC-PapersOnLine 48-28 (2015) 1196–1201

(ai0 , ai1 , . . . , aiq ), where aiw = [γiw , δiw ] ∈ ℑ = {[γ, δ] : γ, δ ∈ R, γ ≤ δ}, with w = 0, 1, . . . , q and i = 1, . . . , c. Moreand ahr over, assume acen i i represent the centre and half = (ai,cen , ai,cen , . . . , ai,cen ), range values of ai ; where acen q i 0 1 i,hr i,hr i,cen w w hr aw = (γi + δi )/2; and ai = (a0 , a1 , . . . , ai,hr q ), w w i,hr aw = (δi − γi )/2; w = 0, 1, . . . , q and i = 1, . . . , c,

X hr := [xhr (1), . . . , xhr (n)]⊺ ∈ Rn×q . Moreover, define cen X′ := [M1 · Xecen, . . . , Mc · Xecen] ∈ Rn×(q+1)·c , X′

hr

:= [M1 · Xehr , . . . , Mc · Xehr ] ∈ Rn×(q+1)·c ,

and the interval output y(k) = [y l (k), y u (k)] ∈ ℑ = {[a, b] : a, b ∈ R, a ≤ b}. Define y cen (k) = (y l (k) + y u (k))/2 and y hr (k) = (y u (k) − y l (k))/2, consequently ycen = [y cen (1), . . . , y cen (n)]⊺ ∈ Rn×1 and yhr = [y hr (1), . . . , y hr (n)]⊺ ∈ Rn×1 . The centre (acen ) and radius (ahr ) consequent parameters are calculated as acen = [(X ′ hr

a

cen ⊺

) · X′

= [(X

cen −1

′ hr ⊺

) ·X

]

(X ′

′ hr −1

]

cen ⊺

(X

) · ycen ,

′ hr ⊺

hr

) ·y .

(17) (18)

3.6 Derivation of the NOE Model from the NARX model

30 Duty cycle in %

The vectors lumping all the centre and radius consequent ⊺ cen ⊺ ⊺ parameters are given by acen = [(acen 1 ) , . . . , (ac ) ] ∈ (q+1)·c×1 cen (q+1)×1 , where ai ∈ R , i = 1, . . . , c; and ahr = R hr ⊺ hr ⊺ ⊺ (q+1)·c×1 (q+1)×1 , where ahr , [(a1 ) , . . . , (ac ) ] ∈ R i ∈ R i = 1, . . . , c. These vectors are estimated globally by using the OLS method (NARX model). Denote Mi ∈ Rn×n , the diagonal matrix having membership grades µi (z(k)) as its k-th diagonal element with 1 ≤ i ≤ c and 1 ≤ k ≤ n. Define the matrices Xecen := [X cen, 1] ∈ Rn×(q+1) , Xehr := [X hr , 1] ∈ Rn×(q+1) , where 1 is a unitary column vector in Rn×1 , X c and X r are the input matrices for the center and radius consequent part, respectively, given as follows: X cen := [xcen (1), . . . , xcen (n)]⊺ ∈ Rn×q ,

40

20 10 0 -10

0

2

4 6 Time in sec.

8

10

Fig. 1. The multisine input signal for the throttle optimized multisine input signal (Ren et al. (2012)), shown in Fig. 1, was chosen as the input signal with the following parameters: sampling time=10 ms, length of the experiment(n)=1000, number of repetitions of the experiment(m)=80. The resulting output time series is shown in Fig. 3 for the case of α = 0.1. The inputs to the consequent variable were selected to be z(k) = [u(k − 1), y(k − 1) − y(k − 2)]⊺ with p = 2, and the inputs to the antecedent variable are to be x(k) = [u(k − 1), y(k − 1), y(k − 2)]⊺ with q = 3. The number of local models and the value of fuzziness parameter were chosen to be c = 8 and ν = 1.3, respectively. The modeling results for the mean, upper and lower output time series are shown in Fig. 2. The assessment criteria used include the MAXimum Absolute Error (MaxAE), Variance Accounted For (VAF), and Root Mean Squared Error (RMSE). The Table 1 shows the modeling performance of this model with the previous modeling approach (Zaidi and Kroll (2014b)). Not only has the framework of modeling been improved by incorporating directly the interval data in modeling, but also the modeling performance has been improved using this approach. Table 1. Comparison of the modeling performance between the Previous (Prev.) and the Current (Curr.) model for the Upper Bound (UB), the Lower Bound (LB) and the Mean time series

In applications like model predictive control or simulation, the NOE model is required (Jelali and Kroll (2002)). The parameters obtained for the NARX case are used as the initial estimates for the NOE model and nonlinear optimization was applied to estimate optimal parameters for the NOE case. The parameters to be optimized were both center and range values of both clustering prototypes and parameters of local models. Denoting the lumped parameter vector for NARX model as θ NARX ∈ R2c(p+q+1)×1 , θNARX := [(gcen )⊺ , (ghr )⊺ , (acen )⊺ , (ahr )⊺ ]⊺ . The optimal set of parameters for the NOE model θ NOE is obtained by minimizing the SSR of NOE model for y = [yl , yu ] as follows n 1 u u θNOE := θ ∗ = arg min (y (k) − yˆNOE (θ, k))2 + n θ

LB

UB

Mean

VAF in % MaxAE in ◦ RMSE in ◦ VAF in % MaxAE in ◦ RMSE in ◦ VAF in % MaxAE in ◦ RMSE in ◦

Identification data Prev. Curr. 99.96 99.9804 1.11 0.8929 0.34 0.2464 99.95 99.9829 2.59 1.2850 0.37 0.2253 99.98 99.9933 1.28 1.0378 0.25 0.1436

Test Prev. 99.63 1.23 0.52 99.69 1.12 0.40 99.91 0.46 0.19

data Curr. 99.8231 0.7778 0.2380 99.7568 0.8621 0.2914 99.9399 0.3528 0.1442

k=1

l (y l (k) − yˆNOE (θ, k))2 (19)

5. CONCLUSION

4. EXPERIMENTAL RESULTS The developed method was demonstrated on the electromechanical throttle (Zaidi and Kroll (2014a)). A phase

An approach to build models that can provide the expected response of an uncertain nonlinear dynamic system along with the spread around it has been presented and

1200

Angular position in ◦

2015 IFAC SYSID October 19-21, 2015. Beijing, China

Salman Zaidi et al. / IFAC-PapersOnLine 48-28 (2015) 1196–1201

Reference time series Model time series

90 80 70 60

Test data

Identification data

50 40 30 20 10 0

1

2

3

4 5 6 Time in sec.

7

8

9

10

Angular position in ◦

Fig. 2. The reference (blue) and model (red) time series for identification (first 9 sec.) and test data (10-th sec.)

100 90 80 70 60 50 40 30 20 10 0

Measured output time series Upper boundary Lower boundary Sample wise expected values

0

1

2

3

4 5 6 Time in sec.

7

8

9

10

Fig. 3. The output time series with extended Chebyshevs Inequality based CIs and mean curves (80 experiments) demonstrated for an electro-mechanical throttle. The approach uses the extended Chebyshev’s Inequality to obtain confidence interval based upper and lower bounds of the output time series. These bounds were then directly used in the T-S fuzzy modeling using the theory of symbolic interval-valued data. The mean response was calculated by averaging out the upper and lower response of the model. The results are compared with an alternative approach where two separate models are used for modeling. The results show that the presented approach provide interval parameters with more accuracy and less modeling efforts in a unified modeling framework. ACKNOWLEDGEMENTS The authors would like to thank the financial grant from German Academic Exchange Service (DAAD). REFERENCES Babuˇska, R. (1998). Fuzzy Modeling for Control. Kluwer Academic Publishers, Boston. Bezdek, J.C. (1981). Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum Press.

1201

Bock, H.H. and Diday, E. (2000). Analysis of Symbolic Data. In: Exploratory Methods for Extracting Statistical Information from Complex Data. Springer. Chebyshev, P.L. (1867). Des valeurs moyennes. J. Math. Pures Appl., 12(2), 177–184. de Carvalho, F.d.A.T. (2007). Fuzzy c-means clustering methods for symbolic interval data. Pattern Recognition Letters, 28(4), 423–437. Jelali, M. and Kroll, A. (2002). Hydraulic Servo-systems: Modelling, Identification and Control. Springer. Kab´an, A. (2012). Non-parametric detection of meaningless distances in high dimensional data. Statistics and Computing, 22(2), 375–385. Khosravi, A., Nahavandi, S., Creighton, D., and Naghavizadeh, R. (2012). Prediction interval construction using interval type-2 fuzzy logic systems. In Procedings of the 2012 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), 1–7. Kroll, A. (1996). Identification of functional fuzzy models using multidimensional reference fuzzy sets. Fuzzy Sets and Systems, 80(2), 149–158. Lima Neto, E.d.A. and de Carvalho, F.d.A. (2008). Centre and range method for fitting a linear regression model to symbolic interval data. Computational Statistics & Data Analysis, 52(3), 1500–1515. Liu, Z. and Li, H.X. (2005). A probabilistic fuzzy logic system for modeling and control. IEEE Trans. on Fuzzy Systems, 13(6), 848–859. Ljung, L. (1999). System Identification-Theory for the User. Prentice-Hall, 2nd edition. Mendel, J.M. (2001). Uncertain rule-based fuzzy logic systems, introduction and new directions. Prentice Hall. Nelles, O. (2001). Nonlinear System Identification. Springer, Berlin. Ren, Z., Kroll, A., Sofsky, M., and Laubenstein, F. (2012). On identification of piecewise-affine models for systems with friction and its application to electro-mechanical throttles. In Proc. of the 16th IFAC Symposium on System Identification, 1395–1400. ˇ Skrjanc, I., Blaˇziˇc, S., and Agamennoni, O. (2005). Identification of dynamical systems with a robust interval fuzzy model. Automatica, 41(2), 327–332. Xu, Z. and Sun, C. (2009). Interval T-S fuzzy model and its application to identification of nonlinear interval dynamic system based on interval data. In Proc. of the 48th IEEE Conference on Decision and Control/Chinese Control Conference, 4144–4149. Ying, H. (1998). General SISO Takagi-Sugeno fuzzy systems with linear rule consequent are universal approximators. IEEE Trans. on Fuzzy Systems, 6(4), 582–587. Zaidi, S. and Kroll, A. (2013). On identifying nonlinear envelop type dynamical T-S fuzzy models for systems with uncertainties: method and application to electromechanical throttle. In 23. Workshop Computational Intelligence, 129–143. Dortmund, Germany. Zaidi, S. and Kroll, A. (2014a). Electro-mechanical throttle as a benchmark problem for nonlinear system identification with friction. In 24. Workshop Computational Intelligence, 173–186. Dortmund, Germany. Zaidi, S. and Kroll, A. (2014b). On identifying envelop type nonlinear output error takagi-sugeno fuzzy models for dynamic systems with uncertainties. In 19th IFAC World Congress, 3226–3231. Cape Town, South Africa.

1201