An Error-Entropy Minimization Algorithm for Tracking Control of Nonlinear Stochastic Systems with Non-Gaussian Variables*

Proceedings of the the 20th World World Congress The International Federation of Congress Automatic Control Proceedings of 20th The International International Federation of Congress Automatic Control Control Proceedings of the 20th9-14, World Toulouse, France, July 2017 The Federation of Automatic Available online at www.sciencedirect.com Toulouse, France,Federation July 9-14, 9-14, 2017 2017 The International of Automatic Control Toulouse, France, July Toulouse, France, July 9-14, 2017

ScienceDirect

IFAC PapersOnLine 50-1 (2017) 10407–10412 An Error-Entropy Minimization An Error-Entropy Minimization Algorithm Algorithm An Error-Entropy Minimization Algorithm An Error-Entropy Minimization Algorithm for Tracking Control of Nonlinear for Tracking Tracking Control Control of of Nonlinear Nonlinear for for Tracking Control Nonlinear Stochastic Systems with Stochastic Systems with⋆ofNon-Gaussian Non-Gaussian Stochastic Systems with Non-Gaussian Stochastic Systems with⋆⋆ Non-Gaussian Variables Variables Variables ⋆ Variables ∗ ∗∗ ∗∗∗ ∗∗∗∗

Yunlong Liu ∗ Aiping Wang ∗∗ Lei Guo ∗∗∗ Hong Wang ∗∗∗∗ Yunlong Liu Liu∗ Aiping Aiping Wang Wang∗∗ Lei Lei Guo Guo∗∗∗ Hong Hong Wang Wang∗∗∗∗ Yunlong ∗ ∗∗ ∗∗∗ Yunlong Liu Aiping Wang Lei Guo Hong Wang ∗∗∗∗ ∗ ∗ State Key Laboratory of Synthetical Automation for Process ∗ State Key Laboratory of Synthetical Automation for Process Key Laboratory of Synthetical Automation Process Industries, University, Shenyang 110819,for China (e-mail: ∗ State Northeastern State Northeastern Key Laboratory of Synthetical Automation Process Industries, Northeastern University, Shenyang 110819,for China (e-mail: Industries, University, Shenyang 110819, China (e-mail: [email protected]) Industries, Northeastern University, Shenyang 110819, China (e-mail: [email protected]) ∗∗ [email protected]) Sciences, Anhui University, Anhui, China ∗∗ Institute of Computer [email protected]) ∗∗ Institute of Computer Sciences, Anhui University, Anhui, China ∗∗∗ Institute of Computer Sciences, Anhui University, Anhui, China ∗∗ Science and Technology on Aircraft Control Laboratory, Beihang ∗∗∗ of Computer Sciences, Anhui University, Anhui,Beihang China ∗∗∗ Institute Science and and Technology on Aircraft Aircraft Control Laboratory, Beihang Science Technology on Control Laboratory, Beijing 100191, China (e-mail:[email protected]) ∗∗∗University, Science and Technology on Aircraft Control Laboratory, Beihang University, Beijing 100191, China (e-mail:[email protected]) ∗∗∗∗ University, Beijing 100191, China (e-mail:[email protected]) National Laboratory, 902 Battelle Blvd ∗∗∗∗ Pacific Northwest University, Beijing 100191, (e-mail:[email protected]) ∗∗∗∗ Pacific WA Northwest National Laboratory, 902 Battelle Battelle Blvd Blvd Pacific Northwest National Laboratory, 902 Richland, 99352 USA China (e-mail: [email protected]) ∗∗∗∗ Pacific WA Northwest Laboratory, 902 Battelle Blvd Richland, WA 99352 National USA (e-mail: (e-mail: [email protected]) Richland, 99352 USA [email protected]) Richland, WA 99352 USA (e-mail: [email protected]) Abstract: This paper presents an error-entropy minimization tracking control algorithm for a Abstract: This paper paper presents presents an error-entropy error-entropy minimization tracking control algorithm for aa Abstract: This an tracking algorithm for class of dynamic stochastic systems. The system isminimization represented by a set ofcontrol time-varying discrete Abstract: This paper presents an error-entropy minimization tracking control algorithm forofa class of dynamic stochastic systems. The system is represented by a set of time-varying discrete class of dynamic stochastic systems. The system is represented by athe set statistical of time-varying discrete nonlinear equations with non-Gaussian stochastic input, where properties class of dynamic stochastic systems. The system is represented by a set of time-varying discrete nonlinear equations with non-Gaussian stochastic input, where the statistical properties of nonlinear equations with non-Gaussian input, where the statistical of stochastic input are unknown. By using stochastic Parzen windowing with Gaussian kernelproperties to estimate nonlinear equations with non-Gaussian stochastic input, where the statistical properties of stochastic input are unknown. By using Parzen windowing with Gaussian kernel to estimate stochastic input are unknown. By using Parzen windowing with Gaussian kernel to estimate the probability densities of errors, recursive algorithms are then proposed to design the stochastic input are unknown. By using Parzen windowing with Gaussian kernel to estimate the probability densities of errors, recursive algorithms are then proposed to design the the probability densities of errors, algorithmsThe areperformance then proposed design the controller such that the tracking errorrecursive can be minimized. of thetoerror-entropy the probability densities of errors, recursive algorithms are thenin proposed design the controller suchisthat that the tracking tracking error can be be minimized. minimized. The performance of the theto error-entropy controller such the error can The performance of error-entropy minimization compared with the mean-square-error minimization the simulation results. controller such that the tracking error can be minimized. The performance of the error-entropy minimization is compared with the mean-square-error minimization in the simulation results. minimization is compared with the mean-square-error minimization in the simulation results. minimization is comparedFederation with the of mean-square-error in the simulation © 2017, IFAC (International Automatic Control)minimization Hosting by Elsevier Ltd. All rights results. reserved. Keywords: Minimum error entropy, information potential, non-Gaussian variables, probability Keywords: Minimum error entropy, information potential, non-Gaussian variables, probability Keywords: Minimum error entropy, density function, stochastic systems.information potential, non-Gaussian variables, probability Keywords: Minimum error entropy, density function, function, stochastic systems.information potential, non-Gaussian variables, probability density stochastic systems. density function, stochastic systems. 1. INTRODUCTION as close as possible to a required distribution shape (see Yi 1. INTRODUCTION INTRODUCTION as al. close as possible possible to[2009]) required distribution shape (see Yi Yi 1. as close as to aa required distribution shape (see et [2007], Yi et al. or to minimize the entropy of 1. INTRODUCTION as close as possible to a required distribution shape (see Yi et al. [2007], Yi et al. [2009]) or to minimize the entropy of et al. [2007], Yirandom et al. [2009]) or (see to minimize entropy of variable Guo andthe Wang [2006], Stochastic control is an active part of control theory which corresponding et al. [2007], Yi et al. [2009]) or to minimize the entropy of corresponding random variable (see Guo and Wang [2006], Stochastic control is an active part of control theory which corresponding random variable (see Guo and Wang [2006], and Yin [2009], Liu et al. [2015]). Unfortunately, Stochastic is an active part of control theory which deals withcontrol data that are polluted by stochastic noise Guo corresponding random variable (see Guo and Wang [2006], Guo and Yin [2009], Liu et al. [2015]). Unfortunately, Stochastic control is an active part of control theory which dealsthe with data that are are polluted pollutedisby byto stochastic stochastic noise most Guo of and [2009], Liu focus et al.on[2015]). Unfortunately, theYin existing studies stochastic distribution deals with that noise and aimdata of stochastic control design optimal Guo and Liu focus et al.on [2015]). Unfortunately, most of of theYin existing studies focus on stochastic distribution deals with data that arethe polluted noise and the the aim ofperforms stochastic control isbyto to stochastic design optimal most the existing studies stochastic distribution based on[2009], PDF functional operator mapping model and aim of stochastic control is design optimal controller that desired control task with control most of the existing studies focus on stochastic distribution control based on PDF functional operator mapping model and the aim of stochastic control is to design optimal controller that performs the desired control task with control based PDF functionalofoperator mapping model needs theon priori knowledge stochastic input, such as controller that despite performs desired task with that minimum cost the the existence of control these disturbance. control based on PDF functional operator mapping model that needs the priori knowledge of stochastic input, such as controller that performs the desired control task with minimum cost despite the existence of these disturbance. that needs the priori knowledge of stochastic input, such as PDF, which is a strong assumption and cannot always minimum cost the existence of these disturbance. Along with thedespite development of Pontryagin’s maximum the that needswhich the priori knowledge of stochastic input, such as thesatisfied PDF, which is aa strong strong assumption and cannot cannot always minimum cost despite the existence of these disturbance. Along with the development of Pontryagin’s maximum the PDF, is assumption and always be in practice. Along with the development of Pontryagin’s maximum principle (MP), Bellman’s dynamic programming (DP) the PDF, which is a strong assumption and cannot always be satisfied in practice. Along with the development of Pontryagin’s maximum principle (MP), Bellman’s dynamic programming (DP) principle (MP),linear-quadratic Bellman’s dynamic (DP) be satisfied in practice. and Kalman’s (LQ) programming control, stochastic Mean in square error (MSE), which only concentrates be satisfied practice. principle (MP),linear-quadratic Bellman’s dynamic programming (DP) Since and Kalman’s Kalman’s linear-quadratic (LQ) control, since stochastic and (LQ) control, stochastic optimal control theory has been well developed early Since Meanorder square error (MSE), (MSE), which only concentrates concentrates Since Mean square error only on second statistics, is ablewhich to extract all possible and Kalman’s linear-quadratic (LQ) control, stochastic optimal control theory has been well developed since early optimal control theory hason been well developed since early on Since Mean square error (MSE), which only 1960s which mainly focus Gaussian stochastic systems on second order statistics, is able to extract all possible second order able to data extract all possible information fromstatistics, a giving istraining setconcentrates under the optimal control theory has been well developed since early 1960s which mainly focus on Gaussian stochastic systems 1960s mainly focusthat on Gaussian stochastic systems information on second order statistics, is able to extract all possible under which the assumption the noises obey Gaussian information from a giving training data set under the a givingassumptions, training data the linearity andfrom Gaussianity it set has under been well 1960s which mainly focus on Gaussian stochastic systems under the assumption that the noises obey Gaussian under the assumption the noises obey Gaussian linearity information from a giving training data set under the distribution (see Athans that [1971]). linearity and Gaussianity assumptions, it has been well and Gaussianity assumptions, it has been well in the training of adaptive systems including under the assumption that the noises obey Gaussian employed distribution (see Athans Athans [1971]). [1971]). distribution (see linearity and Gaussianity assumptions, it has been well employed in the training of adaptive systems including employed in and the artificial training neural of adaptive systems filters networks due toincluding the fact However, most real-life distribution (see Athanssystems [1971]).are governed by nonlinear linear employed inand the artificial training of adaptive systems linear filters and artificial neural networks due to toincluding the fact However, most real-life systemsnoises are governed governed by nonlinear linear filters neural networks due fact Wiener [1949] established the perspective ofthe adapHowever, most real-life systems are nonlinear models and most stochastic are far by from being that linear filters and artificial neural networks due to the fact that Wiener [1949] established the perspective of adapHowever, most real-life systems are governed by nonlinear models and most stochastic noises are far from being that filters Wiener [1949] established perspective However, of adapas statistical function the approximation. models most stochastic noises are far distribution from being tive Gaussianand distributed. Therefore, stochastic that Wiener [1949] established the perspective of adaptive filters as statistical function approximation. However, models and most stochastic noises are far from being Gaussian distributed. Therefore, stochastic distribution tive filters as statistical function approximation. However, densities take complex forms in many applications. Gaussian distributed. Therefore, stochastic control (see Wang [2000], Yue and Wang distribution [2003] and data tive as statistical function approximation. However, datafilters densities take complex complex forms in many manydoes applications. Gaussian distributed. Therefore, stochastic control (see (see Wangfor[2000], [2000], Yue and and Wang distribution [2003] and and data densities take forms in applications. the probability distribution involved not obey control Wang Yue Wang [2003] references therein) non-Gaussian stochastic system has When data densities take complex forms in many applications. When the probability distribution involved does notinforobey control (see Wang [2000], Yue and Wang [2003] and references therein) for non-Gaussian stochastic system has When thedistribution, probability distribution not obey MSE fails toinvolved capturedoes all the references therein) for non-Gaussian stochastic system been studied extensively in recent years in response to has the Gaussian When the probability distribution involved does not obey Gaussian distribution, MSE fails to capture all the inforreferences therein) for non-Gaussian stochastic system has been studied extensively in recent years in response to the Gaussian MSE fails towhich capture the inforin distribution, the data. Since entropy, wasallintroduced been studied extensively of in recent in response the mation increased requirements many years practical systems.toThe Gaussian distribution, MSE fails to capture all the information in the data. Since entropy, which was introduced been studied extensively in recent years in response to the increased requirements of many practical systems. The mation in the data. Since entropy, which was introduced Shannon [1949], is a scalar quantity that provides a increased requirements of manydistribution practical systems. primary purpose of stochastic control isThe to by mation inof the data. Since entropy, which was provides introduced by Shannon Shannon [1949], is aainformation scalar quantity that provides increased requirements of many practical systems. primarycontrol purpose of stochastic stochastic distribution control isThe to measure by [1949], is scalar quantity that the average contained in randomaa primary purpose of control is to design input such thatdistribution the probability density by Shannon [1949], is a scalar quantity that provides measure of the average information contained in randoma primary purpose of stochastic distribution control is to design control input such that the probability density measure with of thea average contained infunction, random certain information probability distribution design such that random the probability functioncontrol (PDF) input of corresponding variable isdensity made variable measure of the average information contained in random variable with a certain probability distribution function, design control input such that the probability density function (PDF) of corresponding random variable is made variable with a certain probability distribution function, as an function (PDF) of corresponding random variable is made minimum error entropy (MEE) is superior to MSE variable with a certain probability distribution function, minimum error entropy (MEE) is superior to MSE asthe an (PDF) of corresponding random variable isScience made optimality ⋆function minimum error entropy (MEE) is superior to MSE as an criterion due to the fact that minimizing This work was supported in part by the National Natural ⋆ This work was supported in part by the National Natural Science minimum error entropy (MEE) is superior to MSE as an optimality criterion due to the fact that minimizing the ⋆ optimality criterionalldue to the of fact minimizing the Foundation China underinGrant 61333007, 61621004, 61573022, This work of was supported part by the National Natural Science entropy constrains moments thethat PDF. In Erdogmus ⋆ Foundation of China under Grant 61333007, 61621004, 61573022, optimality criterion due to the fact that minimizing the entropy constrains all moments of the PDF. In Erdogmus This work was supported in part by the National Natural Science the Chinese of National Foundation under Grants Foundation China Post-doctor under GrantScience 61333007, 61621004, 61573022, entropy constrains moments of the PDF. In Erdogmus and Principe [2002],allthe MEE criteria has been employed the Chinese Chinese National National Post-doctor Science Foundation under Grants Foundation of China under Grant 61333007, 61621004, 61573022, entropy constrains moments ofitthe PDF. In Erdogmus and Principe [2002],all the MEEand criteria has been employed 2015M571322 and CCSI of the Pacific Northwest National Laborathe Post-doctor Science Foundation under Grants and Principe [2002], the MEE criteria has been employed in adaptive systems training, has been proved that 2015M571322 and CCSI CCSI of the the Pacific Pacific Northwest National Laborathe Chinese National Post-doctor Science Foundation underLaboraGrants and Principe [2002],training, the MEEand criteria been employed in adaptive adaptive systems training, and it has hashas been proved that tory. 2015M571322 and of Northwest National in systems it been proved that tory. 2015M571322 and CCSI of the Pacific Northwest National Laborain adaptive systems training, and it has been proved that tory.

tory. Copyright © 2017 IFAC 10894 2405-8963 © 2017, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Copyright © 2017 2017 IFAC IFAC 10894 Copyright © 10894 Peer review under responsibility of International Federation of Automatic Control. Copyright © 2017 IFAC 10894 10.1016/j.ifacol.2017.08.1720

Proceedings of the 20th IFAC World Congress 10408 Yunlong Liu et al. / IFAC PapersOnLine 50-1 (2017) 10407–10412 Toulouse, France, July 9-14, 2017

minimizing the error entropy is equivalent to minimizing the distance between the joint densities of system inputoutput and the desired input-output pairs. Besides, the problem of optimal state estimation for stochastic systems has been considered from the view of information theory in Feng et al. [1997]. Also, in Xu et al. [2005], an adaptive Luenberger observer based on MEE has been designed to deal with the problem of nonlinear state estimation.

∥f (xk + δ) − f (xk ) − Ak δ∥ ≤ a1 ∥δ∥ (2) n×n n where Ak ∈ R is a known constant matrix, δ ∈ R is a vector and a1 is a known positive constant.

In this paper, an error entropy minimization algorithm is investigated in stochastic distribution control for a class of non-Gaussian stochastic systems with no priori knowledge of stochastic input. Since the error entropy minimization algorithm we proposed relied on the use of quadratic Renyi’s entropy and the analytical error density distributions are not available by PDF functional operator mapping model, nonparametric estimation of the PDF of a random variable is required for the evaluation of its entropy. Parzen windowing (also called kernel density estimator) is an typical density estimation scheme, where the PDF is approximated by a sum of kernels whose centers are translated to the sample locations. A commonly used kernel function is the Gaussian, since it is continuously differentiable and it leads to continuously differentiable density estimates, which can provide a computational simplification in the gradient-based algorithm design (see Principle et al. [2000]).

The purpose of this paper is to construct the input signal uk such that the system state xk can track that of the ideal deterministic model (see Athans [1971]) with certain accuracy, following

The organization of this paper is as follows. The tracking control problem is formulated in Section 2 for a class of nonlinear stochastic systems with no priori knowledge of stochastic input. The main results and detailed derivations are given in Section 3. A numerical simulation is given in Section 4 and the performances between MSE and MEE are compared. Finally, some conclusions are given in Section 5. 2. PRELIMINARIES AND PROBLEM FORMULATION Consider the following discrete time nonlinear stochastic system: xk+1 = f (xk ) + Bk uk + Hk ωk , xk |k=0 = x0

yk = Ck xk + vk (1) where xk ∈ Rn , yk ∈ R and uk ∈ R are the state, measured output and control input, respectively. f (·) is a vector-valued nonlinear function. x0 ∈ Rn is the initial condition. ωk ∈ Rs , vk ∈ R are the additive system noise and measurement noise respectively, which are of arbitrary distribution form rather than Gaussian distribution. Bk , Hk , Ck are known parameter matrices with appropriate dimensions. The following assumptions, which are quite general, are required to simplify the controller design procedure. Assumption 1. {ωk }, {vk } (k = 0, 1, · · ·) are bounded, stationary and mutually independent with no priori knowledge of statistical property. Assumption 2. The initial state x0 is independent of ωk , vk . Assumption 3. f (·) is a known Borel measurable and smooth function of its arguments, and assumed to satisfy f (0) = 0 and

Remark 1. The nonlinear description (2), which has been adopted in Yaz and Azemi [1993], quantify the maximum possible derivations from a linear model with Ak as its system parameter matrix.

xr(k+1) = f (xrk ) + Bk rk , (3) yrk = Ck xrk where xrk ∈ R , rk ∈ R and yrk ∈ R are the ideal deterministic state, bounded input, and output, respectively. n

Define the tracking error as ξk = xk − xrk , it can be seen that ξk obeys non-Gaussian distribution since the nonlinearity of system and the non-Gaussianity of random noises. Hence, quadratic Renyi’s entropy, which is given in the following equation, is employed beyond MSE (secondorder statistics) to quantify the statistical property for convenience of calculation. ∫ H2 (X) = − ln p2X (x)dx (4) where pX (x) is the PDF of random variable X.

Since the statistical properties of the stochastic inputs, such as PDFs, are unavailable in this paper, neither the entropy nor the PDF of the tracking error can be calculated by the functional operator mapping model (see Guo and Wang [2006], Guo and Yin [2009], Liu et al. [2015]). Fortunately, Parzen windowing is an efficient way to approximate the PDF of a given sample distribution (see Devroye and Lugosi [2001]), especially in low-dimensional spaces, and it does not need prior knowledge of the system apart from the samples. For a given set of i.i.d samples z1 , · · · , zn drawn from q(z), the Parzen windowing estimate for the distribution, assuming a fixed-size kernel function Kσ (·) for simplicity, is given by n

p(z) =

1∑ Kσ (z − zi ) n i=1

(5)

where σ is the window width, and can be optimized in accordance with the least-square cross-validation, likelihood cross-validation, the test graph method or other rules-ofthumb method (see Devroye and Lugosi [2001], Principe and Erdogmus [2000]). And Gaussian function, which is given in the following equation, is chosen as kernel function in this paper as it is continuously differentiable and can simplify the computation in the algorithm design (see Principe et al. [2000]). (x−µ)T |Σ|−1 (x−µ) 1 − 2 Kσ (x) = G(x − µ, Σ) = √ (6) 1 e n ( 2π) |Σ| 2 where µ and Σ are the mean and covariance matrix, respectively. Since entropy is invariant to the mean of the sample data, µ is chosen as 0 and Σ is selected as

10895

Proceedings of the 20th IFAC World Congress Yunlong Liu et al. / IFAC PapersOnLine 50-1 (2017) 10407–10412 Toulouse, France, July 9-14, 2017

σ 2 I for simplicity in the following formulation. Hence, by substituting (6) into equation (5), we have 1 N

p˜k+1 (ξ) =

k+1 ∑

i=k+1−N

G(ξ − ξi , σ 2 ).

∫∞

p˜2k+1 (ξ)dξ = E[˜ pk+1 (ξ)]

−∞

˜ k+1 (ξ) = − ln( H

∫∞

−∞

p˜2k+1 (ξ)dξ) = − ln(V˜k+1 (ξ))

In the following section, the state observer will be constructed first. Then the value of xk+1 is predicted so as to calculate Vk+1 (e). Finally, the observer-based controller will be designed so that the tracking error can be minimized in some sense of probability.

(8)

3. MAIN RESULTS

(9)

In this section, the states will be estimated at first. For system (1), the full-order observer that is of the following structure is employed.

where p˜k+1 (ξ) is the PDF estimate of sequence {ξi } (i = ˜ k+1 (ξ) is the corresponding quadratk+1−N, · · · , k+1), H ic Renyi’s entropy and V˜k+1 (ξ) is the information potential defined as the argument of ln function in Renyi’s entropy. And from eq. (8), it can be seen that the information potential is actually the expectation of PDF. In this context, minimizing the entropy is equivalent to maximizing the information potential since ln is a monotonous function. Hence, information potential can be used to characterize the statistical property of random variable instead of entropy to simplify the calculation. When the estimated information potential in (8) is applied directly, the algorithm suffers from o((N )2 ) computational complexity since this is a batch method which needs all the data. Dropping the expectation in eq.(8) and stochastically approximating the value of this operation with the instantaneous value of its argument (see Erdogmus [2002]), we obtain the stochastic estimate for information potential as shown in the following equation. 1 Vˆk+1 (ξ) = N

k ∑

i=k+1−N

G(ξk+1 − ξi , σ 2 )

1 N

i=k+1−N

ˆk yˆk = Ck x

G(e′k+1 − ei , σ 2 )

(11)

(12)

n

where x ˆk ∈ R is the state estimate, yˆk ∈ R is the output estimate, and K is the observer gain to be designed. The observer gain K will be designed such that the error between the measurement and estimated output ηk = yk − yˆk is minimized in some sense of probability since the value of xk , i.e. the value of εk = xk − x ˆk is unavailable. In this case, the stochastic quadratic information potential that is estimated from {ηi }(i = k − N1 , · · · , k) is given by k−1

1 ∑ G(ηk − ηi , σ 2 ) ≤ Vk0 (0) N1 i=k−N1 ∑ k−1 1 where Vk0 (0) = N1 i=k−N1 G(0, σ 2 ). Vk0 (η) =

(13)

The cost function used to update the observer gain K is chosen as 1 1 R1k (Vk0 (0) − Vk0 (η))2 + K T R2k K (14) 2 2 T > 0 and R2k = R2k ≥ 0 are weighting matrices.

J1k (K) = where R1k

It is important to note, however, that the true value of state vector cannot be measured directly. Hence, we should estimate the value of {xi }, (i = 1, · · · , k) and predict the value of xk+1 at sample time k based on the input and output data u1 , · · · , uk−1 , y1 , · · · , yk , then replacing the true value with the estimate value and predicted value in eq.(10). That is, eq.(10) should be rewritten in the following form so that the information potential can be used as a criterion for evaluating the randomness of tracking error sequence. Vk+1 (e) =

xk−1 ) + Bk−1 uk−1 + K(yk−1 − yˆk−1 ), x ˆk = f (ˆ

(10)

Remark 2. The length N of the sliding window should be selected in consideration with the length of the duration where the samples can be assumed i.i.d in theory. It is paperworthy that the independence and identicalness of the assumption about sample data cannot be guaranteed in general, but all simulations proved that this violation of the assumption does not cause any problems in practice as the nonparametric estimator itself starts behaving as a suitable finite-sample case cost function in all applications.

k ∑

ˆi − xri , e′k+1 = x ˆ′k+1 − xr(k+1) , and x ˆi is where ei = x ˆ′k+1 is the one step the estimate of xi (i = 1, · · · , k), x prediction of xk+1 .

(7)

Moreover, it can be seen that V˜k+1 (ξ) =

10409

From (14), it can be seen that the positive definiteness of the cost function can be satisfied. Since J1k (K) is a nonlinear function with respect to K, the following damped Newton iteration algorithm will be applied to obtain the update rule of observer gain K at every sample time k. ∆Kk = −(R1k Ξ2k + R2k )−1 (R1k Ξ1k + R2k Kk−1 ), Kk = Kk−1 + λ1k ∆Kk

(15)

where 1 Ξk = (Vk0 (0) − Vk0 (η))2 , 2 ∂Ξk ∂ 2 Ξk |K=Kk−1 , Ξ2k = |K=Kk−1 Ξ1k = ∂K ∂K∂K T and λ1k is the step size. Since eq. (15) only results from a necessary condition for optimization, the following inequality of second-order derivative of J1k (K) with respect to K should be satisfied in order to guarantee the sufficiency.

10896

Proceedings of the 20th IFAC World Congress 10410 Yunlong Liu et al. / IFAC PapersOnLine 50-1 (2017) 10407–10412 Toulouse, France, July 9-14, 2017

∂ 2 J1k (K) = R1k Ξ2k + R2k > 0 ∂K∂K T It can be seen that based on the input and output data up to time step k, only the value of xk can be estimated by the observer (12). Hence, we need to predict the value of xk+1 . In this paper, the following one-step-ahead prediction is adopted. x ˆ′k+1 ′ yˆk+1

= f (ˆ xk ) + Bk uk + Kk (yk − yˆk ),

= Ck+1 x ˆ′k+1 .

update the gains in this paper. However, because the information potential is derived from entropy and has the same properties of entropy, we still call this criteria as MEE. For the analysis of error system, the error εk and ek dynamics can be written in a compact form as follows: ¯ k−1 ϑk−1 zk = A¯k−1 zk−1 + Mk−1 + H where ] ] [ m1(k−1) εk , Mk−1 = , zk = ek m2(k−1) ] ] [ [ ¯ k−1 = Hk−1 −Kk , ϑk−1 = ωk−1 , H 0 Kk υk−1 ] [ 0 Ak−1 − Kk Ck−1 ¯ Ak−1 = , Kk Ck−1 Ak−1 + Bk−1 Lk−1 [

(16)

When the observer-based control law xk − xrk ), uk = uek + rk (17) uek = L(ˆ is applied, we obtain the tracking error dynamics for the closed loop system as follows:

xk−1 ) − A(xk−1 − x ˆk−1 ), m1(k−1) = f (xk−1 ) − f (ˆ

ek = f (ˆ xk−1 ) − f (xr(k−1) ) + Bk−1 L(ˆ xk−1 − xrk−1 ) +Kk (yk−1 − yˆk−1 ),

(18)

xk ) − f (xrk ) + Bk L(ˆ xk − xrk ) e′k+1 = f (ˆ

+Kk (yk − yˆk ) where L is the controller gain need to be designed.

1 1 R3k (Vk+1 (0) − Vk+1 (e))2 + LR4k LT (20) 2 2 T > 0, R4k = R4k ≥ 0 are weighting matrices.

J2k+1 (L) =

Then the update rule of controller gain L can be obtained by applying damped Newton iteration algorithm to (20). ∆Lk = −(R3k Π1k+1 +

R4k LTk−1 )T (R3k Π2k+1

Lk = Lk−1 + λ2k ∆Lk where

+ R4k )

−1

,

(21)

1 Πk+1 = (Vk+1 (0) − Vk+1 (e))2 , 2 ∂Πk+1 ∂ 2 Πk+1 |L=Lk−1 | , Π = Π1k+1 = L=L 2 k−1 k+1 ∂LT ∂LT ∂L

|Mk−1 | ≤ a1 |zk−1 |.

(24)

Theorem 1. Under Assumptions 1-3, if there exist a positive constant 0 < ρ < 1, such that the gain Kk , Lk satisfy ||A¯k + a1 || = ρ < 1, then the stability of the error system ¯ k ϑk || ≤ (1−ρ)2 δ 2 (23) can be guaranteed. Besides, if E{||H is satisfied for a constant δ > 0 , then ∀E{||z0 ||2 } ≤ δ 2 , the solution of system (23) is bounded in the mean square sense, where ρ is called as a convergence exponent and 1 − ρ is regarded as a noise damping exponent. Proof. The proof procedure can be referred to Liu et al. [2015]. 4. SIMULATIONS In this section, a numerical example is presented to illustrate the usefulness and flexibility of the method developed in this paper. Moreover, in order to illustrate the performance, we compare the results based on MEE with the results based on MSE. Let the system model be given by

λ2k is the step size, and to guarantee the sufficiency of optimization, the weight matrixes R3k , R4k shoud satisfy R3k Π2k+1 + R4k > 0.

xk−1 ) − f (xr(k−1) ) − A(ˆ xk−1 − xr(k−1) ). m2(k−1) = f (ˆ From Assumption 3, it can be obtained that

(19)

Substituting (18), (19) into (11), the stochastic quadratic information potential of tracking error can be calculated. Like (14), the cost function used to update the controller gain is chosen as

where R3k

(23)

(22)

Remark 3. In fact, the iteration algorithm of observer gain (15) and controller gain (21) is a hybrid algorithm combined Newton algorithm and steepest descent algorithm. When λ1k = 1 and λ2k = 1, the algorithm is equal to the algorithm that obtained in Guo and Wang [2006]. Moreover, compared with the method adopted in Guo and Wang [2006], the method has features with quick convergence rate and large convergence range. Remark 4. Actually, we use the stochastic estimate of information potential instead of entropy as criteria to

xk+1 = 0.2(1 − x2k ) sin(xk ) + 0.5uk + ωk ,

yk = xk + vk and the reference model state is xrk = sin(k/3).

(25)

The instantaneous MSE cost functions can be chosen as 1 1 (26) J¯1k (K) = R1k ηkT ηk + K T R2k K 2 2 1 1 T ′ (27) J¯2k+1 (K) = R3k e′T k+1 ek+1 + LR4k L 2 2 Then, according to (15) and (21), the observer gain and controller gain can be calculated at every time step k. First, we assumed that ωk , vk are zero-mean Gaussian noise. Figure 1 (a), (b) present the response curves of state xk and estimated state x ˆk . Figure 2 (a), (b) depict the trajectories of ideal model state xrk and dynamical state

10897

Proceedings of the 20th IFAC World Congress Yunlong Liu et al. / IFAC PapersOnLine 50-1 (2017) 10407–10412 Toulouse, France, July 9-14, 2017

xk . Figure 1 (c), (d) and Figure 2 (c), (d) show the estimate error and tracking error of the corresponding PDF and Figure 3 and Figure 4 show the 3D plot of the estimate error PDF and tracking error PDF using MEE with the increase of iterative step.

10411

3D mesh plot of tracking error (MEE)

5 4 3

state and its estimate (MSE)

4

state and its estimate (MEE)

2

2

1

2

1

0 0

−1

−2

−2

0 1

−3 −4 −6

x(k)(M SE) x ˆ(k)(M SE) 0

20

40

60

80

100

−5

0

20

40

60

80

−1

100

0

(b)

(a) estimation error

1

0

x(k)(M EE) x ˆ(k)(M EE)

−4

Fig. 4. 3D mesh plot of the tracking error with Gaussian noise

4

−1

Time

PDF of estimation error

5

0

40

20

100

80

60

3

−2 2

−3

1

−4 −5

M EE M SE 0

20

40

60

80

M EE M SE 100

0 −2

−1

0 (d)

(c)

1

2

Fig. 1. The state, estimate, estimate error and its PDF with Gaussian noise state and ideal model state (MSE)

4

Secondly, we assumed that ωk , vk are non-Gaussian noise. The simulation results are shown in Fig.5-8 when the PDF of ωk is set to be { −6x2 + 3/2 x ∈ [−0.5, 0.5], γω (x) = 0 x ∈ (−∞, −0.5) ∪ (0.5, ∞) and vk is assumed to obey β(3, 2) distribution.

state and ideal model state (MEE)

2

state and its estimate (MSE)

5

2

1 0

0 0

−1

−1

0

−2

−2

−2

−3

−3 −4 −6

xr (k) x(k)(M SE) 0

20

40

60

80

100

0

20

40

60

80

−5

100

tracking error

PDF of tracking error

5

0

3

0

−2

2

−2

−4

1

60

80

0 −3

−2

−1

0 (d)

(c)

1

2

−6

3

5 4 3 2 1 0 1 0 20

1.5

20

40

60 (b)

80

100

PDF of estimation error

1

0.5

40

0

20

40

60

80

100

0 −5

0 (d)

5

Fig. 5. The state, estimate, estimate error and its PDF with non-Gaussian noise From these figures, we can see that the performance using MEE control is better than the results using MSE method no matter whether the noises obey Gaussian distribution or non-Gaussian distribution. This is because errors obey non-Gaussian distribution even if the noises obey Gaussian distribution in the presence of the nonlinearity of system (1), the cost function (14) and (20) extract more information about the errors than the quadratic form. The simulation results have demonstrated better tracking accuracy, smaller tracking error by MEE method than by using MSE method.

3D mesh plot of eatimation error(MEE)

0

0

(c)

Fig. 2. The state, ideal model state, tracking error and its PDF with Gaussian noise

−1

100

M EE M SE

M EE M SE 100

80

−4

M EE M SE 40

60

−5

M EE M SE 2

20

40

estimation error

4

4

0

20

x(k)(M EE) x ˆ(k)(M EE)

−4

(a)

2

−6

0

(b)

(a)

4

x(k)(M SE) x ˆ(k)(M SE)

xr (k) x(k)(M EE)

−4 −5

state and its estimate (MEE)

2

1

60

80

100

5. CONCLUSION

Time

Fig. 3. 3D mesh plot of the estimate error with Gaussian noise

In this paper, the tracking control problem for a class of nonlinear non-Gaussian stochastic systems is considered. The stochastic inputs are subjected to obey arbitrary distributions rather than Gaussian distributions and do

10898

Proceedings of the 20th IFAC World Congress 10412 Yunlong Liu et al. / IFAC PapersOnLine 50-1 (2017) 10407–10412 Toulouse, France, July 9-14, 2017

state and ideal model state (MSE)

5

REFERENCES

state and ideal model state (MEE)

2 1 0 −1

0

−2 −3 xr (k) x(k)(M SE)

−5

0

20

40

60

80

xr (k) x(k)(M EE)

−4 100

−5

0

20

40

(a) tracking error

4

60 (b)

100

PDF of tracking error

1.4

M EE M SE

1.2

2

80

1 0

0.8

−2

0.6 0.4

−4

0.2

M EE M SE −6

0

20

40

60

80

0 −5

100

0 (d)

(c)

5

Fig. 6. The state, ideal model state, tracking error and its PDF with non-Gaussian noise 3D mesh plot of eatimation error (MEE)

5 4 3 2 1 0 1 0 −1 0

20

40

60

80

100

Time

Fig. 7. 3D mesh plot of the estimate error PDF with nonGaussian noise 3D mesh plot of tracking error (MEE)

4 3 2 1 0 1 0 −1 0

20

40

60

80

100

Time

Fig. 8. 3D mesh plot of the tracking error PDF with nonGaussian noise not require any prior knowledge of statistical properties. The parzen windowing with Gaussian kernel is used to estimates the PDFs of errors and the concept of MEE criteria is adopted to update the gains. However, this method need more storage space and the computing complexity is also high, and this problem will be studied in the future.

M. Athans, The role and use of the stochastic LinearQuadratic-Gaussian problem in control system design, IEEE Trans. Autom. Control, vol. 16, no. 6, pp. 529-552, 1971. H. Wang, Bounded Dynamic Stochastic Systems: Modeling and Control. London, U.K: Springer-Verlag, 2000. H. Yue and H. Wang, Recent developments in stochastic distribution control: a review, Measurement and Control, vol. 36, no. 7, pp. 209-15, 2003. Y. Yi, T. Li, L. Guo, and H. Wang, Adaptive tracking control for the output PDFs based on dynamic neural networks, Proc. 4th International Symposium on Neural Networks, pp. 93-101, 2007. Y. Yi, L. Guo, and H. Wang, Constrained PI tracking control for output probability distributions based on teo-step neural networks, IEEE Trans. Circuits and Systems, vol. 56, no. 7, pp. 1416-1426, 2009. L. Guo and H. Wang, Minimunm entropy filtering for multivariable stochastic systems with non-Gaussian noises, IEEE Trans. Autom. Control, vol. 51, no. 4, pp. 695-700, 2006. L. Guo and L.P. Yin, Robust PDF control with guaranteed stability for non-linear stochastic systems under modeling errors, IET Control Theory and Applications, vol. 3, no. 5, pp. 575-582, 2009. Y.L. Liu, H. Wang, and L. Guo, Observer-based feedback controller design for a class of stochastic systems with non-Gaussian Variables, IEEE Trans. Autom. Control, vol. 60, no. 5, pp. 1445-1450, 2015. N. Winer, Extrapolation, Interpolation, and Smoothing of Stationary Time Series with Engineering Applications. MIT Press, Cambridge, MA, 1949. C.E. Shannon and W. weaver, The Mathematical Theory of Communications. U. Illinois Press, 1949. D. Erdogmus and J.C. Principe, An error-entropy minimization algorithm for supervised trainning of nonlinear adaptive systems, IEEE Trans. Signal Processing, vol. 50, no. 7, pp. 1780-1786, 2002. X. Feng, K.A. Loparo, and Y. Fang, Optimal state estimation with active probing for stochastic systems: an information theoretic approach, IEEE Trans. Autom. Control, vol. 42, no. 6, pp. 771-785, 1997. J.W. Xu, D. Erdogmus, and J.C. Principe, Minimum Error Entropy Luenberger Observer, Proc. Amer. Control Conf. (ACC), June, pp. 1923-1928, 2005. J.Principle, D. Xu, and J. Fisher, Information Theoretic Learning: in Unsupervised Adaptive Filtering. S. Haykin, Ed. New York: Wiley, 2000. E. Yaz and A. Azemi, Observer design for discrete and continuous nonlinear stochastic systems, Int. J. Syst. Sci., vol. 24, pp. 2289–2302, 1993. L. Devroye and G. Lugosi, Combinatorial Methods in Density Estimation. New York: Springer, 2001. J.C. Principe and D. Erdogmus, From adaptive linear to information filtering, IEEE Conf. Adaptive Systems for Signal processing, Communications, and Control Symposium, pp. 99-104, 2000. D. Erdogmus, Information Theoretic Learning: Renyi’s Entropy and its Applications to Adaptive System Training. phD Dissertation, University of Florida, Gainesville, FL, 2002.

10899