- Email: [email protected]
An Improved Approach to the LQ non-Gaussian Regulator Problem
Proceedings of the 20th World Congress Proceedings of 20th The International Federation of Congress Automatic Control Proceedings of the the 20th World World Congress Proceedings of the 20th World Congress Control The of Toulouse, France,Federation July 9-14, 2017 The International International Federation of Automatic Automatic Control Available online at www.sciencedirect.com The International Federation of Automatic Control Toulouse, Toulouse, France, France, July July 9-14, 9-14, 2017 2017 Toulouse, France, July 9-14, 2017
ScienceDirect
IFAC PapersOnLine 50-1 (2017) 11808–11813
An Improved Approach to the LQ An Improved Approach to the An Improved Approach to the LQ LQ non-Gaussian Regulator Problem non-Gaussian Regulator Problem non-Gaussian Regulator Problem ∗ ∗∗ ∗
S. Battilotti ∗ F. Cacace ∗∗ M. d’Angelo ∗ A. Germani ∗∗∗ ∗ A. Germani ∗∗∗ ∗∗∗ S. Battilotti ∗∗ F. Cacace ∗∗ M. d’Angelo S. S. Battilotti Battilotti F. F. Cacace Cacace ∗∗ M. M. d’Angelo d’Angelo ∗ A. A. Germani Germani ∗∗∗ ∗ a di Roma La Sapienza, Roma, Italy (e-mail: ∗ DIAG, Universit` ∗ Universit` a di Roma La Sapienza, Roma, Italy (e-mail: DIAG, Universit` a (e-mail: ∗ DIAG, {battilotti,mdangelo}@diag.uniroma1.it, ) DIAG, Universit` a di di Roma Roma La La Sapienza, Sapienza, Roma, Roma, Italy Italy (e-mail: {battilotti,mdangelo}@diag.uniroma1.it, ) ∗∗ {battilotti,mdangelo}@diag.uniroma1.it, ) Universit` a Campus Bio-Medico di Roma, Roma, Italy (e-mail: {battilotti,mdangelo}@diag.uniroma1.it, ) ∗∗ ∗∗ Universit` a Campus Campus Bio-Medico di di Roma, Roma, Roma, Roma, Italy Italy (e-mail: (e-mail: a Bio-Medico ∗∗ Universit` [email protected]). Universit` a Campus Bio-Medico di Roma, Roma, Italy (e-mail: [email protected]). ∗∗∗ [email protected]). DISIM, Universit` a degli studi dell’Aquila, L’Aquila, Italy (e-mail: [email protected]). ∗∗∗ ∗∗∗ DISIM, Universit` a degli studi dell’Aquila, L’Aquila, Italy (e-mail: a degli studi dell’Aquila, L’Aquila, Italy ∗∗∗ DISIM, Universit` [email protected]). DISIM, Universit` a degli studi dell’Aquila, L’Aquila, Italy (e-mail: (e-mail: [email protected]). [email protected]). [email protected]). Abstract: In this paper, an improved approach for the solution of the regulator problem Abstract: In this paper, an improved for the solution of the and regulator problem Abstract: In an approach for of regulator problem for linear discrete-time dynamical systemsapproach with non-Gaussian disturbances quadratic cost Abstract: In this this paper, paper, an improved improved approach for the the solution solution of the the and regulator problem for linear discrete-time dynamical systems with non-Gaussian disturbances quadratic cost for linear discrete-time dynamical systems with non-Gaussian disturbances and quadratic cost function isdiscrete-time proposed. Itdynamical is known that a sub-optimal control can be derivedand from the classical for linear systems with non-Gaussian disturbances quadratic cost function is proposed. It is known sub-optimal can be derived from the classical function is It that sub-optimal control can from the classical LQG solution by substituting the that linearaaa filtering part control with a quadratic optimal filter. However, function is proposed. proposed. It is is known known that sub-optimal control can be be derived derived from theHowever, classical LQG solution by substituting the linear filtering part with a quadratic optimal filter. LQG solution by substituting the linear filtering part with a quadratic optimal filter. However, classical quadratic filters have some critical drawbacks when the system is not asymptotically LQG solution by substituting the linear filtering part with a quadratic optimal filter. However, classical quadratic filters have critical drawbacks when the is not asymptotically classical quadratic filters some critical drawbacks when system is stable and, as a consequence in some that case, there is no guarantee on system the stochastic stability of the classical quadratic filters have have some critical drawbacks when the the system is not not asymptotically asymptotically stable and, as a consequence in that case, there is no guarantee on the stochastic stability the stable and, as a consequence in that case, there is no guarantee on the stochastic stability of the controlled system. In order to enlarge the class of systems that can be controlled we will of make stable and, as a consequence in that case, there is no guarantee on the stochastic stability of the controlled system. In order to enlarge the class of systems that can be controlled we will make controlled system. In order to enlarge the class of systems that can be controlled we will make use of the Feedback Quadratic Filter and a quadratically optimal controller is designed also for controlled system. In order to enlarge the class of systems that can be controlled we will make use of the Feedback Quadratic Filter and aa quadratically optimal controller is designed also for use of the Feedback Quadratic Filter and quadratically optimal controller is designed also for non asymptotically stable systems. Numerical results show the performance of these methods. use of the Feedback stable Quadratic FilterNumerical and a quadratically optimal controller isofdesigned also for non asymptotically systems. results the performance methods. non asymptotically stable systems. results show show the performance of these these methods. non asymptotically stable Federation systems. Numerical Numerical show the by performance these © 2017, IFAC (International of Automaticresults Control) Hosting Elsevier Ltd.ofAll rightsmethods. reserved. Keywords: Non-Gaussian systems; Discrete-time systems; Polynomial methods; Kalman filters; Keywords: Non-Gaussian systems; Polynomial methods; Kalman Keywords: filters; Non-Gaussian systems; Discrete-time Discrete-time systems; Polynomialcontrol; methods; Kalman filters; filters; Nonlinear Output injection; LQG control systems; method; Stochastic Separation Keywords: Non-Gaussian systems; Discrete-time systems; Polynomial methods; Kalman filters; Nonlinear filters; Output injection; LQG control method; Stochastic control; Separation Nonlinear filters; Output injection; LQG control method; Stochastic control; Separation principle; filters; Output injection; LQG control method; Stochastic control; Separation Nonlinear principle; principle; principle; 1. INTRODUCTION terference includes noise components that are essentially 1. terference includes that 1. INTRODUCTION INTRODUCTION terference includes noise components that are areatessentially essentially non-Gaussian whichnoise is acomponents common situation frequen1. INTRODUCTION terference includes noise that are essentially non-Gaussian which is aacomponents common situation at frequennon-Gaussian which is common situation at frequencies below 100 MHz (Rappaport and Kurtz, 1966). Newhich is a common situation at frequenThe importance of the optimal control policy in the non-Gaussian cies below 100 MHz (Rappaport and Kurtz, 1966). Necies below 100 MHz (Rappaport and Kurtz, 1966). Neglecting these components is a major source of error in The importance of the optimal control policy in the The importance of the the isoptimal optimal control policy in the the cies below 100 components MHz (Rappaport and Kurtz, 1966). Neengineering applications well known, andpolicy the problem glecting these is aa In major source of error in The importance of control in glecting these components is major source of error in communication systems design. these cases the condiengineering applications is well known, and the problem engineering applications is well well known, known, and the the problem these components is a In major source of error in is usually modeled as a nonlinear programming communication systems these the engineering applications is and problem glecting communication systems design. In optimal these cases cases the condiconditional expectation, whichdesign. gives the minimum variis usually as aa nonlinear programming problem is usually modeled modeled as the nonlinear programming problem systems design. In these cases the condiconsisting in finding minimum of a function with communication tional expectation, which gives the optimal minimum variis usually modeled as a nonlinear programming problem tional expectation, which gives the optimal minimum variance estimation, is the solution of an infinite dimensional consisting in finding the minimum of a function with consisting in finding the minimum of a function with tional expectation, which gives the optimal minimum varidynamical constraints. A common case is that the system estimation, the an infinite consisting in finding the minimum of athat functionsystem with ance ance estimation, is the solution solution ofbe ansolved infinitebydimensional dimensional problem (Zakai, is 1969) that canof numerical dynamical constraints. A common case dynamical constraints. A and common case is is that that the the system estimation, is the solution of an infinite dimensional to be controlled is linear the performance criterion is ance problem (Zakai, 1969) that can be solved by numerical dynamical constraints. A common case is the system problem (Zakai, 1969) that can be solved by numerical approximate solutions, with high computational burden. to be controlled is linear and the performance criterion is to be controlled controlled isinlinear linear and the the performance criterion isa problem (Zakai, 1969) that can be solved by numerical a quadratic formis state and control. The aim criterion is to find is approximate solutions, with high computational burden. to be performance approximate solutions, with high computational burden. Therefore, in order to design any control strategy, it is aaminimum quadratic form in state and control. The aim is to find a quadratic form in state and control. The aim is to find a approximate solutions, with high computational burden. energy feedback control law The that aim keeps the statea Therefore, in order to design any control strategy, it is a quadratic form feedback in state and control. is the to find Therefore, in order order to design design any any control strategy, strategy, it of is important to find satisfactory sub-optimal estimatesit minimum energy control law that keeps state minimum energy feedback control law that keeps the state Therefore, in to control is of the system close to the state-space origin. In the Linear important to find satisfactory sub-optimal estimates of minimum energy feedback control law that keeps the state important to find satisfactory sub-optimal estimates of the state variables that are actually computable. Methods of the system close to the state-space origin. In the Linear of the system system close to to(LQG) the state-space state-space origin. In the the Linear to find that satisfactory sub-optimal estimates of Quadratic Gaussian regulatororigin. problem theLinear noise important the state are actually of the close the In the state variables variables that areconditional actually computable. computable. Methods to approximate thethat state probabilityMethods density Quadratic Gaussian (LQG) regulator problem the noise Quadratic Gaussian (LQG) regulator problem the noise the state variables are actually computable. Methods statistics are assumed to be Gaussian. approximate state conditional probability density Quadratic Gaussian (LQG) regulator problem the noise to to approximate the state Carlo conditional probability density function includethe Monte methods (Arulampalam, statistics statistics are are assumed assumed to to be be Gaussian. Gaussian. to approximate the state conditional probability density include Monte Carlo methods (Arulampalam, statistics are assumed be Gaussian. A well known property to of the discrete LQG optimal control function function include Monte Carlo methods (Arulampalam, 2002), sums of Gaussian densities (Arasaratnam et al., function include Monte Carlo methods (Arulampalam, A well known property of the discrete LQG optimal control 2002), sums of Gaussian densities (Arasaratnam et al., A well known property of the discrete LQG optimal control problem with partial state information is that the optimal 2002), sums of Gaussian densities (Arasaratnam et al., 2007) and weighted sigma points (Julier and Uhlmann, A well known property of the discrete LQG optimal control 2002), sumsweighted of Gaussian densities (Arasaratnam et al., problem with partial state information is that the optimal 2007) and sigma points (Julier and Uhlmann, problem with partial state information is that the optimal regulator,with synthesized by information the LQ optimal technique, is 2007) 2007) and weighted sigma points (Julier and Uhlmann, 2004) among others. These general solutions can cope with problem partial state is that the optimal and weighted These sigmageneral points solutions (Julier and Uhlmann, regulator, synthesized by LQ is among can cope regulator, synthesized by the thelinear LQ optimal optimal technique, is 2004) generated from the optimal estimate technique, of the state 2004) among others. others. These general solutions canoutliers cope with with nonlinearities and/orThese with the presence of noise or regulator, synthesized by the LQ optimal technique, is 2004) among others. general solutions can cope with generated from the optimal linear estimate of the state nonlinearities and/or with the presence of noise outliers or generated from the optimal linear estimate of the state (Kwakernaak, 1972). For linear Gaussian systems, the nonlinearities and/or with the presence of noise outliers or unknown parameters (Stojanovic and Nedic, 2016), and generated from the optimal linear estimate of the state nonlinearities and/or with the presence of noise outliers or (Kwakernaak, 1972). parameters (Stojanovic and (Kwakernaak, 1972).is For For linear Gaussian systems, the Kalman Filter (KF) the linear optimalGaussian recursive systems, estimatorthe in unknown unknown parameters (Stojanovic and Nedic, Nedic, 2016), and and they generally have high computational cost. 2016), (Kwakernaak, 1972). For linear Gaussian systems, the unknown parameters (Stojanovic and Nedic, 2016), and Kalman Filter (KF) is the optimal recursive estimator in they generally have high computational cost. Kalman Filter (KF) is the optimal recursive estimator in the minimum mean-square error sense. On the other hand, they generally have high computational cost. Kalman Filter (KF) is the optimal recursive estimator in they have cost.control probFromgenerally the point of high viewcomputational of the optimal the minimum mean-square error On the minimum mean-square error sense. sense. Onisthe the other hand, for linear non-Gaussian systems the KF theother best hand, affine From the point of view of the optimal control the minimum mean-square error sense. On the other hand, From the point of view of the optimal control problem the same issue arises, as it is known sinceprobthe for linear non-Gaussian systems the KF is the best affine for linear non-Gaussian non-Gaussian systemstothe the KF is isestimators the best best affine affine thesame pointissue of view of as the itoptimal control probestimator but it is yet possible develop that From lem the arises, is known since the for linear systems KF the lem the same issue arises, as it is known since the work of Mortensen (1966), that the original incompleteestimator but it is yet possible to develop estimators that estimator but it is yet possible to develop estimators that the same issue(1966), arises,that as the it is knownincompletesince the are more accurate. work of estimator but it is yet possible to develop estimators that lem work of Mortensen Mortensen (1966), that the original original incompleteinformation stochastic optimal control problem with nonare are more more accurate. accurate. work of Mortensen (1966), that the original incompleteinformation stochastic optimal control problem with nonare more accurate. In many important technical areas the widely used Gaus- information information stochastic optimal noises controlisproblem problem with to non-a linearities orstochastic non Gaussian equivalent optimal control with nonIn many important technical areas the widely used Gauslinearities or non Gaussian noises is equivalent to aa In many important technical areas the widely used Gaussianmany assumption cannot be accepted aswidely a realistic statislinearities or non Gaussian noises is equivalent to complete-information but infinite-dimensional one. AlIn important technical areas the used Gauslinearities or non Gaussian noises is equivalent toAla sian assumption cannot be accepted as aa realistic statiscomplete-information but infinite-dimensional one. sian assumption cannot be accepted as realistic statistical description of the random quantities involved. As complete-information but infinite-dimensional one. Although the results in but Charalambous and Elliotone. (1998) sian assumption cannot be accepted as a realistic statiscomplete-information infinite-dimensional Altical description of the random quantities involved. As though the results in Charalambous and Elliot (1998) tical description ofpapers, the random random quantities involved. As though showndescription in various of increasing attention has been thein results results in Charalambous Charalambous andsufficient Elliot (1998) (1998) show that some cases finite-dimensional statistical the quantities involved. As though the in and Elliot shown in various papers, increasing attention has been show that in some cases finite-dimensional sufficient statisshown in various papers, increasing attention has been given to non-Gaussian systems in control engineering (Yaz, show that in some cases finite-dimensional sufficient statistics are available and allow to reduce the original inshown in various papers, increasing attention has (Yaz, been showare thatavailable in some cases finite-dimensional sufficient statisgiven to in engineering and allow to reduce the original ingiven to non-Gaussian non-Gaussian systems in control control engineering (Yaz, tics 1987a,b; Arulampalam,systems 2002; Gordon et al., 1993; Maryak tics are available and allow to reduce the original incomplete information optimal control problem to a finitegiven to non-Gaussian systems in control engineering (Yaz, tics are available and allow control to reduce the original in1987a,b; Arulampalam, 2002; et Maryak finite1987a,b; Arulampalam, 2002; Gordon Gordon et al., al., 1993; 1993; Maryak et al., 2004). In particular, non-Gaussian problems of- complete complete information information optimal optimal control control problem problem to to a finite1987a,b; Arulampalam, 2002; Gordon et al., 1993; Maryak complete information optimal problem to aa finiteet al., 2004). In particular, non-Gaussian problems ofet al., 2004). In particular, particular, non-Gaussian problems oftenal., arise in digital communications when the noise inet 2004). In non-Gaussian problems often arise in digital communications when the noise inten arise in digital communications when the noise inten arise in digital communications when the noise inCopyright © 2017, 2017 IFAC 12314 2405-8963 © IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Copyright © 2017 IFAC 12314 Copyright ©under 2017 responsibility IFAC 12314 Peer review of International Federation of Automatic Control. Copyright © 2017 IFAC 12314 10.1016/j.ifacol.2017.08.1992
Proceedings of the 20th IFAC World Congress Toulouse, France, July 9-14, 2017 S. Battilotti et al. / IFAC PapersOnLine 50-1 (2017) 11808–11813
dimensional and complete-information one, the general case remains challenging. As a consequence, a sensible alternative is to look for sub-optimal and more easily computable solutions. The first step in this direction is to improve the state estimate provided by the KF. Since this estimate is a linear function of the measurements, in the minimum variance framework a natural development is to design filters that make use of quadratic or polynomial functions of the observations to improve the estimation accuracy, at the same time preserving easy computability and recursion. This was first proposed in Verriest (1985), De Santis et al. (1995) by building an augmented system that has as observations both the observations of the original system and their squares. The suboptimal quadratic estimate of the state for the original system is obtained by applying the KF to the augmented system. This approach has been extended in Carravetta et al. (1996) to polynomial estimates. The estimate provided by quadratic or polynomial systems can be exploited in the finite horizon control problem of nonGaussian systems, as described in Germani and Mavelli (1999, 2002); Cacace et al. (2013). The resulting quadratic optimal controller yields better performance in terms of the standard quadratic cost functional with respect to the standard linear optimal controller, and it is obtained by replacing the linear optimal estimate of the KF with the quadratic (or polynomial) optimal one, in virtue of the separation principle proved in Germani and Mavelli (1999). In the case of non asymptotically stable systems this approach has a conceptual drawback. Indeed, the state noise of the augmented systems depends on the state and its variance may become unbounded when the original system is not strictly stable. Thus, the stability of the Quadratic Filter (QF) is guaranteed only for asymptotically stable systems, and the same holds for the controller based on it. In order to enlarge the class of systems that can be processed and controlled by using the QF, in this paper we apply a novel approach called Feedback Quadratic Filter (FQF) which is based on a suitable rewriting of the system model through an output injection term (Cacace et al. (2016)). As it has been shown, the main novelty of this filter is that it can be applied also to non asymptotically stable systems. Moreover, by tuning the gain of the injection term, the FQF attains better performance than the QF in a mean-square error sense. Thus, the FQF provides a new method to process and control non asymptotically stable systems. In this paper we show that the separation result of the QF can be extended to the FQF. This guarantees that the optimal controller based on quadratic estimates remains linear in the state. In other words, the optimal quadratic controller is straightforwardly obtained by using the prediction of the FQF instead of the one of the KF together with the same feedback control law as in the linear optimal regulator (LQG) for the Gaussian case. Notation. Throughout the paper E denotes the expectation. If A and B are two matrices, then A ⊗ B denotes the Kronecker product of A and B, while A[i] is the i-th Kronecker power of A (Bellman, 1970), clearly the same
11809
definitions holds for vectors. Finally, ρ(A) is the spectrum of a square matrix A and In is the identity matrix in Rn×n . 2. PROBLEM STATEMENT The control problem we solve concerns the class of linear, detectable and stabilizable systems driven by nonGaussian additive noise described by the following equations: (1) x(k + 1) = Ax(k) + Bu(k) + fk , x(0) = x0 (2) y(k) = Cx(k) + gk with the associated cost function N 1 T T E x (k)Qx(k) + u (k)Ru(k) . (3) J = lim N →∞ N k=0
As usual, x(k), fk ∈ Rn , u(k) ∈ Rp , y(k), gk ∈ Rq , A ∈ Rn×n , B ∈ Rn×p , C ∈ Rq×n , Q ∈ Rn×n and R ∈ Rp×p . Matrices Q and R are assumed symmetric nonnegative definite (strictly positive definite in the case of R). Moreover, the initial state and the non-Gaussian random sequences {fk } and {gk } satisfy the following conditions for k ≥ 0: x0 ∼ N (¯ x0 , Ψx0 ); {fk } and {gk } are zero mean i.i.d. sequences; {fk }, {gk } and x0 are statistically independent; fk and gk have finite fourth moments; (i) [i] (i) [i] ψf := E[fk ] and ψg := E[gk ], for i = 2, 3, 4, are known vectors; (vi) [C Ψg ], where Ψg = E[gk gkT ], is full row rank.
(i) (ii) (iii) (iv) (v)
(i)
We define similarly for x0 the vectors ψx0 := E[(x0 − x ¯0 )[i] ], known since x0 is Gaussian. In this framework, the infinite-horizon (sub-)optimal control problem for nonGaussian discrete-time linear systems with partial state information is considered. The aim is to find a minimum energy feedback control law, in the class of recursively computable quadratic feedback, that keeps the state of the system close to the state-space origin. It is well known (Kwakernaak, 1972) that, for the infinitehorizon regulator problem (1)–(3), the output feedback optimal control u∗ (k) among all affine transformations of y(0), y(1), . . . , y(k − 1) is given by u∗ (k) = −M x ˆ(k|k − 1), (4) where (5) M = (R + B T P B)−1 B T P A, P is the solution of the Riccati equation P = Q + AT P A − AT P B(R + B T P B)−1 B T P A (6) and x ˆ(k|k − 1) is the prediction of x(k) provided by the KF (Balakrishnan, 1984). We now clarify some issues on polynomial filtering and recursion. 3. QF WITH OUTPUT INJECTION 3.1 The Geometric Approach We introduce some basic notions and issues about polynomial filtering using the geometrical approach. Let (Ω, F, P) be a probability space, G a given sub σ-algebra of F and L2 (G, n) the Hilbert space of the n-dimensional, G measurable random variables with finite second order moment.
12315
Proceedings of the 20th IFAC World Congress 11810 S. Battilotti et al. / IFAC PapersOnLine 50-1 (2017) 11808–11813 Toulouse, France, July 9-14, 2017
We write L2 (X, n) to denote L2 (σ(X), n). We denote with Π [·| M] the orthogonal projection onto a given Hilbert space M.
Given the output sequence Yk = col(y(0), · · · , y(k)) and the auxiliary vector Yk = col(1, Yk ) ∈ Rl+1 , with l = (k + 1)q, the minimum variance estimate of the state x(k) of system (1)–(2), can be defined as the orthogonal projection of x(k) onto the Hilbert space L2 (Yk , n): x ˆ(k) = E [x(k)| σ(Yk )] = Π x(k)| L2 (Yk , n) . (7) If the sequences {x(k)} and {y(k)} are jointly Gaussian, this projection is equivalent to the projection on the subspace Lky ⊂ L2 (Yk , n) of all affine functions of Yk , Lky = {z : Ω → Rn : ∃T ∈ Rn×l+1 : z = T Yk }.
(8)
Π[x(k)|Lky ]
The KF recursively computes the projection which is the best affine estimate of x(k) in the minimum variance sense. In the Gaussian case, this coincides with E [x(k)| σ(Yk )].When {x(k)} and/or {y(k)} are nonGaussian, the computation of (7) is highly difficult. Since the best affine estimate is obtained by projecting onto Lky , better suboptimal estimates can be obtained by projecting the state x(k) onto larger sub-spaces. For example, we may consider the space of second order polynomial (quadratic) transformations of Yk , here denoted by Qky . Let Yk ˜ (2) l ˜l = 1 + l + l2 , Yk = (9) [2] ∈ R , Yk then ˜ (2) Qky = {z : Ω → Rn : ∃T ∈ Rn×l : z = T Yk }. (10) Remark 1. Since Lky ⊂ Qky ⊂ L2 (Yk , n), it follows from the definition of the norm in L2 (X, n), v2L2 (X,n) := T v vdP = E[v T v], that projecting the state onto Qky (a Ω larger subspace) will return an estimate, having an error variance equal or smaller than that of the affine estimate, that we may well name the optimal quadratic estimate.
However, the computation of Π[x(k)|Qky ] would require a growing filter size, due to the presence in Qky of terms of the kind y i (k1 )y j (k2 ), with i, j ≤ 0, 1, 2. Thus, we replace (2) Yk with Yk[2] y(0) (2) ¯ Y k = . ∈ Rl , ¯l = 1 + l + (k + 1)q 2 , (11) . . y(k)[2]
that is, a vector containing all the observations and their Kronecker squares from time 0 to k. We obtain the projection subspace k
¯
(2)
Qy = {z : Ω → Rn : ∃T ∈ Rn×l : z = T Y k },
(12)
k Qy
with Lky ⊂ ⊂ Qky . Thus the resulting estimate will not be the optimal quadratic estimate, but it will still have an error not larger than the best linear one. Since this projection is recursively computable we shall refer to it as recursive quadratic estimate. As a consequence, we shall refer to the optimal recursive quadratic control when the recursive quadratic prediction,
k−1
i.e. Π[x(k)|Qy control law.
], of the state is used to generate the
3.2 Output Injection As in Cacace et al. (2016), the state equation (1) is transformed using the output equation (2): x(k + 1) = Ax(k) + Bu(k) + fk = Ax(k) + Bu(k) + fk + Ly(k) − LCx(k) − Lgk ˜ = Ax(k) + Bu(k) + Ly(k) + hk , (13) where A˜ = A − LC, L ∈ Rn×q is an arbitrarily chosen (i) [i] matrix and hk = fk − Lgk . Moreover, ψh = E hk , (i)
i = 2, 3, 4, can be computed as a function of ψf (i) ψg
and
as follows (2)
(2)
(14)
(3)
(3)
(15)
ψh =ψf + L[2] ψg(2) , ψh =ψf − L[3] ψg(3) , (4) (4) (2) ψh =ψf + M24 ψf ⊗ L[2] ψg(2) + L[4] ψg(4) ,
(16)
M24
where is the coefficient matrix for the expansion of the binomial Kronecker power (see Carravetta et al. (1996)). As the second step, taking advantage of linearity, the state sequence is split into two sequences, {xd (k)} and {xs (k)}. The deterministic component xd (k) is the solution of ˜ d (k) + Bu(k) + Ly(k), xd (0) = xd , (17) xd (k + 1) = Ax 0 with xd0 = E[x0 ] = x ¯0 , i.e. k−1 (18) A˜k−τ −1 (Bu(τ ) + Ly(τ )) , xd (k) = A˜k xd0 + τ =0
which can be computed at time k, since the deterministic initial value xd0 and the output sequence Yk−1 are available. The stochastic component xs (k) is the solution of ˜ s (k) + hk , xs (0) = xs , xs (k + 1) = Ax (19) 0 (i) s s [i] with x0 ∼ N (0, Ψx0 ), and, therefore ψxs = E x0 = 0
(i) ψ x0 .
From (17) and (19) trivially follows that x(k) = xd (k) + xs (k) ∀k ≥ 0. (20) The output map (2) is arranged as ys (k) = y(k) − Cxd (k) = Cxs (k) + gk , (21) where ys (k) is an available quantity at time k. We can also define the corresponding sequence vector Ys,k := col(ys (0), ys (1), . . . , ys (k)) (22) and the auxiliary vector Ys,k = col(1, Ys,k ). Moreover, (2)
k
also vector Y s,k and spaces Lkys , Qkys , and Qys can be defined using Ys,k instead of Yk in (11), (8), (10), and (12), respectively. In Cacace et al. (2016) it is proved that Ys,k is an affine transformation of the original sequence vector Yk and viceversa. Thus, Lky ≡ Lkys . The same fact holds for the quadratic transformations, thus Qky ≡ Qkys . However this is not true for the recursive quadratic transformations, k k i.e. Qy ≡ Qys (Cacace et al., 2016). Since (18) is an affine transformation of Yk , the projection k of xd (k) onto Qys trivially corresponds to itself. The opk
timal recursive quadratic estimate x ˜s (k) = Π(xs (k)|Qys )
12316
Proceedings of the 20th IFAC World Congress Toulouse, France, July 9-14, 2017 S. Battilotti et al. / IFAC PapersOnLine 50-1 (2017) 11808–11813
can be obtained by processing the system (19)-(21). Analogously, we can obtain the optimal recursive quadratic k−1 prediction x ˜s (k|k − 1) = Π(xs (k)|Qys ). 4. SEPARATION PRINCIPLE AND OPTIMAL RECURSIVE QUADRATIC CONTROL We prove in this section that the structure of the controller of the LQG regulator (4)–(6) remains optimal in the class of quadratic feedback control laws. To this end, let us introduce the extended state and output (2) , (23) Xe (k) = col xd (k), xs (k), x[2] (k), ψ s h Ye (k) = col y(k), y˜s[2] (k) , (24) [2]
where xd (k) and xs (k) are defined in (17)–(19), and xs [2] and y˜s satisfy (2) (2) x[2] (k + 1) = A˜[2] x[2] (k) + ψ + h , (25) s
s
h (2)
k
y˜s[2] := ys[2] (k) − ψg(2) = C [2] x[2] s (k) + gk , where (2) ˜ s (k) ⊗ hk + hk ⊗ Ax ˜ s (k) + h[2] − ψ (2) , hk = Ax k h (2) gk
[2] gk
(26) (27)
ψg(2)
(28) = Cxs (k) ⊗ gk + gk ⊗ Cxs (k) + − are zero-mean, mutually correlated white noise sequences uncorrelated with the initial state x0 . The dynamics of the extended system is (29) Xe (k + 1) = AXe (k) + Bu(k) + Wk , (30) Ye (k) = CXe (k) + Vk , where A LC 0 0 Lgk B hk 0 A˜ 0 0 0 A= 0 0 A˜[2] I , B = 0 , Wk = h(2) , k 0 0 0 0 0 I C=
C C 0 0 , 0 0 C [2] 0
Vk =
where M is defined in (5). Then, the optimal recursive quadratic control for (1) is ˆ˜(k|k − 1) . u∗ (k) = φ x (34)
ˆ˜(k|k − 1) = xd (k) + x ˜s (k|k − 1) is obtained as the where x sum of the known term xd (k) obtained from (18) and of the optimal quadratic recursive prediction x ˜s (k|k − 1).
Proof. The optimal recursive quadratic control for the regulator problem (1)–(3) coincides with the LQG solution for the extended system (29)–(31), i.e. e (k|k − 1), (35) u∗ (k) = − (R + B T PB)−1 B T PA X
where P is given by P = Q + AT PA − AT PB(R + B T PB)−1 B T PA (36) e (k|k − 1) is the optimal linear prediction of Xe (k). and X It is easy to verify, by expanding the terms of (36), that P P 0 0 P P 0 0 P= , (37) 0 0 0 0 0 0 0 0
where P is the solution of the Riccati equation (6) associated to (1)–(3), solves (36). The resulting control law is e (k|k − 1)1 + X e (k|k − 1)2 ), u∗ (k) = −M (X (38) e (k|k −1)1 and X e (k|k −1)2 are the optimal linear where X predictions of the first two components of Xe (k), that is xd (k) and xs (k). The first component is deterministic e (k|k − 1)1 = xd (k). The second and known, hence X component can be obtained by the optimal prediction of the stochastic part of (29)–(30), i.e.
gk (2) . gk
Let Je be the cost function associated to the extended state dynamics (29) N 1 E XeT (k)QXe (k) + uT (k)Ru(k) , Je = lim N →∞ N k=0 (31) where Q Q00 Q Q 0 0 Q= . (32) 0 0 0 0 0 0 00 Clearly, Je ≡ J. Remark 2. Since we would like to perform an estimate of the state as a quadratic transformation of the output map, we need to consder the evolution of the second Kronecker power of the state, third row of system (29). Moreover, we (2) extend the state with a forth constant component ψh in order to have the system in the classical form (29) driven by a zero mean noise. Theorem 1. Let φ : Rn → Rp be the map φ(ξ) = −M ξ, (33)
11811
hk 0 xs (k + 1) xs (k) A˜ 0 = + (2) + (2) x[2] ψh hk 0 A˜[2] x[2] s (k + 1) s (k) (39) gk ys (k) C 0 xs (k) = + (2) . (40) (k) y˜s[2] (k) 0 C [2] x[2] g s k
e (k|k − 1)2 is a projection on the space of the Since X e (k|k − quadratic transformations of ys (0), . . . , ys (k −1), X k−1 1)2 = Π(xs (k)|Qys ) = x ˜s (k|k − 1). Remark 3. Note that, even though the optimal recursive quadratic control (35) depends a priori on the prediction [2] e (k|k − 1), Theorem 1 shows that it only of xs in X depends on xd (k) and on the prediction x ˜s (k|k − 1) = k−1 Π(xs (k)|Qys ). Remark 4. Note that the map φ defined in (33) is invariant with respect to output injection whereas u∗ of (34) ˆ depends on the output injection gain L through x ˜. k−1
The prediction x ˜s (k|k − 1) = Π(xs (k)|Qys ) is obtained by applying the KF algorithm to the system (39)–(40). It is easy to prove that the state and measurement noise of this system are zero mean, temporally uncorrelated and mutually correlated sequences. Since (A, C) is detectable, we can choose L to have A˜ Schur stable. In this case system (39)–(40) is detectable. Moreover, if A˜ is Schur
12317
Proceedings of the 20th IFAC World Congress 11812 S. Battilotti et al. / IFAC PapersOnLine 50-1 (2017) 11808–11813 Toulouse, France, July 9-14, 2017
stable then the state, output and noises in (39)–(40) are all second-order asymptotically stationary processes. In these conditions the KF for correlated state and measurement noises applied to (39)–(40) is asymptotically stationary ˆ˜(k) = and internally asymptotically stable. The estimate x ˜s (k) is quadratic and recursively computable and xd (k) + x it is called the FQF of (1)–(2).
MSE KF
10
MSE
2
KF
F
6
600 200 500 150 400
30
mean(JKF)
50
Pq∞
JKF
MSE q
mean(JFQF) J
40
mean(MSE q)
FQF
100
00
10 10
20 20
30 30
40 40
50 50
Fig. 1. MSE and cost function J for Example 1. 40
α FQF mean(α FQF)
30 20 10 0
0
10
20
30
40
50
Fig. 2. Performance index for Example 1. gk ∈ R.
(44)
The probability densities of the state and output noise sequences are shown in Table 1, in which σ ¯ = (11/2)1/2 . state noise
output noise 0 9/20
20
50 200
Example 1 We consider an unstable system of the form with 1.94 −0.46 0 A= , B= , C = [1 0] (43) 0 101.68 0.18 20 30 1 40 50
1/¯ σ 1/2
10
100 300
50(1)–(2)
-10/¯ σ 1/20
2 0
KF
where u∗ (k) is the control law defined in (4) and N is the number of sampling points. Analogous definitions for J QF F QF ∗ for ) which the control law u (k) is generated by 200and Jmean(JKF KF using Ja QF and the FQF, respectively. Clearly, we initialize FQF mean(Jwith ) the same initial conditions which are not of the filters FQF 150relevance J with regard to the results. Both examples have been simulated with a time horizon N = 150 across 50 100realizations.
(i)
3
2
k=0
fk (i) p[(fk )]
4
4
α = 10 · (J − J )/J , (41) FQF ) wheremean(MSE J KF , J QF and J F QF are the cost functions asso8 ciated to a single realization of the controlled dynamics when the predictions of KF, QF and FQF are used by 6the controller, respectively. Therefore, given a probability space (Ω, F, P) on which are defined the stochastic noise 4sequences and the initial condition x , for a fixed ω ∈ Ω, 0 N 1 T 2 x (k)Qx(k) + u∗ T (k)Ru∗ (k), (42) J KF = 0 10 N 20 30 40 50
and the state and output noises (1) fk fk = (2) ∈ R2 , fk
5
mean(MSE FQF)
8
We now test the effectiveness of the proposed approach with two numerical examples. We consider also non asymptotically stable system since they can be processed by the FQF. We define a relative performance index αF , MSE KF F ∈ {QF, F QF }, as mean(MSE KF) F
MSE FQF
10
5. NUMERICAL EXAMPLES
FQF
6
mean(MSE KF)
gk p[(gk )]
-1 0.5
5 0.1
0 0.4
Table 1. Probability mass functions of the noise sequences of Example 1. The cost function is defined by Q = I2 and R = 0.2. Since the spectrum of the matrix A is ρ(A) = {1.1, 1.02}, we use output injection and we choose the gain L that ˜ Figure 1 (top) plots assigns the spectrum {0.4, 0.45} to A. the MSE and the average of MSE of the state estimation across realizations for the KF and the FQF. Figure 1 (bottom) plots J KF and J F QF and their average values. Figure 2 plots the relative performance index αF QF across realizations and its average value 18.6%.
Example 2 In this second example we show the dependence of the control (34) on the output injection gain L. To compare the result with the QF we consider a stable MIMO system of the form (1)–(2) defined by 0.6 0 1 0 0.5 1 1000 0 −0.4 1 1 0 0.4 A= , B= , C= 0 0 0.8 0 0.2 1 0100 0 0 0 0.9 0 1 (45) and the state and output noises (1) (2) (3) (4) fk = col fk , fk , fk , fk ∈ R4 , (46) (1) (2) gk = col gk , gk ∈ R2 .
The components of {fk } and {gk } are i.i.d. sequences and their probability densities are shown in Table 2 in which σ ¯ = (11/2)1/2 .
The cost function is characterized by the matrices Q = I4 and R = 0.2 · I2 . Since ρ(A) = {0.6, −0.4, 0.8, 0.9} we can control the system by using a standard QF without output
12318
1
Proceedings of the 20th IFAC World Congress Toulouse, France, July 9-14, 2017 S. Battilotti et al. / IFAC PapersOnLine 50-1 (2017) 11808–11813
state noise
600
(i) fk
500
300
1/¯ σ 1/2
0 9/20
-11.5 0.1
(i)
p[(g MSEkq )]
1.5 0.7
0.5 0.2
q
injection. We make a comparison with an optimized FQF where the output injection gain Lopt is such that Lopt = argmin|λ|<1 ∀λ∈ρ(A) (47) ˜ trace S∞ (L),
200 0
40 30 20 10 0
-10/¯ σ 1/20
mean(MSE ) Table 2. Probability mass functions of the noise sequences of Example 2.
400
100
(i)
p[(fk )]
output noise (i) gPkq ∞
0
covariance where S∞ is20 the steady-state 10 30 40 50 matrix of the estimation error. Note that in this example the pair (A, C) is observable, thus the minimization over the whole spectrum of A˜ makes sense. In Table 3 the first column α FQF is the percentage reduction of the MSE of the state FQF mean(α ) estimation with respect to the KF. The second column presents the average values of J KF , J QF and J F QF , and the third column shows the values of the relative performance index α in (41). It is evident that, even in the case of stable systems, the proposed approach improves significantly the performance.
10
Filter KF 20QF FQF
% MSE -21.45% 30 -33.01%
J 94.87 85.27 40 79.60
α -10.17% 50 -15.67%
Table 3. MSE and cost function. 6. CONCLUSION The optimal quadratic infinite-horizon regulator for linear stochastic non asymptotically stable non-Gaussian systems is obtained by a recursive algorithm. It has been proved that the optimal quadratic regulator is obtained by replacing the Kalman Filter prediction with the one of the Feedback Quadratic Filter and by using the same feedback control law as in the linear optimal regulator. We have explicitly proved the validity of the separation principle for this case. The increase of the computational cost of the proposed algorithm with respect to the linear one is moderate. Numerical simulations show the effectiveness of this method with respect to standard optimal linear control and also with respect to the standard Quadratic Filter. We finally stress the fact that by using a polynomial filter it is possible to further improve the performance. REFERENCES I. Arasaratnam, S. Haykin and R. J. Elliott. Discrete-Time Nonlinear Filtering Algorithms Using Gauss-Hermite Quadrature. Proc. of the IEEE, 95(5), 953–977, 2007. M. S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp. A tutorial on particle filters for online nonlinear/nonGaussian Bayesian tracking. IEEE Trans. on Sign. Proc. 50(2), 174–188, 2002. A.V. Balakrishnan. Kalman Filtering Theory. Optimization Software, Inc., New York, 1984. R. R. Bellman. Introduction to Matrix Analysis. McGrawHill. 1970. F. Cacace, A. Fasano, A. Germani, and A. Monteriu. Optimal reduced-order quadratic solution for the nonGaussian finite-horizon regulator problem. Proc. of the
11813
52nd IEEE Conf. on Decision and Control, CDC 2013, 3085–3090, 2013. F. Cacace, F. Conte. A. Germani, and G. Palombo. Feedback Quadratic Filtering. (accepted for publication in Automatica), 2016. F. Carravetta, A. Germani, and M. Raimondi. Polynomial filtering for linear discrete time non-Gaussian systems. SIAM J. on Control and Optim., 34 (5), 1666-1690, 1996. C. Charalambous and R.J. Elliot. Classes of Nonlinear Partially Observable Stochastic Optimal Control Problems with Explicit Optimal Control Laws. SIAM J. on Control and Optim., 36 (2), 542–578, 1998. A. De Santis, A. Germani, and M. Raimondi. Optimal quadratic filtering of linear discrete-time non-Gaussian systems. IEEE Trans on Autom. Contr., 40 (7), 1274– 1278, 1995. A. Germani and G. Mavelli. Optimal Quadratic Solution for the non-Gaussian Finite-Horizon Regulator Problem. Systems & Control Letters, 38, 321–331, 1999. A. Germani and G. Mavelli. The Polynomial Approach to the LQ non-Gaussian Regulator Problem. IEEE Trans on Autom. Contr., vol. 47 (8), 1385–1391, 2002. N.J. Gordon, D.J. Salmond, and A.F.M. Smith. Novel approach to nonlinear/non-Gaussian Bayesian state estimation. Proc. of the Institution of Electrical Engineers F140 (2), 107–113, 1993. S. J. Julier and J. K. Uhlmann. Unscented Filtering and Nonlinear Estimation. Proceedings of the IEEE, 92(3), 401–422, 2004. H. Kwakernaak and R. Sivan Linear optimal control systems. Vol. 1. New York: Wiley-interscience J. Maryak, J.C. Spall, and B. Heydon. Use of the Kalman filter for inference in state-space models with unknown noise distributions. IEEE Trans on Autom. Contr., 49 (1), 87–90, 2004. R. E. Mortensen. Stochastic Optimal Control with Noisy Observations. Int. Jour. of Control, 4(5), 455–464, 1966. S.S. Rappaport and L. Kurtz. An optimal nonlinear detector for digital data transmission through nonGaussian channels. IEEE Trans on Commun. Technol., 3, 266–274, 1966. V. Stojanovic, and N. Nedic. Robust Kalman filtering for nonlinear multivariable stochastic systems in the presence of non-Gaussian noise. Int. Jour. of Robust and Nonlinear Control, 26, 445–460, 2016. E. I. Verriest. Linear filters for linear systems with multiplicative noise and nonlinear filters for linear systems with non-Gaussian noise. In: IEEE American Contr. Conf., 182–184, 1985. E. Yaz. Relationship between several novel control schemes proposed for a class of nonlinear stochastic systems. Internat. J. Control 45, 1447–1454, 1987. E. Yaz. A control scheme for a class of discrete nonlinear stochastic systems. IEEE Trans on Autom. Contr., 32 (1), 77–80, 1987. M. Zakai. On the optimal filtering of diffusion processes. Probab. Theory Related Fields, 11 (3)3, 230–243, 1969.
12319
ScienceDirect
IFAC PapersOnLine 50-1 (2017) 11808–11813
An Improved Approach to the LQ An Improved Approach to the An Improved Approach to the LQ LQ non-Gaussian Regulator Problem non-Gaussian Regulator Problem non-Gaussian Regulator Problem ∗ ∗∗ ∗
S. Battilotti ∗ F. Cacace ∗∗ M. d’Angelo ∗ A. Germani ∗∗∗ ∗ A. Germani ∗∗∗ ∗∗∗ S. Battilotti ∗∗ F. Cacace ∗∗ M. d’Angelo S. S. Battilotti Battilotti F. F. Cacace Cacace ∗∗ M. M. d’Angelo d’Angelo ∗ A. A. Germani Germani ∗∗∗ ∗ a di Roma La Sapienza, Roma, Italy (e-mail: ∗ DIAG, Universit` ∗ Universit` a di Roma La Sapienza, Roma, Italy (e-mail: DIAG, Universit` a (e-mail: ∗ DIAG, {battilotti,mdangelo}@diag.uniroma1.it, ) DIAG, Universit` a di di Roma Roma La La Sapienza, Sapienza, Roma, Roma, Italy Italy (e-mail: {battilotti,mdangelo}@diag.uniroma1.it, ) ∗∗ {battilotti,mdangelo}@diag.uniroma1.it, ) Universit` a Campus Bio-Medico di Roma, Roma, Italy (e-mail: {battilotti,mdangelo}@diag.uniroma1.it, ) ∗∗ ∗∗ Universit` a Campus Campus Bio-Medico di di Roma, Roma, Roma, Roma, Italy Italy (e-mail: (e-mail: a Bio-Medico ∗∗ Universit` [email protected]). Universit` a Campus Bio-Medico di Roma, Roma, Italy (e-mail: [email protected]). ∗∗∗ [email protected]). DISIM, Universit` a degli studi dell’Aquila, L’Aquila, Italy (e-mail: [email protected]). ∗∗∗ ∗∗∗ DISIM, Universit` a degli studi dell’Aquila, L’Aquila, Italy (e-mail: a degli studi dell’Aquila, L’Aquila, Italy ∗∗∗ DISIM, Universit` [email protected]). DISIM, Universit` a degli studi dell’Aquila, L’Aquila, Italy (e-mail: (e-mail: [email protected]). [email protected]). [email protected]). Abstract: In this paper, an improved approach for the solution of the regulator problem Abstract: In this paper, an improved for the solution of the and regulator problem Abstract: In an approach for of regulator problem for linear discrete-time dynamical systemsapproach with non-Gaussian disturbances quadratic cost Abstract: In this this paper, paper, an improved improved approach for the the solution solution of the the and regulator problem for linear discrete-time dynamical systems with non-Gaussian disturbances quadratic cost for linear discrete-time dynamical systems with non-Gaussian disturbances and quadratic cost function isdiscrete-time proposed. Itdynamical is known that a sub-optimal control can be derivedand from the classical for linear systems with non-Gaussian disturbances quadratic cost function is proposed. It is known sub-optimal can be derived from the classical function is It that sub-optimal control can from the classical LQG solution by substituting the that linearaaa filtering part control with a quadratic optimal filter. However, function is proposed. proposed. It is is known known that sub-optimal control can be be derived derived from theHowever, classical LQG solution by substituting the linear filtering part with a quadratic optimal filter. LQG solution by substituting the linear filtering part with a quadratic optimal filter. However, classical quadratic filters have some critical drawbacks when the system is not asymptotically LQG solution by substituting the linear filtering part with a quadratic optimal filter. However, classical quadratic filters have critical drawbacks when the is not asymptotically classical quadratic filters some critical drawbacks when system is stable and, as a consequence in some that case, there is no guarantee on system the stochastic stability of the classical quadratic filters have have some critical drawbacks when the the system is not not asymptotically asymptotically stable and, as a consequence in that case, there is no guarantee on the stochastic stability the stable and, as a consequence in that case, there is no guarantee on the stochastic stability of the controlled system. In order to enlarge the class of systems that can be controlled we will of make stable and, as a consequence in that case, there is no guarantee on the stochastic stability of the controlled system. In order to enlarge the class of systems that can be controlled we will make controlled system. In order to enlarge the class of systems that can be controlled we will make use of the Feedback Quadratic Filter and a quadratically optimal controller is designed also for controlled system. In order to enlarge the class of systems that can be controlled we will make use of the Feedback Quadratic Filter and aa quadratically optimal controller is designed also for use of the Feedback Quadratic Filter and quadratically optimal controller is designed also for non asymptotically stable systems. Numerical results show the performance of these methods. use of the Feedback stable Quadratic FilterNumerical and a quadratically optimal controller isofdesigned also for non asymptotically systems. results the performance methods. non asymptotically stable systems. results show show the performance of these these methods. non asymptotically stable Federation systems. Numerical Numerical show the by performance these © 2017, IFAC (International of Automaticresults Control) Hosting Elsevier Ltd.ofAll rightsmethods. reserved. Keywords: Non-Gaussian systems; Discrete-time systems; Polynomial methods; Kalman filters; Keywords: Non-Gaussian systems; Polynomial methods; Kalman Keywords: filters; Non-Gaussian systems; Discrete-time Discrete-time systems; Polynomialcontrol; methods; Kalman filters; filters; Nonlinear Output injection; LQG control systems; method; Stochastic Separation Keywords: Non-Gaussian systems; Discrete-time systems; Polynomial methods; Kalman filters; Nonlinear filters; Output injection; LQG control method; Stochastic control; Separation Nonlinear filters; Output injection; LQG control method; Stochastic control; Separation principle; filters; Output injection; LQG control method; Stochastic control; Separation Nonlinear principle; principle; principle; 1. INTRODUCTION terference includes noise components that are essentially 1. terference includes that 1. INTRODUCTION INTRODUCTION terference includes noise components that are areatessentially essentially non-Gaussian whichnoise is acomponents common situation frequen1. INTRODUCTION terference includes noise that are essentially non-Gaussian which is aacomponents common situation at frequennon-Gaussian which is common situation at frequencies below 100 MHz (Rappaport and Kurtz, 1966). Newhich is a common situation at frequenThe importance of the optimal control policy in the non-Gaussian cies below 100 MHz (Rappaport and Kurtz, 1966). Necies below 100 MHz (Rappaport and Kurtz, 1966). Neglecting these components is a major source of error in The importance of the optimal control policy in the The importance of the the isoptimal optimal control policy in the the cies below 100 components MHz (Rappaport and Kurtz, 1966). Neengineering applications well known, andpolicy the problem glecting these is aa In major source of error in The importance of control in glecting these components is major source of error in communication systems design. these cases the condiengineering applications is well known, and the problem engineering applications is well well known, known, and the the problem these components is a In major source of error in is usually modeled as a nonlinear programming communication systems these the engineering applications is and problem glecting communication systems design. In optimal these cases cases the condiconditional expectation, whichdesign. gives the minimum variis usually as aa nonlinear programming problem is usually modeled modeled as the nonlinear programming problem systems design. In these cases the condiconsisting in finding minimum of a function with communication tional expectation, which gives the optimal minimum variis usually modeled as a nonlinear programming problem tional expectation, which gives the optimal minimum variance estimation, is the solution of an infinite dimensional consisting in finding the minimum of a function with consisting in finding the minimum of a function with tional expectation, which gives the optimal minimum varidynamical constraints. A common case is that the system estimation, the an infinite consisting in finding the minimum of athat functionsystem with ance ance estimation, is the solution solution ofbe ansolved infinitebydimensional dimensional problem (Zakai, is 1969) that canof numerical dynamical constraints. A common case dynamical constraints. A and common case is is that that the the system estimation, is the solution of an infinite dimensional to be controlled is linear the performance criterion is ance problem (Zakai, 1969) that can be solved by numerical dynamical constraints. A common case is the system problem (Zakai, 1969) that can be solved by numerical approximate solutions, with high computational burden. to be controlled is linear and the performance criterion is to be controlled controlled isinlinear linear and the the performance criterion isa problem (Zakai, 1969) that can be solved by numerical a quadratic formis state and control. The aim criterion is to find is approximate solutions, with high computational burden. to be performance approximate solutions, with high computational burden. Therefore, in order to design any control strategy, it is aaminimum quadratic form in state and control. The aim is to find a quadratic form in state and control. The aim is to find a approximate solutions, with high computational burden. energy feedback control law The that aim keeps the statea Therefore, in order to design any control strategy, it is a quadratic form feedback in state and control. is the to find Therefore, in order order to design design any any control strategy, strategy, it of is important to find satisfactory sub-optimal estimatesit minimum energy control law that keeps state minimum energy feedback control law that keeps the state Therefore, in to control is of the system close to the state-space origin. In the Linear important to find satisfactory sub-optimal estimates of minimum energy feedback control law that keeps the state important to find satisfactory sub-optimal estimates of the state variables that are actually computable. Methods of the system close to the state-space origin. In the Linear of the system system close to to(LQG) the state-space state-space origin. In the the Linear to find that satisfactory sub-optimal estimates of Quadratic Gaussian regulatororigin. problem theLinear noise important the state are actually of the close the In the state variables variables that areconditional actually computable. computable. Methods to approximate thethat state probabilityMethods density Quadratic Gaussian (LQG) regulator problem the noise Quadratic Gaussian (LQG) regulator problem the noise the state variables are actually computable. Methods statistics are assumed to be Gaussian. approximate state conditional probability density Quadratic Gaussian (LQG) regulator problem the noise to to approximate the state Carlo conditional probability density function includethe Monte methods (Arulampalam, statistics statistics are are assumed assumed to to be be Gaussian. Gaussian. to approximate the state conditional probability density include Monte Carlo methods (Arulampalam, statistics are assumed be Gaussian. A well known property to of the discrete LQG optimal control function function include Monte Carlo methods (Arulampalam, 2002), sums of Gaussian densities (Arasaratnam et al., function include Monte Carlo methods (Arulampalam, A well known property of the discrete LQG optimal control 2002), sums of Gaussian densities (Arasaratnam et al., A well known property of the discrete LQG optimal control problem with partial state information is that the optimal 2002), sums of Gaussian densities (Arasaratnam et al., 2007) and weighted sigma points (Julier and Uhlmann, A well known property of the discrete LQG optimal control 2002), sumsweighted of Gaussian densities (Arasaratnam et al., problem with partial state information is that the optimal 2007) and sigma points (Julier and Uhlmann, problem with partial state information is that the optimal regulator,with synthesized by information the LQ optimal technique, is 2007) 2007) and weighted sigma points (Julier and Uhlmann, 2004) among others. These general solutions can cope with problem partial state is that the optimal and weighted These sigmageneral points solutions (Julier and Uhlmann, regulator, synthesized by LQ is among can cope regulator, synthesized by the thelinear LQ optimal optimal technique, is 2004) generated from the optimal estimate technique, of the state 2004) among others. others. These general solutions canoutliers cope with with nonlinearities and/orThese with the presence of noise or regulator, synthesized by the LQ optimal technique, is 2004) among others. general solutions can cope with generated from the optimal linear estimate of the state nonlinearities and/or with the presence of noise outliers or generated from the optimal linear estimate of the state (Kwakernaak, 1972). For linear Gaussian systems, the nonlinearities and/or with the presence of noise outliers or unknown parameters (Stojanovic and Nedic, 2016), and generated from the optimal linear estimate of the state nonlinearities and/or with the presence of noise outliers or (Kwakernaak, 1972). parameters (Stojanovic and (Kwakernaak, 1972).is For For linear Gaussian systems, the Kalman Filter (KF) the linear optimalGaussian recursive systems, estimatorthe in unknown unknown parameters (Stojanovic and Nedic, Nedic, 2016), and and they generally have high computational cost. 2016), (Kwakernaak, 1972). For linear Gaussian systems, the unknown parameters (Stojanovic and Nedic, 2016), and Kalman Filter (KF) is the optimal recursive estimator in they generally have high computational cost. Kalman Filter (KF) is the optimal recursive estimator in the minimum mean-square error sense. On the other hand, they generally have high computational cost. Kalman Filter (KF) is the optimal recursive estimator in they have cost.control probFromgenerally the point of high viewcomputational of the optimal the minimum mean-square error On the minimum mean-square error sense. sense. Onisthe the other hand, for linear non-Gaussian systems the KF theother best hand, affine From the point of view of the optimal control the minimum mean-square error sense. On the other hand, From the point of view of the optimal control problem the same issue arises, as it is known sinceprobthe for linear non-Gaussian systems the KF is the best affine for linear non-Gaussian non-Gaussian systemstothe the KF is isestimators the best best affine affine thesame pointissue of view of as the itoptimal control probestimator but it is yet possible develop that From lem the arises, is known since the for linear systems KF the lem the same issue arises, as it is known since the work of Mortensen (1966), that the original incompleteestimator but it is yet possible to develop estimators that estimator but it is yet possible to develop estimators that the same issue(1966), arises,that as the it is knownincompletesince the are more accurate. work of estimator but it is yet possible to develop estimators that lem work of Mortensen Mortensen (1966), that the original original incompleteinformation stochastic optimal control problem with nonare are more more accurate. accurate. work of Mortensen (1966), that the original incompleteinformation stochastic optimal control problem with nonare more accurate. In many important technical areas the widely used Gaus- information information stochastic optimal noises controlisproblem problem with to non-a linearities orstochastic non Gaussian equivalent optimal control with nonIn many important technical areas the widely used Gauslinearities or non Gaussian noises is equivalent to aa In many important technical areas the widely used Gaussianmany assumption cannot be accepted aswidely a realistic statislinearities or non Gaussian noises is equivalent to complete-information but infinite-dimensional one. AlIn important technical areas the used Gauslinearities or non Gaussian noises is equivalent toAla sian assumption cannot be accepted as aa realistic statiscomplete-information but infinite-dimensional one. sian assumption cannot be accepted as realistic statistical description of the random quantities involved. As complete-information but infinite-dimensional one. Although the results in but Charalambous and Elliotone. (1998) sian assumption cannot be accepted as a realistic statiscomplete-information infinite-dimensional Altical description of the random quantities involved. As though the results in Charalambous and Elliot (1998) tical description ofpapers, the random random quantities involved. As though showndescription in various of increasing attention has been thein results results in Charalambous Charalambous andsufficient Elliot (1998) (1998) show that some cases finite-dimensional statistical the quantities involved. As though the in and Elliot shown in various papers, increasing attention has been show that in some cases finite-dimensional sufficient statisshown in various papers, increasing attention has been given to non-Gaussian systems in control engineering (Yaz, show that in some cases finite-dimensional sufficient statistics are available and allow to reduce the original inshown in various papers, increasing attention has (Yaz, been showare thatavailable in some cases finite-dimensional sufficient statisgiven to in engineering and allow to reduce the original ingiven to non-Gaussian non-Gaussian systems in control control engineering (Yaz, tics 1987a,b; Arulampalam,systems 2002; Gordon et al., 1993; Maryak tics are available and allow to reduce the original incomplete information optimal control problem to a finitegiven to non-Gaussian systems in control engineering (Yaz, tics are available and allow control to reduce the original in1987a,b; Arulampalam, 2002; et Maryak finite1987a,b; Arulampalam, 2002; Gordon Gordon et al., al., 1993; 1993; Maryak et al., 2004). In particular, non-Gaussian problems of- complete complete information information optimal optimal control control problem problem to to a finite1987a,b; Arulampalam, 2002; Gordon et al., 1993; Maryak complete information optimal problem to aa finiteet al., 2004). In particular, non-Gaussian problems ofet al., 2004). In particular, particular, non-Gaussian problems oftenal., arise in digital communications when the noise inet 2004). In non-Gaussian problems often arise in digital communications when the noise inten arise in digital communications when the noise inten arise in digital communications when the noise inCopyright © 2017, 2017 IFAC 12314 2405-8963 © IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Copyright © 2017 IFAC 12314 Copyright ©under 2017 responsibility IFAC 12314 Peer review of International Federation of Automatic Control. Copyright © 2017 IFAC 12314 10.1016/j.ifacol.2017.08.1992
Proceedings of the 20th IFAC World Congress Toulouse, France, July 9-14, 2017 S. Battilotti et al. / IFAC PapersOnLine 50-1 (2017) 11808–11813
dimensional and complete-information one, the general case remains challenging. As a consequence, a sensible alternative is to look for sub-optimal and more easily computable solutions. The first step in this direction is to improve the state estimate provided by the KF. Since this estimate is a linear function of the measurements, in the minimum variance framework a natural development is to design filters that make use of quadratic or polynomial functions of the observations to improve the estimation accuracy, at the same time preserving easy computability and recursion. This was first proposed in Verriest (1985), De Santis et al. (1995) by building an augmented system that has as observations both the observations of the original system and their squares. The suboptimal quadratic estimate of the state for the original system is obtained by applying the KF to the augmented system. This approach has been extended in Carravetta et al. (1996) to polynomial estimates. The estimate provided by quadratic or polynomial systems can be exploited in the finite horizon control problem of nonGaussian systems, as described in Germani and Mavelli (1999, 2002); Cacace et al. (2013). The resulting quadratic optimal controller yields better performance in terms of the standard quadratic cost functional with respect to the standard linear optimal controller, and it is obtained by replacing the linear optimal estimate of the KF with the quadratic (or polynomial) optimal one, in virtue of the separation principle proved in Germani and Mavelli (1999). In the case of non asymptotically stable systems this approach has a conceptual drawback. Indeed, the state noise of the augmented systems depends on the state and its variance may become unbounded when the original system is not strictly stable. Thus, the stability of the Quadratic Filter (QF) is guaranteed only for asymptotically stable systems, and the same holds for the controller based on it. In order to enlarge the class of systems that can be processed and controlled by using the QF, in this paper we apply a novel approach called Feedback Quadratic Filter (FQF) which is based on a suitable rewriting of the system model through an output injection term (Cacace et al. (2016)). As it has been shown, the main novelty of this filter is that it can be applied also to non asymptotically stable systems. Moreover, by tuning the gain of the injection term, the FQF attains better performance than the QF in a mean-square error sense. Thus, the FQF provides a new method to process and control non asymptotically stable systems. In this paper we show that the separation result of the QF can be extended to the FQF. This guarantees that the optimal controller based on quadratic estimates remains linear in the state. In other words, the optimal quadratic controller is straightforwardly obtained by using the prediction of the FQF instead of the one of the KF together with the same feedback control law as in the linear optimal regulator (LQG) for the Gaussian case. Notation. Throughout the paper E denotes the expectation. If A and B are two matrices, then A ⊗ B denotes the Kronecker product of A and B, while A[i] is the i-th Kronecker power of A (Bellman, 1970), clearly the same
11809
definitions holds for vectors. Finally, ρ(A) is the spectrum of a square matrix A and In is the identity matrix in Rn×n . 2. PROBLEM STATEMENT The control problem we solve concerns the class of linear, detectable and stabilizable systems driven by nonGaussian additive noise described by the following equations: (1) x(k + 1) = Ax(k) + Bu(k) + fk , x(0) = x0 (2) y(k) = Cx(k) + gk with the associated cost function N 1 T T E x (k)Qx(k) + u (k)Ru(k) . (3) J = lim N →∞ N k=0
As usual, x(k), fk ∈ Rn , u(k) ∈ Rp , y(k), gk ∈ Rq , A ∈ Rn×n , B ∈ Rn×p , C ∈ Rq×n , Q ∈ Rn×n and R ∈ Rp×p . Matrices Q and R are assumed symmetric nonnegative definite (strictly positive definite in the case of R). Moreover, the initial state and the non-Gaussian random sequences {fk } and {gk } satisfy the following conditions for k ≥ 0: x0 ∼ N (¯ x0 , Ψx0 ); {fk } and {gk } are zero mean i.i.d. sequences; {fk }, {gk } and x0 are statistically independent; fk and gk have finite fourth moments; (i) [i] (i) [i] ψf := E[fk ] and ψg := E[gk ], for i = 2, 3, 4, are known vectors; (vi) [C Ψg ], where Ψg = E[gk gkT ], is full row rank.
(i) (ii) (iii) (iv) (v)
(i)
We define similarly for x0 the vectors ψx0 := E[(x0 − x ¯0 )[i] ], known since x0 is Gaussian. In this framework, the infinite-horizon (sub-)optimal control problem for nonGaussian discrete-time linear systems with partial state information is considered. The aim is to find a minimum energy feedback control law, in the class of recursively computable quadratic feedback, that keeps the state of the system close to the state-space origin. It is well known (Kwakernaak, 1972) that, for the infinitehorizon regulator problem (1)–(3), the output feedback optimal control u∗ (k) among all affine transformations of y(0), y(1), . . . , y(k − 1) is given by u∗ (k) = −M x ˆ(k|k − 1), (4) where (5) M = (R + B T P B)−1 B T P A, P is the solution of the Riccati equation P = Q + AT P A − AT P B(R + B T P B)−1 B T P A (6) and x ˆ(k|k − 1) is the prediction of x(k) provided by the KF (Balakrishnan, 1984). We now clarify some issues on polynomial filtering and recursion. 3. QF WITH OUTPUT INJECTION 3.1 The Geometric Approach We introduce some basic notions and issues about polynomial filtering using the geometrical approach. Let (Ω, F, P) be a probability space, G a given sub σ-algebra of F and L2 (G, n) the Hilbert space of the n-dimensional, G measurable random variables with finite second order moment.
12315
Proceedings of the 20th IFAC World Congress 11810 S. Battilotti et al. / IFAC PapersOnLine 50-1 (2017) 11808–11813 Toulouse, France, July 9-14, 2017
We write L2 (X, n) to denote L2 (σ(X), n). We denote with Π [·| M] the orthogonal projection onto a given Hilbert space M.
Given the output sequence Yk = col(y(0), · · · , y(k)) and the auxiliary vector Yk = col(1, Yk ) ∈ Rl+1 , with l = (k + 1)q, the minimum variance estimate of the state x(k) of system (1)–(2), can be defined as the orthogonal projection of x(k) onto the Hilbert space L2 (Yk , n): x ˆ(k) = E [x(k)| σ(Yk )] = Π x(k)| L2 (Yk , n) . (7) If the sequences {x(k)} and {y(k)} are jointly Gaussian, this projection is equivalent to the projection on the subspace Lky ⊂ L2 (Yk , n) of all affine functions of Yk , Lky = {z : Ω → Rn : ∃T ∈ Rn×l+1 : z = T Yk }.
(8)
Π[x(k)|Lky ]
The KF recursively computes the projection which is the best affine estimate of x(k) in the minimum variance sense. In the Gaussian case, this coincides with E [x(k)| σ(Yk )].When {x(k)} and/or {y(k)} are nonGaussian, the computation of (7) is highly difficult. Since the best affine estimate is obtained by projecting onto Lky , better suboptimal estimates can be obtained by projecting the state x(k) onto larger sub-spaces. For example, we may consider the space of second order polynomial (quadratic) transformations of Yk , here denoted by Qky . Let Yk ˜ (2) l ˜l = 1 + l + l2 , Yk = (9) [2] ∈ R , Yk then ˜ (2) Qky = {z : Ω → Rn : ∃T ∈ Rn×l : z = T Yk }. (10) Remark 1. Since Lky ⊂ Qky ⊂ L2 (Yk , n), it follows from the definition of the norm in L2 (X, n), v2L2 (X,n) := T v vdP = E[v T v], that projecting the state onto Qky (a Ω larger subspace) will return an estimate, having an error variance equal or smaller than that of the affine estimate, that we may well name the optimal quadratic estimate.
However, the computation of Π[x(k)|Qky ] would require a growing filter size, due to the presence in Qky of terms of the kind y i (k1 )y j (k2 ), with i, j ≤ 0, 1, 2. Thus, we replace (2) Yk with Yk[2] y(0) (2) ¯ Y k = . ∈ Rl , ¯l = 1 + l + (k + 1)q 2 , (11) . . y(k)[2]
that is, a vector containing all the observations and their Kronecker squares from time 0 to k. We obtain the projection subspace k
¯
(2)
Qy = {z : Ω → Rn : ∃T ∈ Rn×l : z = T Y k },
(12)
k Qy
with Lky ⊂ ⊂ Qky . Thus the resulting estimate will not be the optimal quadratic estimate, but it will still have an error not larger than the best linear one. Since this projection is recursively computable we shall refer to it as recursive quadratic estimate. As a consequence, we shall refer to the optimal recursive quadratic control when the recursive quadratic prediction,
k−1
i.e. Π[x(k)|Qy control law.
], of the state is used to generate the
3.2 Output Injection As in Cacace et al. (2016), the state equation (1) is transformed using the output equation (2): x(k + 1) = Ax(k) + Bu(k) + fk = Ax(k) + Bu(k) + fk + Ly(k) − LCx(k) − Lgk ˜ = Ax(k) + Bu(k) + Ly(k) + hk , (13) where A˜ = A − LC, L ∈ Rn×q is an arbitrarily chosen (i) [i] matrix and hk = fk − Lgk . Moreover, ψh = E hk , (i)
i = 2, 3, 4, can be computed as a function of ψf (i) ψg
and
as follows (2)
(2)
(14)
(3)
(3)
(15)
ψh =ψf + L[2] ψg(2) , ψh =ψf − L[3] ψg(3) , (4) (4) (2) ψh =ψf + M24 ψf ⊗ L[2] ψg(2) + L[4] ψg(4) ,
(16)
M24
where is the coefficient matrix for the expansion of the binomial Kronecker power (see Carravetta et al. (1996)). As the second step, taking advantage of linearity, the state sequence is split into two sequences, {xd (k)} and {xs (k)}. The deterministic component xd (k) is the solution of ˜ d (k) + Bu(k) + Ly(k), xd (0) = xd , (17) xd (k + 1) = Ax 0 with xd0 = E[x0 ] = x ¯0 , i.e. k−1 (18) A˜k−τ −1 (Bu(τ ) + Ly(τ )) , xd (k) = A˜k xd0 + τ =0
which can be computed at time k, since the deterministic initial value xd0 and the output sequence Yk−1 are available. The stochastic component xs (k) is the solution of ˜ s (k) + hk , xs (0) = xs , xs (k + 1) = Ax (19) 0 (i) s s [i] with x0 ∼ N (0, Ψx0 ), and, therefore ψxs = E x0 = 0
(i) ψ x0 .
From (17) and (19) trivially follows that x(k) = xd (k) + xs (k) ∀k ≥ 0. (20) The output map (2) is arranged as ys (k) = y(k) − Cxd (k) = Cxs (k) + gk , (21) where ys (k) is an available quantity at time k. We can also define the corresponding sequence vector Ys,k := col(ys (0), ys (1), . . . , ys (k)) (22) and the auxiliary vector Ys,k = col(1, Ys,k ). Moreover, (2)
k
also vector Y s,k and spaces Lkys , Qkys , and Qys can be defined using Ys,k instead of Yk in (11), (8), (10), and (12), respectively. In Cacace et al. (2016) it is proved that Ys,k is an affine transformation of the original sequence vector Yk and viceversa. Thus, Lky ≡ Lkys . The same fact holds for the quadratic transformations, thus Qky ≡ Qkys . However this is not true for the recursive quadratic transformations, k k i.e. Qy ≡ Qys (Cacace et al., 2016). Since (18) is an affine transformation of Yk , the projection k of xd (k) onto Qys trivially corresponds to itself. The opk
timal recursive quadratic estimate x ˜s (k) = Π(xs (k)|Qys )
12316
Proceedings of the 20th IFAC World Congress Toulouse, France, July 9-14, 2017 S. Battilotti et al. / IFAC PapersOnLine 50-1 (2017) 11808–11813
can be obtained by processing the system (19)-(21). Analogously, we can obtain the optimal recursive quadratic k−1 prediction x ˜s (k|k − 1) = Π(xs (k)|Qys ). 4. SEPARATION PRINCIPLE AND OPTIMAL RECURSIVE QUADRATIC CONTROL We prove in this section that the structure of the controller of the LQG regulator (4)–(6) remains optimal in the class of quadratic feedback control laws. To this end, let us introduce the extended state and output (2) , (23) Xe (k) = col xd (k), xs (k), x[2] (k), ψ s h Ye (k) = col y(k), y˜s[2] (k) , (24) [2]
where xd (k) and xs (k) are defined in (17)–(19), and xs [2] and y˜s satisfy (2) (2) x[2] (k + 1) = A˜[2] x[2] (k) + ψ + h , (25) s
s
h (2)
k
y˜s[2] := ys[2] (k) − ψg(2) = C [2] x[2] s (k) + gk , where (2) ˜ s (k) ⊗ hk + hk ⊗ Ax ˜ s (k) + h[2] − ψ (2) , hk = Ax k h (2) gk
[2] gk
(26) (27)
ψg(2)
(28) = Cxs (k) ⊗ gk + gk ⊗ Cxs (k) + − are zero-mean, mutually correlated white noise sequences uncorrelated with the initial state x0 . The dynamics of the extended system is (29) Xe (k + 1) = AXe (k) + Bu(k) + Wk , (30) Ye (k) = CXe (k) + Vk , where A LC 0 0 Lgk B hk 0 A˜ 0 0 0 A= 0 0 A˜[2] I , B = 0 , Wk = h(2) , k 0 0 0 0 0 I C=
C C 0 0 , 0 0 C [2] 0
Vk =
where M is defined in (5). Then, the optimal recursive quadratic control for (1) is ˆ˜(k|k − 1) . u∗ (k) = φ x (34)
ˆ˜(k|k − 1) = xd (k) + x ˜s (k|k − 1) is obtained as the where x sum of the known term xd (k) obtained from (18) and of the optimal quadratic recursive prediction x ˜s (k|k − 1).
Proof. The optimal recursive quadratic control for the regulator problem (1)–(3) coincides with the LQG solution for the extended system (29)–(31), i.e. e (k|k − 1), (35) u∗ (k) = − (R + B T PB)−1 B T PA X
where P is given by P = Q + AT PA − AT PB(R + B T PB)−1 B T PA (36) e (k|k − 1) is the optimal linear prediction of Xe (k). and X It is easy to verify, by expanding the terms of (36), that P P 0 0 P P 0 0 P= , (37) 0 0 0 0 0 0 0 0
where P is the solution of the Riccati equation (6) associated to (1)–(3), solves (36). The resulting control law is e (k|k − 1)1 + X e (k|k − 1)2 ), u∗ (k) = −M (X (38) e (k|k −1)1 and X e (k|k −1)2 are the optimal linear where X predictions of the first two components of Xe (k), that is xd (k) and xs (k). The first component is deterministic e (k|k − 1)1 = xd (k). The second and known, hence X component can be obtained by the optimal prediction of the stochastic part of (29)–(30), i.e.
gk (2) . gk
Let Je be the cost function associated to the extended state dynamics (29) N 1 E XeT (k)QXe (k) + uT (k)Ru(k) , Je = lim N →∞ N k=0 (31) where Q Q00 Q Q 0 0 Q= . (32) 0 0 0 0 0 0 00 Clearly, Je ≡ J. Remark 2. Since we would like to perform an estimate of the state as a quadratic transformation of the output map, we need to consder the evolution of the second Kronecker power of the state, third row of system (29). Moreover, we (2) extend the state with a forth constant component ψh in order to have the system in the classical form (29) driven by a zero mean noise. Theorem 1. Let φ : Rn → Rp be the map φ(ξ) = −M ξ, (33)
11811
hk 0 xs (k + 1) xs (k) A˜ 0 = + (2) + (2) x[2] ψh hk 0 A˜[2] x[2] s (k + 1) s (k) (39) gk ys (k) C 0 xs (k) = + (2) . (40) (k) y˜s[2] (k) 0 C [2] x[2] g s k
e (k|k − 1)2 is a projection on the space of the Since X e (k|k − quadratic transformations of ys (0), . . . , ys (k −1), X k−1 1)2 = Π(xs (k)|Qys ) = x ˜s (k|k − 1). Remark 3. Note that, even though the optimal recursive quadratic control (35) depends a priori on the prediction [2] e (k|k − 1), Theorem 1 shows that it only of xs in X depends on xd (k) and on the prediction x ˜s (k|k − 1) = k−1 Π(xs (k)|Qys ). Remark 4. Note that the map φ defined in (33) is invariant with respect to output injection whereas u∗ of (34) ˆ depends on the output injection gain L through x ˜. k−1
The prediction x ˜s (k|k − 1) = Π(xs (k)|Qys ) is obtained by applying the KF algorithm to the system (39)–(40). It is easy to prove that the state and measurement noise of this system are zero mean, temporally uncorrelated and mutually correlated sequences. Since (A, C) is detectable, we can choose L to have A˜ Schur stable. In this case system (39)–(40) is detectable. Moreover, if A˜ is Schur
12317
Proceedings of the 20th IFAC World Congress 11812 S. Battilotti et al. / IFAC PapersOnLine 50-1 (2017) 11808–11813 Toulouse, France, July 9-14, 2017
stable then the state, output and noises in (39)–(40) are all second-order asymptotically stationary processes. In these conditions the KF for correlated state and measurement noises applied to (39)–(40) is asymptotically stationary ˆ˜(k) = and internally asymptotically stable. The estimate x ˜s (k) is quadratic and recursively computable and xd (k) + x it is called the FQF of (1)–(2).
MSE KF
10
MSE
2
KF
F
6
600 200 500 150 400
30
mean(JKF)
50
Pq∞
JKF
MSE q
mean(JFQF) J
40
mean(MSE q)
FQF
100
00
10 10
20 20
30 30
40 40
50 50
Fig. 1. MSE and cost function J for Example 1. 40
α FQF mean(α FQF)
30 20 10 0
0
10
20
30
40
50
Fig. 2. Performance index for Example 1. gk ∈ R.
(44)
The probability densities of the state and output noise sequences are shown in Table 1, in which σ ¯ = (11/2)1/2 . state noise
output noise 0 9/20
20
50 200
Example 1 We consider an unstable system of the form with 1.94 −0.46 0 A= , B= , C = [1 0] (43) 0 101.68 0.18 20 30 1 40 50
1/¯ σ 1/2
10
100 300
50(1)–(2)
-10/¯ σ 1/20
2 0
KF
where u∗ (k) is the control law defined in (4) and N is the number of sampling points. Analogous definitions for J QF F QF ∗ for ) which the control law u (k) is generated by 200and Jmean(JKF KF using Ja QF and the FQF, respectively. Clearly, we initialize FQF mean(Jwith ) the same initial conditions which are not of the filters FQF 150relevance J with regard to the results. Both examples have been simulated with a time horizon N = 150 across 50 100realizations.
(i)
3
2
k=0
fk (i) p[(fk )]
4
4
α = 10 · (J − J )/J , (41) FQF ) wheremean(MSE J KF , J QF and J F QF are the cost functions asso8 ciated to a single realization of the controlled dynamics when the predictions of KF, QF and FQF are used by 6the controller, respectively. Therefore, given a probability space (Ω, F, P) on which are defined the stochastic noise 4sequences and the initial condition x , for a fixed ω ∈ Ω, 0 N 1 T 2 x (k)Qx(k) + u∗ T (k)Ru∗ (k), (42) J KF = 0 10 N 20 30 40 50
and the state and output noises (1) fk fk = (2) ∈ R2 , fk
5
mean(MSE FQF)
8
We now test the effectiveness of the proposed approach with two numerical examples. We consider also non asymptotically stable system since they can be processed by the FQF. We define a relative performance index αF , MSE KF F ∈ {QF, F QF }, as mean(MSE KF) F
MSE FQF
10
5. NUMERICAL EXAMPLES
FQF
6
mean(MSE KF)
gk p[(gk )]
-1 0.5
5 0.1
0 0.4
Table 1. Probability mass functions of the noise sequences of Example 1. The cost function is defined by Q = I2 and R = 0.2. Since the spectrum of the matrix A is ρ(A) = {1.1, 1.02}, we use output injection and we choose the gain L that ˜ Figure 1 (top) plots assigns the spectrum {0.4, 0.45} to A. the MSE and the average of MSE of the state estimation across realizations for the KF and the FQF. Figure 1 (bottom) plots J KF and J F QF and their average values. Figure 2 plots the relative performance index αF QF across realizations and its average value 18.6%.
Example 2 In this second example we show the dependence of the control (34) on the output injection gain L. To compare the result with the QF we consider a stable MIMO system of the form (1)–(2) defined by 0.6 0 1 0 0.5 1 1000 0 −0.4 1 1 0 0.4 A= , B= , C= 0 0 0.8 0 0.2 1 0100 0 0 0 0.9 0 1 (45) and the state and output noises (1) (2) (3) (4) fk = col fk , fk , fk , fk ∈ R4 , (46) (1) (2) gk = col gk , gk ∈ R2 .
The components of {fk } and {gk } are i.i.d. sequences and their probability densities are shown in Table 2 in which σ ¯ = (11/2)1/2 .
The cost function is characterized by the matrices Q = I4 and R = 0.2 · I2 . Since ρ(A) = {0.6, −0.4, 0.8, 0.9} we can control the system by using a standard QF without output
12318
1
Proceedings of the 20th IFAC World Congress Toulouse, France, July 9-14, 2017 S. Battilotti et al. / IFAC PapersOnLine 50-1 (2017) 11808–11813
state noise
600
(i) fk
500
300
1/¯ σ 1/2
0 9/20
-11.5 0.1
(i)
p[(g MSEkq )]
1.5 0.7
0.5 0.2
q
injection. We make a comparison with an optimized FQF where the output injection gain Lopt is such that Lopt = argmin|λ|<1 ∀λ∈ρ(A) (47) ˜ trace S∞ (L),
200 0
40 30 20 10 0
-10/¯ σ 1/20
mean(MSE ) Table 2. Probability mass functions of the noise sequences of Example 2.
400
100
(i)
p[(fk )]
output noise (i) gPkq ∞
0
covariance where S∞ is20 the steady-state 10 30 40 50 matrix of the estimation error. Note that in this example the pair (A, C) is observable, thus the minimization over the whole spectrum of A˜ makes sense. In Table 3 the first column α FQF is the percentage reduction of the MSE of the state FQF mean(α ) estimation with respect to the KF. The second column presents the average values of J KF , J QF and J F QF , and the third column shows the values of the relative performance index α in (41). It is evident that, even in the case of stable systems, the proposed approach improves significantly the performance.
10
Filter KF 20QF FQF
% MSE -21.45% 30 -33.01%
J 94.87 85.27 40 79.60
α -10.17% 50 -15.67%
Table 3. MSE and cost function. 6. CONCLUSION The optimal quadratic infinite-horizon regulator for linear stochastic non asymptotically stable non-Gaussian systems is obtained by a recursive algorithm. It has been proved that the optimal quadratic regulator is obtained by replacing the Kalman Filter prediction with the one of the Feedback Quadratic Filter and by using the same feedback control law as in the linear optimal regulator. We have explicitly proved the validity of the separation principle for this case. The increase of the computational cost of the proposed algorithm with respect to the linear one is moderate. Numerical simulations show the effectiveness of this method with respect to standard optimal linear control and also with respect to the standard Quadratic Filter. We finally stress the fact that by using a polynomial filter it is possible to further improve the performance. REFERENCES I. Arasaratnam, S. Haykin and R. J. Elliott. Discrete-Time Nonlinear Filtering Algorithms Using Gauss-Hermite Quadrature. Proc. of the IEEE, 95(5), 953–977, 2007. M. S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp. A tutorial on particle filters for online nonlinear/nonGaussian Bayesian tracking. IEEE Trans. on Sign. Proc. 50(2), 174–188, 2002. A.V. Balakrishnan. Kalman Filtering Theory. Optimization Software, Inc., New York, 1984. R. R. Bellman. Introduction to Matrix Analysis. McGrawHill. 1970. F. Cacace, A. Fasano, A. Germani, and A. Monteriu. Optimal reduced-order quadratic solution for the nonGaussian finite-horizon regulator problem. Proc. of the
11813
52nd IEEE Conf. on Decision and Control, CDC 2013, 3085–3090, 2013. F. Cacace, F. Conte. A. Germani, and G. Palombo. Feedback Quadratic Filtering. (accepted for publication in Automatica), 2016. F. Carravetta, A. Germani, and M. Raimondi. Polynomial filtering for linear discrete time non-Gaussian systems. SIAM J. on Control and Optim., 34 (5), 1666-1690, 1996. C. Charalambous and R.J. Elliot. Classes of Nonlinear Partially Observable Stochastic Optimal Control Problems with Explicit Optimal Control Laws. SIAM J. on Control and Optim., 36 (2), 542–578, 1998. A. De Santis, A. Germani, and M. Raimondi. Optimal quadratic filtering of linear discrete-time non-Gaussian systems. IEEE Trans on Autom. Contr., 40 (7), 1274– 1278, 1995. A. Germani and G. Mavelli. Optimal Quadratic Solution for the non-Gaussian Finite-Horizon Regulator Problem. Systems & Control Letters, 38, 321–331, 1999. A. Germani and G. Mavelli. The Polynomial Approach to the LQ non-Gaussian Regulator Problem. IEEE Trans on Autom. Contr., vol. 47 (8), 1385–1391, 2002. N.J. Gordon, D.J. Salmond, and A.F.M. Smith. Novel approach to nonlinear/non-Gaussian Bayesian state estimation. Proc. of the Institution of Electrical Engineers F140 (2), 107–113, 1993. S. J. Julier and J. K. Uhlmann. Unscented Filtering and Nonlinear Estimation. Proceedings of the IEEE, 92(3), 401–422, 2004. H. Kwakernaak and R. Sivan Linear optimal control systems. Vol. 1. New York: Wiley-interscience J. Maryak, J.C. Spall, and B. Heydon. Use of the Kalman filter for inference in state-space models with unknown noise distributions. IEEE Trans on Autom. Contr., 49 (1), 87–90, 2004. R. E. Mortensen. Stochastic Optimal Control with Noisy Observations. Int. Jour. of Control, 4(5), 455–464, 1966. S.S. Rappaport and L. Kurtz. An optimal nonlinear detector for digital data transmission through nonGaussian channels. IEEE Trans on Commun. Technol., 3, 266–274, 1966. V. Stojanovic, and N. Nedic. Robust Kalman filtering for nonlinear multivariable stochastic systems in the presence of non-Gaussian noise. Int. Jour. of Robust and Nonlinear Control, 26, 445–460, 2016. E. I. Verriest. Linear filters for linear systems with multiplicative noise and nonlinear filters for linear systems with non-Gaussian noise. In: IEEE American Contr. Conf., 182–184, 1985. E. Yaz. Relationship between several novel control schemes proposed for a class of nonlinear stochastic systems. Internat. J. Control 45, 1447–1454, 1987. E. Yaz. A control scheme for a class of discrete nonlinear stochastic systems. IEEE Trans on Autom. Contr., 32 (1), 77–80, 1987. M. Zakai. On the optimal filtering of diffusion processes. Probab. Theory Related Fields, 11 (3)3, 230–243, 1969.
12319