- Email: [email protected]
Simultaneous Optimal Identification and Tracking of Stochastic Linear Uncertain Systems
3b-064
Copyright © 1996 IFAC 13th Triennial World Congress, San Francisco, USA
Simultaneous Optimal Identification and Tracking of Stochastic Linear Uncertain Systems Han Rusnak, RAFAEL (88), P.O.Box 2250, Haifa 31021, Israel.
AllonGuez ECE Department, Drexel University, Philadelphia, PA 19104.
Abstract: The problem of optimal simultaneous identification and tracking of stochastic uncertain linear systems on finite time interval is fonnulated and solved. The control objective is minimization of a quadratic tracking criterion. The identification criterion is maximization of the Fisher information matrix. The combined objective is the simultaneous minimization of the quadratic tracking criterion and maximization of the infonnation matrix. The inherent conflict between tracking and identification, as they are competing for the only available resource, the input to the plant, is posed and solved. Keywords: Identification. Optimal control, Uncertain systems. Stochastic systems.
1. The Conflict between Identification and Tracking In has been recognized that estimation-identification and tracking-control are conflicting [Guez, Polderman, Koussoulass, Rusnak 92a, Rusnak 92b]. That is, the objective of the control is to minimize the tracking error which is usually the input to the plant. However, if the input to the plant is zero, or if it is constant, one can not explore the plant and identify its parameters. In other words, only after realizing this basic conflict of competition, on the only available resource, the input to the plant between the two objectives, the tracking and identification, one can fonnulate a problem that will give a framework and attempt to solve it by a compromise. In the current literature several approaches are proposed for providing on line estimation, such as application of dither, dual control [Feldbaum], switching of controllers [Feldbaum, Ljung]. These references lack a discussion on how much is "paid" for the estimation. in tenns of tracking error and no rigorous rationalization was supplied for the application of these heuristics. Recently, the issue of joint design of identification and control is emerging [Guez. Rusnak 92b, Rusnak 92c, Gevers]. It is realized that the only justification for identification is achieving better control perfonnance.
the employment of Multiple Objective Optimization Theory (MOOP) [Koussoulass, Khargonekar) to replace the heuristics (e.g. dual control, dither, persistent excitation, etc.), used in previous treatments of control of uncertain systems. The use of the Multiple Objective Optimization theory advises how to combine several conflicting objectives. The formulation of the problem and the technique of solution immediately provide the cost-price paid to perfonn each of the objectives, thus enabling the required trade-off between these criteria. A theory of multiple objective optimization approach to adaptive control for general nonlinear system is presented in [Guez]. Application of this methodology and a suboptimal approach to discrete linear time-invariant system is in [Rusnak 92c). Application of a suboptimal approach to online identification and control of linearized aircraft dynamics can be found in [Rusnak 92b). Preliminary results with the MOOP approach were presented in [Rusnak 93a, Rusnak 93b]. The problem of simultaneous identification and control-regulation has been presented and solved in [Rusnak: 94], i.e. the control objective is the regulator objective. In contrast, here we apply the same approach with tracking objective. i.e. the output of the plant has to track a trajectory ,
3. Problem Statement 2. MOOP Approach to Control of Uncertain Systems The observations made in section I immediately suggest
In this section we pose a tracking problem of stochastic linear uncertain time-invariant systems. We consider the nth order continuous stochastic linear time-invariant singleinput single-output system
5138
5139
5. Simultaneous Optimal State Estimation and Parameters Identification The states and parameters observability form immediately suggests the use of the well known theory of observers for linear time-variant systems. The optimal estimator of the augmented state is [Kwakemaak]
Remarks: 1) The result in theorem 5.1 means that the dynamic equation ~(t) = [ Ap(t) - Kp(t) cp] ~(t), ~(to) = ~'O'
(5.2)
is exponentially stable. 2) The results in theorem 5.1 are sufficient conditions. 3) It is proved in [Rusnak 92d] that uniform complete observability of (Ap(t),cp) is almost always equivalent to persistency of excitation.
6. Optimal Inputs for Identification This is the "linear" optimal estimator, by what we mean that given (yp(t), up(t), O:$;t:$;t}, there exist no other algorithm of the structure (5.1) that derives better mean square estimate of the augmented state. These equations are easily solved since up to the current time t, Ap(t), bp and cp are known and the integration goes forward. The issue of how the selection of the input to the system, up(t), affects the quality of the estimation is dealt with in [Rusnak 92d, Rusnak 93c]. For example the quality that we may look is the convergence of the estimation error, namely, asymptotic, exponential or in Lyapunov sense convergence, the magnitude of the estimation error covariance, etc. The state observer and parameters identifier that is discussed in this section is the "linear" optimal estimator. However, it is emphasized here that this optimality is with respect to a very specific experiment. This experiment is that the input driving noise and the input and output measurement noises are white stochastic processes, and the true parameters of the system in the observer canonical form are random variables. Namely, the presented optimal state observer and parameters identifier, gives the smallest second order statistics on the average when the average is taken with respect to all systems within the distribution of the parameters, all initial conditions and all noises. This means that when dealing with a specific system which has constant parameters, although unknown, this approach does not necessarily lead to better performance with respect to other identification algorithms. The following presents a result on the performance of the optimal estimator-identifier. Theorem 5.1: If the input to the plant up(t) is such that (Ap(t),cp) is uniformly completely observable, V pI is T 1/2 such that (A p (t), V pI) is uniformly completely controllable and Vp 2>O, then the optimal observer (4.13) is exponentially stable, and the minimal mean square estimation error is Qp(t). Proof: Direct outcome of[Kwakemaak, thm 4.10]. Q.E.D.
In this section we consider design of the input such that the observability condition is taken care off with respect to an optimality objective. The optimal input is based on the maximization of the Fisher information matrix. This section follows closely the approach by [Mehra 74a, 74b]. We deal with the linear single-input single-output stochastic system (3.1). The problem of finding the "optimal" input for identification, as formulated in [Mehra 74a], is to selecl the input (up(t), 0 :$; t :$; tl} to maximize a norm of the Fisher Information matrix, M, subject to a constraint. The Fisher Information matrix M for the unknown parameters 9p is shown to be [Nahi] 11
M = E{
f
V 9YpT
V~12 V9Yp dt }
la where
(6.1)
where V9Yp is a the gradient of the estimate of yp(t) with respect to the unknown parameters, 9 p , V 9Yp =
[~~ CJe!'CJei ...... ]E Rlx2n • T0
d enve . a scal ar measure 0 f the
Fisher information matrix there are several matrix norms that can be used [Mehra 74a]. In this work we use the weighted trace norm: Je = trace(SM)
(6.2)
where S>O is a weighting matrix. We further assume, for simplicity, that S is diagonal, so that the scalar objective,
5140
5141
5142
objective creates a completion to the plant's input, up(t), so as to enhance the identification problem, and it trades-off the tracking quality with the identification quality. This completion depends on the desired trajectory. That is, there are "bad" trajectories from identification point of view, not sufficiently rich, that need lot of completion. On the other extreme, there are trajectories that need very little or no completion. Conclusions The problem of optimal simultaneous identification and tracking of stochastic uncertain linear systems on finite time interval has been formulated and solved. The combined objective is the simultaneous minimization of the quadratic tracking criterion and maximization of the information matrix. The solution is explicit and non causal. It suggests the structure of the causal approximation and clearly states the tradeoff between the tracking and identification costs. References [Boniventol Bonivento, c.:"Structural Insensitivity versus Identifiability," IEEE Trans. on Autom. Cont., AC-18, No. 2, April 1973, pp. 190-193. [Bryson] Bryson, A.E. and Ho, Y.c.: Applied Optimal Control, Hemisphere Publishing Company, 1975. [Feldbaum] Feldbaum, A. A.: Optimal Control Systems, Academic Press Inc., New York, 1965. [Gevers] Gevers, M.:"Towards a Joint Design of Identification and Control?," chapter 5 in Trentelman, H.L. and Willems, J .C., Editors:" Essays on Control: Perspectives in the theory and its Applications," BirkMuser Boston 1993. [Goodwin] Goodwin, G. C. and Payne, R. L.:"Dynamic System Identification: Experiment Design and Data Analysis," Academic Press, New York, 1977. [Guez] Guez, A., Rusnak, I. and BarKana, I.:" Multiple Objective Optimization Approach to Adaptive Learning Control," International Journal of Control, Aug. 1992. [Khargonekar] Khargonekar, P. P. and Rotea, M. A.:"Multiple Objective Optimal Control of Linear Systems: The Quadratic Norm Case," IEEE Trans. on Autom. Contr., Vol. 36, No. 1, January, 1991. [Koussoulass] Koussoulass, N. T. and Dimitriadis, E. K.:" Computational Techniques for Multicriteria Stochastic Optimization and Control," Leondes C. T. Edit., Control and Dynamic Systems, Advances in Theory and Applications, Vol. 30, part 3, Academic Press, 1989. [Kwakemaak] Kwakernaak, and Sivan. R.: Linear Optimal Control, New York: Wiley, 1972. [Ljung] Ljung, L.:System Identification. Prentice-Hall, Inc., 1977. [Mehra 74a] Mehra, R. K.:"Optimal Inputs for Linear Systems Identification," IEEE Trans. on Automatic Control, Vol. AC-19, No. 3, June 1974. [Mehra 74b] Mehra, R. K.:"Optimal Input Signals for
Parameter Estimation in Dynamic Systems - Survey and New Results," IEEE Trans. on Automatic Control, Vol. AC-19, No. 6, December 1974. [Nahi] Nahi, N. E. and Wallis, D. E. Jr.:"Optimal Inputs for Parameter Estimation in Dynamic Systems with White Observation Noise," Joint Automatic Control Conference, Boulder, Colorado, 1969. [Polderman] Polderman, J. W.:"Adaptive Control and Identification: Conflict or Conflux," Stichting Mathematisch Centrum, Amsterdam, Netherlands, 1989. [Rusnak 92a] Rusnak, I., Guez, A. and BarKana, I.: "Multiple Objective Approach to Adaptive and Learning Control of Discrete Linear Time-Invariant Systems," 7-th Workshop on Adaptive and Learning Control, Yale University, May 20-22, 1992. [Rusnak 92b] Rusnak, I., Steinberg, M., Guez, A. and BarKana, I. :"On-Line Identification and Control of Linearized Aircraft Dynamics," IEEE AES Magazine, July 1992. [Rusnak 92c] Rusnak, I., Guez, A. and BarKana, I.:"Multiple Objective Approach to Adaptive and Learning Control of Discrete Linear Time-Invariant Systems," 7-th Workshop on Adaptive and Learning Control, Yale University, May 20-22, 1992. [Rusnak 92d] Rusnak, I., Guez, A. and BarKana, I.:"Necessary and Sufficient Conditions on the Observability and Identifiability of Linear Systems," ACC 1992, American Control Conference, Chicago, Illinois, June 24-26, 1992. [Rusnak 93a] Rusnak, I., Guez, A. and BarKana, I. :"Control of Stochastic Linear Uncertain Systems by Multiple Objective Optimization," 1993 IEEE Regional Conference on Aerospace Control Systems, CACS 93, Thousand Oaks, Los Angeles, May 25-27,1993. [Rusnak 93b] Rusnak, I., Guez, A. and BarKana, I. :"Multiple Objective Optimization Approach to Adaptive Control of Linear Systems," American Control Conference, ACC 93, June 2-5, 1993, San Francisco. [Rusnak 93c] Rusnak, I.:" Optimal Adaptive Control of Uncertain Stochastic Linear Systems," Ph.D. thesis, Drexel University, June 1993. [Rusnak 93d] Rusnak, I., Guez, A. and BarKana, I. :"State Observability and Parameters Identifiability of Stochastic Linear Systems," 1993 IEEE Regional Conference on Aerospace Control Systems, CACS 93, Thousand Oaks, Los Angeles, May 25-27,1993. [Rusnak 94] Rusnak, I. and Guez, A.:":Simultaneous Optimal Identification and Control of Stochastic Linear Uncertain Systems." The 2nd IEEE Symposium on New Directions in Control and Automation, Chania, Crete, 1922 Jun, 1994, Greece.
5143
Copyright © 1996 IFAC 13th Triennial World Congress, San Francisco, USA
Simultaneous Optimal Identification and Tracking of Stochastic Linear Uncertain Systems Han Rusnak, RAFAEL (88), P.O.Box 2250, Haifa 31021, Israel.
AllonGuez ECE Department, Drexel University, Philadelphia, PA 19104.
Abstract: The problem of optimal simultaneous identification and tracking of stochastic uncertain linear systems on finite time interval is fonnulated and solved. The control objective is minimization of a quadratic tracking criterion. The identification criterion is maximization of the Fisher information matrix. The combined objective is the simultaneous minimization of the quadratic tracking criterion and maximization of the infonnation matrix. The inherent conflict between tracking and identification, as they are competing for the only available resource, the input to the plant, is posed and solved. Keywords: Identification. Optimal control, Uncertain systems. Stochastic systems.
1. The Conflict between Identification and Tracking In has been recognized that estimation-identification and tracking-control are conflicting [Guez, Polderman, Koussoulass, Rusnak 92a, Rusnak 92b]. That is, the objective of the control is to minimize the tracking error which is usually the input to the plant. However, if the input to the plant is zero, or if it is constant, one can not explore the plant and identify its parameters. In other words, only after realizing this basic conflict of competition, on the only available resource, the input to the plant between the two objectives, the tracking and identification, one can fonnulate a problem that will give a framework and attempt to solve it by a compromise. In the current literature several approaches are proposed for providing on line estimation, such as application of dither, dual control [Feldbaum], switching of controllers [Feldbaum, Ljung]. These references lack a discussion on how much is "paid" for the estimation. in tenns of tracking error and no rigorous rationalization was supplied for the application of these heuristics. Recently, the issue of joint design of identification and control is emerging [Guez. Rusnak 92b, Rusnak 92c, Gevers]. It is realized that the only justification for identification is achieving better control perfonnance.
the employment of Multiple Objective Optimization Theory (MOOP) [Koussoulass, Khargonekar) to replace the heuristics (e.g. dual control, dither, persistent excitation, etc.), used in previous treatments of control of uncertain systems. The use of the Multiple Objective Optimization theory advises how to combine several conflicting objectives. The formulation of the problem and the technique of solution immediately provide the cost-price paid to perfonn each of the objectives, thus enabling the required trade-off between these criteria. A theory of multiple objective optimization approach to adaptive control for general nonlinear system is presented in [Guez]. Application of this methodology and a suboptimal approach to discrete linear time-invariant system is in [Rusnak 92c). Application of a suboptimal approach to online identification and control of linearized aircraft dynamics can be found in [Rusnak 92b). Preliminary results with the MOOP approach were presented in [Rusnak 93a, Rusnak 93b]. The problem of simultaneous identification and control-regulation has been presented and solved in [Rusnak: 94], i.e. the control objective is the regulator objective. In contrast, here we apply the same approach with tracking objective. i.e. the output of the plant has to track a trajectory ,
3. Problem Statement 2. MOOP Approach to Control of Uncertain Systems The observations made in section I immediately suggest
In this section we pose a tracking problem of stochastic linear uncertain time-invariant systems. We consider the nth order continuous stochastic linear time-invariant singleinput single-output system
5138
5139
5. Simultaneous Optimal State Estimation and Parameters Identification The states and parameters observability form immediately suggests the use of the well known theory of observers for linear time-variant systems. The optimal estimator of the augmented state is [Kwakemaak]
Remarks: 1) The result in theorem 5.1 means that the dynamic equation ~(t) = [ Ap(t) - Kp(t) cp] ~(t), ~(to) = ~'O'
(5.2)
is exponentially stable. 2) The results in theorem 5.1 are sufficient conditions. 3) It is proved in [Rusnak 92d] that uniform complete observability of (Ap(t),cp) is almost always equivalent to persistency of excitation.
6. Optimal Inputs for Identification This is the "linear" optimal estimator, by what we mean that given (yp(t), up(t), O:$;t:$;t}, there exist no other algorithm of the structure (5.1) that derives better mean square estimate of the augmented state. These equations are easily solved since up to the current time t, Ap(t), bp and cp are known and the integration goes forward. The issue of how the selection of the input to the system, up(t), affects the quality of the estimation is dealt with in [Rusnak 92d, Rusnak 93c]. For example the quality that we may look is the convergence of the estimation error, namely, asymptotic, exponential or in Lyapunov sense convergence, the magnitude of the estimation error covariance, etc. The state observer and parameters identifier that is discussed in this section is the "linear" optimal estimator. However, it is emphasized here that this optimality is with respect to a very specific experiment. This experiment is that the input driving noise and the input and output measurement noises are white stochastic processes, and the true parameters of the system in the observer canonical form are random variables. Namely, the presented optimal state observer and parameters identifier, gives the smallest second order statistics on the average when the average is taken with respect to all systems within the distribution of the parameters, all initial conditions and all noises. This means that when dealing with a specific system which has constant parameters, although unknown, this approach does not necessarily lead to better performance with respect to other identification algorithms. The following presents a result on the performance of the optimal estimator-identifier. Theorem 5.1: If the input to the plant up(t) is such that (Ap(t),cp) is uniformly completely observable, V pI is T 1/2 such that (A p (t), V pI) is uniformly completely controllable and Vp 2>O, then the optimal observer (4.13) is exponentially stable, and the minimal mean square estimation error is Qp(t). Proof: Direct outcome of[Kwakemaak, thm 4.10]. Q.E.D.
In this section we consider design of the input such that the observability condition is taken care off with respect to an optimality objective. The optimal input is based on the maximization of the Fisher information matrix. This section follows closely the approach by [Mehra 74a, 74b]. We deal with the linear single-input single-output stochastic system (3.1). The problem of finding the "optimal" input for identification, as formulated in [Mehra 74a], is to selecl the input (up(t), 0 :$; t :$; tl} to maximize a norm of the Fisher Information matrix, M, subject to a constraint. The Fisher Information matrix M for the unknown parameters 9p is shown to be [Nahi] 11
M = E{
f
V 9YpT
V~12 V9Yp dt }
la where
(6.1)
where V9Yp is a the gradient of the estimate of yp(t) with respect to the unknown parameters, 9 p , V 9Yp =
[~~ CJe!'CJei ...... ]E Rlx2n • T0
d enve . a scal ar measure 0 f the
Fisher information matrix there are several matrix norms that can be used [Mehra 74a]. In this work we use the weighted trace norm: Je = trace(SM)
(6.2)
where S>O is a weighting matrix. We further assume, for simplicity, that S is diagonal, so that the scalar objective,
5140
5141
5142
objective creates a completion to the plant's input, up(t), so as to enhance the identification problem, and it trades-off the tracking quality with the identification quality. This completion depends on the desired trajectory. That is, there are "bad" trajectories from identification point of view, not sufficiently rich, that need lot of completion. On the other extreme, there are trajectories that need very little or no completion. Conclusions The problem of optimal simultaneous identification and tracking of stochastic uncertain linear systems on finite time interval has been formulated and solved. The combined objective is the simultaneous minimization of the quadratic tracking criterion and maximization of the information matrix. The solution is explicit and non causal. It suggests the structure of the causal approximation and clearly states the tradeoff between the tracking and identification costs. References [Boniventol Bonivento, c.:"Structural Insensitivity versus Identifiability," IEEE Trans. on Autom. Cont., AC-18, No. 2, April 1973, pp. 190-193. [Bryson] Bryson, A.E. and Ho, Y.c.: Applied Optimal Control, Hemisphere Publishing Company, 1975. [Feldbaum] Feldbaum, A. A.: Optimal Control Systems, Academic Press Inc., New York, 1965. [Gevers] Gevers, M.:"Towards a Joint Design of Identification and Control?," chapter 5 in Trentelman, H.L. and Willems, J .C., Editors:" Essays on Control: Perspectives in the theory and its Applications," BirkMuser Boston 1993. [Goodwin] Goodwin, G. C. and Payne, R. L.:"Dynamic System Identification: Experiment Design and Data Analysis," Academic Press, New York, 1977. [Guez] Guez, A., Rusnak, I. and BarKana, I.:" Multiple Objective Optimization Approach to Adaptive Learning Control," International Journal of Control, Aug. 1992. [Khargonekar] Khargonekar, P. P. and Rotea, M. A.:"Multiple Objective Optimal Control of Linear Systems: The Quadratic Norm Case," IEEE Trans. on Autom. Contr., Vol. 36, No. 1, January, 1991. [Koussoulass] Koussoulass, N. T. and Dimitriadis, E. K.:" Computational Techniques for Multicriteria Stochastic Optimization and Control," Leondes C. T. Edit., Control and Dynamic Systems, Advances in Theory and Applications, Vol. 30, part 3, Academic Press, 1989. [Kwakemaak] Kwakernaak, and Sivan. R.: Linear Optimal Control, New York: Wiley, 1972. [Ljung] Ljung, L.:System Identification. Prentice-Hall, Inc., 1977. [Mehra 74a] Mehra, R. K.:"Optimal Inputs for Linear Systems Identification," IEEE Trans. on Automatic Control, Vol. AC-19, No. 3, June 1974. [Mehra 74b] Mehra, R. K.:"Optimal Input Signals for
Parameter Estimation in Dynamic Systems - Survey and New Results," IEEE Trans. on Automatic Control, Vol. AC-19, No. 6, December 1974. [Nahi] Nahi, N. E. and Wallis, D. E. Jr.:"Optimal Inputs for Parameter Estimation in Dynamic Systems with White Observation Noise," Joint Automatic Control Conference, Boulder, Colorado, 1969. [Polderman] Polderman, J. W.:"Adaptive Control and Identification: Conflict or Conflux," Stichting Mathematisch Centrum, Amsterdam, Netherlands, 1989. [Rusnak 92a] Rusnak, I., Guez, A. and BarKana, I.: "Multiple Objective Approach to Adaptive and Learning Control of Discrete Linear Time-Invariant Systems," 7-th Workshop on Adaptive and Learning Control, Yale University, May 20-22, 1992. [Rusnak 92b] Rusnak, I., Steinberg, M., Guez, A. and BarKana, I. :"On-Line Identification and Control of Linearized Aircraft Dynamics," IEEE AES Magazine, July 1992. [Rusnak 92c] Rusnak, I., Guez, A. and BarKana, I.:"Multiple Objective Approach to Adaptive and Learning Control of Discrete Linear Time-Invariant Systems," 7-th Workshop on Adaptive and Learning Control, Yale University, May 20-22, 1992. [Rusnak 92d] Rusnak, I., Guez, A. and BarKana, I.:"Necessary and Sufficient Conditions on the Observability and Identifiability of Linear Systems," ACC 1992, American Control Conference, Chicago, Illinois, June 24-26, 1992. [Rusnak 93a] Rusnak, I., Guez, A. and BarKana, I. :"Control of Stochastic Linear Uncertain Systems by Multiple Objective Optimization," 1993 IEEE Regional Conference on Aerospace Control Systems, CACS 93, Thousand Oaks, Los Angeles, May 25-27,1993. [Rusnak 93b] Rusnak, I., Guez, A. and BarKana, I. :"Multiple Objective Optimization Approach to Adaptive Control of Linear Systems," American Control Conference, ACC 93, June 2-5, 1993, San Francisco. [Rusnak 93c] Rusnak, I.:" Optimal Adaptive Control of Uncertain Stochastic Linear Systems," Ph.D. thesis, Drexel University, June 1993. [Rusnak 93d] Rusnak, I., Guez, A. and BarKana, I. :"State Observability and Parameters Identifiability of Stochastic Linear Systems," 1993 IEEE Regional Conference on Aerospace Control Systems, CACS 93, Thousand Oaks, Los Angeles, May 25-27,1993. [Rusnak 94] Rusnak, I. and Guez, A.:":Simultaneous Optimal Identification and Control of Stochastic Linear Uncertain Systems." The 2nd IEEE Symposium on New Directions in Control and Automation, Chania, Crete, 1922 Jun, 1994, Greece.
5143