- Email: [email protected]
Reduction of the Sensitivity of Optimal Control Systems by Using Two Degrees of Freedom
Reduction of the Sensitivity of Optimal Control Systems by Using Two Degrees of Freedom
Naresh K. Sinha Professor Department of Electrical Engineering and Communication Research Laboratory McMaster University Hamilton, Ontario Canada ABSTRACT This paper considers the implementation of optimal control using two degrees of freedom, i.e., a combination of the open- and closed-loop configurations. It has been shown that with this arrangement, one can minimize the integral state sensitivity, while at the same time, either the performance sensitivity, or the terninal-state sensitivity may be reduced to zero. In practice, some compromise must be accepted between the reductions in the various sensitivities. An example of a linear system with an integral quadratic cost function is used to illustrate the procedure.
Consider a system described by the vector differential equation i(t,u) = f(x.t.u.u) (1) where x(t,p) is the n-dimensional state vector u(t ,p) is the m-dimensional plant-input (or control) vector, andu is the plant parameter vector with nominal value u 0' The optimal control problem is the determination of the control vector which will minimize the cost function
INTRODUCTION
An interesting feature of optimal control theory is that, although it enables the designer to calculate the optimal control input to the plant, it does not specify the manner in which this input should be obtained. This allows considerable flexibility in the implementation of the optimal control, which may, therefore, be used with advantage in the reduction of the sensitivity of the system to small variations in the parameters of the plant. In the past, many authors ( 1 ~ ~ 9 3have ) considered the relative merits of implementation of the optimal control in open-loop and closed-loop configurations, but very little appears to have been done on implementation with two degrees of freedom, using a suitable combination of open-loop and closed-loop implementations. It is surprising that this configuration has not been considered in the literature on optimal con 01 theory, although, quite a while ago, Horowitz had shown that it allows the reduction of sensitivity while maintaining the desired overall transfer function.
(8
The object of this paper is to demonstrate that with the two degrees of freedom available, it is possible to considerably reduce the sensitivity of the performance index as well as the trajectory sensitivity to plant parameter variations in a large number of cases. In general, some compromise must be accepted between reduction of performance sensitivity, terminal-state sensitivity and trajectory sensitivity. An example of a linear regulator with an integral quadratic cost function will be used to illustrate the procedure and the various trade-offs which are possible.
-0
where the initial time, t , the final time, t , the initial state x(to)=x , a% the final state xftf)=xf. are all specified, an8v has the nominal value u 0' The sensitivity problem may be stated as thereduction of the variation in the state trajectory if u varies For the in the infinitesimal neighbourhood of u 0' measure of sensitivity, a number of sensitivity functions are defined. The cost sensitivity, St is defined as
sJ
= aJ G 'u=!J0 P The state sensitivity, St , is defined as
and the terminal-state sensitivity SXf is given by ! J
All these sensitivities are functions of an implementation vector, v(t), defined as
Let the optimal control input for the nominal case (u=u ) be denoted by
It can be determined by a direct application of
Pontryagin's minimum principle, i.e., by obtaining u*(t) such that the Hamiltonian, H is minimized, where T H = L(x,u,t) f (x,u.t.uo) p(t) (8) aH and p(t) = - ax (9) +
.
In general, the optimal control input may be implemented by using a combination of open- and closedloop formulations, leading to a system with two degrees of freedom. Thus, one may write
wherel(x, (t.u)) is a function of the state of the system, and represents state-variable feedback and g(t) represents the open-loop portion of the control input. For the particular case of a linear system with an integral quadratic cost function, equation (10) takes the following simpler form
THE OPTIMAL IMPLJ3ENTATION PROBLEM Kokotovic and Heller ('Ihave shown that the cost sensitivity is given by
Hence, the cost sensitivity can be made zero if
In general, the initial state is independent of the parameter vector, u , hence,
Therefore, cost insensitivity can be assured if the following equation is satisfied where H(t) is the matrix representing linear statevariable feedback. Since u(t,p) must be optimal for the nominal case (u=u ) , the following equation must be satisfied by a71 admissible bl(t) and g(t), u(t,uo) = u*(t) = bl(t) x*(t) + g(t) (12) where x* (t) is the optimal state trajectory for the nominal case, i.e.,
The "open-loop1'implementation of optimal control is obtained when M(t)=O, whereas the llclosed-loop" implementation is obtained when g(t)=O. These have been investigated by many authors and compared from the point of view of sensitivity to variations in plant parameters. It appears natural that the general case, represented by equation (12). provides more flexibility which could be utilized for the reduction of various sensitivities defined in equations (3), (4) and (5).
The state sensitivity vector, y(t), is obtained by differentiating equation (1) partially with respect to U. Hence,
The implementation vector. v(t), is obtained by differentiating equation (11) with respect to u. Ilence , v(t) = bly From equations (19) and (20).
af I af af I The problem may, therefore, be stated as the selThe functions ax u=po* au lu=uoand au p=0 are ection of M(t) in such a manner that these sensitivities are minimized. In order that such a minassumed to be known, and depend upon the nominal imization may be carried out, it is necessary to optimum trajectory and control, x* (t) and v* (t) , define a scalar measure for each of the sensitivities. respectively. If the components of h, ahand If u is a scalar, sJ is also a scalar, but SE is a dt ah continuous over the interval (to,tf), then -are time-varying vectnru and y f is a vector independent a Y of time. Thus * One may consider the norm as an optimal implementation problem can be formulated the measure of terminal-state sensitivity, and one as follows: may define the "integral state sensitivity" as Find M(t) driving y(t) from y(t )=O to a given point or set y(t ) satisfying tRe constraints in S = yT(t) R y (t) dt equation (21) an8 minimizing the integral state Y sensitivity, S defined in equation (14). 0
stf
1:'
Y'
where R is a given positive definite weighting matrix. If p is a vector, scalar measures can be obtained in each case by defining suitable norms for the various sensitivity vectors or matrices. In general, therefore, one would determine M(t) to minimize a scalar sensitivity function which is weighted linear combination of the scalar measures for the three sensitivities defined earlier.
In the design for terminal-state insensitivity, y (tf) is the null vector (or the null matrix if p is a vector), whereas in the cost insensitive design, y(tf) is a point on the hyperplane defined by equation (18). In general, the right-hand side of equation (18) is non-zero, so that y(tf) cannot be made zero. Therefore, it can be concluded that
cost insensitivity can be obtained at the expense of increased terminal-state sensitivity. It also follows that the cost and terminal-state insensitivities cannot be realized at the same time in any implementation of optimal control. By making y(t ) a point on the hyperplane defined by equation (18f at a minimum distance from the origin, one may achieve cost insensitivity, while at the same time minimizing terminal-state sensitivity and the integral state sensitivity.
case are readily evaluated as
The state sensitivities can be obtained directly as
From practical considerations one must also include some constraints on the magnitudes of the elements One may, for instance, require that the of M(t). integral of the norm-squared of M(t), i .e.,
to must be finite. Alternatively, a more restrictive magnitude constraint on the elements of El(t) may also be imposed. EXAMPLE
It may be noted that since the final state never exactly reaches the origin of the state space, the integral state sensitivity approaches infinity. The cost sensitivity, as defined in equation (3). can be evaluated by using equation (15) as
*.
The state sensitivity could have been evaluated more easily by using equation (21). which leads to
An example will now be considered of a second-order linear system with one variable parameter. Consider a system described by Solving equation (36) for the initial conditions y1 (0)=0, y2 (0)=0 leads to equations (34) and (35) immediately. where po=O for the nominal case. The initial and the final states are given by
and the cost function to be minimized is given by m
The optimal control for the nominal case is easily obtained by applying Pontryagin's minimum principle, and is given by u* (t) =-2e-2t (25) and the optimum state and costate variables are given by (for p=p )
For the closed-loop implementation, we have u = -2x -2x 1 2 so that M = ml m2 = -2 -2
C 1
(38) Substituting this value of M into equation (22), the state equations now take the form
One may solve eqn. (39) and then determine the state sensitivities. Alternatively, using equation (21) the following differential equations are obtained
which may be solved for y(O)= k)to give and
p2*(t) = e
-2t
(29)
The minimum value of the cost function for the nominal case is obtained as
Consider now the system with p not equal to zero. For the open-loop implementation, we make u(t) = -2e-2t (311 as in equation (25). The state variables for this
It will be seen that unlike the open-loop case, both the terminal-state sensitivities are zero. The integral state sensitivity is, therefore, finite, and can be evaluated for any givcn R in accordance with equation (14). Taking R as the unit matrix, wc have 5 S (y12 + y22)dt = (43) Y 72
=I:
The cost sensitivity is also finite, and can be computed using equation (15). Foru =0, the cost sensitivity is the same for both the open-loop and the closed-loop configurations. Finally, the implementation using two degrees of freedom will be considered. In this case we have where m , m2 and g must be selected in such a manner thai for the nominal case u(t) satisfies equation (25) for optimality. As there are many possible choices, mk,and,m2will be selected to minimize some sensi ivities, and then g(t) can be determined such that the condition for optimality is satisfied. From equation (25) the following differential equations are obtained for the state sensitivitv
with yl(0)=O, y2(0)=0. One approach may be to determine ml and.m2 which will minimize the integral state sensitivity while satisfying the constraints
Y(-1
=El
and
, 1m21<-
.
This will lead to a system with a minimum value of the integral state sensitivity defined in equation (43), while keeping the terminal-state sensitivity zero. Another approach may be to select ml and m2 in such a manner that tile resulting system satisfies equation (18) for cost insensitivity. This will be achieved, however, at the expense of the terminal state sensitivity which will no longer be zero. Also, the intcgral state sensitivity will be much greater in this case. In solving the problem of minimization of tile integral state sensitivity, it was found that by making ml and m2 large negative quantities, S can be reduced to any dcsired value. In practixe, the gains ml and m 2 must be limited due to other considerations, hence the reduction is possible only to a certain extent. For instance, with m1=-20 and m2=-8, SY was less than 0.001.
It has been shown that by using the configuration with two degrees of freedom in thc implementation of optimal control, the additional flexibility can be used for the reduction of the state sensitivity. It is generally possible to make either the terminalstate sensitivity or the cost sensitivity zero while minimizing the integral state sensitivity.
(1)
U.C. Youla and P. Dorato: "On the comparison of sensitivities of open-loop and closed-loop control systems", IEEE Trans. Automatic Control,
vol. AC-13, April 1068, pp. 186-188. (2) E. Kriendler, "Closed-loop sensitivity reduction of linear optimal control systems", IEEE Trans. Automatic Control, vol. AC-13, June 1968, pp. 254-262. (3) P.V. Kokotovic, J. Ileller and P. Sannuti, "Sensitivity Comparison of Optimal Controls", Int. J. of Control, vol. 9, January 1969, pp. 111-115. (4) 1.M. llorowitz: "Fundamental Theory of Automatic Linear Feedback Control Systems", IRE Trans. Automatic Control, vol. AC-4, 1959, pp. 5-19. (5) P. Kokotovic and J. tleller: "Direct and adjoint sensitivity equations for parameter optimization", IEEE Trans. Automatic Control, vol. AC-12, pp.609610, October 1967.
Naresh K. Sinha Professor Department of Electrical Engineering and Communication Research Laboratory McMaster University Hamilton, Ontario Canada ABSTRACT This paper considers the implementation of optimal control using two degrees of freedom, i.e., a combination of the open- and closed-loop configurations. It has been shown that with this arrangement, one can minimize the integral state sensitivity, while at the same time, either the performance sensitivity, or the terninal-state sensitivity may be reduced to zero. In practice, some compromise must be accepted between the reductions in the various sensitivities. An example of a linear system with an integral quadratic cost function is used to illustrate the procedure.
Consider a system described by the vector differential equation i(t,u) = f(x.t.u.u) (1) where x(t,p) is the n-dimensional state vector u(t ,p) is the m-dimensional plant-input (or control) vector, andu is the plant parameter vector with nominal value u 0' The optimal control problem is the determination of the control vector which will minimize the cost function
INTRODUCTION
An interesting feature of optimal control theory is that, although it enables the designer to calculate the optimal control input to the plant, it does not specify the manner in which this input should be obtained. This allows considerable flexibility in the implementation of the optimal control, which may, therefore, be used with advantage in the reduction of the sensitivity of the system to small variations in the parameters of the plant. In the past, many authors ( 1 ~ ~ 9 3have ) considered the relative merits of implementation of the optimal control in open-loop and closed-loop configurations, but very little appears to have been done on implementation with two degrees of freedom, using a suitable combination of open-loop and closed-loop implementations. It is surprising that this configuration has not been considered in the literature on optimal con 01 theory, although, quite a while ago, Horowitz had shown that it allows the reduction of sensitivity while maintaining the desired overall transfer function.
(8
The object of this paper is to demonstrate that with the two degrees of freedom available, it is possible to considerably reduce the sensitivity of the performance index as well as the trajectory sensitivity to plant parameter variations in a large number of cases. In general, some compromise must be accepted between reduction of performance sensitivity, terminal-state sensitivity and trajectory sensitivity. An example of a linear regulator with an integral quadratic cost function will be used to illustrate the procedure and the various trade-offs which are possible.
-0
where the initial time, t , the final time, t , the initial state x(to)=x , a% the final state xftf)=xf. are all specified, an8v has the nominal value u 0' The sensitivity problem may be stated as thereduction of the variation in the state trajectory if u varies For the in the infinitesimal neighbourhood of u 0' measure of sensitivity, a number of sensitivity functions are defined. The cost sensitivity, St is defined as
sJ
= aJ G 'u=!J0 P The state sensitivity, St , is defined as
and the terminal-state sensitivity SXf is given by ! J
All these sensitivities are functions of an implementation vector, v(t), defined as
Let the optimal control input for the nominal case (u=u ) be denoted by
It can be determined by a direct application of
Pontryagin's minimum principle, i.e., by obtaining u*(t) such that the Hamiltonian, H is minimized, where T H = L(x,u,t) f (x,u.t.uo) p(t) (8) aH and p(t) = - ax (9) +
.
In general, the optimal control input may be implemented by using a combination of open- and closedloop formulations, leading to a system with two degrees of freedom. Thus, one may write
wherel(x, (t.u)) is a function of the state of the system, and represents state-variable feedback and g(t) represents the open-loop portion of the control input. For the particular case of a linear system with an integral quadratic cost function, equation (10) takes the following simpler form
THE OPTIMAL IMPLJ3ENTATION PROBLEM Kokotovic and Heller ('Ihave shown that the cost sensitivity is given by
Hence, the cost sensitivity can be made zero if
In general, the initial state is independent of the parameter vector, u , hence,
Therefore, cost insensitivity can be assured if the following equation is satisfied where H(t) is the matrix representing linear statevariable feedback. Since u(t,p) must be optimal for the nominal case (u=u ) , the following equation must be satisfied by a71 admissible bl(t) and g(t), u(t,uo) = u*(t) = bl(t) x*(t) + g(t) (12) where x* (t) is the optimal state trajectory for the nominal case, i.e.,
The "open-loop1'implementation of optimal control is obtained when M(t)=O, whereas the llclosed-loop" implementation is obtained when g(t)=O. These have been investigated by many authors and compared from the point of view of sensitivity to variations in plant parameters. It appears natural that the general case, represented by equation (12). provides more flexibility which could be utilized for the reduction of various sensitivities defined in equations (3), (4) and (5).
The state sensitivity vector, y(t), is obtained by differentiating equation (1) partially with respect to U. Hence,
The implementation vector. v(t), is obtained by differentiating equation (11) with respect to u. Ilence , v(t) = bly From equations (19) and (20).
af I af af I The problem may, therefore, be stated as the selThe functions ax u=po* au lu=uoand au p=0 are ection of M(t) in such a manner that these sensitivities are minimized. In order that such a minassumed to be known, and depend upon the nominal imization may be carried out, it is necessary to optimum trajectory and control, x* (t) and v* (t) , define a scalar measure for each of the sensitivities. respectively. If the components of h, ahand If u is a scalar, sJ is also a scalar, but SE is a dt ah continuous over the interval (to,tf), then -are time-varying vectnru and y f is a vector independent a Y of time. Thus * One may consider the norm as an optimal implementation problem can be formulated the measure of terminal-state sensitivity, and one as follows: may define the "integral state sensitivity" as Find M(t) driving y(t) from y(t )=O to a given point or set y(t ) satisfying tRe constraints in S = yT(t) R y (t) dt equation (21) an8 minimizing the integral state Y sensitivity, S defined in equation (14). 0
stf
1:'
Y'
where R is a given positive definite weighting matrix. If p is a vector, scalar measures can be obtained in each case by defining suitable norms for the various sensitivity vectors or matrices. In general, therefore, one would determine M(t) to minimize a scalar sensitivity function which is weighted linear combination of the scalar measures for the three sensitivities defined earlier.
In the design for terminal-state insensitivity, y (tf) is the null vector (or the null matrix if p is a vector), whereas in the cost insensitive design, y(tf) is a point on the hyperplane defined by equation (18). In general, the right-hand side of equation (18) is non-zero, so that y(tf) cannot be made zero. Therefore, it can be concluded that
cost insensitivity can be obtained at the expense of increased terminal-state sensitivity. It also follows that the cost and terminal-state insensitivities cannot be realized at the same time in any implementation of optimal control. By making y(t ) a point on the hyperplane defined by equation (18f at a minimum distance from the origin, one may achieve cost insensitivity, while at the same time minimizing terminal-state sensitivity and the integral state sensitivity.
case are readily evaluated as
The state sensitivities can be obtained directly as
From practical considerations one must also include some constraints on the magnitudes of the elements One may, for instance, require that the of M(t). integral of the norm-squared of M(t), i .e.,
to must be finite. Alternatively, a more restrictive magnitude constraint on the elements of El(t) may also be imposed. EXAMPLE
It may be noted that since the final state never exactly reaches the origin of the state space, the integral state sensitivity approaches infinity. The cost sensitivity, as defined in equation (3). can be evaluated by using equation (15) as
*.
The state sensitivity could have been evaluated more easily by using equation (21). which leads to
An example will now be considered of a second-order linear system with one variable parameter. Consider a system described by Solving equation (36) for the initial conditions y1 (0)=0, y2 (0)=0 leads to equations (34) and (35) immediately. where po=O for the nominal case. The initial and the final states are given by
and the cost function to be minimized is given by m
The optimal control for the nominal case is easily obtained by applying Pontryagin's minimum principle, and is given by u* (t) =-2e-2t (25) and the optimum state and costate variables are given by (for p=p )
For the closed-loop implementation, we have u = -2x -2x 1 2 so that M = ml m2 = -2 -2
C 1
(38) Substituting this value of M into equation (22), the state equations now take the form
One may solve eqn. (39) and then determine the state sensitivities. Alternatively, using equation (21) the following differential equations are obtained
which may be solved for y(O)= k)to give and
p2*(t) = e
-2t
(29)
The minimum value of the cost function for the nominal case is obtained as
Consider now the system with p not equal to zero. For the open-loop implementation, we make u(t) = -2e-2t (311 as in equation (25). The state variables for this
It will be seen that unlike the open-loop case, both the terminal-state sensitivities are zero. The integral state sensitivity is, therefore, finite, and can be evaluated for any givcn R in accordance with equation (14). Taking R as the unit matrix, wc have 5 S (y12 + y22)dt = (43) Y 72
=I:
The cost sensitivity is also finite, and can be computed using equation (15). Foru =0, the cost sensitivity is the same for both the open-loop and the closed-loop configurations. Finally, the implementation using two degrees of freedom will be considered. In this case we have where m , m2 and g must be selected in such a manner thai for the nominal case u(t) satisfies equation (25) for optimality. As there are many possible choices, mk,and,m2will be selected to minimize some sensi ivities, and then g(t) can be determined such that the condition for optimality is satisfied. From equation (25) the following differential equations are obtained for the state sensitivitv
with yl(0)=O, y2(0)=0. One approach may be to determine ml and.m2 which will minimize the integral state sensitivity while satisfying the constraints
Y(-1
=El
and
, 1m21<-
.
This will lead to a system with a minimum value of the integral state sensitivity defined in equation (43), while keeping the terminal-state sensitivity zero. Another approach may be to select ml and m2 in such a manner that tile resulting system satisfies equation (18) for cost insensitivity. This will be achieved, however, at the expense of the terminal state sensitivity which will no longer be zero. Also, the intcgral state sensitivity will be much greater in this case. In solving the problem of minimization of tile integral state sensitivity, it was found that by making ml and m2 large negative quantities, S can be reduced to any dcsired value. In practixe, the gains ml and m 2 must be limited due to other considerations, hence the reduction is possible only to a certain extent. For instance, with m1=-20 and m2=-8, SY was less than 0.001.
It has been shown that by using the configuration with two degrees of freedom in thc implementation of optimal control, the additional flexibility can be used for the reduction of the state sensitivity. It is generally possible to make either the terminalstate sensitivity or the cost sensitivity zero while minimizing the integral state sensitivity.
(1)
U.C. Youla and P. Dorato: "On the comparison of sensitivities of open-loop and closed-loop control systems", IEEE Trans. Automatic Control,
vol. AC-13, April 1068, pp. 186-188. (2) E. Kriendler, "Closed-loop sensitivity reduction of linear optimal control systems", IEEE Trans. Automatic Control, vol. AC-13, June 1968, pp. 254-262. (3) P.V. Kokotovic, J. Ileller and P. Sannuti, "Sensitivity Comparison of Optimal Controls", Int. J. of Control, vol. 9, January 1969, pp. 111-115. (4) 1.M. llorowitz: "Fundamental Theory of Automatic Linear Feedback Control Systems", IRE Trans. Automatic Control, vol. AC-4, 1959, pp. 5-19. (5) P. Kokotovic and J. tleller: "Direct and adjoint sensitivity equations for parameter optimization", IEEE Trans. Automatic Control, vol. AC-12, pp.609610, October 1967.