- Email: [email protected]
Minimising conservatism in infinite-horizon LQR control
Systems & Control Letters 46 (2002) 271 – 279
www.elsevier.com/locate/sysconle
Minimising conservatism in in!nite-horizon LQR control Ognjen Marjanovic, Barry Lennox ∗ , Peter Goulding, David Sandoz School of Engineering, Control Technology Centre, University of Manchester, Manchester, UK M13 9PL Received 19 February 2001; received in revised form 1 March 2002
Abstract This paper studies the formulation of the constrained in!nite horizon linear quadratic regulator control law (CIHLQR). Results from recent studies in this area are extended to show that conditions used in the standard formulation of the CIHLQR law are not necessary, but merely su:cient. Through the use of a novel proof it is shown that for a general SISO system with input constraints and certain conditions imposed, saturated LQR provides the same control sequence as CIHLQR. It is further shown that saturated LQR is equivalent to the CIHLQR in the case of !rst-order systems, subject to both state and control constraints. Finally, the region of constrained stabilisability is characterised for the case of open-loop unstable c 2002 Elsevier Science B.V. All rights reserved. !rst-order systems. Keywords: Model predictive control; In!nite horizon; Linear quadratic regulator
1. Introduction A highly successful approach to the control of constrained linear systems that has been widely accepted by academia and industry alike is model predictive control (MPC). In this approach constrained regulation is achieved by employing repetitive on-line optimisation of a system’s performance index (cost function), subject to limiting inequality constraints being imposed onto the system variables. Whilst there have been many successful applications of MPC technology in industry, issues such as feasibility, robustness and stability remain important areas of research. In recent years several MPC formulations have been proposed that through the imposition of certain conditions on the system variables ∗ Corresponding author. Tel.: +44-161-275-4324; fax: +44+161-275-4346. E-mail address: [email protected] (B. Lennox).
and=or the structure of the performance index ensure feasibility and nominal stability. A thorough review of these techniques is presented in [11]. One example of an MPC formulation that ensures nominal stability is constrained in!nite horizon linear quadratic regulator (CIHLQR) [5,14]. This approach considers behaviour of the system over an in!nite horizon in the future, hence addressing the convergence property of the system variables and hence stability. In addition, CIHLQR establishes feasibility and stability results, which state that if there is no feasible=stable CIHLQR solution for the given problem, then the problem itself is insoluble by any other controller. To solve an in!nite-dimensional problem using a !nite-horizon formulation, inclusion condition is imposed onto the terminal state vector. This, in turn, dictates the length of the prediction horizon for the !nite-horizon equivalent of the CIHLQR and hence the complexity of the resulting controller. The typical method for solving MPC control problems, such as
c 2002 Elsevier Science B.V. All rights reserved. 0167-6911/02/$ - see front matter PII: S 0 1 6 7 - 6 9 1 1 ( 0 2 ) 0 0 1 5 1 - 2
272
O. Marjanovic et al. / Systems & Control Letters 46 (2002) 271 – 279
CIHLQR, is through the use of non-analytical methods such as quadratic programming, see [12,13]. This can introduce a signi!cant computational burden, particularly in situations where the length of the prediction horizon, required to guarantee CIHLQR performance, is large. Recently, a number of papers have addressed the issue of reducing the on-line computation requirements of the standard MPC control problem. In particular, it has been shown [7,8] that for a general SISO system with no state constraints, saturated linear MPC, designed for the unconstrained case, is equivalent to its constrained counterpart for a certain non-trivial region of the state space and under a certain set of conditions, demonstrating the equivalence between simple anti-windup scheme and the constrained MPC. In [6], the solution to the !nite-horizon MPC controller with no penalty on control energy is derived for the case of !rst-order systems with constraints on control variables alone, reducing the on-line computational burden of constrained MPC controller implementation to a minimum. The solution takes the form of the standard linear MPC controller, designed for the unconstrained case. Furthermore, the explicit analytical solution to the standard MPC has recently been proposed in [3,4,15], wherein the controller takes the form of a piecewise-a:ne mapping between state and the corresponding optimal control move. In this way on-line computation is reduced to a minimum, allowing MPC to be implemented in situations where relatively fast dynamics may exist. A major disadvantage of this approach is that, as the authors state, the complexity of the problem increases signi!cantly as the number of control and state variables increases and therefore the approach may only be suitable for relatively simple systems. This paper focuses on CIHLQR for which it is shown that a crucial condition in its standard formulation may in many situations be relaxed. Relaxation of this condition leads to a reduction in the complexity of the resulting controller, which results in a signi!cant reduction in the on-line computation requirements of the controller. This paper is organised as follows. In Section 2, the standard unconstrained LQR and CIHLQR are introduced. In Section 3 it is proved, using diJerent arguments to those in [8], that in the case of a general SISO system with input constraints and under certain conditions, saturated LQR provides the same control
sequence as CIHLQR. This result is analysed further to show that a crucial condition in the standard CIHLQR formulation may in many situations be relaxed, leading to a simpler controller. In Section 4, the analytic solution to the CIHLQR with both control and state constraints is developed for a class of sampled-data continuous-time !rst-order systems, using the arguments presented in Section 3. The solution is shown to be the same as the saturated LQR controller, con!rming the original hypothesis regarding the lack of necessity in the standard CIHLQR formulation. Furthermore, the region of constrained stabilisability is characterised for the case of open-loop unstable !rst-order systems. This region is shown to be independent on the particular choice of cost function. Finally, conclusions and directions for future work are presented in Section 5. 2. Problem formulation Consider a SISO discrete linear time-invariant system of the following form: xk+1 = Axk + Buk ;
(1)
where x ∈ Rn is the state vector, u ∈ R is the control input, A ∈ Rn×n is the state transition matrix, B ∈ Rn×1 is the control distribution vector and k ∈ Z+ is a non-negative integer denoting the sampling time instant. The control problem considered in this paper is that of state regulation to the origin. In particular, the requirement is that the state is regulated in an optimal manner, de!ned by the weighted quadratic cost function of the state and control trajectories. The main focus of this paper is on CIHLQR, however, for the bene!t of clarity, the standard unconstrained LQR problem is !rst presented. (P1) Unconstrained in!nite-horizon linear quadratic regulation problem: Given an initial state x0 , the task is to !nd the unconstrained optimal control sequence ∗ ∗ u∗ = {uk∗ }∞ k=0 = {u0 ; u1 ; : : :}, which minimises the following cost function: ∞ J (x0 ; u) = (xkT Qxk + ukT Ruk ) k=0
subject to xk+1 = Axk + Buk for ∀k ∈ Z + .
O. Marjanovic et al. / Systems & Control Letters 46 (2002) 271 – 279 ∞
273
The assumptions made on the weightings Q and R are as follows:
J (x0 ; uP∗ ) = min
(A1) Q = QT ¿ 0, (A2) R ¿ 0.
subject to xk+1 = Axk + Buk and u− 6 uk 6 u+ for ∀k ∈ Z + .
The solution to (P1) is readily available, assuming that (A1) and (A2) hold, in the linear and time-invariant state feedback form [1,9,10]: uk∗ = −KLQ xk ;
(2) 1×n
where KLQ ∈ R is obtained by solving the discrete algebraic Riccati equation (DARE), P = AT P − PB(BT PB + R)−1 BT PA + Q, with P = P T ¿ 0 being its unique solution. This solution is then substituted into KLQ = (BT PB + R)−1 BT PA in order to obtain the optimal LQR controller gain, KLQ . Optimal value of J (x0 ; u) is given by the following relation: J (x0 ; u∗ ) = x0T Px0 ;
(3)
where P is the solution to the DARE. Denition 1. The closed-loop dynamics; resulting from the application of (2); are given as follows: ∗
ACL = A − BKLQ : Remark 1. Anderson and Moore [1]; Kwakernaak ∗ and Sivan [9] and Levine [10] state that ACL is stable with sound robustness and performance properties; provided that the standard assumptions on unconstrained stabilisability and detectability as well as (A1) and (A2) are respected. To enable the controller to account for constraints that will exist on a system it is necessary to consider the constrained extension to (P1). Before that, however, the following assumption is made regarding the structure of the constraints: (A3) The states are assumed to be unbounded while the control input is constrained by non-zero, time-invariant upper and lower bounds, i.e. u− 6 uk 6 u+ for k ¿ 0, with u− ¡ 0 ¡ u+ . Note, however, that the CIHLQR formulation, as presented in [5,14], addresses the state as well as control constraints. (P2) Constrained in!nite-horizon linear quadratic regulation problem (explicit form):
u
(xkT Qxk + ukT Ruk )
k=0
Due to the incorporation of an in!nite horizon and under the assumptions of unconstrained stabilisability and detectability as well as (A1) and (A2), this approach guarantees nominal stability so long as the optimal cost is !nite, i.e. J (x0 ; uP∗ ) ¡ ∞ ⇒ limk→∞ xk → 0 [5,14]. Unfortunately, the CIHLQR formulation, as presented in (P2), results in an insoluble in!nite-dimensional optimisation problem. However, by using the principle of optimality, it is possible to recast (P2) into the following !nite-horizon formulation. (P3) Constrained in!nite-horizon linear quadratic regulation problem (implicit form): N −1 ∗ T T T JN (x0 ; uP ) = min (xk Qxk + uk Ruk ) + xN PxN u
k=0
subject to xk+1 = Axk + Buk and u− 6 uk 6 u+ for 0 6 k 6 N − 1, and xN ∈ XKLQ . By choosing the solution of DARE to be equal to the value of the terminal weight, given as P in (P3), it is possible, according to (3), to substitute T J (xN ; {uP∗k }∞ k=N ) with xN PxN , under the assumption that KLQ is applied for k ¿ N with no subsequent violations of the control constraints [1,5,9,10,14]. This assumption is met by ensuring that the terminal state, xN , belongs to a certain set, formally de!ned below. Denition 2. XKLQ is the subset of state space within which evolution of the closed-loop system; given in De!nition 1; does not violate any constraints: XKLQ = {x0 | u− 6 − KLQ (A − BKLQ )k x0 6 u+ ; ∀k ∈ Z + }: In this paper, it is argued that in certain cases the terminal state inclusion condition, xN ∈ XKLQ , can be relaxed, removing the lower bound on N , required to guarantee CIHLQR performance. The general consequence of this result is simpli!cation of the implemented CIHLQR controller.
274
O. Marjanovic et al. / Systems & Control Letters 46 (2002) 271 – 279
3. Results: general SISO systems In this section an alternative proof to that provided in [8] is presented showing that for a class of SISO systems with hard input constraints and no state constraints, the saturated LQR control move is equivalent to that of the CIHLQR in many circumstances. By taking into account the fact that the saturated LQR control sequence is equivalent to a receding horizon control sequence of the solution to (P3) with no terminal state inclusion condition and with N = 1 so long as (A3) holds, see [16], the lack of necessity of the standard CIHLQR formulation is demonstrated. 3.1. Preliminary de/nitions To begin, a set of preliminary de!nitions is invoked for the bene!t of exposition of the main results. Denition 3. Saturated LQR controller move; denoted by uˆ k ; is formally de!ned as + if uk∗ ¿ u+ ; u uˆ k = uk∗ if u− 6 uk∗ 6 u+ ; u− if u∗ 6 u− : k Denition 4. The diJerence between the value of control move; produced by the saturated LQR; and the value obtained by the unconstrained LQR control law; is de!ned as duk = uˆ k − uk∗ : ∗ Remark 2. Note that uk+1 = −KLQ xk+1 where xk+1 = Axk + Buˆ k ; i.e. the optimal value of the control input at any time is dependent solely on the corresponding state rather than the past control input pro!le.
Next, it is assumed that saturated LQR control sequence, {uˆ k }∞ k=0 , has a particular form which is termed unidirectional saturation. Denition 5. The control sequence of saturated LQR is assumed to take the following unidirectional saturation form: uˆ 0 = uˆ 1 = · · · = uˆ M −1 = usat ; uˆ k =
uk∗
where u
= −KLQ xk sat
for k ¿ M;
is equal to either u+ or u− and M ¿ 1.
In other words, it is assumed that the !rst M elements of {uˆ k }∞ k=0 are saturated at the same constraint level while the rest are given by (2). Remark 3. Appendix C describes a method for !nding the region of state-space for which the saturated LQR sequence is given by De!nition 5. Remark 4. According to De!nitions 3–5; the !rst M non-zero elements of {duk }∞ k=0 are all of the same sign; k=M −1 ; and of the opposite sign to any element of {uˆ k }k=0 where M is as de!ned in De!nition 5. (A4) It is assumed that the terminal weighting matrix, P, is given as the solution to the corresponding DARE. It is worth noting that the results in this paper can be extended to MPC control problems with other terminal weights, as shown in [8] using an alternative proof and set of conditions. 3.2. Preliminary results In this subsection, a number of preliminary results are presented, which bene!t exposition of the main results presented later. In particular, necessary conditions for the optimality, in the CIHLQR sense, of a saturated LQR control sequence, satisfying (A3), are stated in Lemma 1. Also, in Lemma 2, a straightforward method of evaluating gradients of the cost with respect to the control move for the case of the saturated LQR control sequence is given. Before presenting the preliminary results a new notation is introduced in order to bene!t the exposition of the main results: @JN (x0 ; u) ∇uk JN (x0 ; u) ≡ ; (4) @uk @2 JN (x0 ; u) : (5) ∇2uk ;uj JN (x0 ; u) ≡ @uk @uj The following assumption ensures that all the elements in the corresponding Hessian matrix are positive semide!nite. Note that this condition lacks necessity although for a large class of systems, it can be shown to be satis!ed. Hence, it is not highly restrictive: (A5) ∇2uk ;uj JN (x0 ) ¿ 0 for k; j ∈ [0; N − 1]. In the following lemma, necessary optimality conditions, in terms of the CIHLQR cost function, are
O. Marjanovic et al. / Systems & Control Letters 46 (2002) 271 – 279
275
presented for a saturated LQR control sequence, assuming that (A3) holds.
not saturated. These two cases satisfy their respective optimality conditions; stated in Lemma 1.
Lemma 1. Saturated LQR control sequence; satisfying (A3); is optimal in terms of the standard CIHLQR cost; stated in (P3); if and only if the following two conditions are satis/ed:
Corollary 1. It is implicitly shown through the arguments in this section; Lemmas 1 and 2 in particular; that if the saturated LQR control sequence switches from one control limit to another ˆ · uˆ k ¡ 0 is no longer guaranteed then ∇uk JN (x0 ; u) since elements of {duk }∞ k=0 will be of opposite signs; which may cause optimality conditions not being satis/ed.
ˆ · uˆ k ¡ 0 1. ∇uk JN (x0 ; u) for uk = u+ or u− ; ˆ =0 for u− ¡ uk ¡ u+ . 2. ∇uk JN (x0 ; u) Proof. See Appendix A. Hence, the gradient of the cost with respect to the control input at any particular time must be of the opposite sign to the sign of a constraint bound that the control input is saturated at (condition 1), or it is equal to zero if the corresponding decision variable is not saturated (condition 2). Lemma 2. The gradient of the cost with respect to the input sequence; evaluated for saturated LQR control sequence; is given as follows: ˆ = ∇uk JN (x0 ; u)
N −1
∇2uk ;ui Ji+1 (x0 ; u) dui :
i=k
Proof. See Appendix B. 3.3. Main results The main result of this paper is presented in the following theorem showing the equivalence between the CIHLQR formulation and the saturated LQR control sequence. Theorem 1. Assuming that (A3) and (A5) hold; the saturated LQR control sequence is equivalent to that ∗ ∞ of the CIHLQR controller; i.e. {uˆ k }∞ k=0 ={uP k }k=0 ; in the case of unidirectional saturation control pattern; stated in De/nition 5. Proof. According to Remark 4 and Lemma 2; the gradient of the cost for unidirectional saturation pattern is of the opposite sign to the sign of the corresponding ˆ uˆ k ¡ 0; saturated LQR control move; i.e. ∇uk JN (x0 ; u)· or equal to zero if the corresponding control move is
3.4. Demonstration of conservatism in the standard CIHLQR formulation Since it can be easily shown that the solution to arg minu0 J1 (x0 ; u0 ) for any SISO system, satisfying (A3) – (A5), is given by the saturated LQR [16], it is demonstrated in the context of standard CIHLQR formulation that in some circumstances the terminal state inclusion condition, xN ∈ XKLQ , can be totally removed, allowing a reduction in the length of the prediction horizon while maintaining the same performance, which in turn reduces the complexity of the resulting CIHLQR controller. It is also implicitly shown in Example 4.1 in [15] and Example 5.3 in [3] that the most fundamental change in control law, with an increase in the length of the prediction horizon, occurs in the region of a state-space where saturated LQR control move would switch from one input constraint to another. This observation has been con!rmed in Corollary 1, using the arguments developed primarily in Lemmas 1 and 2. Addressing this issue is the subject of future work. 4. Results: rst-order systems In this section, the result presented in Theorem 1 is extended to !rst-order systems. In particular, it is shown in Theorem 2 that for the class of !rst-order delay-free sampled-data continuous-time systems, CIHLQR is equivalent to the saturated LQR for the entire constrained stabilisable region of state-space. In this way, conservatism of imposing xN ∈ XKLQ is clearly demonstrated. Furthermore, the region of constrained stabilisability for a class of open-loop unstable !rst-order systems is characterised in Theorem 3.
276
O. Marjanovic et al. / Systems & Control Letters 46 (2002) 271 – 279
First of all the following assumption is imposed on the state-space realisation of the !rst-order systems: (A6) A ¿ 0, B = 0. Remark 5. One class of system; which satis!es (A6); is the set of sampled-data continuous-time !rst-order systems. Note that the state weighting matrix, Q, in (P1) – (P3), is replaced by a scalar and the following modi!cation of (A1) is presented: (A7) Q ¿ 0. Remark 6. It can be shown that (A5) is automatically respected for the !rst-order systems obeying (A6) and (A7).
CL ∗ 3. If Aˆ k ¿ 1 ¿ ACL ; then |xk+1 | ¿ |xk |; uˆ k+1 = uˆ k ; CL CL resulting in Aˆ k+1 ¿ Aˆ k .
Corollary 2. The closed-loop dynamics sequence; CL CL∗ ; {Aˆ k }∞ k=0 ; is either monotonically decreasing to A CL ˆ increasing to A or remaining stationary at Ak = 1 CL CL CL depending on whether Aˆ 0 ¡ 1; Aˆ 0 ¿ 1 or Aˆ 0 = 1; respectively.
Proof. It follows from the proof of Lemma 3. The second main result of this paper is presented in the following theorem, clearly demonstrating the conservatism of standard CIHLQR formulation.
Denition 6. The closed-loop dynamics of the !rst-order system; regulated by means of the saturated LQR controller; are given by
Theorem 2. In the case of /rst-order systems; obeying (A3); (A6) and (A7); the CIHLQR control sequence coincides with the saturated LQR control sequence.
uˆ k CL ˆ Ak = A + B : xk
Proof. It follows from Theorem 1; Remark 6 and Lemma 3.
Remark 7. It can be shown; by analysing the corresponding DARE; that Buˆ k =xk is negative provided (A6) and (A7) hold.
Corollary 3. The presence of state constraints does not a:ect the result in Theorem 2 as long as x− ¡ 0 ¡ x+ .
Lemma 3. The saturated LQR control sequence for /rst-order systems; obeying (A3); (A6) and (A7); is always given in the form of unidirectional saturation; as presented in De/nition 5; provided x0 ∈ XKLQ .
Proof. It follows from the fact that both state and control trajectories are monotonic trajectories of state and input; as shown in the proof of Lemma 3.
Proof. CL CL ∗ 1. If Aˆ k ¡ 1 and xk+1 ∈ XKLQ then Aˆ k+j = ACL ; ∗ uˆ k+j = uk+j for j ¿ 1. Furthermore; since CL∗ 0¡A ¡ 1; both {xk+j }∞ j=1 and its correspond∗ }∞ are monotonically approaching the ing {uk+j j=1 origins of their respective 1-dimensional spaces. CL CL ∗ As a consequence Aˆ k ¿ Aˆ k+1 = ACL . CL 2. If Aˆ k ¡ 1 and xk+1 ∈ XKLQ then |xk+1 | ¡ |xk | and xk+1 and xk are of the same sign since CL ∗ Aˆ k ¿ ACL ¿ 0. Furthermore; uˆ k+1 = uˆ k = sat u ; which; together with Remark 7; leads to CL CL ∗ Aˆ k ¿ Aˆ k+1 ¿ ACL .
The !nal result of this paper is presented in the following theorem, characterising the region of constrained stabilisability in the case of open-loop unstable !rst-order systems and in the presence of both state and control constraints. Note, however, that this region is dependent on the open-loop dynamics and the level of control constraints, rather than a choice of feedback controller, as discussed in [5,14]. Theorem 3. The region of attraction for a given constrained control problem; in the case of /rst-order systems satisfying (A6); is given as follows: = {x | xmin ¡ x ¡ xmax };
O. Marjanovic et al. / Systems & Control Letters 46 (2002) 271 – 279
where −Bu+ ; for B ¿ 0: xmin = max x− ; A−1 −Bu− xmax = min x+ ; ; A−1 −Bu− for B ¡ 0: xmin = max x− ; ; A−1 −Bu+ xmax = min x+ ; A−1
277
constrained stabilisability has been de!ned for a class of open-loop unstable !rst-order systems. Appendix A
and with state constraints assumed to be of the following form:
The Karush–Kuhn–Tucker optimality condition states that if the intersection between a cone describing directions of improving cost and a cone describing feasible directions is an empty set then the particular choice of decision variables has achieved constrained optimal value, as stated on page 151 in [2]. In the context of (P3) this condition is given as q ∗ vi ∇u gi (uP∗ ; 1) = 0; ∇u JN (x0 ; uP ) +
x− ¡ xk ¡ x+
vi gi (uP∗ ; 1) = 0
i=1
for k ¿ 0:
Proof. It follows from proof of Lemma 3 and; in CL particular; Corollary 2 that Aˆ 0 ¡ 1; x− ¡ x0 ¡ x+ ⇒ x0 ∈ . Hence; once the open neighbourhood is found CL for which Aˆ 0 ¡ 1; then the resulting region of attraction is given as the intersection between that subset of state-space and the state admissible region; given by x− ¡ x0 ¡ x+ . Remark 8. It follows that may be combination of closed and open set; i.e. open in one direction and closed in another; if the state constraints are given by x− 6 x0 6 x+ .
vi ¿ 0
for i = 1; 2; : : : ; q;
for i = 1; 2; : : : ; q
with g(u; 1) 6 0 representing q general nonlinear inequality constraints imposed on control inputs, 0 and 1 representing vectors of appropriate dimensions with all entries equal to 0 and 1, respectively. Note that vi = 0 or vi = 0 depending on whether the corresponding constraint is active or inactive, respectively. Taking the particular structure of constraints, stated in (A3), into consideration, it follows that ∇uk g(u; 1)=1, if and only if uk = u+ and ∇uk g(u; 1) = −1, if and only if uk = u− , otherwise ∇uk g(u; 1) = 0. Appendix B
5. Conclusions In this paper the standard CIHLQR formulation has been reviewed. In particular, it has been demonstrated in Sections 3 and 4 that the terminal state inclusion condition can in many circumstances be relaxed without degrading the performance of the resulting control system. This has the advantage of reducing the complexity of the resulting controller. Furthermore, it is shown in Section 4 that for the entire class of sampled-data continuous-time !rst-order systems, with both state and control constraints, the terminal state inclusion condition can be removed without any degradation in subsequent performance. As a !nal result of this paper the region of
ˆ incorpoThis method of computing ∇uk JN (x0 ; u) rates the feedback structure of a saturated LQR and the knowledge of unconstrained optimal state-feedback in order to simplify subsequent evaluation. ˆ is computed in N In this approach, ∇uk JN (x0 ; u) steps, whereby for the jth step it is assumed that only the !rst j control moves are subject to control constraints and the rest are given by unconstrained LQR control gain. More speci!cally, assuming that (A4) holds and in accordance with principle of optimality, the in!nite horizon cost in jth step of evaluation j−1 is equivalent to Jj (x0 ; {uˆ k }k=0 ) and its gradients are j−1 since the remaining condependent only on {uˆ k }k=0 trol moves are assumed, although only temporarily, to be subject to no control constraints and, therefore,
278
O. Marjanovic et al. / Systems & Control Letters 46 (2002) 271 – 279
−1 optimal with respect to JN −j (xj {uk }Nk=j ) as well as ˆ Also note that, during the (j + 1)th step, the JN (x0 ; u). only numerical modi!cation to the gradients for !rst j control moves is due to the possible saturation of uj+1 . Otherwise, they remain unchanged. For example, in the !rst step:
Xi− =
x0 | − KLQ Ai x0 6 1 + KLQ
j=0
and Ai+1 x0 +
i k=0
∇u0 JN (x0 ; u) = ∇2u2 J1 (x0 ; u0 ) du0 ; for 1 6 k 6 N − 1:
In the second step it is assumed that uˆ 0 and uˆ 1 , alone, are subject to control constraints: ∇u0 JN (x0 ; u) = ∇2u2 J1 (x0 ; u0 ) du0 0
+∇2u0 ;u1 J2 (x0 ; u0 ; u1 ) du1 ; ∇u1 JN (x0 ; u) = ∇2u2 J2 (x0 ; u0 ; u1 ) du1 ; 1
∇uk JN (x0 ; u) = 0
Ak Bu− ∈ XKLQ
Aj B u −
for i = 1; : : : ; M − 1;
0
∇uk JN (x0 ; u) = 0
i−1
where Xi± de!nes the set of initial conditions for which every element of {uˆ k }ik=0 is saturated at u± while the remaining control input trajectory is given by the unconstrained LQR law, i.e. uˆ k = −KLQ xk for k ¿ i. Then the set of initial conditions for which unidirec∗ tional saturation occurs, denoted here as XLQ , is given by M −1 M −1 − ∗ + ∪ ; XLQ = Xi Xi i=0
for 2 6 k 6 N − 1:
Finally, the condensed formula for all N steps of evaluation is given as a result of this lemma.
i=0
where M is as given in De!nition 5. Finding the maximum value of M for which XM± is not empty as well as re!ning this method is a subject of future work. References
Appendix C One possible method of characterising the set of initial conditions for which saturated LQR control sequence results in unidirectional saturation is given in this appendix. De!ne the following subsets of state space: X0+ = {x0 | − KLQ x0 ¿ u+ and Ax0 + Bu+ ∈ XKLQ }; X0− = {x0 | − KLQ x0 6 u− and Ax0 + Bu− ∈ XKLQ }; i−1 Xi+ = x0 | − KLQ Ai x0 ¿ 1 + KLQ Aj B u + j=0
and Ai+1 x0 +
i k=0
for i = 1; : : : ; M − 1;
Ak Bu+ ∈ XKLQ
[1] B.D.O. Anderson, J.B. Moore, Linear Optimal Control, Prentice-Hall, Englewood CliJs, NJ, 1971. [2] M.S. Bazaraa, H.D. Sherali, C.M. Shetty, Nonlinear Programming – Theory and Algorithms, Wiley, New York, 1993. [3] A. Bemporad, M. Morari, V. Dua, E.N. Pistikopoulos, The explicit linear quadratic regulator for constrained systems, Technical Report AUT99-16, Automatic Control Laboratory, ETH, Switzerland, 1999. [4] A. Bemporad, M. Morari, V. Dua, E.N. Pistikopoulos, The explicit solution of model predictive control via multiparametric quadratic programming, Proceedings of the American Control Conference, Chicago, IL, June 2000, pp. 872–876. [5] D. Chmielewski, V. Manousiouthakis, On constrained in!nite-time linear quadratic optimal control, Systems Control Lett. 29 (1996) 121–129. [6] J.A. De Dona, G.C. Goodwin, Disturbance sensitivity issues in predictive control, Internat. J. Adapt. Control Signal Process. 13 (6) (1999) 507–519. [7] J.A. De Dona, G.C. Goodwin, Elucidation of the state-space regions wherein model predictive control and anti-windup strategies achieve identical control policies, Proceedings of the American Control Conference, Chicago, IL, 2000.
O. Marjanovic et al. / Systems & Control Letters 46 (2002) 271 – 279 [8] J.A. De Dona, G.C. Goodwin, M.M. Seron, Anti-windup and model predictive control: reTections and connections European J. Control 6 (5) (2000) 467–477. [9] H. Kwakernaak, R. Sivan, Linear Optimal Control Systems, Wiley, New York, 1972. [10] W.S. Levine (Ed.), The Control Handbook, CRC Press, Boca Raton, 1996. [11] D.Q. Mayne, J.B. Rawlings, C.V. Rao, P.O.M. Scokaert, Constrained model predictive control: stability and optimality, Automatica 36 (2000) 789–814. [12] M. Minoux, Mathematical Programming – Theory and Algorithms, Wiley, New York, 1986.
279
[13] C.V.D. Panne, Methods for Linear and Quadratic Programming, North-Holland, Amsterdam, 1975. [14] P.O.M. Scokaert, J.B. Rawlings, Constrained linear quadratic regulation, IEEE Trans. Automat. Control 43 (8) (1998). [15] M.M. Seron, J.A. De Dona, G.C. Goodwin, Global analytical model predictive control with input constraints, Proceedings of the 39th IEEE Conference on Decision and Control, Sydney, Australia, 2000, pp. 154 –159. [16] S. Valluri, M. Soroush, M. Nikravesh, Shortest-predictionhorizon non-linear model-predictive control, Chem. Engrg. Sci. 53 (2) (1998) 273–292.
www.elsevier.com/locate/sysconle
Minimising conservatism in in!nite-horizon LQR control Ognjen Marjanovic, Barry Lennox ∗ , Peter Goulding, David Sandoz School of Engineering, Control Technology Centre, University of Manchester, Manchester, UK M13 9PL Received 19 February 2001; received in revised form 1 March 2002
Abstract This paper studies the formulation of the constrained in!nite horizon linear quadratic regulator control law (CIHLQR). Results from recent studies in this area are extended to show that conditions used in the standard formulation of the CIHLQR law are not necessary, but merely su:cient. Through the use of a novel proof it is shown that for a general SISO system with input constraints and certain conditions imposed, saturated LQR provides the same control sequence as CIHLQR. It is further shown that saturated LQR is equivalent to the CIHLQR in the case of !rst-order systems, subject to both state and control constraints. Finally, the region of constrained stabilisability is characterised for the case of open-loop unstable c 2002 Elsevier Science B.V. All rights reserved. !rst-order systems. Keywords: Model predictive control; In!nite horizon; Linear quadratic regulator
1. Introduction A highly successful approach to the control of constrained linear systems that has been widely accepted by academia and industry alike is model predictive control (MPC). In this approach constrained regulation is achieved by employing repetitive on-line optimisation of a system’s performance index (cost function), subject to limiting inequality constraints being imposed onto the system variables. Whilst there have been many successful applications of MPC technology in industry, issues such as feasibility, robustness and stability remain important areas of research. In recent years several MPC formulations have been proposed that through the imposition of certain conditions on the system variables ∗ Corresponding author. Tel.: +44-161-275-4324; fax: +44+161-275-4346. E-mail address: [email protected] (B. Lennox).
and=or the structure of the performance index ensure feasibility and nominal stability. A thorough review of these techniques is presented in [11]. One example of an MPC formulation that ensures nominal stability is constrained in!nite horizon linear quadratic regulator (CIHLQR) [5,14]. This approach considers behaviour of the system over an in!nite horizon in the future, hence addressing the convergence property of the system variables and hence stability. In addition, CIHLQR establishes feasibility and stability results, which state that if there is no feasible=stable CIHLQR solution for the given problem, then the problem itself is insoluble by any other controller. To solve an in!nite-dimensional problem using a !nite-horizon formulation, inclusion condition is imposed onto the terminal state vector. This, in turn, dictates the length of the prediction horizon for the !nite-horizon equivalent of the CIHLQR and hence the complexity of the resulting controller. The typical method for solving MPC control problems, such as
c 2002 Elsevier Science B.V. All rights reserved. 0167-6911/02/$ - see front matter PII: S 0 1 6 7 - 6 9 1 1 ( 0 2 ) 0 0 1 5 1 - 2
272
O. Marjanovic et al. / Systems & Control Letters 46 (2002) 271 – 279
CIHLQR, is through the use of non-analytical methods such as quadratic programming, see [12,13]. This can introduce a signi!cant computational burden, particularly in situations where the length of the prediction horizon, required to guarantee CIHLQR performance, is large. Recently, a number of papers have addressed the issue of reducing the on-line computation requirements of the standard MPC control problem. In particular, it has been shown [7,8] that for a general SISO system with no state constraints, saturated linear MPC, designed for the unconstrained case, is equivalent to its constrained counterpart for a certain non-trivial region of the state space and under a certain set of conditions, demonstrating the equivalence between simple anti-windup scheme and the constrained MPC. In [6], the solution to the !nite-horizon MPC controller with no penalty on control energy is derived for the case of !rst-order systems with constraints on control variables alone, reducing the on-line computational burden of constrained MPC controller implementation to a minimum. The solution takes the form of the standard linear MPC controller, designed for the unconstrained case. Furthermore, the explicit analytical solution to the standard MPC has recently been proposed in [3,4,15], wherein the controller takes the form of a piecewise-a:ne mapping between state and the corresponding optimal control move. In this way on-line computation is reduced to a minimum, allowing MPC to be implemented in situations where relatively fast dynamics may exist. A major disadvantage of this approach is that, as the authors state, the complexity of the problem increases signi!cantly as the number of control and state variables increases and therefore the approach may only be suitable for relatively simple systems. This paper focuses on CIHLQR for which it is shown that a crucial condition in its standard formulation may in many situations be relaxed. Relaxation of this condition leads to a reduction in the complexity of the resulting controller, which results in a signi!cant reduction in the on-line computation requirements of the controller. This paper is organised as follows. In Section 2, the standard unconstrained LQR and CIHLQR are introduced. In Section 3 it is proved, using diJerent arguments to those in [8], that in the case of a general SISO system with input constraints and under certain conditions, saturated LQR provides the same control
sequence as CIHLQR. This result is analysed further to show that a crucial condition in the standard CIHLQR formulation may in many situations be relaxed, leading to a simpler controller. In Section 4, the analytic solution to the CIHLQR with both control and state constraints is developed for a class of sampled-data continuous-time !rst-order systems, using the arguments presented in Section 3. The solution is shown to be the same as the saturated LQR controller, con!rming the original hypothesis regarding the lack of necessity in the standard CIHLQR formulation. Furthermore, the region of constrained stabilisability is characterised for the case of open-loop unstable !rst-order systems. This region is shown to be independent on the particular choice of cost function. Finally, conclusions and directions for future work are presented in Section 5. 2. Problem formulation Consider a SISO discrete linear time-invariant system of the following form: xk+1 = Axk + Buk ;
(1)
where x ∈ Rn is the state vector, u ∈ R is the control input, A ∈ Rn×n is the state transition matrix, B ∈ Rn×1 is the control distribution vector and k ∈ Z+ is a non-negative integer denoting the sampling time instant. The control problem considered in this paper is that of state regulation to the origin. In particular, the requirement is that the state is regulated in an optimal manner, de!ned by the weighted quadratic cost function of the state and control trajectories. The main focus of this paper is on CIHLQR, however, for the bene!t of clarity, the standard unconstrained LQR problem is !rst presented. (P1) Unconstrained in!nite-horizon linear quadratic regulation problem: Given an initial state x0 , the task is to !nd the unconstrained optimal control sequence ∗ ∗ u∗ = {uk∗ }∞ k=0 = {u0 ; u1 ; : : :}, which minimises the following cost function: ∞ J (x0 ; u) = (xkT Qxk + ukT Ruk ) k=0
subject to xk+1 = Axk + Buk for ∀k ∈ Z + .
O. Marjanovic et al. / Systems & Control Letters 46 (2002) 271 – 279 ∞
273
The assumptions made on the weightings Q and R are as follows:
J (x0 ; uP∗ ) = min
(A1) Q = QT ¿ 0, (A2) R ¿ 0.
subject to xk+1 = Axk + Buk and u− 6 uk 6 u+ for ∀k ∈ Z + .
The solution to (P1) is readily available, assuming that (A1) and (A2) hold, in the linear and time-invariant state feedback form [1,9,10]: uk∗ = −KLQ xk ;
(2) 1×n
where KLQ ∈ R is obtained by solving the discrete algebraic Riccati equation (DARE), P = AT P − PB(BT PB + R)−1 BT PA + Q, with P = P T ¿ 0 being its unique solution. This solution is then substituted into KLQ = (BT PB + R)−1 BT PA in order to obtain the optimal LQR controller gain, KLQ . Optimal value of J (x0 ; u) is given by the following relation: J (x0 ; u∗ ) = x0T Px0 ;
(3)
where P is the solution to the DARE. Denition 1. The closed-loop dynamics; resulting from the application of (2); are given as follows: ∗
ACL = A − BKLQ : Remark 1. Anderson and Moore [1]; Kwakernaak ∗ and Sivan [9] and Levine [10] state that ACL is stable with sound robustness and performance properties; provided that the standard assumptions on unconstrained stabilisability and detectability as well as (A1) and (A2) are respected. To enable the controller to account for constraints that will exist on a system it is necessary to consider the constrained extension to (P1). Before that, however, the following assumption is made regarding the structure of the constraints: (A3) The states are assumed to be unbounded while the control input is constrained by non-zero, time-invariant upper and lower bounds, i.e. u− 6 uk 6 u+ for k ¿ 0, with u− ¡ 0 ¡ u+ . Note, however, that the CIHLQR formulation, as presented in [5,14], addresses the state as well as control constraints. (P2) Constrained in!nite-horizon linear quadratic regulation problem (explicit form):
u
(xkT Qxk + ukT Ruk )
k=0
Due to the incorporation of an in!nite horizon and under the assumptions of unconstrained stabilisability and detectability as well as (A1) and (A2), this approach guarantees nominal stability so long as the optimal cost is !nite, i.e. J (x0 ; uP∗ ) ¡ ∞ ⇒ limk→∞ xk → 0 [5,14]. Unfortunately, the CIHLQR formulation, as presented in (P2), results in an insoluble in!nite-dimensional optimisation problem. However, by using the principle of optimality, it is possible to recast (P2) into the following !nite-horizon formulation. (P3) Constrained in!nite-horizon linear quadratic regulation problem (implicit form): N −1 ∗ T T T JN (x0 ; uP ) = min (xk Qxk + uk Ruk ) + xN PxN u
k=0
subject to xk+1 = Axk + Buk and u− 6 uk 6 u+ for 0 6 k 6 N − 1, and xN ∈ XKLQ . By choosing the solution of DARE to be equal to the value of the terminal weight, given as P in (P3), it is possible, according to (3), to substitute T J (xN ; {uP∗k }∞ k=N ) with xN PxN , under the assumption that KLQ is applied for k ¿ N with no subsequent violations of the control constraints [1,5,9,10,14]. This assumption is met by ensuring that the terminal state, xN , belongs to a certain set, formally de!ned below. Denition 2. XKLQ is the subset of state space within which evolution of the closed-loop system; given in De!nition 1; does not violate any constraints: XKLQ = {x0 | u− 6 − KLQ (A − BKLQ )k x0 6 u+ ; ∀k ∈ Z + }: In this paper, it is argued that in certain cases the terminal state inclusion condition, xN ∈ XKLQ , can be relaxed, removing the lower bound on N , required to guarantee CIHLQR performance. The general consequence of this result is simpli!cation of the implemented CIHLQR controller.
274
O. Marjanovic et al. / Systems & Control Letters 46 (2002) 271 – 279
3. Results: general SISO systems In this section an alternative proof to that provided in [8] is presented showing that for a class of SISO systems with hard input constraints and no state constraints, the saturated LQR control move is equivalent to that of the CIHLQR in many circumstances. By taking into account the fact that the saturated LQR control sequence is equivalent to a receding horizon control sequence of the solution to (P3) with no terminal state inclusion condition and with N = 1 so long as (A3) holds, see [16], the lack of necessity of the standard CIHLQR formulation is demonstrated. 3.1. Preliminary de/nitions To begin, a set of preliminary de!nitions is invoked for the bene!t of exposition of the main results. Denition 3. Saturated LQR controller move; denoted by uˆ k ; is formally de!ned as + if uk∗ ¿ u+ ; u uˆ k = uk∗ if u− 6 uk∗ 6 u+ ; u− if u∗ 6 u− : k Denition 4. The diJerence between the value of control move; produced by the saturated LQR; and the value obtained by the unconstrained LQR control law; is de!ned as duk = uˆ k − uk∗ : ∗ Remark 2. Note that uk+1 = −KLQ xk+1 where xk+1 = Axk + Buˆ k ; i.e. the optimal value of the control input at any time is dependent solely on the corresponding state rather than the past control input pro!le.
Next, it is assumed that saturated LQR control sequence, {uˆ k }∞ k=0 , has a particular form which is termed unidirectional saturation. Denition 5. The control sequence of saturated LQR is assumed to take the following unidirectional saturation form: uˆ 0 = uˆ 1 = · · · = uˆ M −1 = usat ; uˆ k =
uk∗
where u
= −KLQ xk sat
for k ¿ M;
is equal to either u+ or u− and M ¿ 1.
In other words, it is assumed that the !rst M elements of {uˆ k }∞ k=0 are saturated at the same constraint level while the rest are given by (2). Remark 3. Appendix C describes a method for !nding the region of state-space for which the saturated LQR sequence is given by De!nition 5. Remark 4. According to De!nitions 3–5; the !rst M non-zero elements of {duk }∞ k=0 are all of the same sign; k=M −1 ; and of the opposite sign to any element of {uˆ k }k=0 where M is as de!ned in De!nition 5. (A4) It is assumed that the terminal weighting matrix, P, is given as the solution to the corresponding DARE. It is worth noting that the results in this paper can be extended to MPC control problems with other terminal weights, as shown in [8] using an alternative proof and set of conditions. 3.2. Preliminary results In this subsection, a number of preliminary results are presented, which bene!t exposition of the main results presented later. In particular, necessary conditions for the optimality, in the CIHLQR sense, of a saturated LQR control sequence, satisfying (A3), are stated in Lemma 1. Also, in Lemma 2, a straightforward method of evaluating gradients of the cost with respect to the control move for the case of the saturated LQR control sequence is given. Before presenting the preliminary results a new notation is introduced in order to bene!t the exposition of the main results: @JN (x0 ; u) ∇uk JN (x0 ; u) ≡ ; (4) @uk @2 JN (x0 ; u) : (5) ∇2uk ;uj JN (x0 ; u) ≡ @uk @uj The following assumption ensures that all the elements in the corresponding Hessian matrix are positive semide!nite. Note that this condition lacks necessity although for a large class of systems, it can be shown to be satis!ed. Hence, it is not highly restrictive: (A5) ∇2uk ;uj JN (x0 ) ¿ 0 for k; j ∈ [0; N − 1]. In the following lemma, necessary optimality conditions, in terms of the CIHLQR cost function, are
O. Marjanovic et al. / Systems & Control Letters 46 (2002) 271 – 279
275
presented for a saturated LQR control sequence, assuming that (A3) holds.
not saturated. These two cases satisfy their respective optimality conditions; stated in Lemma 1.
Lemma 1. Saturated LQR control sequence; satisfying (A3); is optimal in terms of the standard CIHLQR cost; stated in (P3); if and only if the following two conditions are satis/ed:
Corollary 1. It is implicitly shown through the arguments in this section; Lemmas 1 and 2 in particular; that if the saturated LQR control sequence switches from one control limit to another ˆ · uˆ k ¡ 0 is no longer guaranteed then ∇uk JN (x0 ; u) since elements of {duk }∞ k=0 will be of opposite signs; which may cause optimality conditions not being satis/ed.
ˆ · uˆ k ¡ 0 1. ∇uk JN (x0 ; u) for uk = u+ or u− ; ˆ =0 for u− ¡ uk ¡ u+ . 2. ∇uk JN (x0 ; u) Proof. See Appendix A. Hence, the gradient of the cost with respect to the control input at any particular time must be of the opposite sign to the sign of a constraint bound that the control input is saturated at (condition 1), or it is equal to zero if the corresponding decision variable is not saturated (condition 2). Lemma 2. The gradient of the cost with respect to the input sequence; evaluated for saturated LQR control sequence; is given as follows: ˆ = ∇uk JN (x0 ; u)
N −1
∇2uk ;ui Ji+1 (x0 ; u) dui :
i=k
Proof. See Appendix B. 3.3. Main results The main result of this paper is presented in the following theorem showing the equivalence between the CIHLQR formulation and the saturated LQR control sequence. Theorem 1. Assuming that (A3) and (A5) hold; the saturated LQR control sequence is equivalent to that ∗ ∞ of the CIHLQR controller; i.e. {uˆ k }∞ k=0 ={uP k }k=0 ; in the case of unidirectional saturation control pattern; stated in De/nition 5. Proof. According to Remark 4 and Lemma 2; the gradient of the cost for unidirectional saturation pattern is of the opposite sign to the sign of the corresponding ˆ uˆ k ¡ 0; saturated LQR control move; i.e. ∇uk JN (x0 ; u)· or equal to zero if the corresponding control move is
3.4. Demonstration of conservatism in the standard CIHLQR formulation Since it can be easily shown that the solution to arg minu0 J1 (x0 ; u0 ) for any SISO system, satisfying (A3) – (A5), is given by the saturated LQR [16], it is demonstrated in the context of standard CIHLQR formulation that in some circumstances the terminal state inclusion condition, xN ∈ XKLQ , can be totally removed, allowing a reduction in the length of the prediction horizon while maintaining the same performance, which in turn reduces the complexity of the resulting CIHLQR controller. It is also implicitly shown in Example 4.1 in [15] and Example 5.3 in [3] that the most fundamental change in control law, with an increase in the length of the prediction horizon, occurs in the region of a state-space where saturated LQR control move would switch from one input constraint to another. This observation has been con!rmed in Corollary 1, using the arguments developed primarily in Lemmas 1 and 2. Addressing this issue is the subject of future work. 4. Results: rst-order systems In this section, the result presented in Theorem 1 is extended to !rst-order systems. In particular, it is shown in Theorem 2 that for the class of !rst-order delay-free sampled-data continuous-time systems, CIHLQR is equivalent to the saturated LQR for the entire constrained stabilisable region of state-space. In this way, conservatism of imposing xN ∈ XKLQ is clearly demonstrated. Furthermore, the region of constrained stabilisability for a class of open-loop unstable !rst-order systems is characterised in Theorem 3.
276
O. Marjanovic et al. / Systems & Control Letters 46 (2002) 271 – 279
First of all the following assumption is imposed on the state-space realisation of the !rst-order systems: (A6) A ¿ 0, B = 0. Remark 5. One class of system; which satis!es (A6); is the set of sampled-data continuous-time !rst-order systems. Note that the state weighting matrix, Q, in (P1) – (P3), is replaced by a scalar and the following modi!cation of (A1) is presented: (A7) Q ¿ 0. Remark 6. It can be shown that (A5) is automatically respected for the !rst-order systems obeying (A6) and (A7).
CL ∗ 3. If Aˆ k ¿ 1 ¿ ACL ; then |xk+1 | ¿ |xk |; uˆ k+1 = uˆ k ; CL CL resulting in Aˆ k+1 ¿ Aˆ k .
Corollary 2. The closed-loop dynamics sequence; CL CL∗ ; {Aˆ k }∞ k=0 ; is either monotonically decreasing to A CL ˆ increasing to A or remaining stationary at Ak = 1 CL CL CL depending on whether Aˆ 0 ¡ 1; Aˆ 0 ¿ 1 or Aˆ 0 = 1; respectively.
Proof. It follows from the proof of Lemma 3. The second main result of this paper is presented in the following theorem, clearly demonstrating the conservatism of standard CIHLQR formulation.
Denition 6. The closed-loop dynamics of the !rst-order system; regulated by means of the saturated LQR controller; are given by
Theorem 2. In the case of /rst-order systems; obeying (A3); (A6) and (A7); the CIHLQR control sequence coincides with the saturated LQR control sequence.
uˆ k CL ˆ Ak = A + B : xk
Proof. It follows from Theorem 1; Remark 6 and Lemma 3.
Remark 7. It can be shown; by analysing the corresponding DARE; that Buˆ k =xk is negative provided (A6) and (A7) hold.
Corollary 3. The presence of state constraints does not a:ect the result in Theorem 2 as long as x− ¡ 0 ¡ x+ .
Lemma 3. The saturated LQR control sequence for /rst-order systems; obeying (A3); (A6) and (A7); is always given in the form of unidirectional saturation; as presented in De/nition 5; provided x0 ∈ XKLQ .
Proof. It follows from the fact that both state and control trajectories are monotonic trajectories of state and input; as shown in the proof of Lemma 3.
Proof. CL CL ∗ 1. If Aˆ k ¡ 1 and xk+1 ∈ XKLQ then Aˆ k+j = ACL ; ∗ uˆ k+j = uk+j for j ¿ 1. Furthermore; since CL∗ 0¡A ¡ 1; both {xk+j }∞ j=1 and its correspond∗ }∞ are monotonically approaching the ing {uk+j j=1 origins of their respective 1-dimensional spaces. CL CL ∗ As a consequence Aˆ k ¿ Aˆ k+1 = ACL . CL 2. If Aˆ k ¡ 1 and xk+1 ∈ XKLQ then |xk+1 | ¡ |xk | and xk+1 and xk are of the same sign since CL ∗ Aˆ k ¿ ACL ¿ 0. Furthermore; uˆ k+1 = uˆ k = sat u ; which; together with Remark 7; leads to CL CL ∗ Aˆ k ¿ Aˆ k+1 ¿ ACL .
The !nal result of this paper is presented in the following theorem, characterising the region of constrained stabilisability in the case of open-loop unstable !rst-order systems and in the presence of both state and control constraints. Note, however, that this region is dependent on the open-loop dynamics and the level of control constraints, rather than a choice of feedback controller, as discussed in [5,14]. Theorem 3. The region of attraction for a given constrained control problem; in the case of /rst-order systems satisfying (A6); is given as follows: = {x | xmin ¡ x ¡ xmax };
O. Marjanovic et al. / Systems & Control Letters 46 (2002) 271 – 279
where −Bu+ ; for B ¿ 0: xmin = max x− ; A−1 −Bu− xmax = min x+ ; ; A−1 −Bu− for B ¡ 0: xmin = max x− ; ; A−1 −Bu+ xmax = min x+ ; A−1
277
constrained stabilisability has been de!ned for a class of open-loop unstable !rst-order systems. Appendix A
and with state constraints assumed to be of the following form:
The Karush–Kuhn–Tucker optimality condition states that if the intersection between a cone describing directions of improving cost and a cone describing feasible directions is an empty set then the particular choice of decision variables has achieved constrained optimal value, as stated on page 151 in [2]. In the context of (P3) this condition is given as q ∗ vi ∇u gi (uP∗ ; 1) = 0; ∇u JN (x0 ; uP ) +
x− ¡ xk ¡ x+
vi gi (uP∗ ; 1) = 0
i=1
for k ¿ 0:
Proof. It follows from proof of Lemma 3 and; in CL particular; Corollary 2 that Aˆ 0 ¡ 1; x− ¡ x0 ¡ x+ ⇒ x0 ∈ . Hence; once the open neighbourhood is found CL for which Aˆ 0 ¡ 1; then the resulting region of attraction is given as the intersection between that subset of state-space and the state admissible region; given by x− ¡ x0 ¡ x+ . Remark 8. It follows that may be combination of closed and open set; i.e. open in one direction and closed in another; if the state constraints are given by x− 6 x0 6 x+ .
vi ¿ 0
for i = 1; 2; : : : ; q;
for i = 1; 2; : : : ; q
with g(u; 1) 6 0 representing q general nonlinear inequality constraints imposed on control inputs, 0 and 1 representing vectors of appropriate dimensions with all entries equal to 0 and 1, respectively. Note that vi = 0 or vi = 0 depending on whether the corresponding constraint is active or inactive, respectively. Taking the particular structure of constraints, stated in (A3), into consideration, it follows that ∇uk g(u; 1)=1, if and only if uk = u+ and ∇uk g(u; 1) = −1, if and only if uk = u− , otherwise ∇uk g(u; 1) = 0. Appendix B
5. Conclusions In this paper the standard CIHLQR formulation has been reviewed. In particular, it has been demonstrated in Sections 3 and 4 that the terminal state inclusion condition can in many circumstances be relaxed without degrading the performance of the resulting control system. This has the advantage of reducing the complexity of the resulting controller. Furthermore, it is shown in Section 4 that for the entire class of sampled-data continuous-time !rst-order systems, with both state and control constraints, the terminal state inclusion condition can be removed without any degradation in subsequent performance. As a !nal result of this paper the region of
ˆ incorpoThis method of computing ∇uk JN (x0 ; u) rates the feedback structure of a saturated LQR and the knowledge of unconstrained optimal state-feedback in order to simplify subsequent evaluation. ˆ is computed in N In this approach, ∇uk JN (x0 ; u) steps, whereby for the jth step it is assumed that only the !rst j control moves are subject to control constraints and the rest are given by unconstrained LQR control gain. More speci!cally, assuming that (A4) holds and in accordance with principle of optimality, the in!nite horizon cost in jth step of evaluation j−1 is equivalent to Jj (x0 ; {uˆ k }k=0 ) and its gradients are j−1 since the remaining condependent only on {uˆ k }k=0 trol moves are assumed, although only temporarily, to be subject to no control constraints and, therefore,
278
O. Marjanovic et al. / Systems & Control Letters 46 (2002) 271 – 279
−1 optimal with respect to JN −j (xj {uk }Nk=j ) as well as ˆ Also note that, during the (j + 1)th step, the JN (x0 ; u). only numerical modi!cation to the gradients for !rst j control moves is due to the possible saturation of uj+1 . Otherwise, they remain unchanged. For example, in the !rst step:
Xi− =
x0 | − KLQ Ai x0 6 1 + KLQ
j=0
and Ai+1 x0 +
i k=0
∇u0 JN (x0 ; u) = ∇2u2 J1 (x0 ; u0 ) du0 ; for 1 6 k 6 N − 1:
In the second step it is assumed that uˆ 0 and uˆ 1 , alone, are subject to control constraints: ∇u0 JN (x0 ; u) = ∇2u2 J1 (x0 ; u0 ) du0 0
+∇2u0 ;u1 J2 (x0 ; u0 ; u1 ) du1 ; ∇u1 JN (x0 ; u) = ∇2u2 J2 (x0 ; u0 ; u1 ) du1 ; 1
∇uk JN (x0 ; u) = 0
Ak Bu− ∈ XKLQ
Aj B u −
for i = 1; : : : ; M − 1;
0
∇uk JN (x0 ; u) = 0
i−1
where Xi± de!nes the set of initial conditions for which every element of {uˆ k }ik=0 is saturated at u± while the remaining control input trajectory is given by the unconstrained LQR law, i.e. uˆ k = −KLQ xk for k ¿ i. Then the set of initial conditions for which unidirec∗ tional saturation occurs, denoted here as XLQ , is given by M −1 M −1 − ∗ + ∪ ; XLQ = Xi Xi i=0
for 2 6 k 6 N − 1:
Finally, the condensed formula for all N steps of evaluation is given as a result of this lemma.
i=0
where M is as given in De!nition 5. Finding the maximum value of M for which XM± is not empty as well as re!ning this method is a subject of future work. References
Appendix C One possible method of characterising the set of initial conditions for which saturated LQR control sequence results in unidirectional saturation is given in this appendix. De!ne the following subsets of state space: X0+ = {x0 | − KLQ x0 ¿ u+ and Ax0 + Bu+ ∈ XKLQ }; X0− = {x0 | − KLQ x0 6 u− and Ax0 + Bu− ∈ XKLQ }; i−1 Xi+ = x0 | − KLQ Ai x0 ¿ 1 + KLQ Aj B u + j=0
and Ai+1 x0 +
i k=0
for i = 1; : : : ; M − 1;
Ak Bu+ ∈ XKLQ
[1] B.D.O. Anderson, J.B. Moore, Linear Optimal Control, Prentice-Hall, Englewood CliJs, NJ, 1971. [2] M.S. Bazaraa, H.D. Sherali, C.M. Shetty, Nonlinear Programming – Theory and Algorithms, Wiley, New York, 1993. [3] A. Bemporad, M. Morari, V. Dua, E.N. Pistikopoulos, The explicit linear quadratic regulator for constrained systems, Technical Report AUT99-16, Automatic Control Laboratory, ETH, Switzerland, 1999. [4] A. Bemporad, M. Morari, V. Dua, E.N. Pistikopoulos, The explicit solution of model predictive control via multiparametric quadratic programming, Proceedings of the American Control Conference, Chicago, IL, June 2000, pp. 872–876. [5] D. Chmielewski, V. Manousiouthakis, On constrained in!nite-time linear quadratic optimal control, Systems Control Lett. 29 (1996) 121–129. [6] J.A. De Dona, G.C. Goodwin, Disturbance sensitivity issues in predictive control, Internat. J. Adapt. Control Signal Process. 13 (6) (1999) 507–519. [7] J.A. De Dona, G.C. Goodwin, Elucidation of the state-space regions wherein model predictive control and anti-windup strategies achieve identical control policies, Proceedings of the American Control Conference, Chicago, IL, 2000.
O. Marjanovic et al. / Systems & Control Letters 46 (2002) 271 – 279 [8] J.A. De Dona, G.C. Goodwin, M.M. Seron, Anti-windup and model predictive control: reTections and connections European J. Control 6 (5) (2000) 467–477. [9] H. Kwakernaak, R. Sivan, Linear Optimal Control Systems, Wiley, New York, 1972. [10] W.S. Levine (Ed.), The Control Handbook, CRC Press, Boca Raton, 1996. [11] D.Q. Mayne, J.B. Rawlings, C.V. Rao, P.O.M. Scokaert, Constrained model predictive control: stability and optimality, Automatica 36 (2000) 789–814. [12] M. Minoux, Mathematical Programming – Theory and Algorithms, Wiley, New York, 1986.
279
[13] C.V.D. Panne, Methods for Linear and Quadratic Programming, North-Holland, Amsterdam, 1975. [14] P.O.M. Scokaert, J.B. Rawlings, Constrained linear quadratic regulation, IEEE Trans. Automat. Control 43 (8) (1998). [15] M.M. Seron, J.A. De Dona, G.C. Goodwin, Global analytical model predictive control with input constraints, Proceedings of the 39th IEEE Conference on Decision and Control, Sydney, Australia, 2000, pp. 154 –159. [16] S. Valluri, M. Soroush, M. Nikravesh, Shortest-predictionhorizon non-linear model-predictive control, Chem. Engrg. Sci. 53 (2) (1998) 273–292.