Necessary and sufficient condition for convergence of iterative learning algorithm

Automatica 38 (2002) 1257 – 1260

www.elsevier.com/locate/automatica

Technical Communique

Necessary and su$cient condition for convergence of iterative learning algorithm
S.N. Huanga; ∗ , K.K. Tana , T.H. Leea a Department

of Electrical and Computer Engineering, National University of Singapore, 10 Kent Ridge Crescent, Singapore 119260, Singapore

Received 9 January 2001; received in revised form 6 November 2001; accepted 12 January 2002

Abstract This correspondence is concerned with an iterative learning algorithm for MIMO linear time-varying systems. We provide a necessary and su$cient condition for the existence of a convergent algorithm. The result extends the main result in Saab (IEEE Trans. Automat. Control 40(6) (1995) 1138). ? 2002 Published by Elsevier Science Ltd. Keywords: Iterative learning control; Linear time-varying systems; MIMO control

1. Introduction In recent years, iterative learning control (ILC) has been a very active research area. The interested readers may refer to linear ILC algorithms proposed by Arimoto, Kawamura, and Miyazaki (1984), Bien and Huh (1989), Lucibello (1992), and Sogo and Adachi (1994), and also to nonlinear ILC algorithms proposed by Chien (1998), French and Rogers (2000), Lee, Tan, Lim, and Dou (2000), and Tan et al. (2001). The monograph by Moore (1992) contains more details on the background of ILC. A recent book (Chen & Wen, 1999) on ILC surveys the status of the Aeld as in 1998. However, the convergence conditions of most ILC algorithms discussed are usually su$cient ones. Motivated to produce a less stiB condition, Kurek and Zaremba (1993) derived a necessary and su$cient condition for linear time-invariant systems. Furthermore, Saab (1995) derived another necessary and su$cient condition. However, none of the papers, which addresses linear time-varying systems, considers the formulation of a necessary and su$cient condition for convergence. In this correspondence, we deal with the learning control problem for a multivariable linear time-varying system. A discrete-time
This paper was not presented at any IFAC meeting. This paper was recommended for publication in revised form by Associate Editor Geir E. Dullerud under the direction of Editor Paul Van den Hof. ∗ Corresponding author. Tel.: +65-8744460; fax: +65-777-3117. E-mail address: [email protected] (S.N. Huang).

0005-1098/02/$ - see front matter ? 2002 Published by Elsevier Science Ltd. PII: S 0 0 0 5 - 1 0 9 8 ( 0 2 ) 0 0 0 1 4 - 6

learning algorithm is proposed. The proposed control can be viewed as an extended version of the ILC algorithm presented by Saab (1995). A necessary and su$cient condition for the existence of convergent algorithm is given. Based on this fundamental result, explicit control gains K(t) can be obtained which can guarantee an iteration convergence. 2. Necessary and sucient condition for convergence Consider a iterative linear discrete-time system as follows: xi (t + 1) = A(t)xi (t) + B(t)ui (t);

yi (t) = C(t)xi (t);

(1)

where i denotes the ith iterative operation of the system; xi (t) ∈ Rn ; ui (t) ∈ Rm , and yi (t) ∈ Rr are the state, control input, and output of the system, respectively; t ∈ [0; N ] is the time interval; and A(t); B(t); and C(t) are matrices with appropriate dimensions. Given a desired output trajectory yd (t), t ∈ [1; N ], the objective is to And the optimal input ud (j); j=0; 1; 2; : : : ; N −1; by iterative learning, so that when ui (t) → ud (t), the system output yi (t) will track yd (t) as close as possible. Denote ei (t) = yd (t) − yi (t). Saab (1995) derived an iterative learning law for linear time-invariant systems, given by ui (t) = ui−1 (t) + K[ei−1 (t + 1) − ei−1 (t)];

(2)

where K is a constant, and provided a necessary and su$cient condition for the convergence of the algorithm.

1258

S.N. Huang et al. / Automatica 38 (2002) 1257–1260

However, the algorithm is applicable only to time-invariant cases. In this paper, we extend the result of Saab (1995) to the case of linear time-varying systems. Following (2), an iterative control for the linear time-varying case can be one described by

.. .

ui (t) = ui−1 (t) + K(t)[ei−1 (t + 1) − ei−1 (t)]:

ei−1 (N ) − ei−1 (N − 1)

(3)

ei−1 (2) − ei−1 (1) = C(2)B(1)ui−1 (1) + [C(2)A(1) − C(1)]B(0)ui−1 (0);

=C(N )B(N − 1)ui−1 (N − 1)   N
−1 N
−2 + C(N ) A(k) − C(N − 1) A(k)

When this ILC (3) is applied to system (1), the analysis of convergence of the learning scheme is to show that a similar conclusion can be drawn for the time-varying system (1). Theorem 1. Consider the iterative linear time-varying system (1); under the assumption that at each iteration; the control system (1) is exactly re-initialized at xd (0); i.e.; xi (0) = xd (0). There exists a sequence {K(t); t = 0; 1; : : : ; N − 1}; where the learning law given by (3) will generate a sequence of inputs ui (t) such that the input error ud (t) − ui (t) and the state error xd (t) − xi (t) converge to zero as i → ∞; if and only if (i5) all matrices C(t + 1)B(t); t = 0; 1; : : : ; N − 1 have full column rank. Proof. The general solution of (1) is of the form yi (t + 1) = C(t + 1)

t


t
k=1

+ · · · + C(t + 1)A(t)B(t − 1)ui (t − 1)

×B(N − 2)ui−1 (N − 2):

(4)

(5)

Using (4); we can compute ei−1 (t) by deAning xi (t) = xd (t) − xi (t) ei−1 (t + 1) = C(t + 1)

t


G(t + 1) = I − K(t)C(t + 1)B(t);

+C(t + 1)

(12)

where I is the unit matrix. Next; [ui (0); ui (1); : : : ; ui (N − 1)]T will be derived over the whole interval [0; N − 1]. For the case t = 0, substituting (9) into (5) yields

ui (1) = [I − K(1)C(2)B(1)]ui−1 (1) − K(1) ×[C(2)A(1) − C(1)]B(0)ui−1 (0) = G(2)ui−1 (1) + G2; 1 ui−1 (0); where G2; 1 = −K(1)[C(2)A(1) − C(1)]B(0). Proceeding along the same fashion, the following equation can be obtained for t = N − 1. ui (N − 1) = G(N )ui−1 (N − 1) + GN; 1 ui−1 (0) + · · ·

A(k)xi−1 (0)

k=0

t


(11)

DeAne

DeAne ui (t) = ud (t) − ui (t). Thus; (3) gives ui (t) = ui−1 (t) − K(t)[ei−1 (t + 1) − ei−1 (t)]:

k=1

where the notation G(1) and ei−1 (0) = 0 are deAned in (12) and (8), respectively. For the case t = 1, substituting (10) into (5) produces

A(k)B(0)ui (0)

+ C(t + 1)B(t)ui (t):

k=1

×B(0)ui−1 (0)+ · · · +[C(N )A(N −1)−C(N −1)]

ui (0) = [I − K(0)C(1)B(0)]ui−1 (0) = G(1)ui−1 (0);

A(k)xi (0)

k=0

+ C(t + 1)

(10)

+GN; N −1 ui−1 (N − 2); A(k)B(0)ui−1 (0) + · · ·

where

k=1

+C(t + 1)A(t)B(t − 1)ui−1 (t − 1) + C(t + 1)B(t)ui−1 (t):

(6)

Since; by the assumption; the control system (1) is exactly re-initialized at xd (0); this implies that xi (0) = 0;

(7)

ei (0) = yd (0) − yi (0) = C(0)(xd (0) − xi (0)) = 0:

(8)

GN; 1 = −K(N − 1)   N
−1 N
−2 × C(N ) A(k) − C(N − 1) A(k) B(0); k=1

.. .

k=1

GN; N −1 = −K(N − 1)[C(N )A(N − 1) − C(N − 1)] B(N − 2)

By virtue of the above initial conditions; it can be obtained from expression (6) that

These equations can be further rewritten in the following composite form:

ei−1 (1) − ei−1 (0) = C(1)B(0)ui−1 (0);

Ui = GUi−1 ;

(9)

(13)

S.N. Huang et al. / Automatica 38 (2002) 1257–1260

1259

where Ui = [uiT (0); uiT (1); uiT (2); : : : ; uiT (N − 1)]T and   G(1) 0 ···  G2; 1 G(2) 0 ··· 0     G3; 1 G3; 2 G(3) · · · 0  G= :  .. .. .. .. ..   . . . . . 

Expanding (16), the following equations can be obtained:

It should be emphasized that the matrix G is an Nm × Nm constant matrix with respect to the iteration i. Thus, the system becomes a discrete time-invariant system. According to Theorem 5.D4 of Chen (1999), Ui is convergent iB G is stable, i.e., all |k [G]| ¡ 1. Since the matrix G is a lower block triangular, it follows that

ui (2) = [B(2)T B(2)]−1 B(2)T xi (3)

GN; 1

k [G] =

N

GN; 2

GN; 3

ui (0) = [B(0)T B(0)]−1 B(0)T xi (1); ui (1) = [B(1)T B(1)]−1 B(1)T xi (2) − [B(1)T B(1)]−1 B(1)T A(1)B(0)ui (0);

· · · G(N )

{k [G(j)]}:

(14)

xi (t + 1) =

t


A(k)xi (0) +

k=0

t


A(k)B(0)ui (0) + · · ·

+ A(2)B(1)ui (1)];

ui (N − 1) = [B(N − 1)T B(N − 1)]−1 B(N − 1)T xi (N ) − [B(N − 1)T B(N − 1)]−1 B(N − 1)T N −1
× A(k)B(0)ui (0) + · · · + A(N − 1) (20)

Since limi→∞ xi (t) = 0 for each t = 1; 2; : : : ; N , using (17) it follows lim ui (0) = 0:

i→∞

(21)

Using this result and limi→∞ xi (t) = 0; t ∈ [1; N ]; and using (18) it is straightforward to obtain

i→∞

k=1

(15)

Note that xi (0)=0 is used. This implies that limi→∞ xi (t+ 1) = 0, if ui (j) = 0 (j = 0; 1; 2; : : : ; N − 1:) Necessity: Since each C(t + 1)B(t) is assumed to be of full column rank, each B(t); t = 0; 1; : : : ; N − 1; is implicitly assumed to be of full column rank (Saab, 1995). Hence Eq. (15) can be written as ui (t) = [B(t)T B(t)]−1 B(t)T xi (t + 1)  t
−1 T T − [B(t) B(t)] B(t) A(k)B(0)ui (0) + · · · k=1



+A(t)B(t − 1)ui (t − 1) :

× B(N − 2)ui (N − 2) :

lim ui (2) = 0:

A(k)B(0)ui (0) + · · ·

+A(t)B(t − 1)ui (t − 1) + B(t)ui (t):



(22)

Similarly, using conclusions (21) and (22), and the condition limi→∞ xi (t) = 0; t ∈ [1; N ], and also using (19) it follows that

it follows that t


k=1

lim ui (1) = 0:

+A(t)B(t − 1)ui (t − 1) + B(t)ui (t);

xi (t + 1) =

(19)

.. .

i→∞

k=1

(18)

− [B(2)T B(2)]−1 B(2)T [A(2)A(1)B(0)ui (0)

j=1

This implies that Ui is convergent iB G(j) = I − K(j − 1)C(j)B(j − 1) for each j = 1; 2; : : : ; N , is a stable matrix, i.e., all the eigenvalues are inside the unit circle. For each I − K(j − 1)C(j)B(j − 1); j = 1; 2; : : : ; N; there exists a corresponding gain matrix K(j − 1) which moves all the eigenvalues of I − K(j − 1)C(j)B(j − 1) inside the unit circle iB C(j)B(j −1) has full column rank. Thus, there exists a sequence {K(j − 1); j = 1; 2; : : : ; N } such that Ui is convergent iB each C(j)B(j − 1) has full column rank. Next, it will be shown that as i → ∞, ui (t) converges to zero if and only if xd (t) − x(t) converges to zero. Su7ciency: Since

(17)

(16)

(23)

Proceeding in the same fashion, and using the earlier results till (20), it can be shown that lim ui (N − 1) = 0

i→∞

(24)

if limi→∞ xi (t) = 0; t ∈ [1; N ]: To summarize, it can be concluded that limi→∞ ui (t) = 0 over the whole interval [0; N − 1] if limi→∞ xi (t) = 0; t ∈ [1; N ]. The proof is completed. Remark 1. Since a solution to the problem exists only if matrix C(t + 1)B(t) has full column rank; the learning gain K(t) can be calculated in a simple way. From (13); the eigenvalues ofG determine the convergence rate of Ui . Since N i [G] = j=1 {k [G(j)]} from (14); this implies that the eigenvalues of G(j) = I − K(j − 1)C(j)B(j − 1) can determine the convergence rate. So one can obtain a sequence

1260

S.N. Huang et al. / Automatica 38 (2002) 1257–1260

{K(t); t = 0; 1; 2; : : : ; N − 1} such that the eigenvalues of I − K(t)C(t + 1)B(t) are placed in the desired form G ∗ ; i.e.; K(t) = (I − G ∗ ){[C(t + 1)B(t)]T [C(t + 1)B(t)]}−1 ×[C(t + 1)B(t)]T :

(25)

Remark 2. It is possible to build a more compact system description by using the discrete-time impulse response model. In this paper; it is a main objective to extend the result of Saab (1995) to linear time-varying system. Therefore; we still follow a similar form of the system description used by Saab (1995). 3. Conclusions In this correspondence, an iterative learning algorithm, applicable to linear time-varying multivariable systems is studied. A necessary and su$cient condition for the convergence of the algorithm is derived. Based on this condition, to assess if a convergent learning law under the proposed control structure exists, it is only necessary to check if each element of the output=input coupling matrices at a time point is of full column rank. The results presented extends the main results of Saab (1995). Acknowledgements The Arst author would like to thank Dr. Y.Q. Chen for his valuable discussions on the topic of iterative learning algorithms. The authors also like to thank Prof. P.M.J. Van den Hof, Associate Editor, and the anonymous reviewers for their valuable suggestions on this paper.

References Arimoto, S., Kawamura, S., & Miyazaki, F. (1984). Bettering operating operation of robots by learning. Journal of Robotic Systems, 1(1), 123–140. Bien, Z., & Huh, K. M. (1989). High-order iterative control algorithm. IEEE Proceedings, Part-D, Control Theory and Applications, 13(3), 105–112. Chen, C. T. (1999). Linear system theory and design (3rd (ed).). New York: Oxford University Press, Inc. Chen, Y. Q., & Wen, C. (1999). Iterative learning control: Convergence, robustness, and applications. Berlin: Springer. Chien, C. J. (1998). A discrete iterative learning control for a class of nonlinear time-varying systems. IEEE Transactions on Automatic Control, 43(5), 748–752. French, M., & Rogers, E. (2000). Nonlinear iterative learning by an adaptive Lyapunov technique. International Journal of Control, 73(10), 840–850. Kurek, J. E., & Zaremba, M. B. (1993). Iterative learning control synthesis based on 2-D system theory. IEEE Transactions on Automatic Control, AC-38(1), 121–125. Lee, T. H., Tan, K. K., Lim, S. Y., & Dou, H. F. (2000). Iterative learning control of permanent magnet linear motor with relay automatic tuning. Mechatronics, 10, 169–190. Lucibello, P. (1992). Learning control of linear systems, In Proceedings of American Control Conference (pp. 1888–1892). Moore, K. L. (1992). Iterative learning control for deterministic systems Advances in Industrial Control Series. London, UK: Springer. Saab, S. S. (1995). A discrete-time learning control algorithm for a class of linear time-invariant systems. IEEE Transactions on Automatic Control, 40(6), 1138–1142. Sogo, T., & Adachi, N. (1994). A gradient-type learning control algorithm for linear systems. Proceedings of ASCC, Vol. 3, Tokyo, Japan, pp. 227–230. Tan, K. K., Dou, H. F., Chen, Y. Q., & Lee, T. H. (2001). High precision linear motor control via relay tuning and iterative learning based on zero phase Altering. IEEE Trans. on Control Systems Technology, 9(2), 244–253.