Chaotic waveform relaxation methods for dynamical systems

N O ~ - ~

Chaotic Waveform Relaxation Methods for Dynamical Systems Yongzhong Song

Department of Mathematics Nanjing Normal University Nanjing 210097, People's Republic of China Transmitted by John Casti

ABSTRACT For solving the large dynamical systems of initial value problems that can be described by a system of mixed implicit differential equations, the waveform relaxation (WR) methods are proposed and investigated by some mathematicians. In this paper, in order to compute the WR methods on parallel processors and to avoid the synchronization costs, the chaotic waveform relaxation (WR) methods are presented and their convergence is investigated. As for the special cases, the nonlinear ODEs, linear ODEs and semi-explicit DAEs are discussed, respectively.

1.

INTRODUCTION

Consider the large dynamical systems of initial value problems that can be described by a system of mixed implicit differential equations (DAEs) of the form F( 5, x; t) = 0,

t E [0, T ]

(1.1a)

E(x(O) - x °) = 0

(1.1b)

where x(t) ~ ~ " is the vector of unknown variables at time t, ~(t) ~ ~ n is the time derivative of x at time t, x ° ~ . ~ n is the given initial value of x,

F: ~ " × ~ "

× [0, T] - * ~ "

is a continuous function, and E ~a~m×n with

APPLIED MATHEMATICS AND COMPUTATION 78:83-100 (1996) © Elsevier Science Inc., 1996 655 Avenue of the Americas, New York, NY 10010

0096-3003/96/$15.00 SSDI 0096-3003(95)00250-L

84

Y. SONG

m ~< n is a matrix of rank m such that Ex(t) is the state of the system at time t. If 3 / ~ yF( y, x; t) exists and is nonsingular, then (1.1a) can, in principle, be inverted into the explicit ODEs

= ¢(x; t)

(1.2)

so that in this case, we call (1.1a) an implicit ODEs. However, if either 0 / a yF( y, x; t) does not exist, or it exists but it is singular, then (1.1a) can not be inverted into the explicit ODEs (1.2). In order to solve the system (1.1) directly, the waveform relaxation (WR) methods, which are also called dynamic iteration methods, are proposed and investigated by m a n y mathematicians. Waveform relaxation methods were originally introduced in [1] for the time domain analysis of large-scale problems arising from the modelling of integrated circuits. Some convergence theorems have been developed [1-9]. One of the reasons for interest in the W R methods is their potential for parallelism. Consider the W R methods that can be transformed into the following canonical form:

{

~lk+l(t)

= GI(X

k, f(1k, x2k; t)

X~+i( t)

G2( Xlk, Xlk, X2k; t)

x?l(0)

x °,

(1.3)

where X~ ~ 9 2 m, X~ ~ 2 t, m + l = n. Written in integral form we have t

X~+ 1(t) = f o GI( gk' xk, xk; T) dT "{- X°l

x ? l( t) = e2( x:,

(1.4)

x:; t).

In parallelism across space, each component of X 1 and X 2 can be computed on different processors. However, it is necessary to add a synchronization mechanism to ensure that the W R method (1.4) is carried out correctly. Hence, the time necessary to carry out the synchronization, as well as the time that a processor must wait until its data is ready to continue the computation, adds an overhead to the computation.

85

Chaotic Waveform Relaxation Methods

In addition, for the W R methods which can be transformed into the following canonical form [1]:

{

2~+ 1(t) = vl(x, ~+1, x~, 2~, xf; t) x~+ 1(t) v2( x~ + 1, Xl~, ~:1~, x}; t) Xl~+1(o) x °,

(1.5)

the parallelism is not direct. In order to avoid the synchronization costs and to compute the W R methods having the form (1.5) on parallel processors directly, we consider chaotic waveform relaxation (WR) methods. In Section 2, some chaotic W R methods for solving the system (1.1) are presented and their convergence are investigated. In Section 3, Section 4, and Section 5, as the special cases, we discuss, respectively, some ODEs and the semi-explicit DAEs. 2.

GENERAL CHAOTIC W R METHODS AND T H E I R CONVERGENCE

In this section, we consider the dynamical systems (1.1). First, assume that the W R method has the form (1.5). Denote X~, i = 1,2, by =

Vl = ('//'1,

..,

. . .

4)

,

, U m) T ,

x:

=

z~+l,...,z~)

U2 = (?Am+l,

. . .

,

,

Un ) T

and Xf = (Xl°,...,

X°m)T.

Then the chaotic W R method can be defined as follows.

{

:~V'(t) = ~,(Y~"), xV "-~, ~?t'(~)-', x V ' - ' ; t), • V'(o) = x °, i= 1,..., ,~, xV'(t) = u,( xt, ")- 1, xt,<,-,, ~t,<,- ~, x~,<'-~; t), i=m+

1,...,n,

(2.1)

Y. SONG

86 where ki( s) denotes the iteration index of xi at time s and jyk,(s)

1

=

(x;,(s), . . .

yk,(d = ( x;‘(s)-

)

xyy,

1, . . .

i=l

) xc,kl-l’+ 1) xy))

I*‘*, n, x;;+;(s)- 1) . . . )

i= XV.9 2

= (x~y+f"),...,

x:“(‘))~,

xy+

1) T,

1 >.**I m,

i = 1,

. . . . 71.

(2.2)

Then, similarly, for the WR method with the form (1.3), we ca.n derive the following chaotic WR method ~~X”‘(

I

q

x;,(yO) = p(q

&wl,

= gi( xy-1,

xf,

i=

= 9,( x;‘(s)-l,

l,...,

&k(s)-1;

t),

xp-1;

t),

m,

&k(V)

i=m+l

,***> n, (2.3)

where

i=l

,*.*, 72.

Now we discuss the convergence of the chaotic WR methods given by (2.1) and (2.3). Let V,(t) E 9"' be continuously differentiabIe. We define set ‘F by

Let

v, = (Wl,..., qJ’,

v, = (%a+l,...,~n)T.

Chaotic Waveform Relaxation Methods

87

For any a > 0, /31,/33 > 0, 132 > / 0 we define n o r m by

te [o, T]

j= 1

t~[0, T]

j=m+l

and

I[ VII/3 -~- ]3111Vll[~ + ]32[[ Vllla -[- ~3]1V211a. T h e n the space ~

with the norm [[ • I1~ is a Banach space.

THEOREM 2.1. Suppose that the functions u~, 1 <~ i <~ n, in (2.1) are continuous with respect to the last variable and globally Lipschitz continuous with respect to the first n + 2 m variables, i.e., there are nonnegative constants Kj~, 1<~ j <. 4, 1 <~ i <~ m, and )tj~, 1 ~ j <. 4, m + 1<~ i <~ n, such that for any

Xl

( Xl'

Xm) T

X:=(~+l,.

Xl

,_

T

X: -- (~m÷l,.-., ~,)~,

xo) ~

(2.4) -~

Ym)T,

Y= (Yl,'",

( Xl,

=

(~1,...,

__ y~) T ,

~ = (~1,..., ~m)~

z = ( z , , . . . , zm)~, it holds

u,( v, xl, z, x2; t) - u,(~, x,, z, x2; t) Kli ~ lYj -- Yjl -{- K2i ~ IXj - ~,j[ -{- K3i ~ IZj - -Zjl j=l

+K4,

j=l

~ j=m+l

Ix~-~jL,

j=l

88

Y. SONG

for i = 1 , . . . , m, and

j=l

+X4~

j=l

j=l

~ Iz j - 5 : j l j=m+l

for i = m + 1, . . . , n. Further, assume that the constants

A3 =

Kj = ~ K j i , i=1

~ hji , i=m+l

l~j~4,

satisfy A4 < 1 and either

~2 K4

~ + K 3 < l 1 - A4

Or ~3 K4

~+K3 0 and any initial guess ( X ° ( t ) , X ° ( t)) with X°(O) = X °, the sequence {X1k(8), X~ (8)} generated by the chaotic W R method (2.1) converges uniformly to (X](t), X2(t)), which satisfies the DAEs:

[

"~"1 = Ul( Xl, Xl, -'~1, X2; t),

=

xl,

Yc ,

Xl(0 ) ~-- XlO.

PROOF.

Define P: V--* W

t),

(2.5)

Chaotic Waveform Relaxation Methods

89

whenever

{

,~,=

.,( Y', v,, ~'~, v~; t),

w,(O) = x °,

i=

1,...,

Wi = ui(g 1, gl,

~rl, V2; t),

m, i=

m+

1,...,n,

and

w=

w~

(wm+,,...,wo)

r

'

y i = ( V l , . " ., v i _ l ' wi ' Vi+l,. " ., Vm) T. m

For W=

P(V)

and W=

<~ K,,

~

P ( V ) we h a v e

Ivj - 7~jl + Klilw, - gvil + K2, ~ lvj - 7~jl ( 2 . 6 )

j=l, j~i

j=l

/: j=l for i = 1 , . . . ,

j=m+l

m, and

I Wi -~(~li

-t- )[2i)

~Ivj--~Jj I-t- A3i~I?Jj--VjI-]- ~tai ~ Ivj--TJjl j=l j=l j=m+l

for i = m + l , . . . , n. F r o m (2.6) we get for i = 1 , . . . ,

Iw, - ~,l < fo

(

j=l,

m

jg-i

j=l dT.

j=l

j=m+l

90

Y. SONG Let x = m a x l ~ i~ mKli" T h e n

[[Wl-

"WI]]~ ~ 4[.L(KIIIV1

-- "Vl"a q- ffllWl -

W l l l a "]- K211 V 1 -

"Pllla

+,,~, ;1- ~1,o + K,,v2- ~,o). Hence, for a > K, we o b t a i n

II Wl -

__

Wllh~ ~<

1 [ Of

-

(

K

K 1 + K:), v~ - p , , o + ~ , ~ 1

- 91,.

+ K4II V2 -

"V211.],

,ec, - ~ 1 , . -< K,,v, - P,,o + ~ , w , - will. + ~:ltv, - Pill.

L,o

+ K3,¢1 - ~1,o + K , , v 2 [ o~-

K

(K1 -[- K2)I[v1 --

Pill.

+ K3II 191 -

-I-K4II V 2 -

When h 4 < land

~2 K4

- -

A4

1 -

-I- K3 < 1 ,

we h a v e X2 1

-

K 3

1 - h4 < - -

K4

Therefore, t h e r e exists a c o n s t a n t ~ s u c h t h a t A2

1 - h4

o . < ~ < ~ < - 1 -

K3

tq

-"

Viii.

V211~],

Chaotic Waveform Relaxation Methods

91

This shows t h a t

(2.7) K 4/~ + A 4 < 1.

Now it derives

IIW- WII~ =/3111W~ - W~ll~ +/3211 w~ - w~ll~ +/3311w2

-

w211~

-<<[ ~ - ~ _ ~(K1 + K2)(/% +/32K) + (K1 + K2)~2

+[1

]

~-'-'K~K K4( ~1 Jr ~2 K) Jr" K4.8 2 Jr ~t4~ 3 IIV2 - W211~.

(2.s) Set

~1 = 2 [ ( K 1 Jr K2),B Jr- ~1.4],

~2 = ~,

,83 = 1.

Because a can be chosen arbitrarily large, from (2.7), there exists a n u m b e r ~/ with ½ < ? < 1 such t h a t 1

--(~

O/-- K

+ ~)(/3~ + / 3 ~ ) + ( ~ + ~)/3~ + ( ~ + x~)~ ~< ~/% ~K3(~l OL-- K 1

--K4( iT-- K

Jr ~2 K) Jr K3~ 2 Jc ~3~3 ~ "~2

/31 Jr ,~2 K) Jr K4/3 2 Jr '~4~3 ~'< ~/~3"

92

Y. SONG

Consequently, from (2.8), m

IIw -

wile < ~11 v -

vile.

This shows that the mapping P is strictly contractive with respect to the norm ]1" lie. Similarly, if A4 < 1 and ~3 K4 -

i

-

+ K 3 < l

,

A4

we can also prove that P is strictly contractive. Hence, by Banach's fixed point theorem, it has a unique fixed point, satisfying (2.5). Moreover, for any X ° ( t ) = ( X l ( t ) T, X2(t)T) T with XI(0) = X °, the sequence {X~ (s), X~(°}, generated by (2.1), converges uniformly to ()(l(t), )(2(t)). The chaotic WR method (2.3) is the special case of (2.1). Hence, we have the following convergence theorem.

THEOREM 2.2. Suppose that the functions g,, 1 <~ i < n, in (2.3) are continuous with respect to the last variable, and globally, Lipschitz continuous with respect to the first n + m variables, i.e., there are nonnegative constants Kji, j = 1,2,3, 1 <~ i < m, and Aj,, j = 1,2,3, m + 1 ~ i < n, such that f o r any _

x, = ( ~,,..., ~m) ~,

X', = ( ~ , . . . ,

x~ = ( ~ m + ~ , . . . ,

X~ = ( ~ , , + , , . . . ,

Z=(z,,

x°) :~ ,

,zm) ~

~= (~,,..,

T

~,,) , .~,) =~,

_

T

zm) ,

it holds

z,

t) - 9,(x,, z,

j=l

t)l

j=l

j=m+l

Chaotic Waveform Relaxation Methods

93

for i = 1 , . . . , m, and

j=l

j=l

j=m+l

for i = m + 1 , . . . , n. Further, assume that the constants

Kj= ~ Kfi,

hj=

i=1

~

hji ,

j = 1,2,3,

i=m+l

satisfy h a < 1 and either ~t 1

K3

~

1 - h3

"+ K2 < l

or )i, 2 K 3

~+K2<1. 1 -

X3

Then, for any T > 0 and any initial guess ( X ° ( t ) , X ° ( t)) with X°(O) = X °, the sequence {X1k(s), X~ (°} generated by the chaotic WR method (2.3) converges uniformly to ( X l ( t ) , X2( t)), which satisfies the DAEs:

2, = a,( x,, 21, x~; t)

x~ = a~( x,, x~, x~; t) Xl(O) = x °. 3.

NONLINEAR ODEs Now we consider the nonlinear ordinary differential equations (ODEs) k = F( x; t), x(0) = x °,

t ~ [0, T]

where F: ~gP × [0, T] --).9~" is a continuous function.

(3.1)

94

Y. SONG

In order to construct WR methods, we split the right-hand side F( x; t) into two parts, i.e., k = G(x, x; t) + [ F ( x ; t) - G(x, x; t)],

(3.2)

where G: ~ n × ~ × [0, T] - - - ) ~ is called splitting function, moving G( x, x; t) to the left-hand side k-

V(x,x;t)

= F ( x ; t ) - G(x, x ; t ) .

Adding iteration indices, we obtain the iteration formula of the chaotic WR method

~V~)(0)

~0

i= 1,...,n, (3.3)

where F= (f~,...,

xk,(s)

I~)T,

(xlkl(S)

G = ( g l . . . . , g~)T,

-k,(s)\ T

yki(s ) = (xkl(S) -1,...,

xki:_;(s)-l, xki,(s), xk~+l,(S)-1,...,

Xnka(s)-l)T.

For the convergence, we have the following theorem.

THEOREM 3.1. Suppose that the functions f~, gi, 1 <~ i <~ n, in (3.3) are continuous with respect to the last variable and globally Lipschitz continuous with respect to the first n variables o f f i and n + m variables orgy, i.e., there are nonnegative constants ,q, 1 ~ i <~ n, and hj~, j = 1, 2, 1 ~ i ~ n, such that f o r any x = (~1 .....

~o)~,

y = ( yl . . . . , y~) T,

--

~

~ = ( y l , . . . , yn)

T

Chaotic Waveform Relaxation Methods

95

and for i = 1 , . . . , n, it holds

E( x; t) - L( 2; t) l < ~ ~ I xj - ~1, j=l

Ig~( y, x;t)

- g , ( ~ , .~; t ) l ~ )[li~_~Iyj- yj[ -'~j=l

*2i~lxj

- ~jl.

j~l

Then, for any T > 0 and any initial guess x°(t) with x ° ( 0 ) = x °, the sequence { x k(s)} generated by the chaotic W R method (3.3) converges uniforvnly to x(t), which satisfies the ODEs (3.1).

PROOF.

Let v(t) ~ ' "

is continuously differentiable. W e define set ~7

by ~=

{ v ( t ) = (v,, . . . , Vn) T ,

t ~ [o, r l ,

v(O) = x°}.

For v, w ~ ~ and yi = ( V l , . . . , V,_l ' wi ' vi+l,... ' vn)T define

P: v---) w whenever

{

,~, = L( v; t) - g,( v, v; t) + g,( V, v; t)

wi(o ) = xT,

i = 1 , . . . , n.

For w = P ( v ) and ~ -- P(~), we have

I@, - ~w,I < l L ( v ; t) - L(~; t) l + l a ( v , v; t) - g,(~, ~; t) l + l g~( Y~, v; t) - gi( y', v; t)l

j=l

j=l

j=l

j=l ~< (K~ +

2A,~ + 2A2z) ~ 15 - 51 + A~,lw~- YJ,I. j=l

Y. SONG

96 Hence, for K = ~¢, + 2Ali + 2A2, , h

=max,

l
1

IIw-

wIl,~ < - ~ ( K I I v -

vii,. + A I I w -

WIl,~),

and, consequently, for a > K K

II w -

wII. < -a- - ~-II v - vll,~.

Because a can be chosen arbitrarily large, we can choose a so t h a t - A) < 1. This shows t h a t the m a p p i n g P is strictly contractive with respect to the norm II " Ila- Hence, using B a n a c h ' s fixed point theorem we can easily finish the proof. Let us consider some special cases. If we take K/(a

G=0, then we have the chaotic W R method:

{ ~,(')( t) = L( z ~'(s)-'; t) z~,(s)(0) z0, i= 1,..., n,

(3.4)

where xki(s) = (xkl(S) '

"'''

(3°5)

xkn(s) ) V

for the chaotic W R Picard method and X ki(S) = (Xl k1($) .....

Xik~-i1(8) , Xk'(s)q- 1 , Xk~_+ll(S)....

,

x~"(~)) T

(3.6)

for the chaotic W R Jacobi method. As the special cases, the convergence theorems for the chaotic W R methods (3.4) with (3.5) or (3.6) can be given from Theorem 3.1.

THEOREM 3.2. Suppose that the functions f~, 1 <<. i <~ n, in (3.4) are continuous with respect to the last variable and globally Lipschitz continuous

Chaotic Waveform Relaxation Methods

97

with respect to the first n variables, i.e., there are nonnegative constants K,, 1 <~ i <~ n, such that for any

• ..~

,

~

...,

Xn)

and for i = 1 , . . . , n, it holds

E( x; t)

-

L( ~; t) l <.< K, E Ixj

-

~jl.

j=l

Then, for any T > 0 and any initial guess x°( t) with x°(O)--x °, the sequence { x k(~)} generated by the chaotic WR method (3.4) with either (3.5) or (3.6) converges uniformly to x( t), which satisfies the ODEs (3.1). 4.

LINEAR ODEs For the linear variable coefficient ODEs k+ B(t) x= F(t), x(0)

t ~ [0, T]

= x°

(4.1)

the matrix B(t) is divided into B(t) = M(t) - N(t), where the matrix M(t) = d i a g ( m l ( t ) , . . . , m,(t)) is diagonal, then the chaotic WR iteration scheme can be defined by

k~'<8)( t) + m,( t) x~'(s)( t) = ~ n o ( t ) x] '(~)- 1(t) + fi(t) j=l

xik'(s)(0) = X°,

(4.2)

i= 1,..., n

with N(t) = (nij),

F = ( f l , - . . , f , ) T,

Xk,(,) = ( Xkl,(8), . . . , xk.(s)) T.

When the matrix M(t) and N(t) are continuously on [0, T], it is easy to show that they are bounded on [0, T] so that the function - m i ( t ) x i + E~= 1no(t) xj + f~(t), i = 1 , . . . , n, are globally Lipschitz continuous with

98

Y. SONG

respect to the n variables x,, i = 1 , . . . , n. Hence, similar to the Theorem 2.1, we have the following convergence theorem.

THEOREM 4.1. Suppose that the matrix functions M(t) and N(t) are continuous with respect to t on [0, T]. Then the chaotic WR method (4.2) converges uniformly to x( t), which satisfies the ODEs (4.1). 5.

SEMI-EXPLICIT DAEs For the semi-explicit DAEs of the following type

{

Fa( Xl, )(2;

-~1 =

t)

0 = F2( X 1, )(2; t),

t • [0, T]

(5.1)

Xl(0) = x °,

with X 1 •~a~,m, )(2 •~qp,-m, F1: ..~2" × [0, T] --*~2 '~, F2: ~2" X [0, T] --* .9~,- r~ the general chaotic WR method can be defined by

~y~( t) - g,( v~,~s~, xt,"~-', x ~ , ~ - 1 ; t) = f,( x:,,~)-1 x?'~ 1; t) - g,( x:,"' -1, x~,"~ -1, x:'~-'; t)

~ ()

X ,(s) 0

= X~, 0

i=1,.,

m,

. ,

x~'~( t) = c~( x W "~, x:'~-~; t), (5.2) where F1

= (fl,

=

"'',

f r o ) T,

,...,x~

~

)

,

i=l

....

,n,

i = 1 , . . . , m,

,.

,

,

i--

1,...,n,

Chaotic Waveform Relaxation Methods

99

and G 2 is an iteration operator for solving F2(X1, )(2; t) = 0 with respect

to x~. The chaotic W R method (5.2) can be regarded as a special one of (2.2). Hence, we can derive the convergence conditions.

THEOREM 5.1. Suppose that the functions f~, g,, 1 <~ i ~ n, in (5.2) are continuous with respect to t and globally Lipschitz continuous with respect to Xt, )(2, and Y, i.e., there are nonnegative constants xji , 1 <~j <~ 5, 1 <~ i <~ m, and )tj, j = 1,2, such that for any X 1, X1, )(2, X2, Y, and given by (2.4), and for i = 1 , . . . , m, it holds

fi(X1, X2;t)-f~(-~l, X2;t)

< *qi ~ Ixj- ~jl j=l

+ K2i L

Izj-~jt,

j=m+l

K3i~IYj--YjI+*aiEIXj j=l m

I g i ( Y , X1, X 2 ; t ) -

gi('Y, X1, X 2 ; t )

~

- ~jl

j=t

+ ~5, ~

I xj-:~jl,

j=m+l

[e2(Xl,

X2; t) -- C2(..Xl, X2; t) ~< A, Ix, - 2,1 + ~21x2 - x21.

Then, for any T > 0 and any initial guess ( X ° ( t ) , X°( t)) with X°l (O) = X °, the sequence {X~ s), X2k(s)} generated by the chaotic WR method (5.2), converges uniformly to (Xl(t), X2( t)), which satisfies the DAEs (5.1). If we choose Newton operator for iteration operator G 2 in (5.2), i.e., taking the place of the third formula in (5.2) let

[

× 5( xV

o F (xV t).

t)

]1

(5.3)

Then we obtain the WR-Newton method. The sufficient condition for local convergence can be proposed similarly.

100

Y. SONG

Furthermore, the Newton iteration formula (5.3) can be carried out with parallel computation.

Project supported by the National Natural Science Foundation of China. REFERENCES 1 B. Leimkuhler, A. E. Ruehli, and A. L. Sangiovanni-Vincentelle, The Waveform Relaxation Method for Time-Domain Analysis of Large-Scale Integrated Circuits, IEEE Trans. Computer-Aided Design of ICAS, Vol. CAD-l, 3:131-145 (1982). 2 U. Miekkala, Dynamic Iteration Methods Applied to Linear DAE Systems, J. Comput. Appl. Math., 25:133-151 (1989). 3 U. Miekkala, and O. Nevanlinna, Convergence of Dynamic Iteration Methods for Initial Value Problems, SIAM J. Sci. Stat. Comp. 8:459-482 (1987). 4 U. Miekkala, and O. Nevanlinna, Sets of Convergence and Stability Regions, BIT 27:554-584 (1987). 5 O. Nevanlinna, Remarks on Picard-LindelSf Iteration, Part I, BIT, 29:328-346 (1989). 6 O. Nevanlinna, Remarks on Picard-LindelSf Iteration, Part II, BIT, 29:535-562 (1989). 7 K.R. Schneider, A Note on the Waveform Relaxation Method, Systems Anal. Modelling Simulation, 8:No. 8, 599-605 (1991). 8 Y. Song, Convergence and comparisons of Waveform Relaxation Methods for Linear ODE Systems, Computing, 50:337-352 (1993). 9 Y. Song, Waveform Relaxation Methods for Initial Value Problems, Ann. of Diff. Equa., 9:467-480 (1993).