Convergence theorems for parallel alternating iterative methods

Applied Mathematics and Computation 148 (2004) 497–517 www.elsevier.com/locate/amc

Convergence theorems for parallel alternating iterative methods q Joan-Josep Climent a

a,*

, Carmen Perea b, Leandro Tortosa a, Antonio Zamora a

Departament de Ci
encia de la Computaci o i Intel.lig
encia Artificial, Universitat d’Alacant, Ap. Correus 99, E–03080 Alacant, Spain b Departamento de Estadıstica y Matem atica Aplicada, Universidad Miguel Hern andez, Escuela Polit ecnica Superior de Orihuela, E-03550 Orihuela, Spain

Abstract The parallel multisplitting nonstationary iterative Model A was introduced by Bru, Elsner, and Neumann [Linear Algebra Appl. 103 (1988) 175–192] for solving nonsingular linear system Ax ¼ b using a weak nonnegative multisplitting of the first type. In this paper new results using a weak nonnegative multisplitting of the second type are introduced when A is a monotone matrix, and using P -regular multisplitting when A is a symmetric positive definite matrix. Combining Model A and alternating iterative methods, two new models of parallel multisplitting nonstationary iterations are introduced. It is shown that when matrix A is monotone and the multisplittings are weak nonnegative of the first or second type, both models lead to convergent schemes. When matrix A is symmetric positive definite and the multisplittings are P -regular, the schemes are also convergent.
2003 Elsevier Inc. All rights reserved. Keywords: Nonsingular matrix; Iterative method; Splitting; Multisplitting; Alternating method; Stationary method; Nonstationary method; Convergence conditions

q

This work was partially supported by Spanish DGES grant PB98-0977. Corresponding author. E-mail address: [email protected] (J.-J. Climent).

*

0096-3003/$ - see front matter
2003 Elsevier Inc. All rights reserved. doi:10.1016/S0096-3003(02)00916-5

498

J.-J. Climent et al. / Appl. Math. Comput. 148 (2004) 497–517

1. Introduction For the solution of the large and sparse linear system Ax ¼ b;

ð1Þ

where A is a nonsingular matrix, x an unknown vector, and b a given vector, an iterative method is usually considered. From the general representation A ¼ M
N , called splitting if M is a nonsingular matrix, the majority of the iterative processes appear. In particular, if we consider two splittings A ¼ M
N ¼ P
Q, we obtain the general class of the iterative methods of the form
xðkþ1=2Þ ¼ M
1 N xðkÞ þ M
1 b k ¼ 0; 1; . . . ð2Þ xðkþ1Þ ¼ P
1 Qxðkþ1=2Þ þ P
1 b Different sets of algorithms described by (2) are studied by authors such as Conrad and Wallach [8], Marchuk [14], and more recently by Benzi and Szyld [2]. Moreover, Climent and Perea [7] have introduced the following nonstationary version of the alternating iterative method (2). 9 lðkÞ
1 P > lðkÞ ðkÞ j ðkþ1=2Þ
1
1
1 > x ¼ ðM N Þ x þ ðM N Þ M b; > = j¼0 ð3Þ mðkÞ
1 P
1 j
1 > mðkÞ ðkþ1=2Þ > ðkþ1Þ
1 > x ¼ ðP QÞ x þ ðP QÞ P b: ; j¼0

On the other hand, with the development of parallel computation in recent years, the utilization of the parallel algorithms for the solution of large and sparse nonsingular linear system has become effective. OÕLeary and White [18] introduce the concept of multisplitting for the parallel solution of linear system p (1): fMl ; Nl ; El gl¼1 is a multisplitting of A if • A ¼ Ml
Nl , is a splitting for l ¼ 1; 2; . . . ; p: • El P 0 is a nonnegative diagonal matrix for l ¼ 1; . . . ; p, called weighting matrices. Pp • l¼1 El ¼ I where I is the identity matrix. Suppose that we have a multiprocessor with p processors connected to a host processor, that is, the same number of processors as splittings, and that all processors have the last update vector xðkÞ , then the lth processor only computes those entries of the vector Ml
1 Nl xðkÞ þ Ml
1 b which correspond to the nonzero diagonal entries of El . The processor then scales these entries so as to be able to deliver the vector

J.-J. Climent et al. / Appl. Math. Comput. 148 (2004) 497–517

499

El ðMl
1 Nl xðkÞ þ Ml
1 bÞ to a host processor, performing the parallel multisplitting scheme xðkþ1Þ ¼

p X l¼1

El Ml
1 Nl xðkÞ þ

p X

El Ml
1 b;

k ¼ 0; 1; 2; . . .

ð4Þ

l¼1

Many authors, such as Neumann and Plemmons [17], Elsner [9], Frommer and Mayer [10,11], Wang [22], Bru, Migall on, and Penades [4], Nabben [16], have studied convergence conditions for different parallel algorithms based on the standard parallel iterative scheme (4) and for different types of matrices. Iterative process (4) implicitly assumes that processor l has to be synchronized between the formation of iterates xðkÞ and xðkþ1Þ . Moreover, in practice, it is quite usual to consider the diagonal matrices El P 0 for l ¼ 1; . . . ; p, with zero or one entries, then in this case in each processor we have to solve some subproblems (a system of size less than n), that may be of different sizes. Bru, Elsner, and Neumann [3], with the purpose of avoid loss of time and efficiency in processor utilization, introduce the mathematical formulation, establishing the convergence for a weak nonnegative multisplitting of the first type (see Definition 1) when A is a monotone matrix, of the following general model of the so called synchronous iterations. Model A: Each processor can carry out a varying number of local iterations until a mutual phase time is reached when all the processors are ready to contribute towards the global iteration (see Section 3 for the corresponding mathematical formulation). Combining Model A and the alternating iterative methods (2) and (3) we introduce the following two synchronous parallel methods. Model 1: Consider fMl ; Nl ; El gpl¼1 and fPl ; Ql ; Fl gpl¼1 two multisplittings of A. We alternate one global iteration of Model A obtained from the multisplitting p fMl ; Nl ; El gl¼1 with another one of Model A obtained from the multisplitting p fPl ; Ql ; Fl gl¼1 . p Model 2: Assume that we have two multisplittings fMl ; Nl ; El gl¼1 and p fPl ; Ql ; El gl¼1 with the same weighting matrices for both multisplittings. In each processor we apply the alternating iterative method (3) until a mutual phase time is reached when all processors are ready to contribute towards the global iteration, similar to Model A. With this aim, in the next section, we introduce the notation and preliminaries necessary in this paper. In Section 3, we present convergence conditions for Model A of Bru, Elsner, and Neumann [3] for weak nonnegative multisplittings of the second type of a monotone matrix, and for P -regular multisplittings of a symmetric positive definite matrix. Finally, in Section 4 we introduce the mathematical formulation of Models 1 and 2 and convergence results for both models in the case where the multisplittings are weak nonnegative of the first or second type of a monotone matrix. Moreover, we

500

J.-J. Climent et al. / Appl. Math. Comput. 148 (2004) 497–517

establish convergence results for P -regular multisplittings of a symmetric positive definite matrix.

2. Notation and preliminaries In this section, we give some notation, some definitions and we present the basic tools necessary to establish convergence conditions for iterative methods in general. Let A be an n  n real matrix, by A P 0 (respectively, A > 0) we denote the matrix whose entries are nonnegative (respectively, positive) and by jAj the matrix whose entries are the absolute value of the corresponding entries of A. Let A and B be two n  n real matrices, A P B (respectively, A > B) means that A
B P 0 (respectively, A
B > 0). Similarly for vectors. In the following definition we present the different types of splittings that appear in this paper [1,5,6,15,19,21,23]. Definition 1. Let A be an n  n real matrix. We say that the splitting A ¼ M
N is • weak nonnegative of the first type (respectively, of the second type) if M
1 P 0 and M
1 N P 0 (respectively, M
1 P 0 and NM
1 P 0). • P -regular if M T þ N is definite positive. As a generalization of Definition 1 we say that a multisplitting is weak nonnegative of the first type, weak nonnegative of the second type or P -regular, respectively, if each splitting of the multisplitting is weak nonnegative of the first type, weak nonnegative of the second type or P -regular, respectively. The stationary iterative methods based on the concepts of splitting and multisplitting can be expressed as a standard iterative scheme xðkþ1Þ ¼ T xðkÞ þ c;

k ¼ 0; 1; 2; . . .

ð5Þ

It is well known that iterative method (5) converges to the unique solution of linear system (1) if, and only if, qðT Þ < 1, where qðT Þ denotes the spectral radius of the iteration matrix T . The basic tools in the study of the convergence of the stationary methods of this paper are the following results. Lemma 1 (Theorem 3.1 of [23]). Let A be a nonsingular matrix and A ¼ M
N be a splitting. Then M
1 NA
1 ¼ A
1 NM
1 and the matrices M
1 N and A
1 N (respectively, NM
1 and NA
1 ) commute.

J.-J. Climent et al. / Appl. Math. Comput. 148 (2004) 497–517

501

Lemma 2. Let A be a n  n nonsingular matrix. (i) (Lemma 2.3 of [13]) Let T be an n  n matrix such that I
T is nonsingular. Then, there exists a unique pair of matrices B and C, such that B is nonsingular, T ¼ B
1 C and A ¼ B
C. The matrices are B ¼ AðI
T Þ
1 and C ¼ B
A. (ii) (Lemma 2 of [7]) Let S be a n  n matrix such that I
S is nonsingular. Then, there exists a unique pair of matrices E and F , such that E is nonsin
1 gular, S ¼ FE
1 and A ¼ E
F . The matrices are E ¼ ðI
SÞ A and F ¼ E
A. With the notation of the above lemma, we say that A ¼ B
C is the unique splitting associated with matrix T , and A ¼ F
E is the unique splitting associated with matrix S. Moreover as an immediate consequence of Lemma 3 of Climent and Perea [7] we also have the following result. Lemma 3. Let A be a nonsingular matrix. For the splitting A ¼ M
N , consider p matrix S ¼ ðNM
1 Þ with p P 1. Then p

(i) S ¼ ATA
1 where matrix T ¼ ðM
1 N Þ , and consequently, qðSÞ ¼ qðT Þ. (ii) If qðT Þ < 1, then matrices T and S induce the same splitting A ¼ B
C of A. On the other hand, the nonstationary iterative methods, also based on the concepts of splitting and multisplitting, can be expressed as a standard iterative scheme xðkþ1Þ ¼ Tk xðkÞ þ ck ;

k ¼ 0; 1; 2; . . .

ð6Þ

To establish sufficient conditions for the convergence of the iterative method (6) it is usually to prove that the error vector converges, in norm, to zero, or simply converges to zero. We use the following vector and matrix norm. For an n-vector x > 0, the monotonic vector norm    yi  kykx ¼ minfa > 0 :
ax 6 y 6 axg ¼ max   1 6 i 6 n xi which induces the matrix norm kAkx that satisfies kAkx ¼ kAxkx

and

kjAjxkx ¼ kAkx :

Furthermore, if jAjx 6 bx for some b P 0, then kAkx 6 b [20]. Moreover, a real matrix A is positive semidefinite if xT Ax P 0

for all x 6¼ 0

ð7Þ

502

J.-J. Climent et al. / Appl. Math. Comput. 148 (2004) 497–517

and positive definite if strict inequality holds in (7) [19]. Furthermore, it is wellknown that for a symmetric positive definite matrix A the expression pffiffiffiffiffiffiffiffiffiffiffi kxkA ¼ xT Ax defines a vector norm. We denote by k  kA both the vector norm defined by A and the corresponding induced matrix norm. To finish this section we introduce the following result that is also a useful tool for establishing the convergence of the iterative methods (6). Lemma 4. Let T1 ; T2 ; . . . ; Tk ; . . . ; be a sequence of n  n nonnegative matrices. If there exists a real number 0 6 b < 1, and a vector x > 0 in Rn , such that Tk x 6 bx for k ¼ 1; 2; . . . then qðHk Þ 6 bk < 1, where Hk ¼ Tk    T2 T1 , and therefore limk!1 Hk ¼ 0.

3. Parallel synchronous iterative methods Bru, Elsner, and Neumann [3] establish the convergence of Model A, mentioned in Section 1, for a monotone matrix A with weak nonnegative multisplittings of the first type. In this section, we introduce convergence results for this model for a monotone matrix A with weak nonnegative multisplittings of the second type, and for symmetric positive definite matrix A with a P -regular multisplitting. Both cases with the typical additional hypothesis of El ¼ al I; l ¼ 1; 2; . . . ; p, used before by other authors such as OÕLeary and White [18], Nabben [16], and Climent and Perea [6]. Firstly, we consider the mathematical formulation of parallel Model A. Let p fMl ; Nl ; El gl¼1 be a multisplitting of A. Starting with an arbitrary vector xð0Þ , iteration ! lðk;lÞ
1 p p X X X lðk;lÞ ðkÞ i ðkþ1Þ
1
1
1 b ð8Þ x ¼ El ðMl Nl Þ x þ El ðMl Nl Þ Ml l¼1

l¼1

i¼0

is performed where lðk; lÞ P 1 denotes the number of local iterations in processor l at the global iteration k for k ¼ 0; 1; 2; . . . and l ¼ 1; 2; . . . ; p. Using the same notation as Bru, Elsner, and Neumann [3] we denote by Bk ¼

p X

El ðMl
1 Nl Þlðk;lÞ

ð9Þ

l¼1

and rðkÞ ¼

p X l¼1

El

lðk;lÞ
1 X

! i

ðMl
1 Nl Þ Ml
1 b

i¼0

J.-J. Climent et al. / Appl. Math. Comput. 148 (2004) 497–517

503

that allows us to write iterative method (8) as xðkþ1Þ ¼ Bk xðkÞ þ rðkÞ :

ð10Þ

Now, denoting by n the exact solution to system (1) and defining the error vectors as eðkÞ ¼ xðkÞ
n

ð11Þ

Bru, Elsner, and Neumann [3] establish the relation, eðkÞ ¼ Bk eðk
1Þ :

ð12Þ

Moreover, in the same paper they establish the following bound for kBk kx . p

Lemma 5 (Lemma 2.1 of [3]). Let fMl ; Nl ; El gl¼1 be a multisplitting of the n  n matrix A. If there exists a vector x > 0 and a scalar 0 6 b < 1 such that jMl
1 Nl jx 6 bx, for l ¼ 1; 2; . . . ; p. Then kBk kx 6 b, for k ¼ 1; 2; . . ., where Bk is given in (9). Using, basically, relation (12) and the above lemma, Bru, Elsner, and Neumann [3] introduce the next convergence result for method (10). Theorem 1 (Theorem 2.1 of [3]). Let A be a monotone matrix and let fMl ; Nl ; p El gl¼1 be a weak nonnegative multisplitting of the first type, then the iterative method (10) converges for any initial vector xð0Þ whenever k ¼ 1; 2; . . . ; l ¼ 1; 2; . . . ; p:

lðk; lÞ P 1;

ð13Þ

With the purpose of establishing new sufficient conditions for the convergence of iterative method (8) we introduce the following result. Lemma 6. Let G1 ; G2 ; . . . ; Gk ; . . . be a sequence of n  n nonnegative matrices, let B1 ; B2 ; . . . ; Bk ; . . . be another sequence of n  n matrices. If there exists a vector x > 0 and a scalar 0 6 b < 1 such that Gk x 6 bx;

k ¼ 1; 2; . . .

and there exists an n  n nonsingular matrix P such that Gk ¼ PBk P
1 ;

ð14Þ

then lim ðBk    B2 B1 Þ ¼ 0:

k!1

Proof. By Lemma 4 we have that limk!1 ðGk    G2 G1 Þ ¼ 0 and from (14) we have that Bk    B2 B1 ¼ P ðGk    G2 G1 ÞP
1 , then lim P ðGk    G2 G1 ÞP
1 ¼ 0:

k!1




504

J.-J. Climent et al. / Appl. Math. Comput. 148 (2004) 497–517

If in Theorem 1 we replace ‘‘first type’’ with ‘‘second type’’ and introduce the additional hypothesis of ‘‘El ¼ al I for l ¼ 1; 2; . . . ; p’’, we obtain the following convergence result. Theorem 2. Let A be a monotone matrix and let fMl ; Nl ; El gpl¼1 be a weak nonnegative multisplitting of the second type. If El ¼ al I P 0 for l ¼ 1; 2; . . . ; p then iterative method (10) converges for any initial vector xð0Þ whenever (13) holds. Proof. Since the multisplitting is weak nonnegative of the second type, considering the vector xT ¼ ð1; 1; . . . ; 1ÞA
1 > 0 we have that xT
xT Nl Ml
1 ¼ xT AM
1 > 0

for l ¼ 1; 2; . . . ; p

thus for a suitable constant 0 6 b < 1, xT Nl Ml
1 < xT 6 bxT for l ¼ 1; 2; . . . ; p: ð15Þ Pp lðk;lÞ If we denote by Gk ¼ l¼1 El ðNl Ml
1 Þ P 0, from (15) we have that xT Gk ¼

p X

al xT ðNl Ml
1 Þ

l¼1

lðk;lÞ

<

p X

al xT b ¼ xT b:

ð16Þ

l¼1

Now, taking into account that El ¼ al I for l ¼ 1; 2; . . . ; p, by Lemma 1 for each splitting, we have that Gk ¼ AA
1 Gk ¼ ABk A
1

for k ¼ 1; 2; . . .

ð17Þ

Furthermore, from the above equality, from inequality (16), and by Lemma 6 we have that lim ðBk    B2 B1 Þ ¼ 0:

ð18Þ

k!1

Finally, from (12) and (18) we obtain lim eðkÞ ¼ lim ðBk    B2 B1 Þeð0Þ ¼ 0:

k!1




k!1

Now if we consider a symmetric positive definite matrix A and a P -regular multisplitting with the same additional hypothesis as in Theorem 2 we can establish the following convergence condition. p

Theorem 3. Let A be a symmetric positive definite matrix and fMl ; Nl ; El gl¼1 be a P -regular multisplitting. If El ¼ al I P 0 for l ¼ 1; 2; . . . ; p, then the iterative method (10) converges for any initial vector xð0Þ whenever (13) holds. Proof. By Theorem 2.4 of Frommer and Szyld [12] we have that kMl
1 Nl kA 6 b; for a suitable constant 0 6 b < 1 and l ¼ 1; 2; . . . ; p. Consequently,

J.-J. Climent et al. / Appl. Math. Comput. 148 (2004) 497–517

505

kðMl
1 Nl Þlðk;lÞ kA 6 blðk;lÞ 6 b < 1 for all lðk; lÞ P 1: Now, since

  p p p   X  X X    lðk;lÞ 
1 kBk kA ¼  El ðMl Nl Þ al ðMl
1 Nl Þlðk;lÞ  6 al b ¼ b < 1:  6  l¼1  A l¼1 l¼1 A

By equality (12) we have that lim keðkÞ kA ¼ lim kBk    B2 B1 eð0Þ kA 6 lim bk eð0Þ ¼ 0

k!1

k!1

and the proof is complete.

k!1




Note, that as Bru, Elsner, and Neumann [3] remark for Theorem 1, in Theorems 2 and 3, assumption (13) can be weakened by lðk; lÞ P 0;

k ¼ 1; 2; . . . ; l ¼ 1; 2; . . . ; p

and for infinitely many kÕs, lðk; lÞ P 1

for all l ¼ 1; 2; . . . ; p:

If in the nonstationary iterative method (8) we consider the particular case in which the number of local iterations only depends on the lth processor, that is lðk; lÞ ¼ lðlÞ for k ¼ 1; 2; . . ., we obtain a stationary version of this method with iteration matrix p X lðlÞ B¼ El ðMl
1 Nl Þ : ð19Þ l¼1

Taking into account Lemmas 2 and 3 and as an immediate consequence of Theorems 6 and 9 of Climent and Perea [7] we obtain the following technical result. Lemma 7. Let A be a nonsingular n  n matrix. Let A ¼ M
N be a splitting and T ¼ ðM
1 N Þp with p P 1. (i) If A
1 P 0 and A ¼ M
N is weak nonnegative of the first type (respectively, second type) there exists a unique splitting A ¼ B
C associated with T that is also weak nonnegative of the first type (respectively, second type). (ii) If A is symmetric positive definite and A ¼ M
N is P -regular, there exists a unique splitting A ¼ B
C associated with T that is also P -regular. Now if we consider a multisplitting of a matrix, we have results similar to the above lemma. The proof of the first part can be found in Elsner [9], of the second part in Climent and Perea [6] and the last part in Nabben [16].

506

J.-J. Climent et al. / Appl. Math. Comput. 148 (2004) 497–517 p

Lemma 8. Let A be a nonsingular n  n matrix. Let fMl ; Nl ; El gl¼1 be a multiPp splitting of A and H ¼ l¼1 El Ml
1 Nl . (i) If A
1 P 0 and the multisplitting is weak nonnegative of the first type, then there exists a unique splitting associated with matrix H that is also weak nonnegative of the first type. (ii) If A
1 P 0 and the multisplitting is weak nonnegative of the second type with the additional hypothesis of El A ¼ AEl for l ¼ 1; 2; . . . ; p, then there exists a unique splitting associated with matrix H that is also weak nonnegative of the second type. (iii) If A is symmetric positive definite and the multisplitting is P -regular with the additional hypothesis of El ¼ al I for l ¼ 1; 2; . . . ; p, then there exists a unique splitting associated with matrix H that is also P -regular. Observe that the condition El A ¼ AEl is valid if El ¼ al I. Using the above lemmas as the principal tool we introduce the following results for the stationary version of the iterative process (8). Theorem 4. Let A be a monotone matrix and let fMl ; Nl ; El gpl¼1 be a multisplitting. Consider in iterative method (10) the particular case where lðk; lÞ ¼ lðlÞ P 1 for l ¼ 1; 2; . . . ; p, then (i) If the multisplitting is weak nonnegative of the first type, this method converges for any initial vector xð0Þ . Moreover, the splitting associated with this method is also weak nonnegative of the first type. (ii) If the multisplitting is weak nonnegative of the second type, with the additional hypothesis El ¼ al I P 0 for l ¼ 1; 2; . . . ; p, this method converges for any initial vector xð0Þ . Moreover, the splitting associated with this method is also weak nonnegative of the second type. Proof. (i) By Theorem 1, iterative method (10) converges for any initial vector xð0Þ , that is, qðBÞ < 1. Now, by part (i) of Lemma 7 there exists a unique weak nonnegative splitting of the first type A ¼ Bl
Cl associated with the iteration matrix pðlÞ Tl ¼ ðMl
1 Nl Þ for l ¼ 1; 2; . . . ; p. Moreover, if we consider the multisplitting p fBl ; Cl ; El gl¼1 by Lemma 8 (i) there exists a unique weak nonnegative splitting of the first type associated with the iteration matrix H¼

p X l¼1

El B
1 l Cl ¼

p X

El ðMl
1 Nl ÞpðlÞ :

l¼1

(ii) Similar to part (i) using Theorem 2 instead of Theorem 1 and using part (ii) instead of part (i) of Lemma 8.


J.-J. Climent et al. / Appl. Math. Comput. 148 (2004) 497–517

507

Finally, for a symmetric positive definite matrix A and a P -regular multisplitting, we have the following result. p

Theorem 5. Let A be a symmetric positive definite matrix and fMl ; Nl ; El gl¼1 be a P -regular multisplitting. Suppose that El ¼ aI P 0 for l ¼ 1; 2; . . . ; p. If in iterative method (10) we consider the particular case where lðk; lÞ ¼ lðlÞ P 1 for l ¼ 1; 2; . . . ; p, then this method converges for any initial vector xð0Þ . Moreover, the splitting associated with this method is also P -regular. Proof. Analogous to the proof of Theorem 4 using part (ii) instead of part (i) of Lemma 7 and part (iii) instead of part (i) of Lemma 8.


4. Parallel alternating synchronous iterative methods 4.1. Convergence results for Model 1 As we have mentioned in Section 1, Model 1 consists of: Alternating one global iteration of Model A, from the multisplitting fMl ; Nl ; El g for l ¼ 1; . . . ; p with another one of Model A obtained from the multisplitting fPl ; Ql ; Fl g for l ¼ 1; . . . ; p. The mathematical formulation of this model can be expressed as 9 Pp lðk;lÞ ðkÞ > xðkþ1=2Þ ¼ l¼1 El ðMl
1 Nl Þ x >  > > Pp Plðk;lÞ
1
1 i
1 > = þ l¼1 El ðM N Þ M b; l l l i¼0 k ¼ 0; 1; 2; . . . ð20Þ P p mðk;lÞ ðkþ1=2Þ
1 > xðkþ1Þ ¼ l¼1 Fl ðPl Ql Þ x >  > > Pp Pmðk;lÞ
1
1 i
1 > ; þ l¼1 Fl ðPl Ql Þ Pl b; i¼0 As is usual, to analyze the convergence of iterative process (20) we eliminate xðkþ1=2Þ obtaining in this case ! ! p p X X mðk;lÞ lðk;lÞ ðkþ1Þ
1
1 x ¼ Fl ðPl Ql Þ El ðMl Nl Þ xðkÞ l¼1

" þ

l¼1

p X

! mðk;lÞ Fl ðPl
1 Ql Þ

p X

l¼1

þ

p X l¼1

Fl

El

!#

i

ðPl
1 Ql Þ Pl
1

!!

i ðMl
1 Nl Þ Ml
1

i¼0

l¼1 mðk;lÞ
1 X

lðk;lÞ
1 X

b;

k ¼ 0; 1; 2; . . .

ð21Þ

i¼0

With the purpose of establishing an error analysis of iterative method (21), in a similar way to method (8), we define

508

J.-J. Climent et al. / Appl. Math. Comput. 148 (2004) 497–517 p X

Hk ¼

! Fl ðPl
1 Ql Þmðk;lÞ

l¼1

p X

! El ðMl
1 Nl Þlðk;lÞ

;

k ¼ 1; 2; . . .

ð22Þ

l¼1

Now, considering n the exact solution of linear system (1) and taking into account the error vector relation (11) it is easy to see that eðkÞ ¼ Hk eðk
1Þ :

ð23Þ

Then, we can introduce sufficient conditions for the convergence of iterative method (21). Previously, we introduce the following upper bound for kHk kx , similar to Lemma 5. Lemma 9. Let fMl ; Nl ; El g and fPl ; Ql ; Fl g be two multisplittings of the n  n matrix A. If there exists a vector x > 0 and a constant 0 6 b < 1 such that jMl
1 Nl jx 6 bx

and

jPl
1 Ql jx 6 bx

ð24Þ

for l ¼ 1; 2; . . . ; p, then kHk kx 6 b, where Hk is given in (22) for each k ¼ 1; 2; . . . Proof. By (24) we have that jMl
1 Nl j

lðk;lÞ

x 6 blðk;lÞ x 6 bx

jPl
1 Ql j

and

mðk;lÞ

x 6 bmðk;lÞ x 6 bx

and from (22) we have that  ! ! p p   X X   mðk;lÞ lðk;lÞ Fl ðPl
1 Ql Þ El ðMl
1 Nl Þ jHk jx ¼  x   l¼1 l¼1 ! ! p p X X mðk;lÞ lðk;lÞ 6 jFl jjPl
1 Ql j jEl jjMl
1 Nl j x l¼1

6

p X

!

l¼1

jFl jjPl
1 Ql jmðk;lÞ bx 6 bbx 6 bx

l¼1

showing that kHk kx 6 b.
Now we introduce sufficient convergence conditions for iterative process (21) similar to the sufficient convergence conditions for iterative method (8) of the previous section. Theorem 6. Let A be a monotone matrix. Let fMl ; Nl ; El g and fPl ; Ql ; Fl g be two weak nonnegative multisplittings of the first type. Then iterative method (21) converges for any initial vector xð0Þ whenever lðk; lÞ P 1 hold.

and

mðk; lÞ P 1;

k ¼ 1; 2; . . . ; l ¼ 1; 2; . . . ; p

ð25Þ

J.-J. Climent et al. / Appl. Math. Comput. 148 (2004) 497–517

509

Proof. Since both multisplittings are weak nonnegative of the first type, conT sidering the vector x ¼ A
1 ð1; 1; . . . ; 1Þ > 0 we have that
x
Ml
1 Nl x ¼ Ml
1 Ax > 0 l ¼ 1; 2; . . . ; p ð26Þ x
Pl
1 Ql x ¼ Pl
1 Ax > 0 and as a consequence jMl
1 Nl jx ¼ Ml
1 Nl x < x jPl
1 Ql jx ¼ Pl
1 Ql x < x


l ¼ 1; 2; . . . ; p:

ð27Þ

Now, by Lemma 9 we have that for a suitable constant 0 6 b < 1 kHk kx 6 b;

k ¼ 1; 2; . . .

But then from (23), we have that lim keðkÞ kx ¼ lim kHk    H2 H1 eð0Þ kx

k!1

k!1

6 lim kHk kx    kH2 kx kH1 kx keð0Þ kx k!1

6 lim bk keð0Þ kx ¼ 0 k!1

for all e

ð0Þ

n

in R .


If in Theorem 6 we replace ‘‘first type’’ with ‘‘second type’’ and introduce the additional hypotheses of ‘‘El ¼ al I and Fl ¼ cl I for l ¼ 1; 2; . . . ; p’’ we obtain the following convergence result. Theorem 7. Let A be a monotone matrix. Let fMl ; Nl ; El g and fPl ; Ql ; Fl g be two weak nonnegative multisplittings of the second type. If El ¼ al I and Fl ¼ cl I for l ¼ 1; 2; . . . ; p, then iterative method (21) converges for any initial vector xð0Þ whenever (25) holds. Proof. Since both multisplittings are weak nonnegative of the second type, considering the vector xT ¼ ð1; 1; . . . ; 1ÞA
1 > 0 we have that
xT
xT Nl Ml
1 ¼ xT AMl
1 > 0 l ¼ 1; 2; . . . ; p ð28Þ xT
xT Ql Pl
1 ¼ xT APl
1 > 0 thus for a suitable constant 0 6 b < 1 xT Nl Ml
1 6 bxT

and xT Ql Pl
1 6 bxT ; l ¼ 1; 2; . . . ; p: ð29Þ P P  p mðk;lÞ p lðk;lÞ
1
1 If we denote by Jk ¼ P 0 from l¼1 Fl ðQl Pl Þ l¼1 El ðNl Ml Þ (29) we have that

510

J.-J. Climent et al. / Appl. Math. Comput. 148 (2004) 497–517 T

x Jk ¼ 6b

p X l¼1 p X

! cl x

T

ðQl Pl
1 Þmðk;lÞ

p X

! El ðNl Ml
1 Þlðk;lÞ

l¼1 T

al x

Nl Ml
1

T

ð30Þ T

6 bbx 6 bx :

l¼1

Now, taking into account that El ¼ al I and Fl ¼ cl I for l ¼ 1; 2; . . . ; p, by applying Lemma 1 to each splitting, we have that Jk ¼ AA
1 Jk ¼ AHk A
1 ;

k ¼ 1; 2; . . . :

Furthermore, from the above equality and from inequality (30), by Lemma 6 we have that lim ðHk    H2 H1 Þ ¼ 0:

ð31Þ

k!1

Finally, from (23) and (31), we obtain lim eðkÞ ¼ lim ðHk    H2 H1 Þeð0Þ ¼ 0:

k!1




k!1

If we consider a symmetric positive definite matrix A and two P -regular multisplittings, with the same additional hypotheses as in the above theorem, we can introduce the following convergence result for method (21).

Theorem 8. Let A be a symmetric positive definite matrix. Let fMl ; Nl ; El g and fPl ; Ql ; Fl g be two P -regular multisplittings. If El ¼ al I and Fl ¼ cl I for l ¼ 1; 2; . . . ; p, then iterative method (21) converges for any initial vector xð0Þ whenever (25) holds. Proof. By Theorem 2.4 of Frommer and Szyld [12] we have that kMl
1 Nl kA 6 b and kPl
1 Ql kA 6 b for a suitable constant 0 6 b < 1 and for l ¼ 1; 2; . . . ; p. Consequently, kðMl
1 Nl Þlðk;lÞ kA 6 b

and

  
1 lðk;lÞ  ðPl Ql Þ  6 b; A

l ¼ 1; 2; . . . ; p: ð32Þ

Now, taking into account that El ¼ al I and Fl ¼ cl I for l ¼ 1; 2; . . . and from (22) we have that

J.-J. Climent et al. / Appl. Math. Comput. 148 (2004) 497–517

511

 ! ! p p   X X  mðk;lÞ lðk;lÞ 
1
1 kHk kA ¼  Fl ðPl Ql Þ El ðMl Nl Þ    l¼1 l¼1 A ! ! p p X X mðk;lÞ lðk;lÞ
1
1 6 cl kðPl Ql Þ kA al kðMl Nl Þ kA l¼1

6

p X

! cl b

l¼1

p X

!

l¼1

al b 6 b2 < 1:

l¼1

Finally, from the above inequality and by equality (23) we have that lim keðkÞ kA ¼ lim kHk    H2 H1 eð0Þ kA 6 lim bk eð0Þ ¼ 0

k!1

k!1

and the proof is complete.

k!1




If in parallel iterative Model 1 expressed mathematically in (21) we consider the particular case where the number of local iteration performed from both multisplittings only depend of the processor, that is lðk; lÞ ¼ lðlÞ and mðk; lÞ ¼ mðlÞ, then we obtain a parallel stationary iterative process with the iteration matrix ! ! p p X X mðlÞ lðlÞ
1
1 H¼ Fl ðPl Ql Þ El ðMl Nl Þ : ð33Þ l¼1

l¼1

Using Lemma 2, as a principal tool, Benzi and Szyld [2] and later Climent and Perea [7], introduce parts (i) and (iii), and part (ii), respectively, of the following result. Lemma 10. Let A be a nonsingular n  n matrix. Let A ¼ M
N ¼ P
Q be two splittings and T ¼ P
1 QM
1 N . (i) If A
1 P 0 and both splittings are weak nonnegative of the first type. There exists a unique splitting A ¼ B
C associated with the iteration matrix T that is also weak nonnegative of the first type. (ii) If A
1 P 0 and both splittings are weak nonnegative of the second type. There exists a unique splitting A ¼ B
C associated with the iteration matrix T that is also weak nonnegative of the second type. (iii) If A is symmetric positive definite and both splittings are P -regular. There exists a unique splitting associated with the iteration matrix T that is also P -regular. Now, using the above lemma jointly with Lemmas 7 and 8, we introduce the following results for the stationary version of the iterative process (21).

512

J.-J. Climent et al. / Appl. Math. Comput. 148 (2004) 497–517

Theorem 9. Let A be a monotone matrix. Let fMl ; Nl ; El g and fPl ; Ql ; Fl g be two multisplittings of the matrix A. Suppose that lðk; lÞ ¼ lðlÞ P 1 and mðk; lÞ ¼ mðlÞ P 1 for l ¼ 1; 2; . . . ; p. (i) If both multisplittings are weak nonnegative of the first type then iterative method (21) is convergent and the splitting associated with the iteration matrix (33) is also weak nonnegative of the first type. (ii) If both multisplittings are weak nonnegative of the second type, with the additional hypotheses El ¼ al I and Fl ¼ cl I, for l ¼ 1; 2; . . . ; p, then iterative method (21) is convergent and the splitting associated with the iteration matrix (33) is also weak nonnegative of the second type. Proof. (i) By Theorem 6 iterative scheme (21) converges for any initial vector xð0Þ , that is qðH Þ < 1. Now by Lemma 7 (i) there exist a unique weak nonnegative splittings of the first type associated with the iteration matrices lðlÞ mðlÞ Rl ¼ ðMl
1 Nl Þ and Sl ¼ ðPl
1 Ql Þ for l ¼; 2; . . . ; p. Moreover, by Lemma 8 (i) there exist a unique weak nonnegative splittings of the first P P type associated with the iteration matrices U ¼ pl¼1 ðMl
1 Nl ÞlðlÞ and V ¼ pl¼1 ðPl
1 Ql ÞmðlÞ , respectively. Finally by Lemma 7 (i) there exists a unique weak nonnegative splitting of the first type associated with the iteration matrix T ¼ VU ¼   Pp mðlÞ Pp lðlÞ
1
1 ðP Q Þ ðM N Þ ¼ H , and the proof of this part is complete. l l l¼1 l l¼1 l (ii) Similar to part (i) using Theorem 7 instead of Theorem 6, part (ii) of Lemma 8 instead of (i), and part (ii) of Lemma 7 instead of part (i).
Finally, for a symmetric definite positive matrix A and P -regular multisplittings, with the same additional hypotheses El ¼ al I and Fl ¼ cl I of Theorem 9 (ii), we introduce the following result. Theorem 10. Let A be a symmetric positive definite matrix. Let fMl ; Nl ; El g and fPl ; Ql ; Fl g be two P -regular multisplittings. If El ¼ al I, Fl ¼ cl I, lðk; lÞ ¼ lðlÞ P 1 and mðk; lÞ ¼ mðlÞ P 1 for l ¼ 1; 2; . . . ; p, then iterative method (21) is convergent and the splitting associated with the iteration matrix (33) is also P -regular. Proof. Similar to Theorem 9 (i) using Theorem 8 instead of Theorem 6, part (ii) of Lemma 7 instead of part (i), part (iii) of Lemma 8 instead of part (i), as well as, part (iii) of Lemma 7 instead of part (i).
4.2. Convergence results for Model 2 As we have mentioned in Section 1, Model 2 consists of: considering the multisplittings fMl ; Nl ; El gpl¼1 and fPl ; Ql ; El gpl¼1 , we apply the alternating iterative method (3) in each processor

J.-J. Climent et al. / Appl. Math. Comput. 148 (2004) 497–517

ðkþ1Þ

x

¼

mðk;lÞ lðk;lÞ ðkÞ ðPl
1 Ql Þ ðMl
1 Nl Þ x

þ

þ

mðk;lÞ ðPl
1 Ql Þ

j

ðMl
1 Nl Þ Ml
1

j¼0

!

mðk;lÞ
1 X

lðk;lÞ
1 X

513

j

ðPl
1 Ql Þ Pl
1 b

j¼0

for l ¼ 1; 2; . . . ; p, until a mutual phase time is reached when all the processors are ready to contribute towards the global iteration, obtaining in this way, the following expression of mathematical formulation of Model 2 9 P xðkþ1Þ ¼ pl¼1 El ðPl
1 Ql Þmðk;lÞ ðMl
1 Nl Þlðk;lÞ xðkÞ > > >  > Pp P mðk;lÞ lðk;lÞ
1 j
1 > =
1
1 þ l¼1 El ðPl Ql Þ ðM N Þ M l l l j¼0 k ¼ 1; 2; . . . > > > Pmðk;lÞ
1
1 j
1  > > þ j¼0 ðPl Ql Þ Pl b ; ð34Þ Now, in a similar way to previous subsection, with the purpose of establishing an error analysis of iterative process (34), we define p X mðk;lÞ lðk;lÞ Vk ¼ El ðPl
1 Ql Þ ðMl
1 Nl Þ ; k ¼ 1; 2; . . . ð35Þ l¼1

Considering n the exact solution of linear system (1) and taking into account the error vector relation (11) it is easy to see that in this case ð36Þ eðkÞ ¼ Vk eðk
1Þ : Then, we can introduce sufficient conditions for the convergence of iterative process (34). However, previously with the purpose to introduce the convergence result for weak nonnegative multisplittings of the first type of a monotone matrix A we establish the following upper bound for kVk kx . p

p

Lemma 11. Let fMl ; Nl ; El gl¼1 and fPl ; Ql ; El gl¼1 be two multisplittings of the nonsingular n  n matrix A. If there exists a vector x > 0 and a constant 0 6 b < 1 such that (24) holds for l ¼ 1; 2; . . . ; p, then kVk kx 6 b, where Vk is given in (35) for each k ¼ 1; 2; . . . Proof. From (35) we have that   p X   mðk;lÞ lðk;lÞ 
1
1 jVk jx ¼  El ðPl Ql Þ ðMl Nl Þ x  l¼1  6

p X

jEl jjPl
1 Ql jmðk;lÞ jMl
1 Nl jlðk;lÞ x 6

l¼1

6 bx showing that kVk kx 6 b < 1.


p X l¼1

jEl jjPl
1 Ql jmðk;lÞ bx 6 bbx

514

J.-J. Climent et al. / Appl. Math. Comput. 148 (2004) 497–517

We are now ready to prove the following convergence results for iterative method (34). p

p

Theorem 11. Let A be a monotone matrix. Let fMl ; Nl ; El gl¼1 and fPl ; Ql ; El gl¼1 be two weak nonnegative multisplittings of the first type. Then the iterative method (34) converges for any initial vector xð0Þ whenever (25) holds. Proof. Since both multisplittings are weak nonnegative of the first type, conT sidering the vector x ¼ A
1 ð1; 1; . . . ; 1Þ > 0 we have that (26) holds and consequence, (27) also holds. Now, by Lemma 11 we have for a suitable constant 0 6 b < 1 that kVk kx 6 b;

k ¼; 1; 2; . . .

But then from (36), we obtain lim keðkÞ kx ¼ lim kVk    V2 V1 eð0Þ kx 6 lim kVk kx    kV2 kx kV1 kx keð0Þ kx

k!1

k!1

k!1

k

ð0Þ

6 lim b ke kx ¼ 0 k!1

for all eð0Þ in Rn .
If we replace in Theorem 11 ‘‘first type’’ with ‘‘second type’’ and introduce the additional hypotheses of ‘‘El ¼ al I, for l ¼ 1; 2; . . . ; p’’, we obtain the following convergence result. p

p

Theorem 12. Let A be a monotone matrix. Let fMl ; Nl ; El gl¼1 and fPl ; Ql ; El gl¼1 be two weak nonnegative multisplittings of the second type. If El ¼ al I for l ¼ 1; 2; . . . ; p, then iterative method (34) converges for any initial vector x0 whenever (25) holds. Proof. Since both multisplittings are weak nonnegative of the second type, considering the vector xT ¼ ð1; 1; . . . ; 1ÞA
1 > 0 we have that (28) holds, and as a consequence for a suitable constant 0 6 b < 1, (29) holds. If we denote by Pp mðk;lÞ lðk;lÞ Wk ¼ l¼1 El ðQl Pl
1 Þ ðNl Ml
1 Þ P 0 from (29) we have that xT Wk ¼

p X

al xT ðQl Pl
1 Þ

mðk;lÞ

ðNl Ml
1 Þ

ll;k

l¼1

6b

p X

al xT Nl Ml
1 6 bbxT 6 bxT :

l¼1

ð37Þ Now, taking into account that El ¼ al I for l ¼ 1; 2; . . . ; p, by applying Lemma 1 to each splitting, we have that Wk ¼ AA
1 Jk ¼ AVk A
1 ;

k ¼ 1; 2; . . . :

J.-J. Climent et al. / Appl. Math. Comput. 148 (2004) 497–517

515

Furthermore, from the above equality and from inequality (37), by Lemma 6 we have that lim ðVk    V2 V1 Þ ¼ 0:

ð38Þ

k!1

Finally, from (36) and (38), we obtain lim eðkÞ ¼ lim ðVk    V2 V1 Þeð0Þ ¼ 0:

k!1




k!1

If we consider A symmetric positive definite and both multisplittings P regular with the same additional hypotheses as in the above theorem, we can establish the following sufficient convergence conditions for iterative process (34). p

Theorem 13. Let A be a symmetric positive definite matrix. Let fMl ; Nl ; El gl¼1 p and fPl ; Ql ; El gl¼1 be two P -regular multisplittings. If El ¼ al I for l ¼ 1; 2; . . . ; p, then iterative method (34) converges for any initial vector xð0Þ whenever (25) holds. Proof. In a similar way that in Theorem 8 we have that (32) holds for a suitable constant 0 6 b < 1 and for l ¼ 1; 2; . . . ; p. Consequently, taking into account that El ¼ al I for l ¼ 1; 2; . . . ; p and from (35) we have that   p  X  mðk;lÞ lðk;lÞ 
1
1 El ðPl Ql Þ ðMl Nl Þ kVk kA ¼     l¼1 A

6

p X

al kðPl
1 Ql Þ

mðk;lÞ

kA kðMl
1 Nl Þ

lðk;lÞ

l¼1

kA 6

p X

al bb 6 b2 < 1:

l¼1

Finally, from the above inequality and by equality (36) we have that lim keðkÞ kA ¼ lim kVk    V2 V1 eð0Þ kA 6 lim bk eð0Þ ¼ 0

k!1

k!1

k!1

and the proof is complete.




To finish, in a similar way that in previous subsection if we consider the particular case in that the number of local iteration only depend of the processor, that is lðk; lÞ ¼ lðlÞ and mðk; lÞ ¼ mðlÞ we obtain the stationary version of iterative method (34) with the iteration matrix V ¼

p X l¼1

El ðPl
1 Ql Þ

mðlÞ

ðMl
1 Nl Þ

lðlÞ

:

ð39Þ

516

J.-J. Climent et al. / Appl. Math. Comput. 148 (2004) 497–517

Moreover, with similar proof that in Theorems 9 and 10, respectively, we can introduce the following two results. Theorem 14. Let A be a monotone matrix. Let fMl ; Nl ; El g and fPl ; Ql ; El g be two multisplittings of the matrix A. Suppose that lðk; lÞ ¼ lðlÞ P 1 and mðk; lÞ ¼ mðlÞ P 1 for l ¼ 1; 2; . . . ; p. (i) If multisplittings are weak nonnegative of the first type then iterative method (34) is convergent and the splitting associated with the iteration matrix (39) is also weak nonnegative of the first type. (ii) If the multisplittings are weak nonnegative of the second type, with the additional hypotheses El ¼ al I for l ¼ 1; 2; . . . ; p, then iterative method (34) is convergent and the splitting associated with the iteration matrix (39) is also weak nonnegative of the second type. Theorem 15. Let A be a symmetric positive definite matrix. Let fMl ; Nl ; El g and fPl ; Ql ; El g be two P -regular multisplittings. If El ¼ al I lðk; lÞ ¼ lðlÞ P 1 and mðk; lÞ ¼ mðlÞ P 1 for l ¼ 1; 2; . . . ; p, then iterative method (34) is convergent and the splitting associated with the iteration matrix (39) is also P -regular. Note, that in a similar way that assumption (13) can be weakened in Theorems 1–3 in convergence results presented in this section for the nonstationary iterative method (34) assumption (25) can be weakened by lðk; lÞ P 0

and

mðk; lÞ P 0;

k ¼ 1; 2; . . . ; l ¼ 1; 2; . . . ; p

and for infinitely many kÕs, lðk; lÞ P 1

and

mðk; lÞ P 1

for all l ¼ 1; 2; . . . ; p:

References [1] A. Berman, R.J. Plemmons, Nonnegative Matrices in the Mathematical Sciences, Academic Press, New York, 1979. Reprinted by SIAM, Philadelphia, PA, 1994. [2] M. Benzi, D.B. Szyld, Existence and uniqueness of splittings for stationary iterative methods with applications to alternating methods, Numerische Mathematik 76 (1997) 39–321. [3] R. Bru, L. Elsner, M. Neumann, Models of parallel chaotic iteration methods, Linear Algebra and its Applications 103 (1988) 175–192. [4] R. Bru, V. Migall on, J. Penades, Chaotic methods for the parallel solution of linear systems, Computing Systems in Engineering 6 (1995) 385–390. [5] J.-J. Climent, C. Perea, Some comparison theorems for weak nonnegative splittings of bounded operators, Linear Algebra and its Applications 275/276 (1998) 77–106. [6] J.-J. Climent, C. Perea, Convergence and comparison theorems for multisplittings, Numerical Linear Algebra with Applications 6 (1999) 93–107. [7] J.-J. Climent, C. Perea, Convergence and comparison theorems for a generalized alternating iterative method, Applied Mathematics and Computation, in press.

J.-J. Climent et al. / Appl. Math. Comput. 148 (2004) 497–517

517

[8] V. Conrad, Y. Wallach, Alternating methods for sets of linear equations, Numerische Mathematik 32 (1979) 105–108. [9] L. Elsner, Comparisons of weak regular splittings and multisplitting methods, Numerische Mathematik 56 (1989) 283–289. [10] A. Frommer, G. Mayer, Convergence of relaxed parallel multisplitting methods, Linear Algebra and its Applications 119 (1989) 141–152. [11] A. Frommer, G. Mayer, On the theory and practice of multisplitting methods in parallel computation, Computing 49 (1992) 63–74. [12] A. Frommer, D.B. Szyld, Weighted max norms, splittings, and overlapping additive Schwarz iterations, Numerische Mathematik 83 (1999) 259–278. [13] P.J. Lanzkron, D.J. Rose, D.B. Szyld, Convergence of nested classical iterative methods for linear systems, Numerische Mathematik 58 (1991) 685–702. [14] G.I. Marchuk, Splitting and alternating direction methods, in: P.G. Ciarlet, J.L. Lions (Eds.), Handbook of Numerical Analysis, vol. I, North Holland, New York, NY, 1990, pp. 197–462. [15] I. Marek, D.B. Szyld, Comparison theorems for weak splittings of bounded operators, Numerische Mathematik 58 (1990) 389–397. [16] R. Nabben, A note on comparison theorems for splittings and multisplittings of Hermitian positive definite matrices, Linear Algebra and its Applications 233 (1996) 67–80. [17] M. Neumann, R.J. Plemmons, Convergence of parallel multisplitting iterative methods for M– matrices, Linear Algebra and its Applications 88/89 (1987) 559–573. [18] D.P. OÕLeary, R.E. White, Multi-splittings of matrices and parallel solution of linear systems, SIAM Journal on Algebraic and Discrete Methods 6 (1985) 630–640. [19] J.M. Ortega, Numerical Analysis, A second course, Academic Press, New York, NY, 1972. Reprinted by SIAM, Philadelphia, PA, 1992. [20] W. Rheinboldt, J. Vandergraft, A simple approach to the Perron–Frobenius theory for positive operators on general partially-ordered finite-dimensional linear spaces, Mathematics of Computation 27 (121) (1973) 139–145. [21] R.S. Varga, Matrix Iterative Analysis, Prentice-Hall, Englewood Cliffs, NJ, 1962. [22] D. Wang, On the convergence of the parallel multisplitting AOR algorithm, Linear Algebra and its Applications 154/156 (1991) 473–486. [23] Z.I. Woznicki, Nonnegative splitting theory, Japan Journal on Industrial and Applied Mathematics 11 (1994) 289–342.