- Email: [email protected]
The block version of the reflection method for the solution of linear algebraic systems
THE BLOCK VERSION OF THE REFLECTION METHOD FOR THE SOLUTION OF LINEAR ALGEBRAIC SYSTEMS* V. V. VOEVODIN
and T. L. RUDNEVA Moscow
(Receiued
IN the solution necessary
of large systems
to create
In comparing
of algebraic
and use block versions
the different
(number of arithmetical computer
20 January
methods
operations)
are usually
considered.
been given to questions
equations
by a computer
of known methods
of solution,
their speed
and the convenience However,
of the accuracy
the point of view of the accuracy
1970)
particular
and stability
11,
of operation
of realization attention in a class
on the
has recently
of calculation
of the calculations
it becomes
21.
[3, 41. From
of systems,
methods based on orthogonal transformations are the best, and the reflection method is most efficient of these. The present note describes the block version of the method of reflections. Let there be a rectangular into blocks
where S; is a square S are orthogonal,
real matrix S of size n x I, 1 ,( n. We partition
matrix of order 1. We suppose
that the columns
it
of the matrix
that it, S’S = R,,
(1)
where E, is a unit matrix of order 1. By analogy the matrix
with the usual
version
of the reflection
R = E,, - 2ww’ where w is an n x 1 matrix
and w’w = El-
* Zh. vychisl.
10, 5, 1281-1285,
Mat. mat. Fiz.,
265
1970.
method we will construct (2)
V. V. Voeuodzn and T. L. Rudneva
266
We have R’R = E, -
4ww’+
4ww’ww’
= E,,
that is, R is an orthogonal matrix. We will select w so that RS=Q,
(3)
Since R and S are orthogonal, Q (and consequently S’R’RS
=
Q1) must also be orthogonal:
Q’Q = El.
We rewrite (3), using (2): (E, - 2ww’)S = 0.
(4)
We multiply equation (4) on the left by S’: S’S - ZS’UW’S= SO,
(5)
As the matrix on the left is symmetric, S’Q must also be symmetric. Let Sl = tAr be a singular expansion [51 of the matrix S,. We put QI = tlr, where I is a diagonal matrix with real diagonal elements i,,, with modulus equal to 1. We determine the signs of these elements later. We show that the matrix Q constructed in this way satisfies the necessary conditions. Since t, r and 1 are orthogonal, Q is orthogonal. Moreover, S’Q = Q’S.
Indeed, as in our case S’Q = Sl’Q, and Q’S = Q’&, is, this matrix is symmetric. It is now necessary the new matrix
(6)
we have S,‘Q( =
r’l.t’tlr,
that
to find the matrix w. It is more convenient to pass to (6’)
U=S-Q. We write W’S = k, where k is square, and by hypothesis, (4) we have 2wk =
c,
from which 2w = Uk-’ and 2ww’=
‘/JJ(k’k)-‘U’.
nondegenerate.
From
(7)
Block
But the orthogonality
267
of the reflection method
version
of S and Q and relation (6) imply that U’U= (Y-Q’)(S--0)
(8)
=2(E-TQ).
On the other hand, from equation (5) we have 2k’k
Therefore,
=
E-
YQ.
rr’U = Wk.
Q, = tlr.
R = E, - 2U(U’U)-W’,
In what follows we have to calculate (VU)-‘, hence for the stability of the calculation it is necessary that the conditionality of the matrix U’U be minimal. But U/U = 2(El - S{Ql) = 2(E1- rWr) = 2r’(E - hl)r. (9) Because S, is an I-th order minor of the orthogonal matrix S, we have (A,,/ < 1 for all lz, and the maximal eigenvalue of U’U does not exceed 4. Ve select the signs of the diagonal elements of the matrix I in such a way that the minimal eigenvalue will be as great as possible, namely, we put ihh =
+ 1,
hkk < 0,
{ - $3
hkk 2 0.
We will show that the matrix constructed have
We
R’R = (E - 2U(U’U) -‘U’) (E - 2U (VU) --VP) =: = E -MJ(U’U)-W’ f RS -
But, by (8), U = Q.
in this way is the one required.
U’S =
4U(U’L’)-‘U’U(U’U)-‘U’
(&‘- 2U(li’U)-‘tJ’)S
IS’ -
Q’)S = S’S -
= E.
= S - 2U(U’U)--VS.
Q’S = E -
Q’S = ‘/pY’U,
hence RS = S -
Now let the n x 1 matrix S be arbitrary. By means of orthogonal transformations (for example, rotation matrices) we reduce it to the form S=Nh,
L&“,‘,,
(10)
where N is orthogonal, and -2, is triangular upper. As A is non-zero only in the first 1 rows, only the first 1 columns of the n x n matrix N participate in the expansion (10). We denote them by S. The orthogonality conditions are satisfied by S, hence by the above, we can construct a block reflection matrix R such that
V. V. Voevodin
268
have RS = G :I,.
We finally
We now estimate point calculations
the value
the columns
The exact
or, using
of notation
of the given
errors
on executing
of the reflection
and some results
matrix S are almost
floating
method.
In this we
from [31. We first suppose
orthogonal,
that
that is,
(6’) and (9), we find (S-@r’(Z?~-kZ)-lr(S’-_‘).
R = E, -
We suppose multiplying
version
matrix of the transformation is R - E, - XJ(U’lJ)-‘U’, relations
will arise
of the rounding
by the proposed
will use the system
calculated
and T. L. Rudneva
(11)
that we will store the matrix R as in&vi&al them explicitly.
Hence all errors of representation
only in the calculation values
factors,
of these
factors.
not of the matrix R
We will denote
actually
by a bar above them.
1. We calculate
the components
of the singular
expansion
SI = tlir, f% = s[ +
(12) (13)
F2.
2.
We calculate
the matrix
Q = f12(tar:) E SZ~+ Q.
3.
We calculate
the matrix ci = fl(s - Q) G s - Q + E&.
4.
We calculate
the diagonal
matrix
B =
fZ(E-h)
--i E (E --%I)-’ i- ES.
Unfortunately it cannot be shown t!mt IIR - i?II is small, but the relation R’A = E + Q,, can be obtained, where llrnll is sufficiently small. Indeed,
We now show that L?‘F~~& _ 21;‘_I- F”. We have B’FG’[~~’= B;(S’ - i7’ + E:‘) (,$ - Q + EL)?’ zzzBF(S’ - 0’) (S Let II Ed& G (12 + &)2-t. consider ~(s’-~)(S-~)~‘=~(S’S+o’,T-~Q’S-S’Q)I:’~ z=.zr(E + E, + E + Ed, - (f’lt’
separately + ~~‘)fiir -
(12) and (13) we find
+
EIO.
the expression
=
From equations
o)f’
r’ht’(lZF r2EF’ -
+ &a))?’ = i?‘Zt’thrF
-
ir’kt’lZ+
+ ~1~.
Block version
Moreover
of the reflection
269
method
E=’=? E + E,(. From this we obtain F(S’- Q’)(S - Q)?’ = 2(E-Ii) + 815, B'FD'iP= 2E $ Es.
Consequently,
K’R = E -l- Pi,
and it remains to estimate \lcc*l!,.
Ye will denote by the single fetter c aI the errors of higher order arising in the evaluation. Suppose we know that for some constants f and 6 we have IIS,II~< f2-f and II~2tiz G ‘~25. A matrix t, appears on multiplication of the almost orthogonal matrices with binary accuracy, hence II c3 II2 sg (I+ E)?‘. The matrix c, consists of the errors on adding two almost matrices, consequently, ii ~4112 SZ (2 -I- E)2-‘. The diagonal matrix fg appears in calculations of the type l/a, where all the a > 1, hence IIas ll2G (I+ E)Z-~. Since llBll, < I, taking into account the relation 810= B?&l’(S- Q)r’ + @(S’ - Q’)E&i;‘, we find
iYe estima~
llc,,li,. This error has the form llat,llr?G (4 + E)Z-( &I2= iElF’f FEI# - i;Ed’thr?- i;f’W&3F’.
Consequently, ll&,2112 < (j +
6
taking into account the estimates f 8)2-t. In its turn, &t3= T’E~?.
obtained above, we find Hence
II ~~~~~~G (q + &)2-f.
We now consider
ll ErsllaG (10 + 26p-+ f + &)2-t. Since n’Z’r;,:’ = ((E -iz)-’ + Es)(2(E-zh)+ E,s) -+- tlo + aO, we have -I-(E-G) -fei3 lle& *g (4 f IO+ 2q~ -tf f 8 i e)Fi,
and F$= e52(E- IX)
V. V. Voevodin
270
Or
and
T. L. Rudneua
(22 + 2q + I + e)2-‘.
II Eo II2 <
Finally, FWE~BFU’
8s = It has therefore
II ea II < (88 + 8cp + 4f + ~)2-~.
and
been shown that the matrix !? constructed
We now consider
is almost
orthogonal.
the product = S - i7r’BfB’S
Jj,$ = (E - uf’BrU’)S
=
=S-(S---_~F~)~‘((E-~Z)-~+F~)~(S’--++EC~)S= =s--
(S-Q)ryE--hZ)-~r(S’-~)S+c,,.
Here E,e = -&$(E--;LI)-‘?(S’Consequently,
II 13202
Ye consider F(s’-
zj;,sSZ
(12
+
separately
is--
where
Therefore,
Q)f’(E--?d)-‘?Ec’S.
the product (F’ZT’ + ~~‘)thr)
Q’)s = ?(s’s - o’s) = F(E[ + &i -
Es&
(s-
e)Z-‘.
di7 = Si - res’thr - Eir’Zj’tlir
where -Ii +
@F’&gl.(S’- B’)s-
E~~= t’~
=
i%: -
II elsl12< (cp+ ~)2-*.
FEI
+
+ EI’I,
Since S’thr =
II ~~~11~ < (2 + f + ~)2-~.
and
we have
Zl’dr
we have r(S’El9
i7’)S =
=
El7 -
-
Zi?
El9
=
11 E19112 <
ZE18,
[El
(2
-Ii)?
+
eis,
f + rP+ E)2-‘,
+
from which hs
=
s-
S(E’I
+
ezo)
+
Q(E:
4
EZO) +
El6
il Eeoh
s
(3 i-
(18
+
=
Q+
EN,
where E20 =
Ezi =
Therefore,
by using
Eir
+
r’(E
-k)
84s + &II -
-‘&is,
SEW,
the transformation
II e2lii2
,(
R and replacing
f f
~g+ &)2-I,
2f + 2~ + .5)2-t.
ES by &, we introduce
the an error not greater than E,,. V/e now estimate the error on calculating product R$, where g is no longer assumed to be orthogonal, that is, we estimate the difference F = I73 - fl (RS). We have ,RS zz (E - i7r’BrU’)S As above,
we consider
the successive
=
S -
.Tr’BrU-‘S.
operations
Block
version
of the reflection
211
method
fl2(O’S) = i7’s’ + Fi,
11 Fi IIE<
(2 + E) 11 3 I/.E~-~,
fZz(Ff13(L;‘S)) = iU’S + Fz,
IIFz IIE<
(4 + E) IIs IlE2_‘,
fk (Bfl: (FU’S)) = BE!7’S + F3,
IIFxIIE
fla(r’fl~(BFiT’S)) = F’BFU’.S+ F‘,
11FI
11: (u/1: ( FBFU’S)
II F 1tE G (20 + E) II S llE2-f.
) =
Dr’ByP$
Therefore, the multiplication with good accuracy.
+ F,
of i by the arbitrary
=z (6f~)
I/E <
tISIt132-~,
(8 + E) tl s tt~2-*,
matrix gcan
be carried
Translated
out
by J. Berry
REFERENCES
1.
VOLOVICH, V. M. The solution of systems of linear algebraic equations by block methods and programming Vychisl. metody i methods. In: Computing programmirovani) No 3,Izd-vo MGU,Moscow, 1965.
2.
TAN CHZEN’, A lattice method for the orthogonalization of the solution of systems of simultaneous linear algebraic equations with a large number of unknowns, Zh. uychisl. Mat. Piz., 1, 1, 78-89, 1961.
3.
WILKINSON,
4.
VOEVODIN, V. V. Rounding errors and stability in direct methods of linear algebra (Oshibki okrugleniya i ystoichivost v pryamykh metodakh lineinoi algebry) Rotaprint VTs MGU, 1969.
5.
VOEVODIN, V. V. Numerical methods of algebra ( theory ana zlgorithms) (Chislennye metody algebry (teoriya i algorifmy)), ‘Nauka’, Moscow, 1966.
J. H. The
algebraic
eigenvalue
problem.
Clarendon
Press.
Oxford,
1965.
and T. L. RUDNEVA Moscow
(Receiued
IN the solution necessary
of large systems
to create
In comparing
of algebraic
and use block versions
the different
(number of arithmetical computer
20 January
methods
operations)
are usually
considered.
been given to questions
equations
by a computer
of known methods
of solution,
their speed
and the convenience However,
of the accuracy
the point of view of the accuracy
1970)
particular
and stability
11,
of operation
of realization attention in a class
on the
has recently
of calculation
of the calculations
it becomes
21.
[3, 41. From
of systems,
methods based on orthogonal transformations are the best, and the reflection method is most efficient of these. The present note describes the block version of the method of reflections. Let there be a rectangular into blocks
where S; is a square S are orthogonal,
real matrix S of size n x I, 1 ,( n. We partition
matrix of order 1. We suppose
that the columns
it
of the matrix
that it, S’S = R,,
(1)
where E, is a unit matrix of order 1. By analogy the matrix
with the usual
version
of the reflection
R = E,, - 2ww’ where w is an n x 1 matrix
and w’w = El-
* Zh. vychisl.
10, 5, 1281-1285,
Mat. mat. Fiz.,
265
1970.
method we will construct (2)
V. V. Voeuodzn and T. L. Rudneva
266
We have R’R = E, -
4ww’+
4ww’ww’
= E,,
that is, R is an orthogonal matrix. We will select w so that RS=Q,
(3)
Since R and S are orthogonal, Q (and consequently S’R’RS
=
Q1) must also be orthogonal:
Q’Q = El.
We rewrite (3), using (2): (E, - 2ww’)S = 0.
(4)
We multiply equation (4) on the left by S’: S’S - ZS’UW’S= SO,
(5)
As the matrix on the left is symmetric, S’Q must also be symmetric. Let Sl = tAr be a singular expansion [51 of the matrix S,. We put QI = tlr, where I is a diagonal matrix with real diagonal elements i,,, with modulus equal to 1. We determine the signs of these elements later. We show that the matrix Q constructed in this way satisfies the necessary conditions. Since t, r and 1 are orthogonal, Q is orthogonal. Moreover, S’Q = Q’S.
Indeed, as in our case S’Q = Sl’Q, and Q’S = Q’&, is, this matrix is symmetric. It is now necessary the new matrix
(6)
we have S,‘Q( =
r’l.t’tlr,
that
to find the matrix w. It is more convenient to pass to (6’)
U=S-Q. We write W’S = k, where k is square, and by hypothesis, (4) we have 2wk =
c,
from which 2w = Uk-’ and 2ww’=
‘/JJ(k’k)-‘U’.
nondegenerate.
From
(7)
Block
But the orthogonality
267
of the reflection method
version
of S and Q and relation (6) imply that U’U= (Y-Q’)(S--0)
(8)
=2(E-TQ).
On the other hand, from equation (5) we have 2k’k
Therefore,
=
E-
YQ.
rr’U = Wk.
Q, = tlr.
R = E, - 2U(U’U)-W’,
In what follows we have to calculate (VU)-‘, hence for the stability of the calculation it is necessary that the conditionality of the matrix U’U be minimal. But U/U = 2(El - S{Ql) = 2(E1- rWr) = 2r’(E - hl)r. (9) Because S, is an I-th order minor of the orthogonal matrix S, we have (A,,/ < 1 for all lz, and the maximal eigenvalue of U’U does not exceed 4. Ve select the signs of the diagonal elements of the matrix I in such a way that the minimal eigenvalue will be as great as possible, namely, we put ihh =
+ 1,
hkk < 0,
{ - $3
hkk 2 0.
We will show that the matrix constructed have
We
R’R = (E - 2U(U’U) -‘U’) (E - 2U (VU) --VP) =: = E -MJ(U’U)-W’ f RS -
But, by (8), U = Q.
in this way is the one required.
U’S =
4U(U’L’)-‘U’U(U’U)-‘U’
(&‘- 2U(li’U)-‘tJ’)S
IS’ -
Q’)S = S’S -
= E.
= S - 2U(U’U)--VS.
Q’S = E -
Q’S = ‘/pY’U,
hence RS = S -
Now let the n x 1 matrix S be arbitrary. By means of orthogonal transformations (for example, rotation matrices) we reduce it to the form S=Nh,
L&“,‘,,
(10)
where N is orthogonal, and -2, is triangular upper. As A is non-zero only in the first 1 rows, only the first 1 columns of the n x n matrix N participate in the expansion (10). We denote them by S. The orthogonality conditions are satisfied by S, hence by the above, we can construct a block reflection matrix R such that
V. V. Voevodin
268
have RS = G :I,.
We finally
We now estimate point calculations
the value
the columns
The exact
or, using
of notation
of the given
errors
on executing
of the reflection
and some results
matrix S are almost
floating
method.
In this we
from [31. We first suppose
orthogonal,
that
that is,
(6’) and (9), we find (S-@r’(Z?~-kZ)-lr(S’-_‘).
R = E, -
We suppose multiplying
version
matrix of the transformation is R - E, - XJ(U’lJ)-‘U’, relations
will arise
of the rounding
by the proposed
will use the system
calculated
and T. L. Rudneva
(11)
that we will store the matrix R as in&vi&al them explicitly.
Hence all errors of representation
only in the calculation values
factors,
of these
factors.
not of the matrix R
We will denote
actually
by a bar above them.
1. We calculate
the components
of the singular
expansion
SI = tlir, f% = s[ +
(12) (13)
F2.
2.
We calculate
the matrix
Q = f12(tar:) E SZ~+ Q.
3.
We calculate
the matrix ci = fl(s - Q) G s - Q + E&.
4.
We calculate
the diagonal
matrix
B =
fZ(E-h)
--i E (E --%I)-’ i- ES.
Unfortunately it cannot be shown t!mt IIR - i?II is small, but the relation R’A = E + Q,, can be obtained, where llrnll is sufficiently small. Indeed,
We now show that L?‘F~~& _ 21;‘_I- F”. We have B’FG’[~~’= B;(S’ - i7’ + E:‘) (,$ - Q + EL)?’ zzzBF(S’ - 0’) (S Let II Ed& G (12 + &)2-t. consider ~(s’-~)(S-~)~‘=~(S’S+o’,T-~Q’S-S’Q)I:’~ z=.zr(E + E, + E + Ed, - (f’lt’
separately + ~~‘)fiir -
(12) and (13) we find
+
EIO.
the expression
=
From equations
o)f’
r’ht’(lZF r2EF’ -
+ &a))?’ = i?‘Zt’thrF
-
ir’kt’lZ+
+ ~1~.
Block version
Moreover
of the reflection
269
method
E=’=? E + E,(. From this we obtain F(S’- Q’)(S - Q)?’ = 2(E-Ii) + 815, B'FD'iP= 2E $ Es.
Consequently,
K’R = E -l- Pi,
and it remains to estimate \lcc*l!,.
Ye will denote by the single fetter c aI the errors of higher order arising in the evaluation. Suppose we know that for some constants f and 6 we have IIS,II~< f2-f and II~2tiz G ‘~25. A matrix t, appears on multiplication of the almost orthogonal matrices with binary accuracy, hence II c3 II2 sg (I+ E)?‘. The matrix c, consists of the errors on adding two almost matrices, consequently, ii ~4112 SZ (2 -I- E)2-‘. The diagonal matrix fg appears in calculations of the type l/a, where all the a > 1, hence IIas ll2G (I+ E)Z-~. Since llBll, < I, taking into account the relation 810= B?&l’(S- Q)r’ + @(S’ - Q’)E&i;‘, we find
iYe estima~
llc,,li,. This error has the form llat,llr?G (4 + E)Z-( &I2= iElF’f FEI# - i;Ed’thr?- i;f’W&3F’.
Consequently, ll&,2112 < (j +
6
taking into account the estimates f 8)2-t. In its turn, &t3= T’E~?.
obtained above, we find Hence
II ~~~~~~G (q + &)2-f.
We now consider
ll ErsllaG (10 + 26p-+ f + &)2-t. Since n’Z’r;,:’ = ((E -iz)-’ + Es)(2(E-zh)+ E,s) -+- tlo + aO, we have -I-(E-G) -fei3 lle& *g (4 f IO+ 2q~ -tf f 8 i e)Fi,
and F$= e52(E- IX)
V. V. Voevodin
270
Or
and
T. L. Rudneua
(22 + 2q + I + e)2-‘.
II Eo II2 <
Finally, FWE~BFU’
8s = It has therefore
II ea II < (88 + 8cp + 4f + ~)2-~.
and
been shown that the matrix !? constructed
We now consider
is almost
orthogonal.
the product = S - i7r’BfB’S
Jj,$ = (E - uf’BrU’)S
=
=S-(S---_~F~)~‘((E-~Z)-~+F~)~(S’--++EC~)S= =s--
(S-Q)ryE--hZ)-~r(S’-~)S+c,,.
Here E,e = -&$(E--;LI)-‘?(S’Consequently,
II 13202
Ye consider F(s’-
zj;,sSZ
(12
+
separately
is--
where
Therefore,
Q)f’(E--?d)-‘?Ec’S.
the product (F’ZT’ + ~~‘)thr)
Q’)s = ?(s’s - o’s) = F(E[ + &i -
Es&
(s-
e)Z-‘.
di7 = Si - res’thr - Eir’Zj’tlir
where -Ii +
@F’&gl.(S’- B’)s-
E~~= t’~
=
i%: -
II elsl12< (cp+ ~)2-*.
FEI
+
+ EI’I,
Since S’thr =
II ~~~11~ < (2 + f + ~)2-~.
and
we have
Zl’dr
we have r(S’El9
i7’)S =
=
El7 -
-
Zi?
El9
=
11 E19112 <
ZE18,
[El
(2
-Ii)?
+
eis,
f + rP+ E)2-‘,
+
from which hs
=
s-
S(E’I
+
ezo)
+
Q(E:
4
EZO) +
El6
il Eeoh
s
(3 i-
(18
+
=
Q+
EN,
where E20 =
Ezi =
Therefore,
by using
Eir
+
r’(E
-k)
84s + &II -
-‘&is,
SEW,
the transformation
II e2lii2
,(
R and replacing
f f
~g+ &)2-I,
2f + 2~ + .5)2-t.
ES by &, we introduce
the an error not greater than E,,. V/e now estimate the error on calculating product R$, where g is no longer assumed to be orthogonal, that is, we estimate the difference F = I73 - fl (RS). We have ,RS zz (E - i7r’BrU’)S As above,
we consider
the successive
=
S -
.Tr’BrU-‘S.
operations
Block
version
of the reflection
211
method
fl2(O’S) = i7’s’ + Fi,
11 Fi IIE<
(2 + E) 11 3 I/.E~-~,
fZz(Ff13(L;‘S)) = iU’S + Fz,
IIFz IIE<
(4 + E) IIs IlE2_‘,
fk (Bfl: (FU’S)) = BE!7’S + F3,
IIFxIIE
fla(r’fl~(BFiT’S)) = F’BFU’.S+ F‘,
11FI
11: (u/1: ( FBFU’S)
II F 1tE G (20 + E) II S llE2-f.
) =
Dr’ByP$
Therefore, the multiplication with good accuracy.
+ F,
of i by the arbitrary
=z (6f~)
I/E <
tISIt132-~,
(8 + E) tl s tt~2-*,
matrix gcan
be carried
Translated
out
by J. Berry
REFERENCES
1.
VOLOVICH, V. M. The solution of systems of linear algebraic equations by block methods and programming Vychisl. metody i methods. In: Computing programmirovani) No 3,Izd-vo MGU,Moscow, 1965.
2.
TAN CHZEN’, A lattice method for the orthogonalization of the solution of systems of simultaneous linear algebraic equations with a large number of unknowns, Zh. uychisl. Mat. Piz., 1, 1, 78-89, 1961.
3.
WILKINSON,
4.
VOEVODIN, V. V. Rounding errors and stability in direct methods of linear algebra (Oshibki okrugleniya i ystoichivost v pryamykh metodakh lineinoi algebry) Rotaprint VTs MGU, 1969.
5.
VOEVODIN, V. V. Numerical methods of algebra ( theory ana zlgorithms) (Chislennye metody algebry (teoriya i algorifmy)), ‘Nauka’, Moscow, 1966.
J. H. The
algebraic
eigenvalue
problem.
Clarendon
Press.
Oxford,
1965.