A note on the condition of a matrix

INT. COMM. H E A T M A S S T R A N S F E R VOI. ii, pp. 191-195, 1984

0735-1933/84 $3.00 + .00 @Pergamon

P r e s s Ltd.

A N O T E ON THE C O N D I T I O N

Printed

in t h e U n i t e d S t a t e s

OF A MATRIX

Graeme Fairweather Departments of M a t h e m a t i c s and Engineering Mechanics U n i v e r s i t y of K e n t u c k y Lexington, K e n t u c k y 40506

(C~L,~nicated

In a recent approximate

paper

b y J.P.

Hartnett

a n d W.J. Minkowycz)

[I], the p r o b l e m of e s t i m a t i n g

solution,

x*, of a system of linear

(I)

Ax=

algebraic

of an

equations

b

was discussed.

In [I], it was e m p h a s i z e d

of the residual

vector

(2)

the accuracy

by means of an example

that the size

r = b - Ax*

is not a reliable

indicator

shown by the following for example,

From

simple

analysis,

of x*.

which

[2].

(I) and

(2), it follows

(3) Then,

of the accuracy

A(x-x*) asstuning that

A

that

= b - Ax* = r.

is nonsingular, x - x* = A-Ir,

and,

if II'll denotes

(4) From

the usual

p-norm,

ilx- x*ll-< ilA-111.ilril. (I), we have

191

This is indeed true,

appears

in several

texts;

as is see,

192

G. F a i r w e a t h e r

Vol.

ilbil -< ilAil.llxll and combining

this

inequality

with

Ilx - x*ll/ilxll

(5) Also,

from

(I) a n d

(4) w e

ii, No.

,

find that

-< ilA-111.11Ail.ilril/llbil-

(3), w e h a v e

Ilxll-<

IIA-111.11bil,

and

JlrH ~ IbH.llx-x'jl, respectively,

and from these i n e q u a l i t i e s

[llA-111.ilAll]-1.ilrlllilbll

(6) From

~llx-x*ilMlxll

(6), we see that if the number

that a small relative

residual,

the size of the residual ensure a small relative number of A, cond(A),

and

(5) it follows

that

~ ilA-lil.llAll.llrlllllbl].

IIA-IN.IIAil

is large there is no guarantee

iirll/llbll, which can be v i e w e d as a measure

vector r e l a t i v e to that of the p r o b l e m error

ilx-x*II/llxll. This number

itself,

of

will

is c a l l e d the c o n d i t i o n

the actual value of w h i c h depends

upon the norm used.

Since

iiIil

I :

it follows

that cond(A) ~ I.

well-conditioned,

and

A

estimate

Gaussian elimination operations,

In

ilA-IAil ~ NA-Iil.ilAII,

If cond(A)

is r e l a t i v e l y

is i l l - c o n d i t i o n e d

(Note that it is not n e c e s s a r y reasonable

:

of cond(A). procedure,

to d e t e r m i n e

if cond(A) A

-I

A is said to be

is r e l a t i v e l y

large.

in order to obtain a

Such an e s t i m a t e for example,

small,

can be o b t a i n e d

during the

at an extra cost of only O(n 2)

where n is the order of the matrix;

see

[I], Raman c o n s i d e r e d the example 0.782x I + 0 . 5 6 4 x 2 = 0.218 0.911x I + 0.658x 2 = 0.253,

[3,4,5,6].)

2

Vol.

ii, NO.

2

A NOTE ON THE ODNDITION

and showed that the vector vector

(0.999,-I.001),

this example,

coefficient since

which

an estimate

the subroutine

(0.339,-0.084) is closer

answers

[7] c l a i m e d

examines

Raman's

of i l l - c o n d i t i o n e d

that Raman's

the determinant

0.752,10 -3 •

Tal c o n c l u d e d

it is a popular determinant

(1.0,-1.0).

For

l-norm p r o v i d e d indicates

example

by

that the

is not too s u r p r i s i n g

equations

much more meaningful that the m a t r i x llAll/cond(A). distance cond(A)

A

that inaccurate

that,

measure differs

from a singular may be c o n s i d e r e d

A

the m a t r i x number

A

is not a f f e c t e d

measure

is as w e l l - c o n d i t i o n e d

triangular

matrix

as possible.

is

example,

of the

It can be shown

A [8]

in n o r m by no more than

as the reciprocal matrices;

of the relative hence,

if

It should be noted

by scaling nor is it directly the determinant since

For example,

of A is 10 -100,

[8] in which

defined by

matrix

[9, page 283],

and the order of the matrix.

then the determinant

number.

"By comparison,

of ill-conditionin@",

from

the m a t r i x

to singularity.

is close to singularity.

terrible

example

is small,

the smallness

to the set of singular

by the size of the matrix.

following

when one

which has the value

of its nearness

is the c o n d i t i o n

affected

the scaling

in general,

is an indication

from the m a t r i x

that the condition

matrix,

explained

While this may be true for this particular

Thus cond(A)

is large,

is very easily

that since the determinant

misconception

of a matrix

example

of the coefficient

close to b e i n g singular.

matrix

in the

than the

can give small residuals.

Tal

= I/I0,

number

193

residual

solution,

[5] is 0.343.103 , which

is ill-conditioned.

it is a characteristic

a smaller

to the exact

of the condition

SGECO in L I N P A C K

matrix

gives

OF A MATRIX

yet,

since

it depends

if the

A

on both

I00xi00 m a t r i x A

cond(A)

On the other hand,

the m a t r i x

is a

= I, this consider

the

is the the nxn unit upper

194

G. F a i r w e a t h e r

a

Vol.

Ii, No.

2

= 0, if i > j, z3

a

= -I, if i < j, 13

a If 2 1-n is s u b t r a c t e d matrix becomes

singular.

of the G a u s s i a n experimentally lost b e c a u s e

Thus,

speaking,

for large n, the matrix

and must be ill-conditioned,

n u m b e r plays an i m p o r t a n t elimination

process

that in G a u s s i a n

A

differs

negligibly

yet its d e t e r m i n a n t

has

role in the r o u n d i n g error analysis

for s o l v i n g

elimination

of r o u n d i n g errors,

can be i m p r o v e d u s i n g iterative I/cond(A)

for example,

In conclusion, important measure In fact,

It can be shown

log(cond(A))

epsilon,

I/cond(A)

decimal

places can be

~ 10 p, then the last p

The a c c u r a c y of a c o m p u t e d

improvement

is larger than m a c h i n e

provided,

roughly

that is, p r o v i d e d

> I;

[4].

it is worth

s t r e s s i n g again that cond(A)

is a far more

of the badness of a linear s y s t e m Ax=b than the d e t e r m i n a n t

Rice

no useful p u r p o s e

(I).

that is, if cond(A)

I +

of A.

in the first column of A, the

digits of the solution may be in error.

solution

see,

I .

I.

The c o n d i t i o n

decimal

=

from each of the elements

from a singular matrix the value

ii

[10, page 23]

goes as far as to say " D e t e r m i n a n t s

serve

in linear a l g e b r a computation".

REFERENCES I.

V. M. Raman, A s s e s s m e n t of s o l u t i o n s of finite d i f f e r e n c e e q u a t i o n s in fluid m e c h a n i c s , L e t t e r s in Heat and Mass Transfer, 9(1982), pp. 73-75.

2.

S. D. Conte and C. de Boor, E l e m e n t a r y N u m e r i c a l Analysis, M c G r a w - H i l l Book Company, New York, 1980.

3.

G. E. Forsythe, M. A. M a l c o l m and C. B. Moler, C o m p u t e r Methods M a t h e m a t i c a l Computations, Prentice-Hall, New Jersey, 1977.

Third Edition,

for

VOI. ii, NO. 2

A NOTS O~ THE CONDITION OF A MATRIX

195

4.

R. L. Johnston, Numerical Methods: A Software Approach, JOhn wiley and Sons, New York, 1982.

5.

J. J. Dongarra, C. B. Moler, J. R. Bunch and G. W. Stewart, L I ~ A ~ Guides ~iAMI Philadelphia, 1979.

6.

A. ~s Cline, A. R. Conn and C. F. Van Loan, Generalizing the LINPACK condition estimator, L e c t u r e N o t e s in Mathematics 909, Springer Verlag, New York, 1982, pp. 73-83.

7.

R. Tal, Reassessment of finite difference equations in fluid mechanics, Letters in Heat and Mass Transfer, 9(1982), pp. 227-229.

8.

W. Kahan, Numerical linear algebra, Canad. Ma£h, Bull., 9(1966), pp. 757-801.

9.

G. Strang, Linear Algebra and Its Applications, Second Editi~8, Academic Press~ i980.

Users'

10. J. R. Rice, Matrix Computations and Mathematical Software, McGraw-Hill Book Company, New York, 1981.