Norms and inequalities for condition numbers, II

LINEAR

ALGEBRA

Norms

and Inequalities for Condition

ALBERT

W.

AND

ITS

167

APPLICATIONS

Numbers,

II* **

MARSHALL

Boeing Scientific Research Laboratories and Cambridge University, Cambridge, England

INGRAM

Stanford

OLKIN

University,

and Cambridge Communicated

Stanford,

University, by Alan

California

Cambridge, England

J. Hoffman

I.

INTRODUCTIOii

The matrices

condition

number

c$ of a matrix

is defined

for all nonsingular

A by c+(A) = $W1$(A-l),

where ordinarily condition

4 is a norm.

of a positive

to it a positive

multiple

According

definite

matrix

to a result of Riley

(1955) the

A can be improved

of the identity.

More precisely,

by adding

he showed that

cc&A+ 4 G c,(A), whenever

A is positive definite,

of the maximum The generally,

purpose

eigenvalue of this

the perturbing

matrix

k > 0, and $(A)

of AA*,

paper

both with regard

(1.1)

or $(A)

is either the square root

= (tr AA*)l”.

is to show that

(1.1) holds

to the norm 4 involved,

to be added to A. c&t

much

more

and with regard to

In particular,

we show that

+ B) < c,(A)

* Work supported in part by the Office of Naval Research. ** Dedicated to Professor -4. M. Ostrowski on his 75th birthday. Linear Algebra and Its Applications Copyright

0

1969

by American

Elsevier

2(1969),

Publishing

167-173

Company,

Inc.

168

A.

W.

MARSHALL

AND

I. OLKIN

whenever A and B are positive definite, c,(B) < c,(A), and 4 is monotone in the sense defined below. 2.

A CONDITION

NUMBER

INEQUALITY

Write B > A(B > A) to mean that B - A is positive definite (semidefinite) ; call 4 a monotone norm if 0 < A < B implies $(A) G+(B), and the usual norm axioms are satisfied [i.e., +(A) > 0 when A # 0, $@A)

=

laMA)

1. definite and if THEOREM

f or all complex a, $(A + B) < $(A) Let 4 be a monotone norm

+ 4(B)].

If A and B are positive

c+(A) G c,(B) 1

(2.1)

c&A + 4 < c&B).

(2.2)

then

Proof. 6 = 1-

8.

Let U = A/&A), V = B/$(B), 8 = $(A)/[+(A) Then b(U) = 4(V) = 1, and (2.1) becomes $(U-l)

< $(V-l),

4(B)],

and (2.3)

while (2.2) becomes #X7 By subadditivity

+ ev) $&(f3U+ &q-l)

< #J-1).

(2.4)

of 4,

+(eu + 0~) G e+(u) + B+(v =

1;

(25)

by convexity of the matrix inverse on the domain of positive definite matrices (easily proved by simultaneously diagonalizing U and V),

(eu + &q-l < From this, the monotonicity

eu-1

+ b-1.

of $, and (2.3),

+((eu + &q-l) < +(eu-1+ Bv-1) < e+(v) + B+(v-l)< +(v-1). By combining (2.5) and (2.6), (2.2) is obtained. Linear

Algebra

and Its Applications

2(1969),

167-1’72

/I

(2.6)

NORMS,

INEQUALITIES,

CONDITION

A slight modification but

this

in the definition

is equivalent

Theorem

to

(2.2)

1 is equivalent

169

NUMBERS

of 0 yields c+(aA + j3B) < c,(B) ;

because

c,(aB)

to the assertion

= c,(B)

for all a > 0.

that

s, = {A : A > 0, c,(A) < t> is convex point

for all t.

out

that

Because

c,((A

this suggests

diag(2, 3, . . . , 3) and +(A) = the eigenvalues Since

c+ itself might

+ B)/2) > +[c,(A)

+ c,(B)],

[41(AA*)]“2,

be convex,

when

A = I,

we B =

where n,(Z) > 1. . > A,,(Z) are

of 2.

the theorem

can be rewritten

in the form

c4(A+ B) < max[c@), c,(B)], it is natural

to ask if c&A + B) can be compared

Counterexamples tions

to show the impossibility

can be obtained

[jlr(AA*)]?

If

by noting

A + B = I, min[c+(A), 3.

then

B = I,

that

with min[c,(A),

of this without

c,(A)

3 cm(l) when,

min[c&A),

c,(B)]

c,(B)]

3 c&A + B).

required

of the norms

c,(B)].

further

condi-

e.g., +(A) =

< c4(A + B),

but

if

MONOTONE NORMS

The monotonicity property

of a large number

here that further

all unitarily

examples

invariant

LEMMA

Proof. invariant that AA*.

2.

matrices

a

We show

are in fact monotone,

and give

requirements,

If 4 is unitarily

According

to a result

for all A, +(A) = @(a), [A function

to the usual positivity,

U, V.

if and only if there

invariant,

then. $ is monotone.

of von Neumann

exists where

a symmetric

(1937),

q5 is unitarily

gauge function

@ such

ar2, . . . , as2 are the eigenvalues

of

@ on a complex vector space is called a symmetric gauge

function if Q’(U) > 0 when u # 0, @(au) = Ial@ D(U) + Q(v),

1 is in fact

norms.

invariant if, in addition

and subadditivity

for all unitary

norms

in Theorem

encountered

to show that other (but not all) norms can be monotone.

A norm q%is wzitarily homogeneity

of commonly

for complex a, @(u + v) <

and @(zli, . . ., u,J = @‘(EMUS,, . . ., &,pz ) whenever n

Linear

Algebra

and Its Applications

2(1969),

ej = f 167-172

1

A. W. IMARSHALL

170 and (ii, . . . , i,) is a permutation (1950)) ; it is also known l,...,n.

Combining

$(A) = W,(A)1 The converse norm $(A) implies

@ (see, e.g., Schatten

0 < A ,( I3 implies we obtain

<@p,(B),

0 < A,(A) < A,(B), that

. . .I ii,(q)

= $5(B).

1:

2 is false, as can be seen by considering

This norm is clearly

if A > 0, then

maxjui,l

not unitarily

= max uzc.

invariant;

Since

the on

0 < A < B

max u,~ < max bii, the norm is monotone.

Two 4(A)

hand,

gauge functions

results,

. . ., i,(A))

= maxjai,l.

the other

that these

of Lemma

I. OLKIN

of (1, . . . , n).] It is known that 0 < $6, < 211

implies &Q(U)< Q(V) for all symmetric i=

AND

other

commonly

= maxi Ci

encountered These

la,J.

norms

norms

are

4(A)

are not monotone

=

2

Iu,~~ and

since for

we have 0 < A < B, but +(A) > 4(B). We remark

that because of the connection

norms and symmetric for symmetric Z -1

=

@-I,

gauge functions. z2-1,

THEOREM

@(y)@(y_,),

gauge functions,

unitarily

invariant

1 yields a similar result

To state this result,

we use the notation

. . . , z,-1).

If @ is a synmetric

3.

between

Theorem

gauge

function

and @(x)@(x_,)

<

tJ2efi @(x + Y)@((X + Y)-1) < @(Y)@(Y-l).

A direct proof of this result can be given which is quite analogous proof of Theorem 4. FURTHER

CONDITION

The duality functions

NUMBER

between

provided

for any nonsingular

Algebra

and

INEQUALITIES

unitarily

invariant

norms and symmetric

the key to our proof (Marshall

matrix

a key to the following Linear

to the

1.

Its

A and unitarily

result

A+@ications

(Marshall,

invariant Olkin,

2(1969), 167-172

gauge

and Olkin,

1965) that

norm $.

It also was

and Proschan,

1967).

NORMS, INEQUALITIES, THEOREM 4. 1
171

CONDITION NUMBERS

Let $(x) =

~~uu,xc’,

. < v, (m < w).

where uu,2 0, i = 1, . . . , m, and

If A > 0 and if 4 is a unitarily

invariant

norm, then c,(A) An immediate

application

COROLLARY 5. then c,(A’)

G c&(A)).

of this theorem

If A is positive

is increasing

in Y 3

definite and 4 is unitarily

with Theorem

COROLLARY 6.

A is a positive

If

Izorm, ui>,O,

invariant,

1.

This can be combined

invariant

is

1 to yield

i=O,l,...,m

definite

further

matrix,

results.

#J is a unitarily

and l
then

c+,(A) < c&q,A + ztlAcl + . e. + u,AVm) < c&A”“). Proof.

The left inequality

be obtained Corollary c,(A”l). 1, it

from

Theorem

5, c+(A) < c&A”) Combining

follows

completes

is Theorem so that

that

A)-l)

Proof.

.i.

by Theorem

can From

1 c+(u,,A + ulAV1) < An

c&u,,A + ulAul + uSA”*) < c&A”~).

the proof.

COROLLARY7. -

The right inequality 5 using induction.

this with c,(A”l) < c,(A”P) and again applying Theorem induction

ii

If A and I -

invariant norm, the matrix A(I c,(A(I

4.

1 and Corollary

A are positive definite and $ is a unitarily -

A)-l

is more ill-conditioned

than A, i.e.,

> c,(A).

This follows directly from Theorem 4, with 9(x) = .x/(1 - x).

~

COMMENTSON AN APPLICATION Riley

(1955) has proposed

of linear equations

a procedure

when A is positive

for solving

definite

procedure

calls first for the solution

derivation

of x from y. A slight generalization

Let y be obtained Then

A = C A-l

a system Ax = d

but ill-conditioned.

His

of (A + k1)y = d, and then for the can be described as follows :

by solving Cy = d where C = A + B;

i.e., y = C-V.

B so that

= C-l

+ (BC-l)C Linear

+ (BC-1)2C Algebra

+ (BC-l)T

and Its Applications

+ *a *. 2(1969), 167-172

172

A. W. MARSHALL

AND

I. OLKIN

Thus, A-ld

x =

=

c-q +

(BC-l)C-ld

= y + BC-ly + (BC-yy One would compared

like to choose

+ (BC-l)Y-ld

+ *. .

+ ....

B so that

C = A + B is well conditioned

with A, while at the same time, the above series for x converges

rapidly.

We have not made a study of how this might be done.

suggests

B = kl where k depends

carried.

But in view of Theorem

may be worthy

upon

the number

of decimal

Riley places

1, a much wider class of matrices

B

of consideration.

REFERENCES A. W. Marshall and I. Olkin, Norms and inequalities J.

Math.

X(1965).

A. W. Marshall, other

I. Olkin, and F. Proschan,

applications

ed. by 0. Univ.,

of majorization,

Shisha, Academic

J. van Neumann, Rev.

for condition

numbers, Pacific

241-247.

Press, New York,

Some matrix-inequalities

1, pp. 286-300

Monotonicity

Inequalities,

of ratios of means and

Proceedings pp. 177-190,

and metrization

of a Symposium, 1967.

of matric-space,

(1937) (in Collected Works, Vol. IV, Pergamon

Tomsk Press,

1962). J. D. Riley, Solving systems of linear equations with a positive definite, symmetric, but possibly R.

Schatten,

ill-conditioned A

Theory

matrix,

Math.

of Cross-Spaces,

Tables Aids Comput.

Princeton

1950. Received A pail 9, 1968

Linear

Algebra

and Its AppZications

2(1969),

167-172

University

9(1955), Press,

96-101.

Princeton,