Optimal regularization of operator equations

OPTIMAL REGULARIZATION

OF OPERATOR

~Q~ATlONS*

V. A.MOROZOV Moscow (Received

12 Nouembcr

1. Statement

1969)

of the Problem

LET A be a linear completely continuous operator acting from II into F, where H and F are Hilbert spaces. We consider the problem

where f is an element of F. Problem (1) is assumed to be solvable. A solution of (1) with a minimum norm will be called a pseudo-solution of the problem 111.

LetjT{be a given class of positive definite self-adjoint such that the quadratic form

uf

operators T in H

where k = k(A, Tf > 0 is a constant independent of the choice of u G D,. We define the functional

where g E F is any element of I;. It is easily seen that the element gT E D,, mininliziIlg @,,lu, gl, satisfies the equation (T + A*A) u = -4*g,

* Zh.

u.ychisl.

jMat. mat.

Fiz.,

10, 4, 818-829,

1970.

(3)

Optimal regularization

11

of operator equations

where A* is the adjoint operator to A. In view of (8, Eq. (3) has the unique solution UT

=

RTg,

RT z (2'+A'A)-iA'.

Let the element f be specified approximately: y= f + E, where 5 is a ‘small’ (in the sense defined below) random process with values in F, for which,Jhe con$ition M[ = 0 is satisfied, where M is the expectation operator. Let uT = R,f; we put

Let lufl and I& be classes tions respectively;

of “permissible”

solutions of (1) and perturba-

The problems to be considered below are: Problem I. To construct the operator Topt E 17’1,defined by the condition

If a solution of this problem exists, the element uopt = R,

f is said to t,e a

(IT), jUf I, 15 D-optimal (approximate) solution of problem (19”.”The quantity Opt ! {uf>, (8 ) = sT,pt ( @f) 7 (8) (!TII), lUfl, 1&optimal solution.

is called in this case the error of the

When {T} = (T: T=aE}, a > 0 is a parameter, problem (4) amounts to the determination of the optimal value aopt of the regularization parameter

El. Problem II. To estimate the error of an optimal solution of problem (1) in the sense of (4). Problem III. To find effective means for realizing the optimal approximation (construction of quasi-optimal approximations). Note.

Problem I was discussed

earlier, in

b-~~l,as applied to particular

V. A. Morozov

12

in L61with reference to linear

cases of problem (1). It was investigated algebraic equations. 2. Auxiliary

Propositions

Let ieil, i = 1, 2, . . . , be an orthonormalized set of eigenelements of the operator A*A, corresponding to eigenvalues hi, A”Aei = h,ei, i = 1, 2,. . . , By Hilbert’s theorem [7l, any element u E H can be hi+l < hi, hi > 0. written as OD A”Au’ = 0, uN = Ui = (U, ei) H. U = u’ + u”, u‘ 1. UN, Uiei, c i-1

(5) Notice that the condition A*Au t 0 is equivalent to Au = 0. Putting

we note that

oi = Aei / l/h,

AA*oi = oiY,hi,

(Oi,

oj)

F =

t&j,

where aij is the Kronecker delta. Lemma 1 Any element g E F can be written as g = g’ + g”,

A’g’ = 0,

g’ 1. g”,

gi=(g,

@i)F-

i=l (6)

From (5) and (6). Lemma 2 If “I is a pseudo-solution

of (l),

a3

Uf =t

c

iiif?i,

fi=(f,

mi)F.

i-l

We shall assume henceforth that {T} = {T: Tei = aiei, i = 1, 2, . . .>-

(7)

Optimal

regularization

of operator

13

equations

Lemma 3 We have

(8)

Notice We shall

that

ai + A; 3

assume

k” >

0 by virtue

of’ (2).

below that

where a’ is a constant.

Hence

5 may be a “generalized’

random process.

Lemma4 Whatever the T E

1Ti and

ZLE E

{j},-

(9) 3.

Construction

(I T I, uf, &Optimal we get

of the Optimal

approximations.

Approximations

On minimizing

(9) with respect

to ail

when ai =

aFPt= XifJi2/fi2,

i=

Hence the (1T\, uI, &?-optimal approximation 2

8x7

uopt=

IX i=l

*j2’l;

1,2,.

.. .

has the form

fi fi2 ) ljhi

ei*

(11)

V. A. Morozov

14

Tfaeorem 1

Let

Thf?n lim &t CT-+0

(uf, {E);)

=

0.

The theorem is proved. ({Z’), u,, {E) ; ) -Optimal approximations.

It may easily be seen that

O” Ai2S2+ ai2fi2 ET’ c”i {&j)

= c

ai tai + A,)?

'

i=l

minimizing

&Z(U~, (El,- )

with respect to aj, we

&t(ujdE>;;)

=

g

get

h, (;:$i’).

when ai =

The ((%

Q,

opt =---, ?A2 ai

fi”

i=f,2,....

{El : ) -o pt imal approximation is then given by

142’)

15

Opt imal regularizat ion of operator equations

It is easily seen that eloD,(uf, {El;) = ctpt (ala, (%>,> lim ({T},

{uj}(., {E},- )

~i~t(uf,(E);)

-0kikal

c4c ={

Uf

=

0.

approximations. H: iii2 < Ci2,

E

Let

c i=l

The

so that

f (T}) (uf} =, f@ 2 ) -optimal ~proximations

Ci2 <

00

> -

are then given by

while

((IF}, (u~}~, {zf; ) -Optimal approximations. L& the system kz,l be (Uj}_v= {Uf EE $1: Ui* < C2, i = finite-dimensional UVdimensions), N), We have =1,2,.... Theorem 2

The ({T),

{u~x,(51:)

-o pt imal solution of problem (1) is a solution

of the equation (UE + A”A) a = A”f where n = a opt = 0’ I’ C”, Here,

The proofs of these propositions 3,

Error Estimates

are fairly simple and may be omitted.

for the Optimal Approximations

V. A. Morozov

16

where the class ([I E {El. We similarly define

Let {u,~}={u~}~P

=

{ uf EH:

L (+)2p’(2x+‘)](2x+1)‘p < R:,,}

[

1

i=l

where

where

k,2 = (2x)2x/(2%+1) / (1 f 2x).

Proof.

where

are constants,

>O,zJZl,f3bl

In accordance

and l/p + l/q = 1. Then,

The bound (13) cannot be improved.

with (9),

gi’ = Ui2/ J*i’X. Let

cp@) =

IJ*g*a** 02 +

g2)&2x+1’

a2

0,

02, g* # 0.

Then, it is easily shown that max

cp (A)

= cp( Amax) = CX*(O*)2xA2x+1) (g’) “(2x+1) ,

h>U

where

hmax =

(2x0* / gz) i@xfl).

Applying Ht)lder’s inequality,

Consequently,

we get

Optimal rcgularization

of operator

17

equalions

This gives us (13). Let uf = uie;, where

E<= aixt?z,n, g = zimi,

2

ox,<,= (1/2x)r.:X+‘.R:t2,x”? where

i.e. the bound (13) cannot be improved. Notes. 1. Let p = 1, q= m. Then,

2. Let

p=Zx.+1,4=

(2~+1)/21c;

Let us give some sufficient to the class I ufj Lemma

and c!J~is such that MT;i2=

It is easily shown that E, E {u~}~~,5 E {%}0’1,

The theorem is proved. &IO” z= (51;

So that

weput

criteria for the pseudo-solution

of (1) to belong

5

If uf = A*& h cz F and 11h /IF $ R, then uf E {z+}~ when x = %. If ~1.~ = (.4 *A)xw, w E EI, )I w llrT < Rx, then zzf E {uJx whatever the

3c>0. The conditions of the lemma are satisfied e.g., if the operator A is normally solvable fll. To prove it, we have to utilize the expansions (5) and (6) for the elements w and h respectively. Note 3. Let fu&” = (Ur : =f = (A*A)Rw, w E ff, llwilw < Rx}. It is easily seen that the class +I~)~* is compact in W whatever the x> 0. In accordance with [Sl, we define a quasi-solution u X of problem (1) as a solution of the problem IInfix - fllr2, ILE (uj),’ - min

V.

18

A. Morozov

(we assume that [ is not random). Putting

where {&}(J* = ($= F :

IiktiF

<

01,

and using the fundamentalresult of [?‘I, we

get

where

(Ix2 =

~WX+*).

Comparing(15) and (161, we see that the orders of the estimates are identical, though the constant in (15) is less than that in (16), at least for x=‘k!and X=1. 5. On the Realization of Optimal Approximations Realization of the ( {T} , uf, {E} -,) -optimal approximations(12). We define the element (17)

which we shall call a ( {T} , uf, {,g} -,) quasi-optimal approximation. Notice that, to realize (17), we only need to know u, i.e. the accuracy to which the element f is specified. We shall now state an assumption which we shall call the linearization principle. Let

We shall assume that a “reasonable” accuracy is then achieved if we confine

ourselves to the linear terms in the expansion P&)

= F(0) + F’(O)& +.

..

Optimal regularization

of operator equations

19

Theorem 4

Let the linearization

principle be satisfied.

MIl&,tProof.

Then,

~rllH2~~&&?

GIZ).

We have

w =&

-

L

a2fi

82 +

fi”

+

3fW

+ fi4 E

1 i=l,2,....

(a”+ fi”)” ’ ’

The theorem is proved Corollary

If Uf E {7.&‘, and we recall (14). we can also obtain an ‘estimate” the accuracy of the quasi+ptimal approximations:

The method of regularization

[21. Let

zzfE {u?}~, E E {t},,

= {l’;} = (7’ : Te, = ah-2xei}, where a > 0 is a parameter.

where

gi’ = ci2hi *’

as above.

Put

k4X-tI

2.2”

v(h)=

We have

where

h

$(A)= (a

+

h2x+‘)2

ind

Then

’

(a

+

I?“+‘)2

’

>

0.

of

{T} =

V. A.

c ,= (4%+ 1)(4~+ww+1) x (4x+2)2

Morozov

C,ff=(;zl

)2(~)2w2n+',

'

while hZax

=

(-5)

w%+l)a1/(2~+l).

Then, -1'(2x+l!_j_ ('x"&2(L2X'(2xtl) =

EaTax(Uf, E) \( L’x’h

&,2 (a) (19)

But

where Cx”’ = Cx’

( -

CX’

2xC,”

-1/(*x+1)

>

w(*x+1)

+CXN (&7)

7

while

ax=

C%’ -

2xcrr,

o*Rx-*.

(20)

Hence itff&“Ta,

({u~>~,

(E}o)<

~x*204xl(2ri1)R2xl((2X+1).

G-3

This bound cannot be improved as regards the orders of (I and Rx. . To see this, we only need to take a =

Et = liiei, iii =

02Rxs2 and evaluate We thus have

AtaRx2, f, = &+wi, M&2 = fl and

E;~” (if, 8.

Optimal regularization

Theorem

5

The errorofa ( {Tax}, {q}%, {Eja) than the right-hand regards

21

of operator- equations

side of the estimate

-0 ptimal approximation

(21), which cannot

the orders of D and Rx . To obtain

quasi-optimal

is not greater

be improved

as

approximations,

we

only need to take a = ax as given by (201 in the ( T,x I_regulmizing algorithm. The accuracy of these approximations is the same, as regards the orders of

, as

uandRx Note.

the accuracy

Evaluation

may be rendered operator cz

of

difficult

A*A.

of the optimal

({r=x}, {u~}~, {uO) quasi-optimal by the need to evaluate

Let the operator

ker A’A. Then

approximations.

G be such that

!T z l-regularization

amounts

(aG + A*A)u

approximations

the spectral

structure

of the

GU’ = 0,

U’

Ge, = hg-zxe,,

to solving

E

the equation

= A*f,

which can be done quite efficiently. Let us consider It is easily

We shall

further the case

of IT(xj-regularization,

where T, = aE.

seen that

assume

that

uf cz (uf}%, E GZ {g}O. Then,

i)=g(aa;;i)2+ a2z

hi2’gi2

&Ta2@fr where

gi2 =

pi? / &ZX

(a

+

Ai)2

as before.

Lemma 6 Let

(P(h) =

)*/ (a $ 3,)‘,

$(h)

=

hex,/ (c~-j-h)~,

h >O,

where

&XIX =

a,

c:“= (+$I

-

X)2,

Then,

V. A.

22

co

c i=l

iiai2

(a+hi12

b/(1 -41a,

O
+m,

x3

1

AN max= Using this lemma,

Morozov

1.

we get

(

u2

02

.x7

w hi2%gi2 ( Ii i=, (a+hi)2 ’

These bounds cannot be improved.

u~~C:“R~~,O
la2R12,

x2

1.

For putting E = EiWir M&2 = a2

and a = Ai, we get 00

Ali& ci=, (a+W2 Similarly, when 0 < x < 1, on putting

[ (1 - x) /xl&,

=-, a2

4c.z uf = uiei,

u? = ?@R,?,

a=

we get a2

It remains to note that, when x 3 1, 00

Iim a+0

xi21

?!3i2%gi2 (a +

hi)2=R12*

Using the bounds obtained above, we get

The right-hand side of this bound is a minimum when

(22) Using this fact, we get

[(2x + 1)/(8x)?K'(2X+l)]a4X!(2Xtl)(C,“‘Rx2)1’(2x~1), 0 < x
& a0pt ({"f)x~ 6%)

6

I 3

$64

~"sR'~'~,

x>l.

(23)

Optimal regutarizat ion of operator equations

23

Theorem 6 Let;ln

be the solution of the equation (czE+A*R)u=A’~

and let the parameter a be given by (22). The accuracy of the approximation is then given by the right-hand side of (23). Note. It is easily seen that, when 0 < x 6 1, the orders of the estimates (21) and (23) are the same, while when x b 1, (23) is at any rate of lower order with respect to (I, though the corresponding constant is independent of X, which makes (23) much easier to use. In addition, the ~T~~-a~~orit~ is simpler to realize than the (2’; balgorithm. 6.

Example

The above results are applicable to a wide class of problems. consider the solution of the evolutionary equation dU

-=Lu, at

O
U(O)=f,

As an example,

(24)

where L is a positive definite self-adjoint operator with discrete non-negative spectrum, acting from W into H. Problem (24) is assumed to be solvable. Put A, = e -Lt.

Then (24) amounts to solving the parametric operator equation

Let iei) be the complete system of eigenfunctions Lej =

Ait?j,

i=l,2,..

,

of the operator L: limAi=foo. i-+03

Then,

where fi =- ff, eif H. The

t,{ Tj, uf, i)

-optimal solution of problem (1’ ) is

V. A. Morozov

24

nopt(q=E ji2 i=, oi2 +

From this we can easily find the optimal, solution. Put x = (2’ - t)/2t. u,(t)=

( (I’},

C?Xp(Ait)fiei.

fi2

uf, (g>,-)

-optimal, and then the quasi-

Then, obviously, 2Ai~t+niT]fiei=(At’At)XLL,(T)=A*2XUJ(T),

Eexp[-i=l

i.e. the conditions of Lemma 4 are satisfied. The {Ti I-regularized approximations &(t)=pexp(A&)

(1+

are given by CteXp(2hiT))-‘f&.

i=l

Noting that bound

{u,(t)),

=

{u, : Iluj(T)

11112 < RxZ,

and using (21), we get the

We showed above that the order of this estimate cannot be improved. Notice in conclusion that our analysis is also valid when the element f is assumed to be random; we only have to demand that M6$‘; = 0, and replace fzi throughout by Mf’i. When A - E, problem I is the same as the optimal filtering problem for the random process f. Translated

by D. E. Brown

REFERENCES 1.

MOROZOV,

V. A.

On pseudo-solutions.

Zh. uychisl.

Mat. mat. Fiz.

9, 5, 1388-1391,

1969. 2.

TIKHONOV, A. N. On regularization Nauk SSSH. 153. 1, 49-52, 1963.

3.

LAVRENT’EV.

M. M. and VASIL’EV,

posed problem6 of mathematical 1966.

of incorrectly

V. G. physics,

posed problems, Dokl. Akad.

On the statement of some incorrectly Sibirskii

matem. zh. 7, 3, 559376,

Optimal

regularization

of operator

4.

ARSENIN, V. YA., and IVANOV, V. V. Nauk SSSR. 182, 1, 9-12, 1968.

5.

ARSENIN,

V. YA.

coefficients, 6.

MOROZOV, tochnosti

SSSR.

On optimal approximate

in: r2ccurcms and Efficiency i effektivnosti

vychisl.

25

On optimal regularization,

summation of Fourier

Dokl. k\!zacf.Nauk.

V. A.

equations,

On optimal

equations

series

182, 2, 257-260, solutions

Akad.

with approximate 1968.

of sets of linear algebraic

of Computational

algoritmov

Dokl.

Algorithms

(tr. simpoziuma)),

(Vopr.

1, Kiev,

88-59.

1969. 7. 8.

IVANOV, IVANOV,

V. K.

On incorrectly

V. K. and KOROLYUK,

incorrectly

posed problems,

posed problems.

Cl&em.

sb. 61, 2, 211-223,

T. 1. On error estimates Zh. u$hisl.

,Mat. mat.

Fiz.

for the solutions 9, 1, 30-41,

1963. of

1969.