A general method of constructing regularizing algorithms for a linear ill-posed equation in Hilbert space

A GENERAL METHOD OF CONSTRUCTING REGULARIZING ALGORITHMS FOR A LINEAR ILL-POSED EQUATION IN HILRERT SPACE* A.

B.

BAKUSHINSKII M0 s c 0 w

(Received

THIS paper deals tion of a linear

28 .4pr il

1966)

with the problems of constructing the approximate ill-posed equation in Hilbert space.

Let H1 and 112 be complete Hilbert spaces, plicitness we shall assume to be real. Consider

an equation

solu-

which for the sake of ex-

of the form Az=f,

ZEHi,

i E Hz,

(1)

where A is a linear, bounded operator, operating from Hl and Hz. If the operator A has no bounded inverse (and this will be assumed in what follows) ) the problem of finding x E ‘I1 for which equation (1) is satisfied, is ill-posed in the classical sense (up to Section 3 of this paper by this we shall mean instability with respect to small perturbations of f in the norm of “2). This gives rise to theoretical difficult ies , for example, when we want to obtain a numerical solution. As shown in [I - 31, under certain conditions so-called regularizing algorithms (abbreviated to RA) exist which make it possible to obtain a numerical solution of (1). For this reason it is of interest to obtain and study the widest possible class of RA’s. First we adopt the following general. a priori, assumptions: (1) equation (1) has a solution (possibly not unique) for a given right-hand side f E H2; (2) problems which are correct in the classical sense (in particular, the calculation of the values of the bounded operator) can in principle be solved quite accurately. *

Zh.

vychisl.

Mat.

mat.

Fiz.

7,

3, 672 - 6’77, 1967. 279

280

A.B.

Bakushinskii

Following [21, we shall call an algorithm a “regularizing algorithm” of problem (1) if for any E > 0 one can find such 6 = E(E. A, f) > 0 !” that for each f, satisfying the condition lij-F[I,, < 6, one can,find a unique x such that (l), corresponding

Ils-41 H, < s,. where x is a certain to the right-hand side f.

Now we shall turn directly structing an RA. We shall

consider

to the description

two separate

fixed

solution

of the method for

of

con-

cases.

Case 1. HI = H2 and the operator A is positive selfadjoint. Let “A be its spectral family. In order that (1) should have a solution it is necessary and sufficient that the following condition should be satisfied

When (2)

is satisfied

equation

(1) has a solution

-’

s

5=

written

in the form

dEnf

-.

0

h

An RA of equation (1) can be constructed using the following scheme. suppose we are given a real function of two variables q(A, a) defined S(A) (the spectrum of A) and a > 0, everywhere bounded and when A measurable with respect to {$,I, and such that snup IN% 4 &ES(A)

I=

K,<

00,

(4)

h

cp(O,a)

=

0,

a

#

(5)

0,

(6)

lim cp(k,a) = 1, a-+0 =.%A) A#0

while in (6) the convergence is uniform for all h where c is a very small positive number. We put

-,

a#

upon Q(A, a),

S(A),

0;

I~I
E (c, 00) n

there

h=O. exists

a bounded operator

Constructing

function

Y(A,

a)

regularizing

[41).

(see

It

appears

281

algorithms

that

the

algorithm

for

finding

& = Y(A, a)f (for

any a > 0) The proof

of

it

is

actually

is

a bounded Let

is

an RA of

this

is

carried

problem

obtained out

operator

the

the

and 11Y(A, Consider

II f - ? II = 6.

the

special

same scheme

as in

function

Indeed,

9.

[31,

where

y(i.,

a)

a) II = K,. the

< IIZ~-

(1).

using

for

(7)

inequality

211< llz -

zorll+ llz, -

zcc!l < 11~- 41 + L.6,

(8)

to formula (3), %a according to formula x is determined_ ,according x - xa in the form of an integral and j;,. = P(A, a)f. We represent with respect to El. Dividing the integral into two parts, in the same (2) and (6), way as it has been in [31 for the sum, and using conditions

where (7),

we can easily If

show that

lim K,6 = 0, a.+

we can choose that

lim gas

obviously

such

that

lim IIs,4

a a > 0 (“adjust

zll < ‘e. From the

IIt-

in order

(I x - xg( II = O.

lim ll& icrd

sll I the

left-hand

511= 0,

it

is

0. Consequent lo, algorithm”)

side

of

(8)

necessary

for

and such it

also

any E > 0 a 6 > 0,

follows

and sufficient

that

that

= O.

a-r 0. Case

2.

equation

We now turn

In view equation

to the

construction

of

of

our

the

where A* is The operator therefore We take the

an RA for

the

general

a priori

assumptions

(1)

is

equivalent

to

an

= A’f,

(9)

an operator, the conjugate of A, operating from H2 into HI. A*A is a positive selfadjoint operator from H1 into H2,

an RA for the

(1)

function

1 (replacing weaker

equation

form A”Ax

Case

of

( 1).

condition

the

can be constructed g~(h, a),

set

S(A)

satisfying by S(A*A).

as in Case the

1.

same conditions

Condition

(4)

is

as in

replaced

by

282

A.B.

3akuski~ski~

(10) we form the function

I&I
h=O.

The operator Y(A*A, a) will now be selfadjoint, but possibly not bounded in Hl. However the operator B = Y(A*A, a)A* will now be bounded and the algorithm of finding G = Y(A*A, Is

an RA of

problem

n)A’f

(iif

(1).

Let us consider the proof of this statement. ?irst of all, using the operational calculus of Riesz [41, it can be shown that the operator f3 is bounded,

and 11B11 = &

Furthermore, one has to use (8), where x is formed according to formula (3). replacing f by A*f, and K, by the quantity K,. The proof of the convergence of z and xo is the same as in Case 1, using (2) and &u&6 = 0 is neces(6), and replacing f by A’f in (2). The condition a-o sary and sufficient (for any perturbation of f) in order that lim II& - sll = 0. a--r0 The class

of RA’s described

We shall give general scheme.

some concrete

here

is fairly

examples of RA’s

wide. obtained

from the

Such a yl(h, a) satisfies all the Example 1. Let yOi, af = fh + al-l. necessary conditions. The RA obtained is widely known. It has been studied in [l - 31, and [51. It is natural to call it Tlkhonov' s RA, For this

RA &=l!U,

%
proved. The concrete form of this the spaces HI and H2. Exanple

2. Let

The last

Inequality

cannot be im-

2A depends on the specification

of

Constructing

In the

case

when the

consists

(1) or (9) selfadjoint

which correspond operator.

Example

A

to

a,

O
(

the

283

fully

is

continuous,

components

small

from the

eigenvalues

of

such exact

the

a regular-

solution

of

corresponding

Let

3.

, 1 -(I

-illrjL)‘l’”

a#

-’

a *(La)=

o

{

operator

in removing

a>

l/h,

cl(ha)=

ization

algorithms

regularizing

+

0,

=z2/llAll

0< p

P

a=

-1

(0 < p G a/ll~*-JIl).

0,

a

In this If

case

l/a

K,=

~Ia,~~=O(il~~)).

= n (an integer),

ing out the and positive)

iterations

such

with

$0 = f, the role of the parameter number of iterations. In the

general

The process obtained

in

choice of examples. 2. our

(12) [61.

the

In this

scheme

case

for

Algorithms

li+i

the

(12)

(7)

and (ll),

@)zi

appear

to

+

of

are

demonstrate of

be two important

symmetrical

(la)

a large

where

by A’A, of

the

iterations

and

n is

f

I kind

depends

number of

some other

regularization

can obviously

by carry-

A is

number l/n,

equations

such

(if

Pf;

must be replaced

cally) to obtain the approximat e solution function u, and a number a with respect to the approximate solution of (1). There

be realized

formulae

by the

form of

we shall

construction

the

(Em

Ml and 112. There

section the

=

solution

The concrete

spaces

to

a is taken

A in

for

an RA can easily

respect

by

the

A’f.

was first

on the

similar

possibilities

of

algorithms.

be-used

(at

of (1). formula

Namely, selecting some (7) or (11). we obtain

characteristics

of

least

such

theoreti-

a method

of

284

A.B.

approximate

Bakushinskii

solution:

(1) the number Aa (or 2x) which indicates how the possible errors on the right-hand side of f influence the approximate solution xo; from (a) it can be seen that this number plays the role of an nampllfication factor” of the errors in f; we shall call it the “correctness index” of the given method of approximate solution: (2) If we substitute into the right-hand side of (7) or (11) the accurate solution of f, then obviously we shall not obtain an accurate solution of (1); the quantity 11n - x, 11 is another characteristic of the chosen method of solution.

If we now fix the correctness index A, we can examine the different methods (formulae) for the approximate solution of (1). obtained from (7) and (11) with different cp(h, a), while a is fixed, so that Ko = K(/ta

= K).

of functions

All these formulae are obviously determined by a certain &I(A)) and the values of ty(A, a) when h = 0.

set

We formulate the following extremum problem: among all formulae for the approximate solution of (l), of the form (7) and (11). having a given correctness coefficient H”, find the one for which the quantity a minimum. where x is determined by a formula of the form (3). while ~(9) is the result of applying (7) or (11) to the accurate right-hand side of f (obviously a is chosen so that the correctness coefficient should be equal to A).

II x - x(9)II is

From (8) it totically best

is obvious that such an extremum formula has the asymp(when 6 -, *O) estimate of the quantity II x - ~(9) II among

all

formulae having the correctness coefficient not influenced by the value of ~(0, shall not make reference to it.

x((p)II Is

The extremum problem is solved In the case when A is a positive,

In the case when A is arbitrary,

by the following selfadjoint

II x -

K. The value of a). For this reason we

functfons operator,

P(h).

Constructing

regularizing

algorithms

285

The proof of this follows immediately from the definition of K and the representation of the difference z - x(q) In the form of an integral with respect to El. The selected extremum problem is obviously not the It simply illustrates those new problems which appear only one possible. in connection with the introduction of the sufficiently wide family of RA’s.

It is possible also to formulate the problem of hand side of (8) with respect to a with a fixed 6 ization algorithm (searching for a nquasi-optimaln case it 3s important to know the behavlour of II X of a. Suppose it

minimizing the rightand a fixed regulard. see [?I). In this - %a II as a function

is known that

0 G jq(h, a) - ii

=G E(a)rl(h),

h # 0,

(h E S(A”A)),

A.ES(A)

where C(Z) + 0 when a + O. In this case if the equation q(A)Ax = f (or q(A*A)A*Ax = A'f) is solvable, we have /x-z~\] =0(5(a)). This easily follows from the representation of x - xa in the form of an integral Thus, in the case of Tikhonov’s RA one can put 0 < E < 1. One obtains a corresponding statement vergence of order aE of Tikhonov’s RA.

with respect concerning

to EA.

the con-

3. For a numerical regularization of the algorithms (‘7) and (11) additional difficulties arise due to the fact that we cannot operate with an accurate function y, and operators d and A’. The errors which appear are determined basically by the selected algorithm and must be here carefully studied in each actual case. We shall confine ourselves of the to the following general statement, concerning the stability algorithm (ll), for a fixed a, with respect to the perturbation of the operator A. Let the function q(A. a) defined everywhere over the interval [O, 4: containing S(A*A), be continuous over this interval, and

[O.dl-t

,‘h

Let the perturbed operator i be such that perturbation is “decreased” so that

S(A”*i)

II (A’ - P)Asll we

have

E

[O,

--t 0,

4.

If now the

-4.5. Bakushinskii

266

This immediately

For particular made (see for

follows

from the inequality

algorithms example

obviously

some more precise

statements

can be

Cd').

4. Finally we remark that for normslly of the kind (11, i.e. those for which the ator A fs a subsPace, the algorithm will to f (6 and I can be chosen independently

solvable ill-Posed equations range of values of the operbe equal to an RA with respect of f within any sphere

II f If < c). This follows from the fact that ators the zero will be an fsolated statement if also true: if for the RA (with respect to f), the problem from the definition of an RA using in Hilbert space.

with corresponding symmetrical operpoint of the spectrum. The converse problem (1) there exists a uniform is normal& solvable. This is proved the weak compactness of a bounded set

Translated

1,

2.

3.

T~K~~N~V, A.N. On the solution of ill-posed of regalarizatlon. Dokl. Akad, Nauk SSM TIKHONOV, A.N. on the regularisation Akad. #auk SSSfi 153, 1, 49 - 52,

of

problems 151,

3,

ill-posed

by 0.

Kiss

and the method

501

-

504,

Problems.

1963.

D&t.

1963.

RAK~6~~~~K~~. A.R. On a method of solvins the Fredholm integral equations of the first kind. Zh. v?chisl. idajrat.mat. Fiz. 5, 4, 144

-

748,

1965.

and NAGY, B.S. translation), 1954.

4.

RIESZ.

5.

~~ASL~V,

F.

V.P,

~e~tar~at~oa

vozmushchenii 1965.

Lectures

theory

i asimptoticheskie

on functional

and

asymptotic

metody),

analssis

l)t~!thads

Mask.

00s.

(in

RUSSian

(Teoriya Univ.

?~~OSCOW,

Constructing

regularizing

287

algorithms

6.

FRIDMAN, V.M. The method of successive Fredholm integral equations of the 11, 1. 233 - 234, 1956.

approximations first kind. usp.

for the mat. Auk

7.

TIKHONOV, A.N. and GLASRO, V.B. On the approximate solution gral equations of the first kind. Zk. vj%kisl. Mat. mat. 3, 564 - 5’71. 1964.

of Fit.

inte4,