Algorithms of optimal accuracy for the approximate solution of operator equations of the first kind

ALGORITHMS OF OPTIMAL ACCURACY FOR THE APPROXIMATE SOLUTION OF OPERATOR EQUATIONS OF THE FIRST KIND* V. V. IVANOV Kiev

(Submitted 4 June 1973; revised 5 November 1973)

ASYMPTOTICALLY optimal algorithms are constructed for solving a wide class of problems, notably, operator equations of the first kind of the type K~+Ff (not necessarily linear), under the assumption that the right side f and the operator K are specified with a known absolute error. By an operator equation of the first kind we mean an equation of the type

where f is a given element of abstract metric space F, @ is the required element of abstract metric space a’, and K is a continuous operator from Q, into F. In general, the problem of solving this equation is incorrectly posed in the classical sense [l-3], so that the ordinary statement of the problem of constructing an optimal algorithm for its solution needs to be suitably modified [4, 51. An analog of the modified statement is to be found in [6-81 as applied to problems of function minimization, or in [9, lo] as applied to the problem of solving Eq. (1). 1. Let us give a generalized statement of the problem. We consider the class 9 (Y,Se) of problems P(1, R) with initial data IE Y and solutions R&Z, where Y and 5I! are subsets of the respective metric spaces X and Y with metrics px and QY . We can write

R=OI, where 0 is the operator of transformation of I into R with domain of definition Y and domain of values .5?.Assume that, instead of 1, we are given the operator J/ and element 7 = $Z. Denote by D the domain of indeterminacy of the initial data IEY:

D-D($,Y,T)={Zl$(I)=T}nY

(2)

and by d the diameter of the domain D;

While it may seem controversial to define D in the form (2), it turns out below that D ,may be-defined in the usual way in every particular case. Let A be an algorithm for transforming IintoR:

*Zh. vjkhisl.

USSRII

A

tit.

mat. Fiz., 15, 1, 3-11, 1975. 1

2

K

v. Ivanov

R=AT=Y.

(3)

Here again some refinement is needed, since the argument and result of the computational algorithm A are assumed to be finite sets of numbers with a finite number of places. Let these number sets be I=cpi-’ r and R=cpz-‘R, where cp~and rpz are one-to-one mappings, which it seems natural to call interpreters of fand3 respectively. We then have, instead of (3),

R=cp2&,-‘r.

(4)

For brevity, however, we shall not distinguish between (3) and (4). We put

where M and Z are given sets of algorithms A and operators J/ respectively. An algorithm A for which I: (r, A, 9, $) =v (r, M, 9, $) , will be called optimal in accuracy in the class of algorithms Mand of problems 9 for a given method of representing the initial data 9 and ?.If u (r, M, 9, J,)lv(K A, 9, $)-+I, as d ($9, r) ‘0, the algorithm A will be called asymptotically optimal in the class of algorithms M and problems 9. The condition v (r, A, 9, I#)/v (r, M, 8, $) as d (9, 9, 7) -+O will imply that A is orderwise optimal. If v (r, A, .9’, 4) -u (r, M, 9, 4) ss q, we say that A is optimal with accuracy up to n > 0. The concepts of optimal operators J/ may be similarly defined. It should be mentioned-that the set M may, in particular, be the same as the set of all possible algorithms for transforming I. In this case we shall omit M in the relevant expressions and notation. Another point is that, if ?is interpreted as a random quantity, there will be a certain probability p=p (9, 9, T) corresponding to the domain (2). We have the same probability corresponding to the other relations involving the quantities introduced. Notice finally that many other criteria could be considered instead of the minimax optimality criterion [l l] ; but we shall not dwell on this. 2. We shall now give an extremely general theorem relevant to a wide class of problems. Later, we apply it to problems of the type (1). Theorem I Let the set E = OD be bounded in Y. Assume that Y is a seperable space and that algorithms can be indicated for constructing, with any assigned absolute error 6 > 0, all the elements of fixed finite segments {Ui},m and {Vi}in of countable sequences {ui} iQ and {vi} i -, everywhere dense in Y and E respectively, and also an algorithm for evaluating the distance PY(yt, $3) for any fixed pair yi, yz=Y, with any previously assigned absolute error u > 0. We then have

(6) ProoJ: By the definition (S), the left-hand side of (6) is obviously not less than its right-hand side. To prove the reverse inequality, it is sufficient to show that the problem of finding inf g(y) VEY is constructively solvable, i.e., that at least one algorithm can be indicated for approximately solving this problem with any assigned absolute error 7> 0.

3

Solution of operator equations of the first kind

By hypothesis, an algorithm A ,,,, n, a, Q can be indicated for evaluating the element y,,, ,?.6. “, for which

where py, u denotes the approximate value of py with an absolute error not exceeding cr. For this algorithm,

+

5

+

PY (%

SUP

PY (Ci,S? Ym,n,G,o) --

l
inf

+l$f_

sup

lz&pm

inf

l
.!/m,n,s,cl) -

PY (Vi,&

l,(?C?ll

1
+

SUP

sup

l,
PY,~ (vi,b PY (w,

SUP PY,o (vi.69 Ym,n,G,a) Idi,
k,~) h,d

Ym,n,G,o)

-

sup pY(vj, u,) -

inf sup l
I;:: ;z;

l
PY (q6,

f.h,b)

PY (vi, us>

PY(R Y)-

0,

as 6, (I + 0 and m, n +=. The theorem is proved. The algorithm mentioned in the proof of Theorem 1 is in essence an algorithm of simple inspection and is not practically efficient for a large number of concrete problems. A number of extremely efficient algorithms for solving minimax problems may be found in [12-151. The development of optimal algorithms for solving problem (6) similar to those in [ 161, is of great interest. In principle, all these algorithms provide for the construction of asymptotically optimal algorithms (in the context of inaccurate initial data) for solving problems of an extremely wide class, provided that at any rate one algorithm is available for solving the problems (when the initial data are exact). 3. Let us now consider some particular cases of problem (6), in the context of solving equations of the type (1). Let K be a completely continuous linear operator, IKll < 1, acting from F into F, while instead of fwe are given fe=$fe’F,

Di:IV-fell+

(7)

where II I( denotes the norm in Hilbert space, and the domain r,cl; elements of the type (see [3]) f=KLu,

is specified as a set of

Ilull~l,

where L is a completely continuous linear operator, (ILII< 1. We assume that there exists, for every f of the type (8), a unique solution of hq. (1)

whilea continuous function oi (~1, Ok

-4

~-4

can be found suchthat

,,rfL$f-$,,S1” ‘p ‘I G al w

(8)

4

K K Ivanov

We denote the class of such problems by LP’,, and the approximate solution of (1) in the present case by

Then

where S(fe, E) is the sphere in F, radius E centre fe, and ZJ(fe,3L 44)

=/Epl (Y),

&(!I)

=ppP-Y/l’ I

E,=K-‘(S(J,,

(9)

E) LlY,).

The last relation holds because, when Et is convex,

We shall call the pointy*, at which (10) is attained, the Chebyshev centre of the domain Et, and
Here, obviously, the upper bound in (11) can be taken with respect to the boundary of Q. The present problem can thus be reduced to a problem of programming with quadratic functional and quadratic constraints. Denote by ue the quasi-solution [2] of (8) and put

In addition, we put IIfs - KLv, II = ;,$ u

II/c -

KLv II

and introduce the notation: x (u,) is the hyperplane tangential to the sphere $0, 1) at the point v,; n+( tie) is the half-space bounded by rc (v,) and containing S( 0, 1) ; E, is the ellipsoidal segment K-’ (S ( fe, E) fl KLd ( v,) ) ; and ‘p+,q- and $+, I#- are the ends of the principal axis of the ellipsoids K-‘(S(f,, E) flKLF and K-‘(S(f,, E) nKLx(v,)). respectively. Theorem 2

If 0 < pe<&, any algorithm for finding the Chebyshev centre of the ellipsoidal segment E, with absolute error o [ b (fe, 9,, $) ] will be asymptotically optimal among all algorithms, while

If Pe=e, then u(fe, Pi, si) =O and any algorithm for fmding Lu, with absolute error n > 0 will be optimal with accuracy up to 77.Additionally, if Ln (u,) does not cut the principal axis of

Solution of operator equations of the first kind

5

the ellipsoid K-’ (S (fe, E) CIKM), the the Chebyshev centre E, will be the same as Lu,. Otherwise, the Chebyshev centre E, is the same as the Chebyshev centre of the triangle $+, $-, cp* where cp*,isthe point cp*, which belongs to E,. Proof: In the case pe=e the theorem is obvious, since the quasi-solution is unique, and hence the domain Q consists of the single point ue. If &0, and let us show that the right-hand side of (12) gives an estimate of the error of approximating Lu, (in which case it estimates, all the more, the least possible error). To this end, we draw the plane n1 through the points Kcp’, KIJ? and the plane 7~ through the points fe, KLu,, Kcp’. Clearly, ni-Ln2. We also introduce the circular segment KE,nn, and the intersection CD=KLS (0, ‘1) flS ( fe,E) nn,. The problem now reduces to an elementary problem of two-dimensional analytical geometry: to show that the distance of any point of the domain a from KLu, is not greater than IIKiu,--$+ll= 11 KLu,-$-II = (E*-P~) “. This problem can easily be solved for boundary points of a, and hence for all points of 9. It remains to prove the auxiliary propositions concerning the position of the Chebyshev centre of the ellipsoidal segment. If Ln (II,) does not cut the principal axis of the ellipsoid E=K-’ (S( fe, E) nKLF) and E, contains the centre of symmetry of E, then the Ch.ebyshev centre of E, is obviously the same as the centre of symmetry of E, which is the same as Lu,.If Ln (u,) does not cut the principal axis of E, and E, does not contain the centre of symmetry of E, then the Chebyshev centre of E, is the same as the centre of symmetry of the ellipsoid EI=K-’ (S ( fe, E) flKLn (v,) ) . This follows from the fact that the centre of symmetry and Chebyshev centre of an ellipsoid are identical, and the latter remains unchanged if we add to the ellipsoid an arbitrary set which does not change the Chebyshev radius. Since the operators L, K, and K-l are linear, the Chebyshev centre of El is obviously the same as Lu,. Finally, if Ln (G) cuts the principal axis of E, the case is possible in which the distance of the centre of symmetry of El from $’ is less than its distance from cp*where cp’ is the point cp* which belongs to E,. In this case the Chebyshev centre of E, is not the same as Lu,. We shall show that it is then the same as the Chebyshev centre of the triangle $+, $-, cp*. We construct a sphere in F, whose centre and radius are the same as the required Chebyshev centre and radius. If the centre of our sphere is not the same as the Chebyshev centre of the triangle, four cases are possible: 1) the surface of the sphere has only two vertices of the triangle in common with E, 2) it has the three vertices in common with E, , 3) it has only one point outside the vertices in common with E, , 4)it has not less than two points outside the vertices in common with E, . In all four cases we arrive at a contradiction: in the first case, because the third vertex is outside the sphere, in the second, because the centre of the largest side of the triangle has all the properties of the Chebyshev centre of E, with radius less than the Chebyshev radius, in the third, because the Chebyshev centre of the triangle has all the properties of the Chebyshev centre of E, with a smaller radius, and in the fourth, because there is at least one point of E, outside the sphere. The theorem follows on eliminating these contradictions.

6

V. V. Ivanov

4. Let us make some comments on, and additions, to Section 3. a. Instead of (12) we can give the more accurate estimate u(fe, 91, Si) ml

(z,

r) =

G’ol

( (EZ--pe2)‘h,

,,l&3tW.

l+llu~ll),

” L” “)

which is a significant improvement on the familiar estimates [ 181. b. Some algorithms for the approximate determination of quasi-solutions are given in [2,19]. For methods of estimating wi (r) and o1 (z, r), see [3,20], c. A theorem similar to Theorem 2 also holds when the role of the sphere S ( fe, E) is played by the ellipsoid E (fe, E) with centre fe and maximum semi-axis E , and the role of the set of elements Lu, IIuII< 1, is played by any set of uniform regularization [2 1] with smooth boundary in any metric space, linear in the small [I 7 1, while K is an arbitrary operator, which is defined, Frkhet continuously differentiable, and invertible in this set, the inverse operator being also continuously differentiable. The class of relevant problems will be called the generalized class 9,.

d. If there are no a priori restrictions on the possible solutions of the equation &=f with ]]f-fC]I
If the diameter d (.EN) ‘0, N+ 03, we obtain a situation similar to that of Section 3. Otherwise, on the basis of the properties of fei and the rate of convergence of et to zero, certain conclusions may often be drawn regarding the properties off; this is done e.g., in the constructive theory of functions, by means of Bemshtein’s converse theorems. If it can be concluded that fEi belongs to a closed compact set M, thenfwill also belong to this set, and we again find ourselves in a favorable situation. e. A more realistic situation is that when fei =fi, ai=&. Here there arises the important problem of planning an experiment, in such a way that EN has a given property, e.g., compactness. 5. Let us now dwell on the case when the operator K, as well as f, is given approximately. We assume that fc,, K,,, Ed, cZ, are known, where

Ilf-f~,ll~El,

f, fst@,

IIK-K.JIez,

K, K,GY',

(13)

and LX!’is a set of linear bounded operators, while 11 II for an operator denotes its spectral norm. We also assume that the solution of (1) belongs to a compact convex set M and is unique for any fured K andf under the conditions (13); for any U, u=M, satisfying the inequality IIKu-Kull
(14)

we also have

Ilu4I~.02(z), uniformly with respect to KEX’, (jK-K,,ll<~~.

f&(T) -0,

T-+0,

(15)

7

Solution of operator equations of the first kind

Denote the class of problems of solving (1) under the conditions (13)-( 15) by PP. Then

Theorem 3 Up to first order terms in ei, Ed the problems 9, are equivalent to the generalized problems 9,, for which X=K,,, and the role of the set {‘p1‘p=Lu, 11~11~ 1) is played by c, M is the projection of fcl onto K,,M. M, fe=fe,, E==Ei+E2llq e,MII, where K,,cp,, M=f hoof: Let (PC?&. Up to principal terms, &,cp-fe,=Kcp-fe,+ (IL-K) cpe, Al and hence II} nM. Now assume the contrary, i.e., that ‘DEB,. It is E&?~={cpI IIKe,cp--fe,ll~&i+iE2ll(pe, 1\4 then sufficient to construct an operator K&l’, IIK-K,,II
(16) It can easily be seen that the required K can be found in the form K=K,,+iSG, where G is any unitary operator, transforming the unit vector q8, M i llcpe, MIIinto the unit vector - (K,,rp--f,,) / IILcp-fcill. BY(16), we have, allowing only for principal terms,

which completes the proof. Theorems 2 and 3 have the corollary that any sufficiently accurate algorithm for finding the quasi-solution cpe,M is in certain cases asymptotically optimal; up to principal terms in ei, EZ we have

6. In conclusion, let us assume that we are dealing with a Pi, problem, but that the information on fis specified in discrete form [23] , i.e., we are given N approximate values & of certain linear functionals v&f):

(17) It can easily be seen that the set QN; e of all elements f of F, for which (17) holds, is an ellipsoid in F. To justify the method above described, it is sufficient to require here that the diameter d (QN ,) -+O, as N-t m and E*O. In the general case of Hilbert space F, this condition is not however satisfied. It is therefore natural to assume that (17) is given in a narrower set, ensuring that the required conditions are satisfied. The narrower set may be e.g., the space FnKLF with

8

Y. V. Ivanov

certain restrictions on the operator L [24]. Translated by

D. E. Brown

REFERENCES 1.

TIKHONOV, A. N., On regularization of incorrectly posed problems, Dokl. Akad. Nauk SSSR, 153, No. 1, 49-52, 1963.

2.

IVANOV, V. K., On linear incorrectly posed problems, Dokl. Akad. Nauk, 145, No. 2, 270-272, 1962.

3.

LAVRENT’EV, M. M., On some incorrectly posed problems of mathematicalphysics (0 nekotorykh nekorrektnykh zadachakh matematicheskoi fiziki), Izd-vo SO Akad. Nauk SSSR, Novosibirsk, 1962.

4.

BAKHVALOV, N. $., On the properties of optimal methods for solving problems of mathematical physics, Zh. vjchisl. Mat. mat. Fiz., 10, No. 3, 555-568, 1970.

5.

CHARUSHNIKOV, V. D., Optimal approximate methods for solving linear problems, Differents. ur-niya, No. 2, 344-352, 1971.

6.

CHERNOUS’KO, F. L., On optimal search for the root of an approximately evaluated function, Dokl. Akad. Nauk SSSR, 177, No. 1,4&51, 1961.

I.

GROMENKO, V. M., and GURIN, L. S., Concerning the best optimization algorithm, Avtomatika i vychisl. tekhn., No. 6, 45-52, 1970.

8.

CHERNOUS’KO, F. L., On the optimal search for unimodal functions, Zh. vjchbl. Mat. mat. Fiz., 10, No. 4, 922-933, 1970.

9.

STRAKHOV, V. N., On the solution of linear incorrectly posed problems in Hilbert space, Differents. ur-niya, No. 8, 149@1495,1970.

10. MOROZOV, V. A., On the optimality of the defect criterion in the problem of computing the values of an unbounded operator, Zh. vjchislMat. mat. Fiz., 11, No. 4, 1019-1024, 1971. 11. CHENTSOV, N. N., Statisticaldecision rules and optimal conclusions (Statisticheskie reshayushchie pravila i optimal’nye vyvody), Nauka, Moscow, 1972. 12. DEM’YANOV, V. F., and MALOZEMOV, V. N., Introduction to the minimax (Vvedenie v minimaks), Nauka, Moscow, 1972. 13. SHOR, N. Z., and SHABASHOVA, L. P., On the solution of minimax problems by the method of generalized gradient descent with expansion of the space, Kibernetika, No. 1, 82-88, 1972. 14. DEM’YANOV, V. F., and PEVNYI, A. B., Numerical methods for finding saddle points, Zh. vjchisl Mat. mat. Fiz., 12, No. 5, 1099-1127, 1972. 15. ZUKHOVITSKII, S. I., POLYAK, R. A., and PRIMAK, M. E., Concave n-person games (numerical methods), Ekonomika i matem. metody, 7, No. 6, 888-900, 1971. 16. IVANOV, V. V., On optimal algorithms for minimizing functions of certain classes, Kibernetika, No. 4, 81-94, 1972. 17. LYUSTERNIK, L. A., and SOBOLEV, V. I., Elements of functional analysis(Elementy funktsional’nogo analiza), Nauka, Moscow, 1965. 18. IVANOV, V. K., On error estimation when solving operator equations of the first kind, in: Aspects of the accuracy and efficiency of computational algorithms (Vopr. tochnosti i effektivnosti vychisl. algoritmov), Proceedings of symposium, No. 2, 102-116, IK AN USSR, Kiev, 1969. 19. IVANOV, V. K., On incorrectly posed problems, Matem. sb.,61, No. 2, 187-199, 1963. 20. IVANOV, V. K., and KOROLYUK, T. I., On error estimation when solving linear incorrectly posed problems, Zh. vjchisl. Mat. mat. Fiz., 9, No. 1, 30-41, 1969. 21. STRAKHOV, V. N., On algorithms for the approximate solution of linear conditionally-correctly problems, Dokl. Akad. Nauk SSSR, 207, No. 5, 1057-1059, 1972.

posed

22. VINOKUROV, V. A., On the error of the approximate solution of linear problems, Zh. vjchisk Mat. mat. Fiz., 12, No. 3, 756-762, 1912.

On regularizing algorithms 23. MOROZOV, V. V., Theory of splines and the problem of stable computation operator, Zh. vychisl. Mat. mat. Fiz., 11,No. 3, 545-558, 1971. 24. STECHKIN, S. B., Best approximation

9 of the values of an unbounded

of linear operators, Matem zametki, 1, No. 2, 137-148, 1967.

ON REGULARIZING ALGORITHMS* YU. N. KHUDAK Moscow (Received 17 May 1973) NECESSARY and sufficient conditions are obtained for a family of linear operators {TV)to give a regularizing algorithm (r. a.) for an operator equation of the first kind Kz= u, z E 2, IL= U, in a pair of Banach spaces 2 and U. The stability of the r. a. when the problem operator K is disturbed is investigated. Necessary and sufficient conditions are obtained, under which the solvability of the initial equation can be judged from the convergence of the r. a. 1.The method of regularization, based on the concept of regularizing algorithm, was proposed by A. N. Tikhonov in [ 1,2] for the approximate solution of an operator equation of the first kind ki=l.&,

ZEZ,

UEU,

(1)

with a continuous, in general non-linear, operator K, acting from Banach space Z into Banach space U, and such that the operator K-l is not continuous. Definition [2] . A family of mappings {Ta}, C-0, of space U into Z is called a regularizing algorithm (r. a.) for the problem (1) if, for any right-hand side u’of (1) for which a solution Z of the equation exists (Kz = ii), and any GO there exists 6 (E, Z) >O, such that, as soon as IIU-GllU
for C-0,

(2)

sinceotherwise the operators Ta will not be defined for all u=U. In the present paper we obtain the necessary and sufficient conditions for a given family of linear bounded operators {Ta} to form an r. a. for the problem (1). Our main result is to obtain the exact attainable growth order of IIT~IIu+z as 6 + 0. An important consequence of this result is a sufficient condition on the disturbance of the operator K, under which the r. a. {Ta}, constructed for the “disturbed” equation Hz=u, where ]]I?--Kll z_,UG6,, yields an approximate solution of the problem (1). We shall also examine the circumstances from the convergence of the I. a. *Zh. vychisl. Mat. mat. Fiz., 15, 1, 12-18, 1975.

in

which the solvability of the problem can be judged