- Email: [email protected]
On the principle of iterative regularization
U.S.S.R. Comput. Maths. Math. Phys. Vol. 19, pp. 256-260 0 Pergamon Press Ltd. 1980. Printed in Great Britain.
0041-5553/0801-0256$07.50/O
ON THE PRINCIPLE OF ITERATIVE REGULARIZATION* A. B. BAKUSHINSKII Moscow (Received 3 May 1978)
THE EFFECT of perturbation of the domain on the algorithms of iterative regularization is investigated. The principle of iterative regularization as a method of constructing iterative algorithms for solving monotonic variational inequalities was described in detail in [I] . For the practical use of the principle it is desirable to know how iterative algorithms based on this principle behave when the operator and the domain of definition of the variational inequality are perturbed. The question of how the perturbation of the operator affects the process of iterative regularization was discussed in [l-3] , where a general scheme of study of the effect of these perturbations was planned. In [3], for example, it was shown that iteratively the regularized Newton method generates a regularizing algorithm with respect to such perturbations in the problem of calculating solutions of a variational inequality. The effect of perturbation of the domain on such algorithms has so far not been studied. One possible way of allowing for the perturbation of the domain in the realization of algorithms based on the iterative regularization principle is presented below. 2. Let Q and Q1 be two closed convex domains in the real Hilbert space H. We will measure the proximity of the domains Q and Ql by the two functionals:
P(Q,Qi, u) =IIP@--PQ,Ull,
(1)
where u is some element of H, and P is the operator of projection onto the corresponding set, and
PD (Q, QJ =
SUP IIPQu-J’c+ll. IIUIKD
(2)
The functionals (l), (2) possess all the properties of a metric, except for the fact that when they are equal to zero it does not follow that the domains Q and Ql are identical. The functionals (1) and (2) are easily evaluated in terms of a more standard measure of proximity of domains - the Hausdorff metric o(Q, Q1), namely, by Lemma 5 of [4], P(Q,Qi,~l
(3)
We note that the requirement of proximity of domains in the sense of (1) or (2) is essentially a weaker requirement than proximity in the sense of the Hausdorff metric. It is easy to give examples of domains Q and Q, for which the value of (1) or (2) equals zero, but o(Q, Qr) = 00.By using the *Zh. vjGhis1.Mat. mat. Fiz., 19,4, 1040-1043,
1979.
256
257
Short communications
functionals (l), (2) we can estimate the closeness of the solutions of two variational inequalities with domains of definition Q and Q1. Let the operator F(x) act in H, be strongly monotonic on
QUQl, (Fh)-F(zz),
Zir
zi-z2)~cllzi-z2ll2,
Z2wJQi,
c>o,
(4)
and also satisfy the Lipschitz condition
Let us consider the variational inequality
(F(z), z-z) GO.
Let w and w1 be solutions of the corresponding inequalities (6) in the domains Q and QI respectively. Lemma
If conditions (4) and (5) are satisfied, then the estimate ,Iw w 11~ midp(Q,Ql, 51, p(Q, Ql, WdI
-
1
1 -(1-P/U)‘”
’
(7)
holds, where W= W- cF ( w) /2L. Proof: Suppose, for example, that p (Q, Qi, w,) G p (Q, Qi, 5). Using the equivalent formula for the variational inequality as an equation containing a projection operator (see [2]), and taking into account (4) and (5), we have
In this inequality fl is an arbitrary positive number. Choosing p = c/2L in (8) we obtain the best estimate (7) for the method chosen. 3. The lemma proved enables us to formulate various theorems on the stability of algorithms obtained on the basis of the principle of iterative regularization in relation to perturbations of the domain. Proofs of the corresponding assertions for algorithms of zeroth and first orders of [ 1 ] will be indicated below.
258
A. B. Bakushinskii
Let the n-th step of the algorithm be carried out by using instead of the fundamental domain Q the “perturbed” domain Q,. The convergence of the corresponding process can be proved by the general scheme described in [l] , namely: when the corresponding additional conditions are satisfied, by proving the stabilization of the trajectories of the processes to the trajectory of the auxiliary points XEn - the solutions of the Brauder inequalities on Q,. In addition it is required that the theorem of convergence of the Brauder-Tikhonov approximations be generalized, that is, that the following relation be established:
(9) It is obvious that
(10) By the lemma PD (Q, Qn)
Il&,-~8,ll~
1 - (1-en2/4L)“a
(11)
’
where
In deriving the estimate (11) it was taken into account that from (11) that the condition
I.m 1
PD (0,
Qn)
Ilx,,llGllyII
(see [l] ). It follows
=o
(12)
etlz
II’m
is sufficient for (9) to be satisfied. In view of the fact that (see [l] ) Ix, n-~e,,+*ll~llYII
from (10) we obtain
(13)
if for some positive constant c PD(Q, Qn)
max
1
--1 1 - (1-~,,~/4L)“2
PD (Qv Qn+d 1 - (l-eZ,+i/4L)“~
We now formulate the principal results.
I
en-en+1 I
en
(14)
259
Short communications
Theorem 1 (on the convergence of the perturbed method of zero order).
On the set
u Qn
let the conditions of Theorem 2 of [l] , and also (12) and (14) be
satisfied; then the iterative sequence
(1%
.,i+i=Pu,,(.~,,-a,,(F(t,,)+Enln))
converges in H to the solution of inequality (6) and Q possessing minimal norm. Theorem 2 (on the convergence of the perturbed first-order method, of Newton type).
On
U QTL
let the conditions of Theorem 5 of [l] be satisfied with the ‘difference that,
for example, n Ilull+
n En21i 7
2c, max-
max [
PD(Q~QA cd
and conditions (5), (12) and (14) are satisfied. Then the iterative sequence of Newton type with the perturbed domains Q, converges in H toy. These theorems are proved in the same way as the corresponding theorems given in [l] , using in the corresponding places the proofs of the inequalities (13). Following the scheme used in [3], it can be proved that the corresponding iterative algorithms generate regularizing algorithms for the problem of solving the variational inequality (6) with a perturbation of the domain in the sense of(l), (2), at least if the prior condition IIy II < k is satisfied, where k is some constant. The theorems formulated can be generalized to the case where the constant L in (5) depends onmax [llzl1l, II 22 II]. Thisis done by the schemeindicatedin [l],usingLemma2 of [l]. We will not reproduce the corresponding results because of their complexity. If the Lipschitz condition for F is absent the question of the effect of a perturbation of the domain on an algorithm of type (15) (the question of an analog of Theorem 1 of [l] ) remains open. In conclusion we note that for the “elementary” case c > 0, e, s 0 the algorithm (15) with the perturbed domains Q, was studied in [5] . The idea of the use of the functionals (1) and (2) for estimating the proximity of the domains was suggested there. Translated by J. Berry.
REFERENCES 1.
BAKUSHINSKII, A. B. Methods of solving monotonic variational inequalities based on the principle of iterative regularization. Zh. ujkhisZ. Mat. mat. Fiz., 17,6, 1350-1362, 1977.
2.
BAKUSHINSKII, A. B. and POLYAK, B. T. On the solution of variational inequalities. Dokl. Akad. Nauk SSSR, 219,5,1038-1041,1974.
3.
BAKUSHINSKII, A. B. A regularizing algorithm based on the Newton-Kantorovich variational inequalities. Zh. vj&sl. Mat. mat. Fiz., 16,6, 1397-1404, 1976.
method for solving
Yu. Ya. Ledyankin
260 4.
LISKOVETS, 0. A. Incorrect problems and the stability of quasi-solutions. Sibirskii matem zh., 10,2, 373-385,1969.
5.
SOIWTAG, Y. Un algoritme iteratif pour la resolution d’une classe d’inequations. A282,5,293-295,1976.
U.S.S.R. Comput. Maths. Math. Phys. Vol. 19, pp. 260-265 0 Pergamon Press Ltd. 1980. Printed in Great Britain.
Compt. rend. Acad. Sci.,
OO41-5553/0801-0260$07.50/O
AN INCREMENT METHOD FOR THE SOLUTION OF THE EQUATION Ay = f* Yu.
Ya. LEDYANKIN Kiev
(Received 22 September 1978; revised3 April 1979)
FOR THE problem Ay = J where A is a self-conjugate operator, a method of solution is presented, an iterative computational process is constructed and a formal description of it is given, and its closeness to the method of simple iterations is demonstrated. 1. To increase the efficiency of the computational process in the solution of mesh analogs of the equations of mathematical physics special parallel action devices are constructed. In this connection it is natural to develop algorithms for solving mesh problems corresponding to these devices. In [ 1,2 ] the importance and usefulness of the processing of the highest-order digits of a number is determined; it is proposed to use operations not with completely digital numbers, but with the highest-order digits or a group of them. Below, using these ideas, a modification of the method of simple iterations [3] is presented - a method of increments oriented to apparatus realization (with the possibility of using it in universal computers), which enables systems of difference equations, for which the simple iteration method is suitable, to be solved economically. In the proposed iterative process the operation of multiplication of numbers is eliminated also. 2. Let wh be a set of numbersxi Jfi the space of the mesh functions y(Xi), defined on wh with the scalar product (,) and norm II - II = t/(,). Let A be an operator which is self-conjugate and positive-definite in xh. We consider the problem Ay=j.
(0
With the properties of the operator A indicated above we can use the method of simple iterations [l] to solve problem (1) S=E-TA,
y(8+‘)=sy(S)+C+,,
y(O)=yo(xi),
XiE
mhr
*Zh. vjkhisl. Mat. mat. Fiz., 19,4, 1043-1047,
1979.
cp=Qf,
s=O, I,...,
N,
(2)
0041-5553/0801-0256$07.50/O
ON THE PRINCIPLE OF ITERATIVE REGULARIZATION* A. B. BAKUSHINSKII Moscow (Received 3 May 1978)
THE EFFECT of perturbation of the domain on the algorithms of iterative regularization is investigated. The principle of iterative regularization as a method of constructing iterative algorithms for solving monotonic variational inequalities was described in detail in [I] . For the practical use of the principle it is desirable to know how iterative algorithms based on this principle behave when the operator and the domain of definition of the variational inequality are perturbed. The question of how the perturbation of the operator affects the process of iterative regularization was discussed in [l-3] , where a general scheme of study of the effect of these perturbations was planned. In [3], for example, it was shown that iteratively the regularized Newton method generates a regularizing algorithm with respect to such perturbations in the problem of calculating solutions of a variational inequality. The effect of perturbation of the domain on such algorithms has so far not been studied. One possible way of allowing for the perturbation of the domain in the realization of algorithms based on the iterative regularization principle is presented below. 2. Let Q and Q1 be two closed convex domains in the real Hilbert space H. We will measure the proximity of the domains Q and Ql by the two functionals:
P(Q,Qi, u) =IIP@--PQ,Ull,
(1)
where u is some element of H, and P is the operator of projection onto the corresponding set, and
PD (Q, QJ =
SUP IIPQu-J’c+ll. IIUIKD
(2)
The functionals (l), (2) possess all the properties of a metric, except for the fact that when they are equal to zero it does not follow that the domains Q and Ql are identical. The functionals (1) and (2) are easily evaluated in terms of a more standard measure of proximity of domains - the Hausdorff metric o(Q, Q1), namely, by Lemma 5 of [4], P(Q,Qi,~l
(3)
We note that the requirement of proximity of domains in the sense of (1) or (2) is essentially a weaker requirement than proximity in the sense of the Hausdorff metric. It is easy to give examples of domains Q and Q, for which the value of (1) or (2) equals zero, but o(Q, Qr) = 00.By using the *Zh. vjGhis1.Mat. mat. Fiz., 19,4, 1040-1043,
1979.
256
257
Short communications
functionals (l), (2) we can estimate the closeness of the solutions of two variational inequalities with domains of definition Q and Q1. Let the operator F(x) act in H, be strongly monotonic on
QUQl, (Fh)-F(zz),
Zir
zi-z2)~cllzi-z2ll2,
Z2wJQi,
c>o,
(4)
and also satisfy the Lipschitz condition
Let us consider the variational inequality
(F(z), z-z) GO.
Let w and w1 be solutions of the corresponding inequalities (6) in the domains Q and QI respectively. Lemma
If conditions (4) and (5) are satisfied, then the estimate ,Iw w 11~ midp(Q,Ql, 51, p(Q, Ql, WdI
-
1
1 -(1-P/U)‘”
’
(7)
holds, where W= W- cF ( w) /2L. Proof: Suppose, for example, that p (Q, Qi, w,) G p (Q, Qi, 5). Using the equivalent formula for the variational inequality as an equation containing a projection operator (see [2]), and taking into account (4) and (5), we have
In this inequality fl is an arbitrary positive number. Choosing p = c/2L in (8) we obtain the best estimate (7) for the method chosen. 3. The lemma proved enables us to formulate various theorems on the stability of algorithms obtained on the basis of the principle of iterative regularization in relation to perturbations of the domain. Proofs of the corresponding assertions for algorithms of zeroth and first orders of [ 1 ] will be indicated below.
258
A. B. Bakushinskii
Let the n-th step of the algorithm be carried out by using instead of the fundamental domain Q the “perturbed” domain Q,. The convergence of the corresponding process can be proved by the general scheme described in [l] , namely: when the corresponding additional conditions are satisfied, by proving the stabilization of the trajectories of the processes to the trajectory of the auxiliary points XEn - the solutions of the Brauder inequalities on Q,. In addition it is required that the theorem of convergence of the Brauder-Tikhonov approximations be generalized, that is, that the following relation be established:
(9) It is obvious that
(10) By the lemma PD (Q, Qn)
Il&,-~8,ll~
1 - (1-en2/4L)“a
(11)
’
where
In deriving the estimate (11) it was taken into account that from (11) that the condition
I.m 1
PD (0,
Qn)
Ilx,,llGllyII
(see [l] ). It follows
=o
(12)
etlz
II’m
is sufficient for (9) to be satisfied. In view of the fact that (see [l] ) Ix, n-~e,,+*ll~llYII
from (10) we obtain
(13)
if for some positive constant c PD(Q, Qn)
max
1
--1 1 - (1-~,,~/4L)“2
PD (Qv Qn+d 1 - (l-eZ,+i/4L)“~
We now formulate the principal results.
I
en-en+1 I
en
(14)
259
Short communications
Theorem 1 (on the convergence of the perturbed method of zero order).
On the set
u Qn
let the conditions of Theorem 2 of [l] , and also (12) and (14) be
satisfied; then the iterative sequence
(1%
.,i+i=Pu,,(.~,,-a,,(F(t,,)+Enln))
converges in H to the solution of inequality (6) and Q possessing minimal norm. Theorem 2 (on the convergence of the perturbed first-order method, of Newton type).
On
U QTL
let the conditions of Theorem 5 of [l] be satisfied with the ‘difference that,
for example, n Ilull+
n En21i 7
2c, max-
max [
PD(Q~QA cd
and conditions (5), (12) and (14) are satisfied. Then the iterative sequence of Newton type with the perturbed domains Q, converges in H toy. These theorems are proved in the same way as the corresponding theorems given in [l] , using in the corresponding places the proofs of the inequalities (13). Following the scheme used in [3], it can be proved that the corresponding iterative algorithms generate regularizing algorithms for the problem of solving the variational inequality (6) with a perturbation of the domain in the sense of(l), (2), at least if the prior condition IIy II < k is satisfied, where k is some constant. The theorems formulated can be generalized to the case where the constant L in (5) depends onmax [llzl1l, II 22 II]. Thisis done by the schemeindicatedin [l],usingLemma2 of [l]. We will not reproduce the corresponding results because of their complexity. If the Lipschitz condition for F is absent the question of the effect of a perturbation of the domain on an algorithm of type (15) (the question of an analog of Theorem 1 of [l] ) remains open. In conclusion we note that for the “elementary” case c > 0, e, s 0 the algorithm (15) with the perturbed domains Q, was studied in [5] . The idea of the use of the functionals (1) and (2) for estimating the proximity of the domains was suggested there. Translated by J. Berry.
REFERENCES 1.
BAKUSHINSKII, A. B. Methods of solving monotonic variational inequalities based on the principle of iterative regularization. Zh. ujkhisZ. Mat. mat. Fiz., 17,6, 1350-1362, 1977.
2.
BAKUSHINSKII, A. B. and POLYAK, B. T. On the solution of variational inequalities. Dokl. Akad. Nauk SSSR, 219,5,1038-1041,1974.
3.
BAKUSHINSKII, A. B. A regularizing algorithm based on the Newton-Kantorovich variational inequalities. Zh. vj&sl. Mat. mat. Fiz., 16,6, 1397-1404, 1976.
method for solving
Yu. Ya. Ledyankin
260 4.
LISKOVETS, 0. A. Incorrect problems and the stability of quasi-solutions. Sibirskii matem zh., 10,2, 373-385,1969.
5.
SOIWTAG, Y. Un algoritme iteratif pour la resolution d’une classe d’inequations. A282,5,293-295,1976.
U.S.S.R. Comput. Maths. Math. Phys. Vol. 19, pp. 260-265 0 Pergamon Press Ltd. 1980. Printed in Great Britain.
Compt. rend. Acad. Sci.,
OO41-5553/0801-0260$07.50/O
AN INCREMENT METHOD FOR THE SOLUTION OF THE EQUATION Ay = f* Yu.
Ya. LEDYANKIN Kiev
(Received 22 September 1978; revised3 April 1979)
FOR THE problem Ay = J where A is a self-conjugate operator, a method of solution is presented, an iterative computational process is constructed and a formal description of it is given, and its closeness to the method of simple iterations is demonstrated. 1. To increase the efficiency of the computational process in the solution of mesh analogs of the equations of mathematical physics special parallel action devices are constructed. In this connection it is natural to develop algorithms for solving mesh problems corresponding to these devices. In [ 1,2 ] the importance and usefulness of the processing of the highest-order digits of a number is determined; it is proposed to use operations not with completely digital numbers, but with the highest-order digits or a group of them. Below, using these ideas, a modification of the method of simple iterations [3] is presented - a method of increments oriented to apparatus realization (with the possibility of using it in universal computers), which enables systems of difference equations, for which the simple iteration method is suitable, to be solved economically. In the proposed iterative process the operation of multiplication of numbers is eliminated also. 2. Let wh be a set of numbersxi Jfi the space of the mesh functions y(Xi), defined on wh with the scalar product (,) and norm II - II = t/(,). Let A be an operator which is self-conjugate and positive-definite in xh. We consider the problem Ay=j.
(0
With the properties of the operator A indicated above we can use the method of simple iterations [l] to solve problem (1) S=E-TA,
y(8+‘)=sy(S)+C+,,
y(O)=yo(xi),
XiE
mhr
*Zh. vjkhisl. Mat. mat. Fiz., 19,4, 1043-1047,
1979.
cp=Qf,
s=O, I,...,
N,
(2)