- Email: [email protected]
The rate of convergence of a method of equalizing maxima
0041-5553/78/1001-0231$07.50/O
U.S.S.R Comput. Maths Math. Phys. Vol. 18, pp. 231-235 o Pergamon Press Ltd. 1979. Printed in Great Britain.
THE RATE OF CONVERGENCE OF A METHOD OF EQUALIZING MAXIMA* V. N. MALOZEMOV and A. V. SOLOMENNIKOV Leningrad (Received 11 JuZy 1977)
THE QUADRATIC rate of convergence of a method of equalizing maxima used in the solution of minimax problems is proved. 1. Consider a non-linear system of the form
where u E Rn, v E Rm, F and G are respectively n- and m-dimensional vector functions possessing the property that the solution (u*, II*} of this system satisfies the relation F&A*, v*) = 0. For the numerical solution of such systems the method of equalization of maxima is used (on the subject of the title see [ 1,2] ):
This method is considerably simpler than Newton’s method, however, as will be shown below it possesses a quadratic rate of convergence. 2. We introduce the notation z= (u, u), ah== {uk, uk}, zk+= {u,,+~,uk}. In all cases the first norm of the vectors is used (the maximum of the moduli of the components) and the norm of the matrix consistent with it. Theorem 1 Let us assume that system (1) has a solution
2*={8*, u*},
and:
1) the mappings F and G are continuously differentiable in some neighbourhood of z*, 2) F,(z*) = 0, 3) the inverse matrices F,-l(z*)
and G,-l(z*)
exist.
If zll={~o, vo) is an initial approximation sufficiently close to’ z*, then the sequence zk. constructed by formula (2), converges to z* at a superlinear rate:
*Zh. @hid.
Mat. mat. Fiz., 18, 5, 1309-1312,1978.
231
K N. Malozemov and A. K Solomennikov
232
whereO
andqk’Oask+=.
If in addition F and G are twice continuously differentiable in the neighbourhood of z*, the rate of convergence is quadratic: 114+,-2’11 ~cllzn-a’llz. roof:
We put
EL=uh-u*,
qlk=vh-v*
and rewrite (2) in the equivalent form:
(3)
where z(t) ={a*+t& v*+@}={u,,- (1-t) En,vk- (l-t)qh},
qh+i
=
tl.b -
G.-'(zi+)
[G(a+) - G(r') I
1
-G,-'(a,,+)
=
{G,(v(t))%~+i
J
(4)
+tW~(t)) - G4a+)lmW,
0
where b,(t) ={u*+tgh+l, v*+hj={h+1-
(1-t)
%h+ir
vh- (+t)qb}.
We now choose 6 > 0 such that in the sphere Ba(2’) ={z ~11~41~6) IIF,(F,(z”)
llF,(z’)-~o(~“)II~E/2, IIF,-*
II~Ee/B,
IlGu(4 II~Jf,
llGu(z’) -Ga(z”)lIa, are satisfied, where
q=Ae(AM+I)
CL
Ii
the relations
lb%-‘(z)
II&A,
We show that k-0,
1, . . . ,
(5)
For k = 0 the statement is trivial. Let us assume that zk=Ba(2’) provided that z&&(0. and that (5) is satisfied. Then by (3) for k + 1 we have
In particular,
z~++,(z’).
Also, by (4), (7)
Combining (6) and (7), we obtain what is required. It has been shown that IA Sk = 6& Eh =
Zk
+
z*
as
k + =. We will establish the superlinear rate of convergence.
max max[2IIF,(z’) - F,(z”) II, 211Fqb’)- Fob”) II, Z’.Z”EB~.(Z’)
II’& (I’)-
‘& (~‘7 III,
qr = Aek(dM+i).
233
Short communications
and 4k + 0 as k + 00. Considering that zk and Zk+ belong to
It is obvious that Eh=$ qk+?
k-0, 1,. . . ,
llzh+,-2*11~qhllzh-2*11,
which proves the superlinear rate of convergence. If it is assumed that F and G are twice continuously differentiable in Bg (z*), then for some L>O
This and (3), (4) imply that bh+i-z*lldL(AM+i)
lbh-~*l12,
that is, in this case the rate of convergence is quadratic. The theorem is proved completely. 3. We prove a similar theorem without the assumption of the existence of a solution. Instead of this some conditions are imposed on the behaviour of F and G in the neighbourhood of the initial approximation. Theorem 2 Let us assume that in the sphere B6 (ZI-J)the mappings F and G are continuously differentiable and the inverse matrices Fu - l(z) and GY- 1(z) exist. Moreover, IIF, IIF&‘)
Ile~i,
llP,-’
IIGu(z) II
-F,(z”
) ll~2Jw-2”
IIG,-‘(z)
(2) ll
IlG,(z’)-G,(z”)llc2Lllz’-2”
II,
II
ll21--2oll=~,
q=max
[A (Lr+e), AW(Lr+e)+ALr]
r/(1-q)d.
{zh}, constructed by formulas
Then in B6 (zo) a solution of system (1) exists, and the sequence (2), converges to some solution z*. If then F,,(z*) = 0 and IIF,
4,
(2 n ) II ~2pll~‘-z
” II
vz’, z” =Ba(zo),
(8)
then the rate of convergence is quadratic. Proofi We put
i$‘=uk+i-uk,
and prove by induction that
qk’=vk+,-vk
ll&lL’il~rqk,
ll9k’ll <‘Qh,
For k = 0 this is true by hypothesis. In particular, Zk+l belong to B6 (zI-J). andletzl,...,
k==O,1,. . . .
nl~&,(ro).
Let (9) be satisfied for some k,
BY (2) IIEwU~
AlWw)
II,
~(zr+t) 1
- F(a++l)
- F(ad
-
s 0
{[MY)
- Pub)
- F,(o) lb’
EC
+ ~4dO)rlh”)~4
(9)
V. N. Malozemovand A. V. Solomennikov
234
where z(t)={zz~+t&,,‘, uc+tqh’}. From this
Taking into account the inequalities 11~~+2-~0ll~ll&:+ill+~l~~~ll+ . . . +II&lrllc we
conclude that
( 8,
G,‘+,=Be (zo). Moreover, Ilqkillc
AllG(&i)II,
+ G(asi+i)= =
where
-!1-P
+ G(ar+i)
’ {Gu(u(t))E 5 e
- G(zk+) - G,(a+)qk k+i +[G&(O)-
G&+)lqr’}dt,
y(t) - {ur+,+t~k+,, vk+tn,,‘}. From this llqi+illG
In particular,
A[MA (b+e)
+ h]rq”
< rqk+‘.
(11)
ah++& (zO). From (10) and (11) the validity of (9) follows for all k.
k-0, 1, . . . . Therefore, the sequence It has been established that 1lz~+~-a~ll6r~, is fundamental and therefore has a limit P, which is the solution of system (1). Then
Il~k-4l<
$q
!l” < W,
k==O,i,...
.
(12)
l)b= u,,-u*, We now assume that F,(z*) = 0 and (8) is satisfied. Writing +uk-u*, by (3) we obtain Il~k+~l16A (L+P) ll~~-z*ll~. Since the sequence {A(L+P) ll~~-~*ll} is bounded, a constant Q > 1 can be found such that 1lr~+-2*ll
l12k+l-2*ll 4Clh-z*l12,
4. We ptit rl = r/(1-q)
Ilqrll) CA (MA
(L+P)+LQ) Ilzjj-2*l12.
where C=max {A(L+P), A (MA (L+P) +LQ)}. The theorem is
and note that by (12) we have ~*43~~(2~).
Theorem 3
If in the conditions of Theorem 2 we also have q C min
then the. solution z* of the system (1) in the sphere B,.I(zu) is unique. Roof: Let us assume that in &I (zu) another solution Fof the system (1) exists. We introduce the notation Elr=uk-& ;jR=uk-a. Then, as for (3) and (4), 1 i h+l=
-Fu-‘bd
s 0
{[F”(r(t))-F,(2A)l~b+Ft,(Z”(t));k}dt,
Short communicationr
where z”(t,={uk-(l-t).&
:k+i
=
235
VI,-(l-t)%},
-Gv-'(zk+)
s
{Gu(y”(t))ik+i +[G,(y”(t))
- ‘%(Zk+)];k}dt,
0
where Y”(t)=(uk+i-(l-t)~k+i,
Uk-(l-t)G}.
This implies that (13)
where q1 =
=
If we put
p=max
Au~ax[ALr
(l-AE,
A2M(Lri+e)
max[A(Lr,+e),
1-A2Me),
+ Ae(l-q),
then
+ ALri]
(A2ML+AL)r
by hypothesis,
+ A2Me(l-q)]
qcl/(p+l).
Therefore,ql
< 1.
By (13), zk -, r, which contradicts the relation zk -+ P. The theorem is proved. 7kanslated by
J. Berry.
REFERENCES 1.
MALOZEMOV, V. N. Equalization of maxima. Zh. #hid.
Mat. mat. Hz., 16,3,781-784,1976.
2.
DEM’YANOV, V. F. and MALOZEMOV, V. N. Editors. Quertio~~ of the theory and elements of the software of minimax problems (Voprosyteorii i elementi programmogo obespqcheniya minimaxnykh zadach), lzd-vo Leningr. un-ta, Leningrad, 1977.
U.S.S.R Comput. Maths Math. Phys. Vol. 18, pp. 231-235 o Pergamon Press Ltd. 1979. Printed in Great Britain.
THE RATE OF CONVERGENCE OF A METHOD OF EQUALIZING MAXIMA* V. N. MALOZEMOV and A. V. SOLOMENNIKOV Leningrad (Received 11 JuZy 1977)
THE QUADRATIC rate of convergence of a method of equalizing maxima used in the solution of minimax problems is proved. 1. Consider a non-linear system of the form
where u E Rn, v E Rm, F and G are respectively n- and m-dimensional vector functions possessing the property that the solution (u*, II*} of this system satisfies the relation F&A*, v*) = 0. For the numerical solution of such systems the method of equalization of maxima is used (on the subject of the title see [ 1,2] ):
This method is considerably simpler than Newton’s method, however, as will be shown below it possesses a quadratic rate of convergence. 2. We introduce the notation z= (u, u), ah== {uk, uk}, zk+= {u,,+~,uk}. In all cases the first norm of the vectors is used (the maximum of the moduli of the components) and the norm of the matrix consistent with it. Theorem 1 Let us assume that system (1) has a solution
2*={8*, u*},
and:
1) the mappings F and G are continuously differentiable in some neighbourhood of z*, 2) F,(z*) = 0, 3) the inverse matrices F,-l(z*)
and G,-l(z*)
exist.
If zll={~o, vo) is an initial approximation sufficiently close to’ z*, then the sequence zk. constructed by formula (2), converges to z* at a superlinear rate:
*Zh. @hid.
Mat. mat. Fiz., 18, 5, 1309-1312,1978.
231
K N. Malozemov and A. K Solomennikov
232
whereO
andqk’Oask+=.
If in addition F and G are twice continuously differentiable in the neighbourhood of z*, the rate of convergence is quadratic: 114+,-2’11 ~cllzn-a’llz. roof:
We put
EL=uh-u*,
qlk=vh-v*
and rewrite (2) in the equivalent form:
(3)
where z(t) ={a*+t& v*+@}={u,,- (1-t) En,vk- (l-t)qh},
qh+i
=
tl.b -
G.-'(zi+)
[G(a+) - G(r') I
1
-G,-'(a,,+)
=
{G,(v(t))%~+i
J
(4)
+tW~(t)) - G4a+)lmW,
0
where b,(t) ={u*+tgh+l, v*+hj={h+1-
(1-t)
%h+ir
vh- (+t)qb}.
We now choose 6 > 0 such that in the sphere Ba(2’) ={z ~11~41~6) IIF,(F,(z”)
llF,(z’)-~o(~“)II~E/2, IIF,-*
II~Ee/B,
IlGu(4 II~Jf,
llGu(z’) -Ga(z”)lIa, are satisfied, where
q=Ae(AM+I)
CL
Ii
the relations
lb%-‘(z)
II&A,
We show that k-0,
1, . . . ,
(5)
For k = 0 the statement is trivial. Let us assume that zk=Ba(2’) provided that z&&(0. and that (5) is satisfied. Then by (3) for k + 1 we have
In particular,
z~++,(z’).
Also, by (4), (7)
Combining (6) and (7), we obtain what is required. It has been shown that IA Sk = 6& Eh =
Zk
+
z*
as
k + =. We will establish the superlinear rate of convergence.
max max[2IIF,(z’) - F,(z”) II, 211Fqb’)- Fob”) II, Z’.Z”EB~.(Z’)
II’& (I’)-
‘& (~‘7 III,
qr = Aek(dM+i).
233
Short communications
and 4k + 0 as k + 00. Considering that zk and Zk+ belong to
It is obvious that Eh=$ qk+?
k-0, 1,. . . ,
llzh+,-2*11~qhllzh-2*11,
which proves the superlinear rate of convergence. If it is assumed that F and G are twice continuously differentiable in Bg (z*), then for some L>O
This and (3), (4) imply that bh+i-z*lldL(AM+i)
lbh-~*l12,
that is, in this case the rate of convergence is quadratic. The theorem is proved completely. 3. We prove a similar theorem without the assumption of the existence of a solution. Instead of this some conditions are imposed on the behaviour of F and G in the neighbourhood of the initial approximation. Theorem 2 Let us assume that in the sphere B6 (ZI-J)the mappings F and G are continuously differentiable and the inverse matrices Fu - l(z) and GY- 1(z) exist. Moreover, IIF, IIF&‘)
Ile~i,
llP,-’
IIGu(z) II
-F,(z”
) ll~2Jw-2”
IIG,-‘(z)
(2) ll
IlG,(z’)-G,(z”)llc2Lllz’-2”
II,
II
ll21--2oll=~,
q=max
[A (Lr+e), AW(Lr+e)+ALr]
r/(1-q)d.
{zh}, constructed by formulas
Then in B6 (zo) a solution of system (1) exists, and the sequence (2), converges to some solution z*. If then F,,(z*) = 0 and IIF,
4,
(2 n ) II ~2pll~‘-z
” II
vz’, z” =Ba(zo),
(8)
then the rate of convergence is quadratic. Proofi We put
i$‘=uk+i-uk,
and prove by induction that
qk’=vk+,-vk
ll&lL’il~rqk,
ll9k’ll <‘Qh,
For k = 0 this is true by hypothesis. In particular, Zk+l belong to B6 (zI-J). andletzl,...,
k==O,1,. . . .
nl~&,(ro).
Let (9) be satisfied for some k,
BY (2) IIEwU~
AlWw)
II,
~(zr+t) 1
- F(a++l)
- F(ad
-
s 0
{[MY)
- Pub)
- F,(o) lb’
EC
+ ~4dO)rlh”)~4
(9)
V. N. Malozemovand A. V. Solomennikov
234
where z(t)={zz~+t&,,‘, uc+tqh’}. From this
Taking into account the inequalities 11~~+2-~0ll~ll&:+ill+~l~~~ll+ . . . +II&lrllc we
conclude that
( 8,
G,‘+,=Be (zo). Moreover, Ilqkillc
AllG(&i)II,
+ G(asi+i)= =
where
-!1-P
+ G(ar+i)
’ {Gu(u(t))E 5 e
- G(zk+) - G,(a+)qk k+i +[G&(O)-
G&+)lqr’}dt,
y(t) - {ur+,+t~k+,, vk+tn,,‘}. From this llqi+illG
In particular,
A[MA (b+e)
+ h]rq”
< rqk+‘.
(11)
ah++& (zO). From (10) and (11) the validity of (9) follows for all k.
k-0, 1, . . . . Therefore, the sequence It has been established that 1lz~+~-a~ll6r~, is fundamental and therefore has a limit P, which is the solution of system (1). Then
Il~k-4l<
$q
!l” < W,
k==O,i,...
.
(12)
l)b= u,,-u*, We now assume that F,(z*) = 0 and (8) is satisfied. Writing +uk-u*, by (3) we obtain Il~k+~l16A (L+P) ll~~-z*ll~. Since the sequence {A(L+P) ll~~-~*ll} is bounded, a constant Q > 1 can be found such that 1lr~+-2*ll
l12k+l-2*ll 4Clh-z*l12,
4. We ptit rl = r/(1-q)
Ilqrll) CA (MA
(L+P)+LQ) Ilzjj-2*l12.
where C=max {A(L+P), A (MA (L+P) +LQ)}. The theorem is
and note that by (12) we have ~*43~~(2~).
Theorem 3
If in the conditions of Theorem 2 we also have q C min
then the. solution z* of the system (1) in the sphere B,.I(zu) is unique. Roof: Let us assume that in &I (zu) another solution Fof the system (1) exists. We introduce the notation Elr=uk-& ;jR=uk-a. Then, as for (3) and (4), 1 i h+l=
-Fu-‘bd
s 0
{[F”(r(t))-F,(2A)l~b+Ft,(Z”(t));k}dt,
Short communicationr
where z”(t,={uk-(l-t).&
:k+i
=
235
VI,-(l-t)%},
-Gv-'(zk+)
s
{Gu(y”(t))ik+i +[G,(y”(t))
- ‘%(Zk+)];k}dt,
0
where Y”(t)=(uk+i-(l-t)~k+i,
Uk-(l-t)G}.
This implies that (13)
where q1 =
=
If we put
p=max
Au~ax[ALr
(l-AE,
A2M(Lri+e)
max[A(Lr,+e),
1-A2Me),
+ Ae(l-q),
then
+ ALri]
(A2ML+AL)r
by hypothesis,
+ A2Me(l-q)]
qcl/(p+l).
Therefore,ql
< 1.
By (13), zk -, r, which contradicts the relation zk -+ P. The theorem is proved. 7kanslated by
J. Berry.
REFERENCES 1.
MALOZEMOV, V. N. Equalization of maxima. Zh. #hid.
Mat. mat. Hz., 16,3,781-784,1976.
2.
DEM’YANOV, V. F. and MALOZEMOV, V. N. Editors. Quertio~~ of the theory and elements of the software of minimax problems (Voprosyteorii i elementi programmogo obespqcheniya minimaxnykh zadach), lzd-vo Leningr. un-ta, Leningrad, 1977.