Stability of discrete processes with respect to specified variables, and the convergence of some optimization algorithms

V. R. Nosov and V. D. Fkrasov REFERENCES 1.

MOISEEV, N. N., Numerical methods in the theory of optimal systems (Chislennye metody v teorii optimal’nykh &tern), Nauka, Moscow, 1971.

2.

PSHENICHNYI, B. N., and DANILIN, Yu. M., Numerical methods in extremal problems (Chislennye metody v ekstremal’nykh zadachakh), Nauka, Moscow, 1975.

3.

CHAN HAN, Approximate methods for solving problems of convex programming, Zh. vjkhisl. Mat. mat. Fiz., 17, No. 4, 805-815, 1977.

4.

POLYAK, B. T., Gradient methods for minimizing functional& Zh. vj%hisl.Mat. mat. Fit., 3, No. 4, 643-653,1963.

5.

LEVITIN, E. S., and POLYAK, B. T., Methods of constrained minimizations, Zb. vj%hisl.Mat. mat. Fiz., 6, No. 5,787-823,1966.

6.

CHAN HAN, Some new methods for solving finite-dimensional IL, VTs Akad. Nauk SSSR, Moscow, 1978.

extremal problems, Disa Kand. fn.-matem.

L9.S.R. Comput. Maths. Math. Phys. VoL 19, pp. 44-58 8 Pergamon Press Ltd. 1980. Printed in Great Britain.

0041-5553/79/0401-0044$07.50/0

STABILITY OF DISCRETE PROCESSES WITH RESPECT TO SPECIFIED VARIABLES, AND THE CONVERGENCE OF SOME OPTIMIZATION ALGORITHMS* V. R. NOSOV and V. D. FURASOV Moscow (Received

15 Mmh

1978)

THEOREMS of Lyapunov’s second method concerning the stability of discrete processes with respect to specified variables are proved. The theorems are employed to establish convergence conditions for certain unconstrained minhnization algorithms. The present paper is concerned with two classes of problems. First, we examine the stability of discrete processes with respect to specified variables, and prove some general propositions with regard to the stability, asymptotic stability, and convergence, of discrete processes with respect to specified variables. The general theorems obtained are then used to establish convergence conditions for some methods of unconstrained minimization of functions. Lyapunov’s method can be looked on as universal when it comes to investigating the different types of stability of a wide variety of processes. The method has often been used (see e.g. [ 11) to study the stability of discrete processes. The stability of the solutions of ordinary differential equations with respect to part of the variables was studied in [2 ] . So far as the present authors are aware, however, the stability of discrete processes with respect to part of the variables or with respect to certain specified variables has not so far been considered. It should be noted that this subject has some special features; in particular, Theorems 2-5 below have an analogues for ordinary differential equations.

*Zh. vjkhisl. Mat. mat. Fiz., 19, 2,316-328,

1979.

Stabilityof discreteprocesses

45

Lyapunov’s method has also been used to establish convergence conditions for various optimization procedures [3,4] . However, our approach is different from that used in [3,4], and leads to a number of new convergence conditions for minimization procedures, and to a refinement of the type of convergence. We shah also consider the influence of computing errors on the priorities of the procedures. As our model minimization procedure we investigate a version of the gradient method, and show that its convergence properties have a close connection with the properties of the minimized function. As the properties of the function deteriorate, so do the convergence properties of the gradient method; the deterioration is described in precise terms. But even in the case of a piecewise smooth function, the sequence of points constructed by the gradient method has an interesting property, leading to finiteness of the computing procedures usually employed. Similar results hold for some other methods, e.g. for Newton’s method with step adjustment, and the method of variable metric, etc.

1. Stability and convergence with respect to specified variables Consider the discrete dynamic process

~(k+l)=~(dW,

k),

z/(‘k)=q(x(k), k),

(1.1)

where x is the state vector, x E Rfl, y is the vector of variables with respect to which the stability and convergence of the process are investigated, y E Rr, r Q n, k is discrete time, k&= (0, 4, 2,. . .}, p: R”XZ-tR” and q: R”XZ+R’ are given functions, continuous with respect to their first argument. By I x I (or Iy I) we denote a norm in Rn (or R3, while x(k, kg, x0) denotes the trajectory of the process, corresponding to the initial state x0 at the initial instant Let x*0 be a given state, and x’(k) -x (k, fko, x;) ko, i.e. x(ko, kO, zO) -x0. the investigated trajectory. We can assume without loss of generality that q (x(k,

ko,

so’), k) -0,

i’c>ko.

Definition 1. Process (1.1) is called stable with respect to the variables y l, . . . , yr, if, given any e > 0, there exists 6 z-6 (E, ko) >O, such that for any states x0 that satisfy the inequality is the condition I y (k) I = I q (z (k, ko, x0), k) I be 1 y(h) I= 1 q (50, Tko) 14, satisfied for all k Z ko.

If a 6 that is independent of kg can be taken, we say that process (1 .l) is uniformly stable with respect to the variables y 1, . . . , yp Notice that Definition 1 is the same as the definition of Lyapunov stability [l] of the trajectory x*(k) if q (x, *k) =z (k) -x’ (k) . Definition 2. Process (1.1) is called asymptotically stable with respect to the variables

Yl, *. * f y, if it is stable in the sense of Definition 1 and its trajectories are convergent with respect to variablesyl, . . . , y,, i.e., they satisfy the condition

ly(k)

I=Iq(x(k,ko,xo),

k) l-4

k--v

for any states xo lying in the domain of attraction of the trajectory x*(k):

(1.2)

46

KRNomvandKD.iWaaov

1Qbo, W 1GA, where A = A&) is a positive number. Let v(x, k) be a continuous function with respect to x, and let V: R”XZ+R’ and I-‘,- (5: ~(3, k) ko. Below,we denote throughout by oj(u), i--l, 2, Of: R’+Ri continuous non-decreasing functions that satisfy the condition Of

(0)‘0,

u>o.

al(u) ‘0,

(1.3)

Theorem1 The necessary and sufficient condition for uniform stability with respect to the variables y1,...9 yr of process (1 .l), is that there exist a non-negative function v(x, k) such that, for all k E 2 and x E Vk, we have the relations ai(l&,

(l-4)

k) I)
(l-5) (l-6) Roof: Sufficiency. When condition (1.5) holds, the inequality

v (5 (k, ko, a),

k) Qv (a,

k) ,

(1.7)

kako,

holds for all x0 E v&). Let E > 0 be such that 01 (e) < Cl. Then, taking 6 = S(e) such that we fmd, in accordance with (1.4)-( 1.7), that, for all k > ku.

o&)-i(a),

Ml&(lk

ko, a&

k) 1)9v(s(k,

ko, so), k)Gv(s,

ko)

~~~(Iq(~o, ko) I)~‘ol(6)~ar,(e). Hence, since 01 (u) is monotonic, we find that I p (k) I -1 q (5 (k, ko, a), k) IGe, k*o. Necessity. We put v(Z,k)==sup

Iq(s(m,k,E)

,m)l.

Obviously, ~(5, kDlqb(k,Ik, and

N, k)I-

Id% 4 I

(1.8)

Stabtli@ of discrete poce$$es

‘vb(% k), k+l)-

41

sup IqMn, k+l,P@, k)), m) 1 mn)k+i

-

sup Iq(2(n,k,Z),m)l mah+l

Qsup Iq(s(m,k,E),m)l=V(~,k). nr)k Hence function (1.8) satisfies conditions (1.4) and (1 S). Let us show that it satisfies condition (1.6). It was shown by Masser that a continuous and strictly increasing function can be chosen for the function h(e) of Definition 1. Consequently, it has an inverse e(S) such that i.e. inequality (1.6) is satisfied for function (1.8). I!++% k, Z), m) lea(l#, k) I), This proves the theorem. Note 1. It is easily shown in a similar way that the existence of a function v(x, k), satisfying conditions (1.4), (1.5) and the condition

is sufficient for the stability (not necessarily uniform) of process (1.1) with respect to variables Yl,.*-*J+. Notice that we do not need an actual knowledge of trajectory x*(k) to apply Theorem 1. This is in fact the case most commonly encountered when applying stability theory to

optimization problems. For the process

dk++=_pw),

k),

!/(k)--+(k).

Theorem 1 is the same as Lyapunov’s theorem and its converse. In the case of the process aG+l)-_p(s(k),

k),

Y(W=4(~(~),

k),

(1.10)

where k=-ko,

k>ko, Theorem 1 can be regarded as an analogue of Rumyantsev’s theorem on stability with respect to part of the variables [2]. nleomn

2

Let C2 > 0 and the non-negative function v(x. k) be such that sup {u (5, ko) : 1q (z, ko) 1GC,} CC,

(1.11)

V. R. Nom and V. D.Fumov

48

and let Y(X,k) satisfy condition (1.9). Process (1.1) is asymptotically stable with respect to the variablesyl,. . . , yr for any xo of the domain (1.12) if, for all k>ko, XEV~ 9 (1.13) k), k+l) 30, and o,(u)>O. conditions(1.4)and(l.S) follow immediately from relation (1.12). The stability thus follows directly from Note 1. Here, in accordance with (1.1 l), on any process trajectory starting in the domain (1.12), we have

proof.Since u(p(x,

w( Iq(x(k, ko, 501,k) IbWxCk, kotx01,k) -u(x(k+l,

ko, xo), k+i).

Consequently, for k > ko, b Hence

c 4-b

0: (I q (x(4 k,,

xo)90 I ) 4~ (x0, k.) --y (x(k, kor xo), k) 5~ (xe, ko).

c-

oi(lq(x(i,ko,xo),i)O~utxo,k.).

Hence or(]q(x(i,

ko, x0:0>,i) /)-to,

(1.14)

i-t-,

which implies (1.2), since wl (u) is monotonic. This proves the theorem. Let us emphasize that the function V(X,k) in Theorem 2 is merely assumed to be non-negative, and not positivedefmite, with respect to variablesyl, . . . , yp Process (1 .I) is asymptotically stable with respect to these variables with the “integral” estimate (1.14). Notice also that no proposition similar to Theorem 2 holds for process with continuous time. Incidentally, in the case of a bounded function p(x, k). Theorem 2 can be regarded as an analogue of Marachkov’s theorem on the asymptotic stability of process (1 .lO) (see [2] ). In the case when a Lyapunov function, independent of k, can be chosen, our theorem is similar to the Lyapunov theorem on asymptotic stability. In the general case, Theorem 2 yields an analogue of the proposition [5] regarding the integral stability of dynamic systems with continuous time. Note 2. It follows from the proof of Theorem 2 that, if all the conditions of the theorem are satisfied, apart from condition (1.9), then process (1 .l) is convergent with respect to variables yr for any xo of the domain (1.12). Yl..... Example. S- {x: Ix 1O, x(k+O=pi(x(k),

k)+f(k),

Y (k) =x(k)

Stability of discrete processes

49

and the function p 1(x, k) satisfies the condition SE&

IP&, k) I-M

kBko,

This process is asymptotically stable if f(k) =O, k>ko. function u(z, k) = Iz 1 for Ci=C2=so.

a=const
If If(k) I f 0, for at any rate one k > ko, then the process is unstable. However, it will be convergent for any xo such that

This can be seen by taking the Lyapunov function

u(x, k)=IzI+st(k),

x(k)--&

If(i) I. (4

Here we obtain u(pJz,

k), k++=lp&,

+x(k)--If(k)

I~p~(s,

k)+f(k)

I+4k+U=thv

k)+x(k)Gv(z,

k)+f(k)

k)-(l-4

1

1~1.

Let us now consider the convergence and stability of process (1 .l) as a whole. Dejhition 3. Process(1 .l) is called uniformly asymptotically stable on the whole with respect to the variables y 1, . . . , yr if it is uniformly stable with respect to these variables, and its trajectories are uniformly convergent with respect to the variables for any ko E Z and x0 from the domain

I Q (xo,ko)IGA

(1.15)

with arbitrary A > 0, i.e., if, for any E > 0, there exists I(E, A) such that I q(x(k,

ko, zo), k) I ko+l(c,

A)

for all initial conditions from the domain (1.15). Theorem 3 F’rocess(1 .l) is convergent on the whole with respect to the variables ~1, . . . , y, though not in general uniformly so, if there exists a function V(X,k) such that, for all k > ko and x E R”. v (x, k) 20, .

V(P(Z, k), k+%v(z,

(1.16) k)--oi(

I+,

k) I,.

If, in addition, condition (1.6) holds, then the process is uniformly asymptotically stable on the whole with respect to the variables ~1, . . . , y,

I_ SSR

lY,?-D

50

V. R. Nosovand V. D. Fumsov

Roof: for all xo E Rn and k. E 2, it follows from (1.16), in the same way as in Theorem 2, that

(1.17)

From (1.17), we fmd that, since wl is monotonic, )4(&,

ko, x0), i) I-4

i-t-,

i.e. the convergence on the whole is proved. Let us,prove the second part of Theorem 3. Since the conditions of Theorem 1 are satisfied, the process will be uniformly stable with respect to variables yl, . . . , yr. Let the initial conditions be taken from the domain (1.15). Then, by (1.6) and (1.17), we find that

(1.18)

We now take an arbitrary E > 0, and let 6(e) be such that, if I q (x0, ko) I 4 6 (e), then In view of the uniform stability of the process, for k>ko. lq(z(k, k,, a), k) IGe such a number S(e) exists. Nowlet

Z(e, A)>o,(A)/oi(6(e)). ==k,, wehave Iq(z(s, lco,z,,), s) IG3(e),

Then,foratleastone sinceotherwise

~~,(lq(z(i,k~,.~),i)I)~~

k-s,

koGs
A)

o,(lq(5(1,k,,s,),i)l) 1-4

f-r, >Z(e,A)o,(6(e))~ot(A), which contradicts relation (1.18).

Hence it follows, in view of the uniform stability, that, for k>k,=ko-f-Z(e, This proves the theorem. have Iq(z(k, s, z(s, ko, G)), k) [Ge.

A) we

Notice that the function V(X,k) in Theorem 3 does not need to have an indefinitely large lower limit. In the case of continuously acting disturbances, stability for discrete processes can be looked on in a somewhat different light from stability for continuous processes. To understand the difference, assume that, instead of process (1. l), we have

f(k+U=p(f(k),

where g(k), h(k) are small disturbances.

k)+g(k),

Y(k)-_q(l(k),

k)+h(k),

(1.19)

Srabiky of discrete processes

51

If we regard the process of finding ?((k) as an actual process of measurements or computations, then it is not possible for us to find the exact values x(k), and we can only find values T(k) and y(k). Hence it is not actually possible to check condition (1.13). Instead of (1.13), we can check the condition u(l(k+l),

kfl)
k)-o,(

IF(k) I).

But it has to be borne in mind that, due to measurement or computing errors, the condition checked in practice is

where a(k) is a vector characterizing the influence of the errors. It is natural to assume that la(k)

IGS,

k&.

Stability of this kind may be termed numerical stability of process (1.1). Theorem 4

Let r(k), 7 (k) be defined from (1.19), let V(X,k) > 0, and u(iZ(k+l),

k+l)
k)-o,(

ly”(k) j)+a(k).

(1.20)

Then, given any E > 0 and y > 0, there exists a number 6(e, 7) such that, for any n >N(xo, E, r), the number of indices I, for which

Iv”(Z)1>e,

(1.21)

will not exceed r(n - ko), provided that (1.22) fioo$ We take arbitrary E > 0 and y > 0. It follows from (1.20) that

Hence

(1.23)

Consider the set of the indices I, ko < I < n, for which (1.2 1) holds, and for which Let the number of such indices be not less than r(n - kg). IW) PbU) I-IW b-6. It then follows from (1.23), in the light of (1.22), that

52

K R. Nom and

V. D. Fumov

y(ra-ko)o~(e--6)-~(n-ko)~V(lo, and n>iVi-ko, where For 6SO.570~ (s-6) lea& to a contradiction. This proves the theorem.

(1.24)

ko).

N>2v (x0, ka) /roi (e-6))

(1.24)

in view of Theorem 4, we can assert that, if the computational errors a(k), h(k) are sufficiently small, then the values x” (2, ko, xo), satisfying (1.2 l), will only rarely be encountered among the values x(k, ko, x0) of process (1.19).

2. Application of stability theory to the study of optimization methods Consider the gradient method of unconstrained minimization

(2.1)

t(k+l)--s(k)-~(k)P(s(k)), where the step A(k) is chosen in such a way that 0 < h (k)
f(+(k)-r(k)t’fz(k)))~~(,(k))-~

]f’(s(N

> I’

(2.2)

Let us examine the properties of this method in terms of the properties of function Ax). Assume initiaily that fix) E @(I@) and that it is a strongly convex function. We know [6] that the matrix f’(x) of such a function is positive deftite, i.e.

(%)P,

PPM,

P),

e-0,

Vx, PER”,

(2.3)

and that Ax) has a unique minimum point xx Assume that the matrix f’(x) is bounded: IfwPI~~lPl

Vx, PER”.

(2.4)

Lemma Assume that j(x) E 62(P)

and that conditions (2.3), (2.4) hold. Then, for all x E R”,

where v(x) = fTx) -fix*). 14ooJ By Taylor’s formula, noting that f(x*) = 0, we obtain p=~.+e

(X-Z.),

Strrbilityof discrete processes

53

Hence it follows that

cm

From the formula of ftite increments

it follows that (2.7) Since

then

whence

?I s-s.MIf’(5)

1.

6931

On substituting (2.7) and (2.8) into (2.6), we obtain the lemma. Proposition 1. Let fix) E @Rn) and let conditions (2.3) and (2.4) be satisfied. Then, process (2.1) in which the step is chosen from condition (2.2) is uniformly asymptotically stable on the whole with respect to the variable Y(k)=q(z(k))e;f’(2(k)). Also, the following convergence rate estimates hold:

If W)

) -fM

I Salk

(2.9)

hoof:

From Taylor’s formula

I(z(k)-l(k)f’(s(k)))PP~(X(k))-h(k) lf’(wa

I”.

+~y”(E)f’(Ef,f’(r(k))) G f(Z(k))+ it follows that, for

[ k2(;)M

1/2MGh (k) <1/M

the

-wcq

If’(r(k)N”

step X(k) satisfies condition (2.2).

54

V.

R. Noxov and Vi D.Furasov

We can write condition (2.2) as v(z(k+l))Gu(s(k))-

+lf’(z(k))

I’=Wz(k)) (2.10)

-&lf’(s$k))

Ia-u(5(k))-o,(lf~(2(k))I),

where 01(u) = u2/4M On now applying Theorem 3, we obtain the first part of the proposition. By (2.8) and the lemma, we have

(2.11)

where d-i-ma/8W<1.

Inequalities (2.9) now follow directly from (2.6) and (2.11).

Reposition 2. Let Ax) E 62(F) be a lower-bounded function, and let x(O) be chosen in suchawaythat,inthedomain D=-{z: f(z)<;f(z(O))} thenormofmatrixf’(x)is Then, process (2.1), with the step chosen in accordance with bounded: Ilf” (5) il
a g

I*GM[f(s(O))-

inf{f(z):

z=R”) 1.

Proof. In the same way as in Proposition 1, we can show that, under the conditions of proposition 2, the step A(&)in condition (2.2) can be taken in such a way that A(k) > (2&f)-‘. and apply Note 2, whence our proposition follows. We put u(z) -f(s) -inf {j(z) : ad?‘}

Nowlet f(z)=CO(R*), f’(s) and let f”(x) have a ftite number of surfaces of discontinuity in every compact domain of Rn; and let Ax) be lower-bounded in Rn. Denote the class of such functions by D2. To seek the minimum of functions of D2 we can we a modified gradient method. We construct a sequence of points x(k) on the basis of the expressions

x(k+l)--s(k)--h(k)g(k),

k(k) l=lf’(4W I, We choose the step h(k), point x(k + l), and also

OGh(k)G4,,

~0s

(g(k), Pb(k)))~%.

(2.12)

(2.13)

in such a way that f(x) and f”(x) exist at the

IP(Nd) I’. f(z(k)-h(k)g(k))~f(,(k))-~

(2.14)

This is the version of the gradient method in which the.descent is in a direction g(k), sufficiently close to the direction of the gradient f ’ (x(k)). If f‘(x(O)) and f(x(0)) exist, then the method

Stability of discrete processes

55

(2.12)-(2.14) will in general construct an inftite sequence of points x(k). For, at every point x(k). For, at every point x(k), condition (2.13) defmes a cone of points, at which we can pass to the next step. For small X(k), using Taylor expansion, we can show that (2.14) is satisfied. Since, moreover, the measure of the set of points at which f(x) and f’(x) do not exist, is zero, then clearly, we can always choose X(k) and g(k) in such a way that all the conditions are satisfied. Proposition

3. Let j(x) E 02, and let f(x(0)) and f’(x(0)) exist. Then, process (2.12j42.14) ~.~nvergent~~re~ctto~ev~ble y(k+l)=ls(k+l)--s(k)f. Proof: Taking

u(z) =f (s) -inf

{f(s) : x&?“),

v(s(k+l))~~(z(k))--4rf’(rfkf)

we can write (2.14) as h(k)

I”.

Hence, in the same way as when proving Theorem 2, we find that

cOa

h(k) -+‘(s(k))

J”~u(x(O))
k-0

Hence

3L-(4IPWW I”iO,

k+=.

But ly(k+1)

]+(k++-s@)

<[h’(k)

If+(k))

I=]WWk)

~“]“ei,“[h(k)

If’@(k))

I [‘F-O,

k+-.

This establishes the prounion. iVore 3. It is recommended in [7] that rules be used, whereby process (2.12) is stopped when one of the quantities ]z(k+l)-z(k) f, ]f(z(kfl)).-f(z(k)) 1, 1s(k+I)-z(k) l,+(k) 1, If ~z(k~~))-Liz} If&(k)) f or a combination of them is less than a preassigned number r > 0. It follows from our Proposition 3 that, when using such stopping rules, algoriti (2.12)-(2.14) gives a ftite sequence of points x(k) for every function j(x) E 02. A method of fastest gradient descent is often considered, in which the points x(k) are constructed in accordance with relation (2.1), while the step A(k) is chosen from the condition

Let us show that, for the method of fastest descent (2.1), (2.19, propositions similar to Propositions 1 and 2 hold. Let condition (2.4) be satisfied. It then follows from Taylor’s formula that

56

(2.16)

=

ftf(k)f--ig:k) If’(r(k)) I”,

where B(k) <&I fitting

where

V(X) =f(X)

~(~(~))

6 -#‘(r(k))

-inf {f(l)

: XER”),

we can write (2.16) as

u(~(k))-@ltl&tw and

0,

al(u) ==a2/2M.

On now applying Theorem 3 or Note 2, we fmd that Proposition 2 and the first part of Proposition 1 hold for the process (2.1), (2.15). Here, instead of estimate (2.9), we have the estimate (2.17) where all the numbers d(i) satisfy the condition

For, on putting

u(z) a!(r)

-f(&),

we obtain from (2.16) and the lemma:

and

Ei)


Hence we immediately obtain estimate (2.17), which implies that the method of fastest descent (2.1), (2.15) has a linear rate of convergence.

srrrbtiityof diwreteprocesses

57

Now let the ~tion algorithm (2.1), (2.2) be realized with certain computational errors rather than exactly. In particular, let f’(x) be computed by means of difference ratios. In fact, assume that we compute the vector 7 (x) with components f G+pe’)-f(s)

,

f(s+pe”)

-f(t)

1. * *,

f (s+pe”)-1 (.z) P

P

P

and that (2-l), (2.2) are replaced by the relations

=j(iiqk))--lj’@((k)) yf

P+a(k).

In these circumstances, Theorem 4 can be applied. We then fmd that, given any E > 0, we can indicate 6, y, such that, if

then, in a sufficiently long sequence of points x(O), . . . , x(n), the majority of points (the number of them is not less than (1 - r)n) will satisfy the condition I f(x(k)) I < e. In other words, the gradient method (2-l), (2.2) is numerically stable in this sense. Note 4. Propositions similar to Propositions l-3, and arguments concerning numerical stability, are applicable to more general unconstrained minimization algorithms of the type

in which the matrix @k) is positive defnite, or else the vector p(k) makes an acute angle with the vector f(x(k)). Here, the step X(k) has to be chosen from the condition (f’(Z(k)),P(M). Algorithms of this hind embrace Newton’s method with step adjustment, and methods of variable metric, etc. The modifications needed to prove these assertions are obvious. ;T)rmsllatedby D. E. Brown.

REFERENCES 1.

HALANAY, A., and WEXLER, D., Quuhrive theory of mmpled datasyaems (Russian translation) Mu, Moscow, 1971.

2.

OZIRANEB, A. S., and R~YA~SEV, V. V., The method of Lyapunov functions in the problem of the stability of motion with respect to part of the variables,Bikl. mot.mekhan, No. L&364-384,1972.

S. K. Zavriev

58 3.

ZANGWILL,V. I., Non-linear ptognrmmfng, A unified appnrrrch, Russiao translationSov. radio, Moscow, 1973.

4.

EVTUSHENKO,Yu. G., and ZHADAN, V. G., Application of the method of Lyapunov functlonr to the study of the convergence of numerical methods, Zh. vjkhisl. Mar. mot. Fk, 15, No. 1, lOl-112.1975.

5.

FURASOV, V. D., St&l& ofmotion, estimation und stabflircrfion(Ustoichivost’ dvizheniya, otsenki i stabilizatsiya), Nauka, Moscow, 1977.

6.

PSHENICHNYI,B. P., and DANILIN, Yu. hi., Numericaf methods &Iextremul problems (Chislennye metody v ekstremal’nykh zadachakh), Nauka, Moscow, 1975.

7.

HIMMELBLAU,D., Applied non-linear ptvgrumming Russian translation Mir,Moscow,1975.

U.S.S.R. Comput. Maths. Math. pylys. Vol. 19, pp. 58-72 8 PergamonPressLtd. 1980. Printedin Great Britain.

HYBRID PENALTY AND STOCHASTIC GRADIENT METHOD FOR SEEKING A MAX-MIN* S. K. ZAVRUW Moscow (Received 6 April 1978)

AN ALGORITHM for seeking a max.min in nonconvex

problems based on a computation of the penalty method and the stochastic gradient method is proposed. The convergence to the set of stationary points, with probability unity, is proved.

1. Formulationof the problem Consider the problem of seeking the max-min u. = maxminF(t, x I

y)

(1)

and the best guaranteed strategy x0 E X: tzg -

minF(xO, y).

(2)

P

In the case when the function F(x, JJ) is continuous in X X Y, X C En, X is a convex compactum, Y C El, and Y is a compacturn, problem (I), (2) reduces to seeking

m=b--c&(z, 24)I

XXU

a.9

C,t+=

(see [l]). Here

w,

u)=-

5Imin(O;F(r,

T

y>-u) lqp(dy),

~34,

and the measure p in En is such that any nonempty intersection of Y with an open set has positive measure (it wiJl be assumed throughout that p is concentrated in Y): *Zh. vjkhisl. Mat. mat. Fiz., 19,2.329-342.

1979.