The penalty method for problems of convex programming

Problems of convex programming

97

REFERENCES 1.

SAMYLOVSKII, A. I., Simulation method of synthesizing the structure of a dynamic object, 7’~.MFTZ, Ser. aerofiz. iprikl. matem., 228-235, 1916.

2.

SAMYLOVSKII, A. I., and SUSHKOV, V. G., Construction of the characteristic function of a non-convex set and the related problem of quadratic integer-valued programming, Zh. vj%hisl.Mat. mat. Fiz.. 18, No. 2,322-331,1978.

3.

FINKEL’SHTEIN, Yu. Yu., Approximate methods and applied problems of discrete programming, (Priblizhennye metody i prikladnye zadachi diskretnogo programmirovaniya), Nauka, Moscow, 1976.

4.

ROZENFEL’D, B. A., Multi-dimensionalspaces (Mnogomernye prostranstva), Nauka, Moscow, 1966.

5.

KOLMOGOROV, A. N., and FOMIN, S. V., Elements of the theory of functions and functional analysis (Elementy teorii funktsii i funktsional’nogo analiza), Nauka, Moscow, 1972.

6.

PONTRYAGIN, L. S., Foundations of combinatorial topology (Osnovy kombinatornoi Nauka, Moscow, 1976.

U.S.S.R. Comput. Maths. Math. Phys. Vol. 19, pp. 97-106 0 Pergamon Press Ltd. 1980. Printed in Great Britain.

topologii),

0041-5553/79/0801-0097$07.50/O

THE PENALTY METHOD FOR PROBLEMS OF CONVEX PROGRAMMING* M. A. SHEPILOV Vladimir (Received 21 Febnmy

1978; revised 26 January 1979)

A METHOD for solving problems of convex programming, related to the penalty and loaded functional methods, is described. The convergence and rate of convergence of two versions of the method are studied. The general scheme of the penalty method for solving problems of mathematical amounts to constructing

for the problem a sequence of functions,

multiplier

xk, and then constructing

functions.

With these constructions

on a penalty points of these

we achieve the convergence relations

lim s(&) i@

=z*

or

where x* is the solution of the initial programming While the underlying

dependent

the sequence x(xk) of absolute minimum

programming

lim 2 (h,) =z’, kk-trn problem.

idea is simple and the methods are easily realized numerically

as distinct from many other methods, have computational

stability,

and,

nevertheless serious difficulties

arise when proving their convergence, due to the unbounded or increase in the penalty multiplier. To avoid this drawback, it has been proposed (see e.g. [l] ) that the variables of the dual problem be used as penalty multipliers (methods linked with modified Lagrange functions).

*Zh. vfihisl. Mat. mat. Fiz., 19,4, 889-895,

1979.

98

M. A. Shepilov

On the other hand, the penalty multipliers may be taken from a numerical sequence convergent to the optimal value of target form [2]. Fairly recently, this method came to be known as the “loaded functional method” [3] .

01 Consider the problem of minimizing the convex function j(x) in the convex set Q:

f’= min f(s),

(1)

XER”.

==Q

For the set Q we construct the penalty function g(x) such that

g(z) 50,

xw.

XEQ, !&p-o,

We assume that a lower bound 01offix) is known in the set the function

cPw= min g(x) f(x)=1

Q, a
VtE[a,

(2)

and we consider

PI,

(3)

w=R’,

(4)

PI.

(5)

the system of equations

grad g(x) +w grad f (x) =0, f(x)=1 and the solution of this system

h(t), w(t) 1

VtE[a,

Lemma 1 If the following conditions hold: (a) functions g(x) and fl x ) are convex and twice continuously differentiable in Rh; (b) the matrix of second derivatives of g(x) is positive definite for any x 6! Q:

yTg(x)

p-0

vy+o;

99

Problems of convex programming (c)

system (4) has a solution for any fared r E [ a, /ii]

(d) grad f( IC)+O

Vx=Rh,

then we can claim: 1) a unique smooth solution (5) exists; 2) w(t)>0

v’t= [a, p) ;

3) (p(t) is a decreasing, convex, twice differentiable function for any t E [(w,0); 4) w(P) =cp-‘(P) =cp-Il (B) =o. Proofi It follows from condition a), c), and d) that problem (3) is equivalent to the problem min g (5)

;

f(X)&

it is easily shown that w(t) > 0 for any c E [a, /3),and that all the conditions of Theorem 6 of [4] , p. 52, are satisfied, whence follows the uniqueness and differentiability of the trajectory [x(0,401 It

formy

t E [ar, LO-

follows at once from definition (3) that (p(t) = g(x(t)), and hence

(6) Multiplying the first of Eqs. (4) scalarly by x’(t), we obtain

(gradg(s(t)),s’)+w(gradf(x(t)),z’)=O.

(7)

Differentiating the second of Eqs. (4) with respect to t, we obtain

(gradf(s(l)),

z/)=1.

(8)

V’t~]% P).

(9)

From (6)-(8) we obtain q’(t) =-w(t)

Consequently, q’(f) < 0 for any t E [a, fl), by assertion 2). Differentiating the first of Eqs. (4) with respect to t, we get

V’gs’+w’(t)

gradf(r(t))+w(t)

V’fx’=O,

(10)

M.A. Shepilov

100 where V2g and

Vzf denote the matrices of second derivatives of g(x) and f(x) at the point x(t)_

Multiplying (9) scalarly by x’(t) and recalling Eq. (7), we get

-w(t)

=(d)q

Tg+w(t)

V”f]Ld.

(11)

Notice that x’(t) f 0, since the contrary would contradict Eq. (7). From conditions a) and b) and Eqs. (9) and (1 I), we finally obtain cp”=-w’(t)>0

VP=

[a,p>.

We obtain the equation cp(p>= 0 from (3), by putting t = fl=f*. For, from condition c), system (4) has in this case a solution (x* , w*), and obviously, ASS= (3: 1 (z) =f’, ZEQ} #@. But cp(t) =g (5 (t) ) =O, if x(t) E Q, by definition (2). It is also obvious that w* = 0, since the contrary would contradict Eqs. (4), since grad by condition d). We obtain the equation g(x*) = 0 by definition (2), while grad f(S) ZO by passing to the limit as t + p in (8). The equation is obtained rp-” @) =o cp-‘(B) =o also by passage to the limit, in (11). The lemma is proved. Let us define the procedure for finding the roots of the function q(t);

&=,a,

@A-l)/fp'(tA-l),

tA=tA-i-9

(12)

where (Yis a lower bound offlx) in the set Q. If the conditions of Lemma 1 hold, then the approximations tk are defined for all k, and process (12) is obviously Newton’s method for finding the root p of the equation @(t) = 0, or the method of simple iteration for finding the root of the equation t=$( t) =t-cp (t) I$( t) . Let us prove the convergence of process (12). Lemma 2

If the conditions of Lemma 1 hold, then lim

tA=r.

A+CC

Proof: By Lemma 1, the function (p(t) is decreasing and convex, and hence it easily follows that, for all k, .

tAptAd,

tA<(f’.

101

Problems of convex programming

is monotonic and bounded, it has a limit t* < f*. Assume that Since the sequence {tk} r* < f‘. Then, passing to the limit in (12), we obtain

cp(t*)lcp’(t*)=o, but this contradicts the properties of the function (p(r) proved in Lemma 1, according to which for all r 0, $0) (0

On the basis of the above assertions, we can construct the following method for solving problem (1). We fm a lower bound to of the function j(x) in the set Q, and assume that an approximation rk has been found. We form the system of equations grad g (5) i-w grad f (2) =O,

and find its solution

[ zk==z ( tk), wk= w ( tk) J ;

f(x)

-t,=o

(13)

we put

(14)

Notice that this method of constructing the sequence since g (xk) =V (LA), and -wR=$(t,,). Theorem

it,,)

is identical with method (1 l),

1

If the conditions of Lemma 1 hold and the set S- {x: f(x) =f’, for method (13) and (14),

x=Q}

is bounded, then,

lim p (xk, S) = lim min Ilxk-yjI =O. k-rcu k-brn $I=.$ Proof: By Lemma 1 and 2, the sequences {f (x,)} = {tk} monotonic, and we have

lim f h+m

(xd ==f’,

and

lim g (xk) =O. k+m

(15)

{g(xk)} ={cp( t,,)}

are

(lo)

102

M. A. Shepilov

Since sequences {j(zk) } and (g(zQ} are monotonic, and the set S is bounded, the sequence {Xk} must be bounded. Let Y be a limit point of sequence {Q} . From (16) we obtain the equations f(Z) =f(x’)

g(E) =o.

=f’,

But in this case, it follows from (1) and (2) that is identical with (15).

S ES-

{x: ;E(x) =f,

xEQ}.

This inclusion

NO&Y.1. In method (13), (14), we can avoid comput~g the Lagrange m~tipliers wk. Multiplying the first of Eqs. (13) scalarly by grad jI&) or by grad &Q), we obtain

WA =

-

(gradg@A),grad f(a) )

Wad ghl II2

llgradf@~k) 11’

= - (gradg(sA),gradf(zA))'

Method (13), (14) is a form of the penalty method, the role of penalty multipliers being of absolute minimum points played by the Lagrange multipliers wk. For, the sequence {x4) of the penalty function F (5, wk)=g (5) + wd (IZ) will be convergent to the solution of problem (1) for a fured sequence (wk}, for which lim w4 A-+Ul

=

0,

However, when solving the equations grad g(S)

fwA

grad f(x)=0

(17)

numerically, different difficuities arise in connection with the decrease of the multipliers wk. It was shown in [5] that, when gradient methods are used to solve system (17), convergence may not be obtained if the sequence decreases too rapidly. The convergence of gradient methods for solving equations of type (17) was investigated in [6,7] , and conditions were stated, which have to be satisfied by the sequence of penalty multip~ers*,The conditions to be imposed on a fured sequence of penalty multipliers strongly restrict the rate of convergence of these methods. In method (13), (14), the parameter t regulates the variation of the penalty multiplier w, ensuring that the method has quite rapid convergence. Below we describe a modification of method (13), (14), whereby we get rid of the penalty multiplier w.

93 We defme the function J/(x, t) as foliows: f(x)<--(t), q =o Vxdt={x: 2c,(z,q =W@) 41 Vew, 9 (4

(18)

*There is a misprint on p. 743 of [6 ] when stating the convergence conditions; condition 5 should read as foUows: 5. Series Zhn ’ is convergent, while ZAn qn is divergent.

Problems of convex programming

103

where h(o) is an increasing function of the variable OL,such that h(0) = 0, and we consider the problem

min {gb> +Q (5, t>1. XERh

(19)

Lemma 3

If conditions a), b), and d) of Lemma 1 hold, along with the conditions: e) function $ (x, t) is convex and twice differentiable with respect to x; f) problem (19) has a solution for any f E [rr, fl] ; then there will exist in [CY, fi) a unique smooth trajectory x(t), which is a solution of the system of equations grad g(z) Sgrad $ (5, t) =O. The present lemma, like assertion 1) of Lemma 1, follows at once from Theorem 6 of [41> P. 52. Along with (19) we consider the problem

(20)

min g(s) f(x)-f(=(t)) where x(t) is the solution of problem (19). We introduce a parameter r, connected with the parameter t by the equation r = fix(t)), and the function q(a)=

min g(z). f(s)=?

(21)

Lemma 4

If the conditions of Lemma 3 hold, we can claim:

2) (P(T)is a decreasing, convex, twice differentiable function for all a~ [a, p) , a< P=f’; 3) cP(ft) =cfJ-‘(f’) =O; (grad g (z (t) 4) cP(r)=g(.N),

cP’(t)=-

) , grad

(Igrad fb(t)

f (z (0 ) 1 > II2

’

Proof: Assertion 1) follows at once from definitions (18) and (19). To prove the other assertions, we have to show that the solution x(r) of problem (21) is the same as the solution x(f) of problem (19), or what amounts to the same thing, show that the solution y(r) of problem (20) is equal to the solution x(t) of problem (19).

M. A. Shepilov

104

In fact, by definition of x(t), we have, for fxed t E [a, /3],

(22) Next, it can easily be shown that f(x(t)) > t for all t E [a, /3), and hence we find that

1I)(Y(t),t)=1C,(s(t),t)=h[f(J:(t))-tl, i.e. by definition, fi(t))

=&c(t)).

We then obtain from (22):

(23) But, by definition of y(t), we have

Vn=Bt={IC: f(s) =f(r(t))}.

g(!l(t))%l(z)

(24)

From inequalities (23) and (24), and the uniqueness of the points x(t) and y(t), we find that x(t) =y(t). Assertions 2)-4) now follow from Lemma 1. The lemma is proved.

We specify the method for solving problem (1) as follows. Let xk be absolute minimum points of the functions g(x) +9(x:, tk) , g (d

+11,(G, h) = min {g Cs) +$ b, b) 1, XEP

(25)

(26)

wf+= -

(grad g (&) , grad f bk) ) (27)

Theorem 2

If the conditions of Lemma 3 hold, and the set S is bounded, then we have for the method (25)-(27): lim o (zk,

s)

=o.

k-.m

Proof: Consider the function (p(r), where r =Xx(t)) (x(t) is the solution of problem (19)). By Lemma 4, this function is decreasing, convex, and twice differentiable for all 7~ [cz, fi) , cdB=f’,

Problems of convex programming

105

and process (26) is Newton’s method (or method of simple iteration) for finding the root f* of the function. For by definition, f (z,)‘=f (5 ( tk) ) =zk, g ( zR) =rp (TV), wk=(p’ ( .GJ . Moreover, for all r O, cp’b)
lim tk= lim rk=y, k-m

k-em

and hence we also have the equations lim f(&)= k-+m It is easily seen that the sequence as the proof of Theorem 1.

lim rk=fl,

lim g (zk) = Iim q ( rk) =o.

k-+m

k-+m

{rk}

k-co,

is monotonic, and the rest of the proof is the same

Notes. 2. The rate of convergence of method (25)-(27) can be estimated if we require that function (p(r) have somewhat stronger properties than those demanded in Lemma 4.

3. It may seem at first sight that the conditions of our propositions are unnecessarily rigid. We shall therefore give an equivalent of the conditions of Lemma 1. Let

Q={x: gj(s) GO; j=1, 2,. . . , m}r S(.Z)>O

for all 22-0, 6(z) =0

for all

~40, z=R’, hj(~)=6kj(s)),

g(X)= +

hj(X).

The conditions of Lemma 1 will be satisfied if: 1) Ax) and g&x) are convex and twice differentiable in Rh , 2) S(z) is a non-decreasing, twice differentiable function in RI, 3) 6(z) is strictly convex in G = {z: a>O}; 4) the set Q is bounded, 5) grad f(r) =+O

BxER”.

Notice in conclusion that (25)-(27) is a penalty method. But it does not require that the value of the penalty multiplier tend to zero or infinity. The penalty functiong(x) + $(x, t) reaches its absolute minimum on the solution of problem (1) for a finite value of the penalty parameter t, with the approach to this value taking place automatically in the process (25-(27). Notice also that method (25)-(27) is suitable for solving non-convex problems. To justify it theoretically in this case, we need the existence of a locally unique isolated absolute minimum x(t) of problem (19), satisfying sufficient conditions for all t E [a 0). Translated by D. E. Brown.

N. S. Kukushkin

106

REFERENCES 1.

POLYAK, B. T., and TRET’YAKOV, N. V., On an iterative method of linear programming, Ekonomika i matem. metody, 8, No. 5, 740-751, 1972.

2.

KOWALIK, J., OSBORN, M. R. and RYAN, D. M., A new method for constrained optimization problems, Operat. Res., 17, No. 6,973-983, 1969.

3.

LEBEDEV, V. Yu., A scheme for seeking the approximate solutions of a problem of convex programming, Zh. v$hisZ. Mat. mat. Fiz., 17, No. 1, 100-108, 1977.

4.

FIACCO, A., and MCCORMICK, G., Nonlinear programming,Wiley, 1968.

5.

SHEPILOV, M. A., Continuous analogues of the penalty method for problems of convex programming, Ekonomika imatem. metody, 11, No. 1, 130-140, 1975.

6.

SHEPILOV, M. A., A generalized gradient method for problems of convex programming, Ekonomika imatem. metody, 11, No. 4,743-747, 1975.

7.

SHEPILOV, M. A., A generalized gradient method for extremal problems, Zh. @h&Z. Mat. mat. Fiz., 16, No. 1,242-247,1976.

U.S.S.R. Comput.Maths.Math. Phys. Vol. 19,~~. 106-121 0 Pergamon Press Ltd. 1980. Printed in Great Britain.

0041-5553/79/0801-0106$07.50/O

NON-COOPERATIVE THREE-PERSON GAMES WITH FIXED HIERARCHICAL STRUCTURE * N. S. KUKUSHKIN Moscow (Received

3 May 1978)

A DEFINITION, generalizing Germeier’s definition for a two-person game, is given for a three-person game with fured hierarchical structure. The (fust player’s) optimal informational extension of the initial game is constructed and the determination of the first player’s best guaranteed result and optimal strategy is reduced to solving a series of optimization problems. The influence of the players’ information patterns on the result of the game is studied.

1. Introduction 1. The concept of a game with fixed sequence of moves (a hierarchical game) was proposed by Germeier [l] , and applications of the concept to the theory of hierarchical systems were indicated in [2-5 ] . While similar discussions, for three or more person games, are to be found in [6-91, in the situations considered only one specific player acts, while the rest either combine in a coalition or else maximize their own pay-offs independently, without trying to influence their partners’ choices. In either case, the hierarchical structure of the game is two-level, i.e. multi-level systems are not considered. In the present paper we generalize Germeier’s hierarchical solution to noncooperative n-person games, so that multi-level hierarchical systems can be analyzed from the game-theoretic stand-point. For simplicity, we in fact confine ourselves to the case n = 3, specially since nothing essentially new is obtained if we take more than 3 players. *Zh, v&h&l. Mat. mat. Fix, 19,4, 896-911,

1979.