Convex Constrained Programmes with Unattained Infima

Journal of Mathematical Analysis and Applications 234, 675᎐694 Ž1999. Article ID jmaa.1999.6407, available online at http:rrwww.idealibrary.com on

Convex Constrained Programmes with Unattained Infima Wiesława T. Obuchowska Department of Mathematics, Uni¨ ersity of Nebraska at Omaha, Omaha, Nebraska 68182 Submitted by E. S. Lee Received November 2, 1998

We consider a problem of minimization of convex function f Ž x . over the convex region R where the objective function and the feasible region have a common direction of recession. In cases when one of these directions is not in the constancy space of the objective function, then the minimal solution is not achieved even if the function f Ž x . is bounded below over the region R. Many algorithms, if applied to this class of programmes, do not guarantee convergence to the global infimum. Our approach to this problem leads to derivation of the equation of the feasible parametrized curve C Ž t ., such that the infimum of the logarithmic penalty function along this curve is equal to the global infimum of the objective function over the region R. We show that if all functions defining the program are analytic, then C Ž t . is also an analytic function. The equation of the curve can be successfully used to determine the global infimum Žin particular, unboundedness. of the convex constrained programmes in cases when the application of classical methods, such as the steepest descent method, fails to converge to the global infimum. 䊚 1999 Academic Press

1. INTRODUCTION We consider the problem minimize f 0 Ž x . subject to: x g R s  x g R n < f i Ž x . F 0, i g I s  1, 2, . . . , m4 4 Ž 1.1. where f i Ž x ., i g I j  04 , are analytic convex functions with unbounded level sets. For simplicity of notation the objective function will also be 675 0022-247Xr99 $30.00 Copyright 䊚 1999 by Academic Press All rights of reproduction in any form reserved.

676

WIESŁAWA T. OBUCHOWSKA

referred to as f Ž x .. We assume that intŽ R . / ⭋. The problem Ž1.1. either has a finite infimum Žalthough there may be no such point at which f Ž x . achieves its minimum., or the function f Ž x . is unbounded from below w2᎐8x. It is well known w11x that the minimum of the convex problem of the form Ž1.1. is not achieved only if the objective function and the feasible region have a common direction of recession. In this case the convex program Ž1.1. is called degenerate. Abrams w1x proposed a method to solve the degenerate problem of the form Ž1.1., with the assumption that the problem is bounded below and that the set m

Cs

F 0q f i is0

of the vectors of recession of the problem is known. The method relies on replacing the original problem with a reduced problem defined by projecting the feasible region R and the epigraph of f Ž x . onto the orthogonal complement of the recession cone of the problem. The projection procedure may be repeated in case the reduced problem is still degenerate, until the objective function and the feasible region of a canonical problem have no directions in common. One of the difficulties in the above approach is that it requires knowledge of the cone of recession of the problem Ž1.1.. Also, the reduction procedure is known in only a few cases: posynomial, geometric programming and quadratic programming and l p-approximation w1, 4x. Furthermore, the procedure fails if the objective function strictly decreases along some direction vector in the cone C. Indeed, if every vector in C belongs to the constancy space of f, then the problem Ž1.1. achieves its infimum, which may be determined by the method in w1x. Our method of the solution of the problem Ž1.1. relies on derivation of the equation of the feasible trajectory such that the infimum of the modified logarithmic penalty function along this curve is equal to the infimum of f Ž x . on R. We will call such a curve an infimal trajectory. We do not impose any restrictions on the cone of recession of the problem Ž1.1.. However, we assume that the direction vector p g C along which the function f Ž x . strictly decreases is given. We also do not impose an assumption on boundedness of f Ž x . over R, which implies, in particular, that the function may be divergent to y⬁ along the trajectory. The method proposed in this paper is an extension of the approach proposed in w10x for the unconstrained minimization problem, where we have derived the equation of the parametrized analytic curve, along which the objective function converges monotonically to its global infimum.

CONVEX PROGRAMMES WITH UNATTAINED INFIMA

677

2. PRELIMINARY RESULTS It is well known that if a closed, proper, convex analytic function is constant along some half-line with a direction vector d, then it is constant along any half-line with this direction. The set of vectors with the latter property forms what is called constancy space of f Ž x ., which is denoted by w x Ž . Ds f , 7, 11 . A vector s is called a direction of recession of f x if for every x the function f Ž x q ts . is a nonincreasing function of t w6, 11x. Since any proper lower semicontinuous convex function is a closed function, then Theorem 8.6 w11x implies that if f Ž x q ts . is nonincreasing for even one x g R n, then it is nonincreasing for every x. We assume in this paper that the intersection of the cones of recession of f and f i , i g I, denoted by 0q f, and 0q f i respectively, is nonempty. The lineality space Ds of f Ž x . f may be defined in terms of the set 0q f w11x as n< q q Ds f s  ygR yg0 fnyyg0 f4.

DEFINITION 2.1. Ži. We say that the function f Ž x . decreases asymptotically along the half-line x Ž t . s x q ts, t G 0 if it has a finite infimum but not a minimum along this half-line. Žii. We say that the vector s g 0q f is an asymptotic direction vector if f has a finite infimum but not a minimum along every half-line with the direction s. Let Dfa denote the set of all vectors s with the above property. Žiii. We say that the vector s g 0q f is a direction of unboundedness, if the function f is unbounded below along every half-line with the direction s. The set of all vectors possessing this property will be denoted by Dfu. LEMMA 2.1 w9x. Ži. If the con¨ ex function f Ž x . is unbounded below along the half-line x aŽ t . s a q ts, t G 0, then for e¨ ery x g R n, f Ž x . is unbounded below along a half-line x Ž t . s x q ts, t G 0 and along a half-line x Ž t . s x q ty, t G 0, for any y g rintŽ0q f .. Žii. Let us assume that f Ž x . g C⬁ . Then, if f Ž x . asymptotically decreases along a half-line x aŽ t . s a q ts, t G 0, then it also asymptotically decreases along any half-line with the direction s. u a u a Žiii. 0q f s Ds f jDf j Df , and Df l Df s ⭋. ⬁ q a Živ. If f g C and rintŽ0 f . l Df / ⭋, then e¨ ery ¨ ector s g rintŽ0q f . is an asymptotic direction. LEMMA 2.2. Let us assume that the function f is con¨ ex, and f g C⬁. Then f achie¨ es an infimum o¨ er R if and only if C ; Ds f . Proof. The backward implication follows from Corollary 27.3.3 w11x. To prove the forward implication, let us assume that x 0 be the point at which the objective function achieves minimum over the region R. Thus for any

678

WIESŁAWA T. OBUCHOWSKA

s g C, f Ž x 0 q ts . s f Ž x 0 . for t G 0. Since f g C⬁ the latter equation s implies that s g Ds f , and consequently C ; Df . We assume in the remaining part of this paper that the vector p g Dfa l C Ž5 p 5 s 1. is given. We also assume that Ds f lC s ⭋. Otherwise, if w x Ds f lC / ⭋, we can apply the algorithm in 7 to determine the constancy s. Ž space Ds f lC or Df , and consequently replace the original problem by Žor Ds its orthogonal projection onto the constancy space Ds f lC f , . Ž . respectively . Since the problem 1.1 has an unattainable infimum, the cone of recession of the new problem will contain only asymptotic directions Ž Dfa . and possible directions of unboundedness Ž Dfu ., for which, by Lemma 2.1, Dfa l Dfu s ⭋. In the lemma below we assume that yi g R n, i s 1, 2, . . . , n y 1, are linearly independent, although more specific assumptions on these vectors will be made later. Let us define Y s Ž y 1 , y 2 , . . . , yny1 . g R Ž ny1.=n. Let x 0 g R be arbitrary but fixed. LEMMA 2.3 w10x. Let f : R n ª R be a con¨ ex function assuming finite ¨ alues for all x g R n. If there exists a sequence  x i 4 such that lim iª⬁ 5 x i 5 s ⬁ and lim sup f Ž x i . - ⬁,

Ž 2.1.

iª⬁

then an arbitrary cluster point of the sequence  x ir5 x i 54 belongs to 0q f. Proof. We will prove the lemma by contradiction. First we note that since f is finite at all x g R n, then it is a proper, closed convex function of x. Let  x i 4 satisfy lim i ª⬁Ž x ir5 x i 5. s ˆ x. Clearly, if ˆ x f 0q f, then it belongs n q to the open set R _ 0 f . We will prove first that the latter fact and the above properties of f Ž x . imply that the function f is unbounded from above along any half-line with the direction ˆ x. To this end let us assume that the opposite is true, that is, ᭚ x, such that f Ž x . is bounded from above along the half-line x Ž t . s x q tx, ˆ t G 0. Since ˆx f 0q f, then ᭚ t G 0, such Ž . that f x is increasing along the half-line x Ž t ., for t G t. Therefore we have lim inf f Ž x q txˆ. - ⬁. tª⬁

Since f is a proper, closed convex function, then by the second part of Theorem 8.6 in w11x, f Ž x . is a nonincreasing function of t along any line x q txˆ for every x g dom f. So, ˆ x g 0qf , which contradicts the assumption and proves that f Ž x . is unbounded from above along any half-line with the direction ˆ x.

679

CONVEX PROGRAMMES WITH UNATTAINED INFIMA

Now, we observe that ᭚␦ 0 ) 0,

x

G 1 y ␦0

¦ˆ ; x,

5 x5

«

x f 0q f .

Therefore, any vector x satisfying the latter relation has the property that along any half-line with the direction vector x, f is unbounded from above. Let f inf s lim inf i ª⬁ f Ž x i .. By assumption Ž2.1. it follows that there exists a subsequence  x i j 4 of the sequence  x i 4 such that lim jª⬁ f Ž x i j . s f inf . Without loss of generality we will use the symbol  x i 4 for the subsequence satisfying the latter equation. Let us define the half-line ˆ x Ž␶ . s ␶ ˆ x, ␶ G 0. Ž Ž .. Ž Since lim ␶ ª⬁ f ˆ x ␶ s ⬁, then for an arbitrarily large but fixed. number W ) 0, such that W ) max  f Ž 0 . q 2, f inf q 3 4 ,

Ž 2.2.

there exists such ␶ W ) 0 that f Ž ˆ x Ž␶ W .. s W. ŽIn the case when f inf s y⬁, the latter inequality is equivalent to W ) f Ž0. q 2.. By continuity of f we have that ᭚⑀ 1 ) 0,

xyˆ x Ž ␶ W . F ⑀1

«

f Ž x . G W y 1.

Ž 2.3.

We assume that ⑀ 1 is a sufficiently small number, so that x s 0 does not satisfy the inequality 5 x y ˆ x Ž␶ W .5 F ⑀ 1. In the case when this assumption is not satisfied, we can simply decrease the value ⑀ 1. We obtain the following implication: ᭚␦ g w 0, 1 x ,

xyˆ x Ž ␶ W . F ⑀1

«

x

G 1 y ␦ . Ž 2.4.

¦ˆ ; x,

5 x5

The value ␦ , which satisfies the latter implication, is not unique, and we define

␦ inf s inf ␦ < ␦ g w 0, 1 x ,

½

xyˆ x Ž ␶ W . F ⑀1

«

x

G1y␦ .

¦ˆ ; x,

5 x5

5

The value ␦ inf is well defined, because the infimum is defined over the set of values ␦ , which by relation Ž2.4. is nonempty and bounded. From the fact that the sequence  x ir5 x i 54 converges to ˆ x, it follows that ᭚ i␦ inf ,

᭙ i ) i␦ inf ,

xi

¦ ;

ˆx, 5 i 5 G 1 y ␦ inf . x

Ž 2.5.

680

WIESŁAWA T. OBUCHOWSKA

Let i 0 be such that i 0 ) i␦ inf and 5 x i0 5 ) ˆ x Ž ␶ W . q ⑀1 ,

Ž 2.6.

Žsuch an i 0 exists because 5 x i 0 54 is an unbounded sequence.. In the case when f inf s y⬁, we determine ı 0 G i 0 , such that x ı 0 satisfies f Ž x ı 0 . - 0.

Ž 2.7.

In the case when f inf ) y⬁, the index ı 0 G i 0 is determined so that x ı 0 satisfies f Ž x ı 0 . - f inf q 1.

Ž 2.8.

From Ž2.5., we conclude that x ı0

¦

;

ˆx, 5 ı 0 5 G 1 y ␦ inf . x

Ž 2.9.

The latter inequality implies that any point x defined by x s ␶ x ı 0 , 0 - ␶ F 1, satisfies x

G 1 y ␦ inf .

¦ˆ ; x,

5 x5

Let us denote BŽ ˆ x Ž ␶ W . , ⑀1 . s  x < x y ˆ x Ž ␶ W . F ⑀14 . From the definition of ␦ inf , it follows that BŽ ˆ x Ž ␶ W . , ⑀1 . ; x < ˆ x,

x

G 1 y ␦ inf

½¦ ;

5

Ž 2.10.

G 1 y ␦ inf .

Ž 2.11.

5 x5

and that for ⑀ ) ⑀ 1 BŽ ˆ xŽ␶W . , ⑀ . o x< ˆ x,

x

½¦ ; 5 x5

5

The inequalities Ž2.6. and 5 x ı 0 5 G 5 x i 0 5 imply x ı 0 f B Ž ˆ x Ž␶ W ., ⑀ 1 .. The inclusions Ž2.9., Ž2.10., and Ž2.11. imply that

Ž 0, x ı 0 . l B Ž ˆx Ž ␶ W . , ⑀ 1 . / ⭋. The latter relation can be written equivalently as ᭚␶ 0 g Ž 0, 1 . , such that x 0 s ␶ 0 x ı 0 ,

x0 y ˆ x Ž ␶ W . F ⑀1 .

CONVEX PROGRAMMES WITH UNATTAINED INFIMA

681

So, relation Ž2.3. yields f Ž x 0 . G W y 1.

Ž 2.12.

From the convexity of f we obtain f Ž␶ 0 x ı 0 . F Ž 1 y ␶ 0 . f Ž 0. q ␶ 0 f Ž x ı 0 . .

Ž 2.13.

Let us first consider the case when f inf s y⬁. Taking into account Eqs. Ž2.7., Ž2.2., and Ž2.12. in the inequality Ž2.13. results in the following double inequality W y 1 F f Ž x 0 . F Ž 1 y ␶ 0 . Ž W y 2. , which yields ␶ 0 - 0. The latter conclusion is inconsistent with our earlier assumption that ␶ 0 g Ž0, 1.. This ends the proof of the lemma in the case when f inf s y⬁. Now suppose that f inf is finite. Taking into account Eqs. Ž2.8., Ž2.2., and Ž2.12. in the inequality Ž2.13. results in the following inequality: W y 1 F f Ž x 0 . F Ž 1 y ␶ 0 . Ž W y 2 . q ␶ 0 Ž f inf q 1 . . Simple calculations along with the use of the inequality Ž2.2. yields the contradiction W y 1 F W y 2, which establishes the proof of Lemma 2.3. LEMMA 2.4. Let f i : R n ª R, i g I j  04 be con¨ ex functions. If there exists an unbounded sequence  x i 4 g R, such that lim sup f Ž x i . - ⬁,

Ž 2.14.

iª⬁ m then an arbitrary cluster point of the sequence  x ir5 x i 54 belongs to F is0 0q f i .

Proof. We will prove the lemma by contradiction. Let  x i 4 satisfy lim iª⬁Ž x ir5 x i 5. s ˆ x. We note that  x i 4 g R implies that lim sup iª⬁ f j Ž x i . - ⬁, j g I. Therefore all functions f j , j g I j  04 , satisfy hypotheses of Lemma 2.3 and consequently ˆ x g F mjs0 0q f j . Remark 1. We note that the result of Lemma 2.4 may be used to generate a vector p g Dfa l C assuming that the sequence  x i 4 g R along which the objective function decrease is given. From now on we assume that yi , i s 1, 2, . . . , n y 1, is the orthonormal basis for the subspace ⵜf Ž x 0 . H , which will be denoted by Y. The symbol Y will be also used to denote the matrix in R Ž ny1.=n formed by the vectors yi , i s 1, . . . , n y 1, and ␰ s Ž ␰ 1 , . . . , ␰ ny1 .T .

682

WIESŁAWA T. OBUCHOWSKA

LEMMA 2.5. Ži.

Let us assume that f is con¨ ex and f g C⬁.

Then the function Gt Ž ␰ . s f Ž x 0 q tp q Y T␰ .

Ž 2.15.

achie¨ es a minimum o¨ er the region Rt s  ␰ < Gti Ž ␰ . s f i Ž x 0 q tp q Y T␰ . F 0,

i s 1, . . . , m4 Ž 2.16.

for any t G 0. Ž . achie¨ es a unique miniIf in addition Ds f s ⭋, then the function Gt ␰ mum o¨ er Rt . Žii. If C l Ds f s ⭋, then the set of the optimal solutions to the problem Ž2.15. ᎐ Ž2.16. is bounded. Žiii. If ␰ Ž t . is an optimal solution to the problem Ž2.15. ᎐ Ž2.16., then lim Gt Ž ␰ Ž t . . s inf f Ž x . .

tª⬁

xg R

Proof. Ži. We observe that Ct , the cone of recession of the problem Ž2.15. ᎐ Ž2.16., is given by Ct s  ␰ g R ny 1 < Y T␰ g C 4 . Since yi H ⵜf Ž x 0 ., ᭙ i s 1, . . . , n y 1, then Y T␰ g C implies Y T␰ g Ds and ␰ g DGs . The f s latter conclusion can be written as Ct ; DG , so from Lemma 2.2 we obtain that the problem Ž2.15. ᎐ Ž2.16. attains its minimum. We note that this part of the lemma remains valid without assumption that f g C⬁. This is because the Lemma 2.2 can be easily restated without assumption that f is an analytic function. Žii. Part Ži. of the lemma implies that the function Gt Ž ␰ . achieves its minimum over the set Rt , at some point ␰ t. We will show that the set T of the optimal solutions to this problem is bounded. If not, then the convexity of the set T implies that there exists a half-line ␩ Ž␶ . s ␰ t q ␩␶ , ␶ G 0, m m ␩ g F is1 0qG ti, contained in T ; Rt . Thus the relations Y T␩ g F is1 0qf i s T u a T q and C l Df s ⭋ imply that either Y ␩ g Df j Df or Y ␩ f 0 f . If the first case holds, then f Ž x . strictly decreases along the half-line x Ž␶ . s x 0 q tp q Y T␩ Ž␶ ., which contradicts the assumption that ␰ t g T. If Y T␩ f 0q f then the function f Ž x . strictly increases along this half-line, which contradicts the inclusion ␩ Ž␶ . g T, ␶ G 0. Žiii. We will show first that f Ž x 0 q tp q Y T␰ Ž t .. monotonically decreases with respect to t. Let t 2 ) t 1. Then, since p g Dfa, we have f Ž x 0 qt 1 pqY T␰ Ž t 1 . . ) f Ž x 0 qt 2 pqY T␰ Ž t 1 . . G f Ž x 0 qt 2 pqY T␰ Ž t 2 . . .

683

CONVEX PROGRAMMES WITH UNATTAINED INFIMA

Suppose that the sequence  x i 4 g R is minimizing for f Ž x ., that is, lim f Ž x i . s inf f Ž x . .

iª⬁

Ž 2.17.

xg R

Each x k can be represented uniquely as x k s x 0 q t k p q Y T␰ k , with t k g R. We will show that there exists a subsequence  t k j 4 of the sequence  t k 4 such that  t k j 4 ª ⬁. Let us suppose that the opposite is true, that is, M s sup k t k - ⬁. Since p g 0qf , then f Ž x 0 qt k pqY T␰ k . G f Ž x 0 qMpqY T␰ k . G f Ž x 0 qMpqY T␰ Ž M . . , ᭙k, where ␰ Ž M . s argmin ␰ g RM f Ž x 0 q Mp q Y T␰ .. Taking k ª ⬁ yields lim k ª⬁ f Ž x 0 q t k p q Y T␰ k . G f Ž x 0 q Mp q Y T␰ Ž M ... Since p g Dfa, then f Ž x 0 q Mp q Y T␰ Ž M .. ) f Ž x 0 q Ž M q 1. p q Y T␰ Ž M ... The latter two inequalities and the equation Ž2.17. give inf f Ž x . ) f Ž x 0 q Ž M q 1 . p q Y T␰ Ž M . . .

xg R

Since p g C, then x 0 q Ž M q 1. p q Y T␰ Ž M . belongs to RMq1 ; R, which contradicts the definition of the infimum of f Ž x . over R. This completes the proof of the statement that there exists  t k j 4 ª ⬁. Without loss of generality, we will use the symbol  t k 4 to denote the subsequence  t k j 4 divergent to q⬁. It follows from part Ži. of the lemma that the problem min  Gt kŽ ␰ . < ␰ g Rt k 4 , where Gt kŽ ␰ . and Rt k are defined in Ž2.15. and Ž2.16., respectively, has a solution for every t k . Let us define ␰ Ž t k . s argmin Gt k < ␰ g Rt k 4 . We observe that x k s x 0 q t k p q Y T␰ k ; Rt k implies f Ž x 0 q t k p q Y T␰ Ž t k . . F f Ž x k . ,

᭙ t k G 0.

Ž 2.18.

Taking the limit with k ª ⬁, of both sides of the inequality Ž2.18., proves that inf f Ž x . s lim Gt k Ž ␰ Ž t k . . ,

xg R

kª⬁

which along with the fact that Gt Ž ␰ Ž t .. strictly decreases with respect to t completes the proof of the lemma.

684

WIESŁAWA T. OBUCHOWSKA

3. INFIMAL TRAJECTORY The problem of minimizing the function Ž2.15. with respect to the constraints Ž2.16. is clearly a parametric program. We observe that to find an approximate infimum of f Ž x ., it is sufficient to solve the problem Ž2.15. ᎐ Ž2.16. for a sufficiently large value of the parameter t. We will prove that the sequence of the optimal solutions to the problem Ž2.15. ᎐ Ž2.16. can be approximated by the points on the trajectory C Ž t . along which the logarithmic penalty function converges to the global infimum of f Ž x . with respect to the set R. Our analysis of the infimal trajectory will be based upon the unconstrained minimization technique which exploits the logarithmic penalty function. Let us define the functions m

L Ž t , ␰ , r . s Gt Ž ␰ . y r Ý ln Ž yGti Ž ␰ . .

Ž 3.1.

is1

and L Ž t , ␰ . s L Ž t , ␰ , t . s Gt Ž ␰ . y

1 t

m

Ý ln Ž yGti Ž ␰ . . .

Ž 3.2.

is1

LEMMA 3.1. Assume that the functions f i : R n ª R, i g I j  04 are con¨ ex and analytic and C l Ds f s ⭋. Let us define the function F Ž t . s min

␰gR ny1

 L Ž t, ␰ . 4 ,

t g Rq .

Ž 3.3.

Then Ži. F Ž t . is well defined, that is, F Ž t . ) y⬁, ᭙ t G 0, and the minimum of the function on the right side of Ž3.3. exists and is unique for e¨ ery t G 0. Žii. F Ž t . is a strictly monotonically decreasing function of t, for t G 0. Proof. Ži. Lemma 2.5 Žii. assures that the set of points that solve the problem of minimizing of Gt Ž ␰ . subject to ␰ g Rt is bounded Žin particuŽ. lar, if Ds f s ⭋, then part i of the lemma states that the set of optimal solutions consists of one point only.; thus the conditions of Theorem 25 in w5x are satisfied. Part Ži. of this theorem implies that the function LŽ t, ␰ , r . has a finite unconstrained minimum ␰ Ž r . with respect to ␰ for every 1 r ) 0, and t ) 0. Letting r s t proves the existence of the minimum of L Ž t, ␰ ., for every t G 0. To prove the uniqueness of the solution we will show that the Hessian matrix of L Ž t, ␰ . is strictly positive definite. To this end we will use the result proved in w12x, that any analytic convex function f i Ž x . can be

CONVEX PROGRAMMES WITH UNATTAINED INFIMA

685

represented as f i Ž x . s Fi Ž c i q Bi x . q ² a i , x : q d i , where Fi is strictly convex analytic function, Bi g R p i=n , c i g R p i , and a i g R n. It has also been shown in w12x that Ds fi s N

Bi

ž / aTi

,

where N Ž⭈. denotes the nullspace of the matrix Ž⭈.. It follows directly that the Hessian of L Ž t, ␰ . is given by the formula ⵜ␰2 L Ž t , ␰ . s YB T ⵜy2 F Ž c q Bx . BY T y

q

1 t

m

Ý

1 t

m

Ý

YBiT ⵜy2i Fi Ž c i q Bi x . Bi Y T fi Ž x .

is1 T

Ž ⵜy Fi Ž ci qBi x . Bi Y T qYai . Ž ⵜy Fi Ž ci qBi x . Bi Y T qYai . , i

i

2

fi Ž x .

is1

where x s x 0 q pt q Y T␰ , and y i g R p i , i g I j  04 . Since f i Ž x . F 0, ᭙ x g R, i g I, then all three terms in the expression on the right side of the latter equation are nonnegative. Thus the Hessian matrix ⵜ␰2 L Ž t, ␰ . is not strictly definite Žsemidefinite. only if there exists a nonzero vector ␰ satisfying the system of equations Bi Y T ␰ s 0, ᭙ i g I j  0 4 ,

ⵜy i Fi Ž c i q Bi x . Bi Y T q Ya i ␰ s 0, ᭙ i g I.

Note that equation Bi Y T ␰ s 0 implies that a i Y T ␰ s 0, ᭙ i, so m

YT␰g

FN is1

Bi

ž / aTi

m

s

F Dsf . i

Ž 3.4.

is1

Since the raws of the matrix Y are orthogonal to ⵜf Ž x 0 ., then w ⵜy F Ž c q Bx 0 . B q axT Y T ␰ s 0, which along with the equation BY T ␰ s 0 yields Ž3.4. contradicts the aT Y T ␰ s 0. Thus Y T ␰ g Ds f , which along with Ž . of the lemma. assumption that C l Ds s ⭋, and establishes part i f

686

WIESŁAWA T. OBUCHOWSKA

Žii. We will show that F Ž t 2 . - F Ž t 1 . if t 2 ) t 1. It follows from Theorem 25 Žvi. in w5x that 1

min L t 1 , ␰ ,

ž

xgR ny1

t1

/

1

) min L t 1 , ␰ ,

ž

␰gR ny1

t2

/

.

Let ␰ 1 , ␰ 2 , and ␰ 3 be such that L t1 , ␰ 1 ,

ž ž

L t1 , ␰ 2 ,

1 t1 1 t2

/ /

s min L t 1 , ␰ ,

/

s min L t 2 , ␰ ,

␰gR ny1

ž ž

s min L t 1 , ␰ , ␰gR

ny1

1 t1 1 t2

/ /

,

/

.

Ž 3.5. Ž 3.6.

and L t2 , ␰ 3 ,

ž

1 t2

␰gR ny1

1

ž

t2

Ž 3.7.

We will prove first that L t2 , ␰ 2 ,

ž

1 t2

- L t1 , ␰ 2 ,

/ ž

1 t2

/

.

Ž 3.8.

To this end we will show that L t2 , ␰ 2 ,

ž

1 t2

/

1

s f Ž x 0 q t 2 p q Y T␰ 2 . y - f Ž x 0 q t 1 p q Y T␰ 2 . y s L t1 , ␰ 2 ,

ž

1 t2

/

t2 1 t2

m

Ý ln Ž yfi Ž x 0 q t 2 p q Y T␰ 2 . . is1 m

Ý ln Ž yfi Ž x 0 q t1 p q Y T␰ 2 . . is1

.

Ž 3.9.

We observe that assumption that p g C l Dfa implies f Ž x 0 q t 2 p q Y T␰ 2 . - f Ž x 0 q t 1 p q Y T␰ 2 .

Ž 3.10.

f j Ž x 0 q t 2 p q Y T␰ 2 . F f j Ž x 0 q t 1 p q Y T␰ 2 . ,

Ž 3.11.

and

i s 1, 2, . . . , m. Furthermore, the equation Ž3.6. and Theorem 25 Ži. in w5x imply that ␰ 2 g int Rt 1, which yields f j Ž x 0 q t 1 p q Y T␰ 2 . - 0,

j s 1, . . . , m.

Ž 3.12.

687

CONVEX PROGRAMMES WITH UNATTAINED INFIMA

The inequalities Ž3.10. ᎐ Ž3.12. prove that inequality Ž3.9., and consequently inequality Ž3.8., holds. Now, by Theorem 25 Žvi. in w5x, we have L t1 , ␰ 2 ,

ž

1 t2

- L t1 , ␰ 1 ,

/ ž

1 t1

/

.

Ž 3.13.

/

.

Ž 3.14.

The definition of ␰ 3 gives L t2 , ␰ 3 ,

ž

1 t2

1

F L t2 , ␰ 2 ,

/ ž

t2

Relations Ž3.3., Ž3.5., and Ž3.7. yield F Ž t 1 . s min L t 1 , ␰ , ␰gR ny1

ž ž

F Ž t 2 . s min L t 2 , ␰ , ␰gR ny1

1 t1 1 t2

s L t1 , ␰ 1 ,

/ ž / ž

s L t2 , ␰ 3 ,

1 t1 1 t2

/ /

,

.

Now inequalities Ž3.14., Ž3.8., and Ž3.13. imply L t2 , ␰ 3 ,

ž

1 t2

- L t1 , ␰ 1 ,

/ ž

1 t1

/

,

which can be written equivalently as F Ž t 2 . - F Ž t 1 .. We note that the equality F Ž t . s L Ž t , ␰ . s Gt Ž ␰ . y

m

1 t

Ý ln Ž yGti Ž ␰ . . is1

holds if and only if ␰ satisfies the equation ⵜ␰ L Ž t , ␰ . s Y ⵜx f Ž x 0 q tp q Y T␰ . y

1 t

m

Ý is1

Y ⵜx f i Ž x 0 q tp q Y T␰ . f i Ž x 0 q tp q Y T␰ .

s 0.

Ž 3.15. For every value of t the equation Ž3.15. consists of n y 1 equations with n y 1 unknown variables ␰ 1 , . . . , ␰ ny1. In the remaining part of the paper we assume that the function L Ž t, ␰ . satisfies the following second order sufficiency condition ᭙ t g Rq , ᭙␰ g R n , ⵜ␰ L Ž t , ␰ . s 0 « ␰ T ⵜ␰2 L Ž t , ␰ . ␰ ) 0, ᭙␰ / 0,

688

WIESŁAWA T. OBUCHOWSKA

which can be expressed in terms of the function f Ž x . and its derivatives as ᭙ y,

x g Rn,

y / 0,

y T ⵜx2 f Ž x . y

1 t

m

Ý

y T ⵜx f Ž x . y

1 t

m

Ý

ⵜx f i Ž x .

is1

ⵜx2 f i Ž x . f i Ž x . y ⵜx f i Ž x . ⵜx f i Ž x . f i2 Ž x .

is1

s0

fi Ž x .

«

T

y ) 0, Ž 3.16.

where x s x 0 q tp q Y T␰ . We note that the Lemma 3.1 remains valid if the assumption that Ž . Ds f lC s ⭋ is replaced with the less restrictive condition 3.16 . In the theorem below we will prove that there exists an analytic function x Ž t . s C Ž t ., such that the infimum of f Ž x . along the curve is equal to the global infimum of f Ž x . over R. THEOREM 3.1. Ži. If f i , i g I j  04 are con¨ ex and analytic, then there exists a unique analytic function hŽ t ., which is a trajectory of unconstrained minima of L Ž t, ␰ .. Furthermore, deri¨ ati¨ es of all orders of hŽ t . can be obtained by subsequent differentiation of ⵜ␰ L Ž t, hŽ t .. s 0 with respect to t. Žii. The parametrized cur¨ e C Ž t . s x 0 q tp q Y T hŽ t . satisfies lim F Ž t . s lim f Ž C Ž t . . y

tª⬁

tª⬁

m

1 t

Ý ln Žyfi Ž C Ž t . . .

s lim L Ž t , h Ž t . .

is1

tª⬁

s inf f Ž x . .

Ž 3.17.

xg R

Proof. Ži. The condition Ž3.16. implies that the Hessian matrix of L Ž t, ␰ . is positive definite at the point Ž t, ␰ . satisfying Eq. Ž3.15.. Since the system of Ž n y 1. equations in n variables ␰ 1 , . . . , ␰ ny1 , t in Ž3.15. has an invertible Jacobian matrix ⵜ␰2 L Ž t, ␰ ., then by the implicit function theorem w13x there exist open sets U g R n and W ; Rq, with Ž ␰ , t . g U and t g W, having the following property: To every t g W, there corresponds a unique ␰ such that Ž ␰ , t . g U and ⵜ␰ L Ž t, ␰ . s 0. Furthermore, if this ␰ is defined to be hŽ t ., then h is a C 1-mapping of W into R ny1 , hŽ t . s ␰ , and ⵜ␰ L Ž t , h Ž t . . s 0,

᭙ t g W.

Differentiating the latter equation with respect to t yields Y ⵜx2 f Ž h 0 Ž t . . Ž Y T h⬘ Ž t . q p . q 1 y Y t

m

Ý

1

m

2

Ý

t

is1

Y ⵜx f i Ž h 0 Ž t . . fi Ž h0 Ž t . .

ⵜx2 f i Ž h 0 Ž t . . f i Ž h 0 Ž t . . y ⵜx f i Ž h 0 Ž t . . ⵜx f i Ž h 0 Ž t . .

is1

= Ž Y T h⬘ Ž t . q p . s 0,

T

f i2 Ž h 0 Ž t . .

Ž 3.18.

689

CONVEX PROGRAMMES WITH UNATTAINED INFIMA

where h 0 Ž t . s x 0 q tp q Y T hŽ t .. By the assumption Ž3.16. the matrix ⵜ␰2 L Ž t , ␰ . s Y ⵜx2 f i Ž h 0 Ž t . .

y

1 t

m

Ý

ⵜx2 f i Ž h 0 Ž t . . f i Ž h 0 Ž t . . y ⵜx f Ž h 0 Ž t . . ⵜx f i Ž h 0 Ž t . .

T

f i2 Ž h 0 Ž t . .

is1

Y T,

has an inverse, so the equation Ž3.18. can be solved for h⬘Ž t . h⬘ Ž t . s y Ž ⵜ␰2 L Ž t , ␰ . . m

1

y Y t

Ý

y1

Y ⵜx2 f Ž h 0 Ž t . . T

ⵜx2 f i Ž h 0 Ž t . . f i Ž h 0 Ž t . . y ⵜx f i Ž h 0 Ž t . . ⵜx f i Ž h 0 Ž t . . f i2 Ž h 0 Ž t . .

is1

q

1 t

2

m

YÝ is1

ⵜx f i Ž h 0 Ž t . . fi Ž h0 Ž t . .

p

.

This proves that the derivative of hŽ t . with respect to t exists for t g W and is a C 1-mapping of W into R n. ŽWe note that the existence and continuity of h⬘Ž t . follows also from the implicit function theorem; however, for the completeness of the proof we include this argument as well.. Now, differentiation of Eq. Ž3.18. with respect to t yields dⵜ␰2 L Ž t , h Ž t . . dt y

1 t

qd

m

d YÝ

h⬘ Ž t . q ⵜ␰2 L Ž t , h Ž t . . h⬙ Ž t . q

m

2

Ý

t

is1

dt T

ⵜx2 f i Ž h 0 Ž t . . f i Ž h 0 Ž t . . y ⵜx f i Ž h 0 Ž t . . ⵜx f i Ž h 0 Ž t . . f i2 Ž h 0 Ž t . .

is1

1

d Y ⵜx2 f Ž h 0 Ž t . .

Y ⵜx f i Ž h 0 Ž t . . fi Ž h0 Ž t . .

p rdt

rdt s 0

The existence of h⬙ Ž t . is assured by the existence of the inverse of the Hessian matrix ⵜ␰2 L Ž t, hŽ t ... Continuing the above process it is possible to obtain explicitly all derivatives hŽ k . Ž t . in terms of the derivatives hŽ i. Ž i s 1, . . . , k y 1., and partial derivatives of the functions f j , j g I j  04 , of degrees i s 1, . . . , k q 1. Since f j g C⬁ , then part Ži. of the theorem follows.

690

WIESŁAWA T. OBUCHOWSKA

Žii. In Lemma 2.5 Žii. we proved that the set of optimal solutions to the problem Ž2.15. ᎐ Ž2.16. is bounded for every t. Thus the assumptions of Theorem 25 Živ. of w5x are satisfied, which yields that for any fixed t k ) 0 lim L t k , ␰ k Ž t . ,

tª⬁

ž

1 t

s min Gt kŽ ␰ . ,

/

Ž 3.19.

␰g Rt k

1 where ␰ k Ž t . denotes the point of unconstrained minimum of LŽ t k , ␰ , t . with respect to ␰ . Lemma 2.5 Žiii. shows that

lim f Ž x 0 q t k p q Y T␰ Ž t k . . s inf f Ž x . ,

kª⬁

xg R

where ␰ Ž t . is an optimal solution to the problem Ž2.15. ᎐ Ž2.16.. It follows from the latter equation that inf f Ž x . s lim min Gt kŽ ␰ . ,

Ž 3.20.

kª⬁ ␰g Rt k

xg R

which, using Eq. Ž3.19., yields lim lim L t k , ␰ k Ž t . ,

kª⬁ tª⬁

ž

1 t

/

s inf f Ž x . . xg R

It follows from the equation Ž3.20. that ᭙⑀ ) 0,

᭚k ,

᭙k G k,

0 - min Gt k y inf f Ž x . - ⑀ . Ž 3.21. ␰g Rt k

xg R

1 Equation Ž3.19., along with the fact that for every fixed t k , LŽ t k , ␰ k Ž t ., t . is monotonically decreasing with respect to t Žsee Theorem 25 Žvi. in w5x., yields

᭙␦ ) 0,

᭚ t␦ ,

᭙ t ) t␦ ,

1

0 F L tk , ␰ k Ž t . ,

ž

t

/

y min Gt k - ␦ . ␰g Rt k

Ž 3.22. Taking ␦ s ⑀ in Ž3.22. and adding inequalities in Ž3.21. and Ž3.22. gives ᭙⑀ ) 0, ᭚k , ᭚ t⑀ , ᭙k G k , ᭙ t G t⑀ , 0 F L t k , ␰ k Ž t . ,

ž

1 t

/

y inf f Ž x . - 2 ⑀ . xg R

If for given ⑀ ) 0, t k ) t⑀ , then we substitute t⑀ [ t k ; otherwise, i.e., if t k - t⑀ , we find the smallest index ˜ k, such that t ˜k ) t⑀ , and take k [ ˜ k.

691

CONVEX PROGRAMMES WITH UNATTAINED INFIMA

The latter modification leads to the relation ᭙⑀ ) 0,

᭚k ,

᭙k G k,

0 F L tk , ␰ k Ž tk . ,

ž

1 tk

/

y inf f Ž x . - 2 ⑀ , xg R

which proves that lim L Ž t k , ␰ k Ž t k . . s inf f Ž x . .

kª⬁

xg R

Since F Ž t k . s L Ž t k , ␰ k Ž t k .. and F Ž t . is a strictly decreasing function of t Žsee Lemma 3.1Žii.., and lim k ª⬁ t k s ⬁, then lim k ª⬁ F Ž t k . s lim t ª⬁ F Ž t ., which implies lim F Ž t . s inf f Ž x . .

tª⬁

xg R

This completes the proof of the equalities in Ž3.17.. In the lemma below we will show that the curve x s C Ž t . does not depend on the choice of the vector p, as long as the vector p is in the cone of recession C and it belongs to Dfa. Let Hi Ž t . s  x g R n < x s x 0 q tpi q Y T␰ 4 ,

i s 1, 2,

t G 0.

Let C1Ž t . and C2 Ž t . denote two parametrized curves constructed for two different vectors p1 , p 2 g C l Dfa. It follows that Ci Ž t . s x 0 q tpi q Y T h i Ž t . ,

i s 1, 2,

Ž 3.23.

where h i Ž t . sargmin  Li Ž t , ␰ . < ␰ gHi Ž t . lint R 4 sargmin  Li Ž t , ␰ . < ␰ gR ny 1 4 ,

Ž 3.24. where Li Ž t , ␰ . s f Ž x 0 q tpi q Y T␰ . y

1 t

m

Ý ln Ž yf j Ž x 0 q tpi q Y T␰ . . . js1

LEMMA 3.2. Ži. If p1 , p 2 g C l Dfa, then C1Ž t . and C2 Ž t . defined in Ž3.23. ᎐ Ž3.24. are equi¨ alent representations of the same parametrized cur¨ e.

692

WIESŁAWA T. OBUCHOWSKA

Žii. For a gi¨ en x 0 g R and p g C l Dfa there is exactly one cur¨ e satisfying C Ž t . s x 0 q tp q Y T hŽ t . g R where h Ž t . sargmin  L Ž t , ␰ . < ␰ gHi Ž t . lint R 4 sargmin  L Ž t , ␰ . < ␰ gR ny 1 4 , with H Ž t . s  x g R n < x s x 0 q tp q Y T␰ , ␰ g R ny1 4 , t G 0, passing through x 0 . The cur¨ e satisfies Eq. Ž3.17.. Proof. Ži. Note that for a fixed value of t, Hi Ž t ., i s 1, 2, represents a linear manifold and ᭙ t 1 g Rq ,

᭚ t 2 g Rq ,

H1 Ž t 1 . s H 2 Ž t 2 . .

Ž 3.25.

Equalities Ž3.23. ᎐ Ž3.25. yield C1 Ž t 1 . s C 2 Ž t 2 . so C1Ž t . and C2 Ž t . are equivalent representations of the same parametrized curve. Since relation Ž3.25. is symmetric with respect to t 1 and t 2 , it follows that there is a unique correspondence between the points on the curves C1Ž t . and C2 Ž t .. This ends the proof of part Ži. of the lemma. Žii. Lemma 2.1 Žii. implies that f Ž x . is strictly decreasing along a half-line x 0 Ž t . s x 0 q tp, t G 0, for any x 0 g R n. Therefore replacing the initial point x 0 with another point x 1 requires one to redefine the vectors y 1 , . . . , yny1 as the orthonormal basis for the subspace  ⵜf Ž x 1 .4 H . Thus all results proved for x 0 remain valid for the function Gt Ž ␰ . s f Ž x 1 q tp q Y T␰ . and respectively modified functions L Ž t, ␰ ., F Ž t . and C Ž t ., and the linear manifold H Ž t .. Furthermore, since C l Ds f s ⭋, Lemma 3.1 implies that the curve C Ž t . is uniquely defined.

4. EXAMPLE AND CONCLUDING REMARKS We will illustrate the method proposed in Section 3 with the example of the problem with two constraints in two variables. Let us consider the problem min f Ž x . s

1 x1 q 1

subject to f 1 Ž x . s yx 1 q x 2 F 1 f 2 Ž x . s yx 2 F y2.

693

CONVEX PROGRAMMES WITH UNATTAINED INFIMA

We have p s Ž1, 0.T , x 0 s Ž0, 0.T , ⵜf Ž0. s Žy1, 0.T , Y s Ž0, 1.. Therefore, min gxgR

ny1

L Ž t , ␰ . s min ␰

ž

ⵜ␰ L Ž t , ␰ . s

1 tq1

y

1 t

ln Ž 1 q t y ␰ . y

1 t Ž1 q t y ␰ .

y

1 t Ž ␰ y 2.

1 t

ln Ž ␰ y 2 . ,

/

s 0.

The latter equation yields

␰ s hŽ t . s

1 2

Ž t q 3. .

Finally C Ž t . s Ž t, 12 Ž t q 3..T , which yields lim t ª⬁ L Ž t, hŽ t .. s 0 and consequently inf x g R Ž1rŽ x 1 q 1.. s 0. Concluding, the aim of this paper is to show that for a convex program Ž1.1. with unattained infimum and for a given initial feasible point, there exists a unique parametrized curve C Ž t ., along which the logarithmic penalty function converges to the infimum of the function f Ž x . over the region R. We show that if all functions defining the program are analytic, then C Ž t . is an analytic function as well. We also prove that an approximate solution to the degenerate program Ž1.1. can be determined by solving the auxiliary parametric problem, defined in Lemma 2.5, for a sufficiently large value of the parameter t. The performance of the proposed method was investigated on a number of convex constrained problems with unattained minimum. The application of the classical methods, such as the steepest descent or the gradient projection methods, fails to provide a sequence convergent to the constrained infimum in these problems. The results obtained for all tested problems seem to be promising, which suggests that the method proposed in this paper deserves more study both from the theoretical and the computational points of view.

REFERENCES 1. R. A. Abrams, Projections of convex programs with unattained infima, SIAM J. Control 13, No. 3 Ž1975., 706᎐718. 2. A. Auslender, R. Cominetti, and J.-P. Crouzeix, Convex functions with unbounded level sets and applications to duality theory, SIAM J. Optim. 3 Ž1993., 669᎐687. 3. R. J. Caron and W. T. Obuchowska, Unboundedness of a convex quadratic function subject to concave and convex quadratic constraints, European J. Operational Res. 63 Ž1992., 114᎐123.

694

WIESŁAWA T. OBUCHOWSKA

4. R. J. Caron and W. T. Obuchowska, An algorithm to determine boundedness of quadratically constrained convex quadratic programmes, European J. Operational Res. 80 Ž1995., 431᎐438. 5. A. V. Fiacco and G. P. McCormick, ‘‘Nonlinear Programming; Sequential Unconstrained Minimization Techniques,’’ Wiley, New York, 1968. 6. J.-B. Hiriart-Urutty and C. Lemarechal, ‘‘Convex Analysis and Minimization Algorithms I,’’ Springer-Verlag, BerlinrNew York, 1993. 7. M. Lamoureux and H. Wolkowicz, Numerical decomposition of a convex function, J. Optim. Theory Appl. 47 Ž1985., 51᎐64. 8. K. G. Murty, ‘‘Linear Complementarity and Nonlinear Programming,’’ Helderman, West Berlin, 1988. 9. W. T. Obuchowska and K. G. Murty, Cone of recession and unboundedness of convex functions, submitted for publication. 10. W. T. Obuchowska, On the infimal trajectory of convex functions with unbounded level sets, submitted for publication. 11. R. T. Rockafeller, ‘‘Convex Analysis,’’ Princeton Univ. Press, Princeton, NJ, 1979. 12. R. T. Rockafeller, Some convex programs whose duals are linearly constrained, in ‘‘Nonlinear Programming’’ ŽJ. B. Rosen, O. L. Mangasarian, and K. Ritter, Eds.., Academic Press, New York, 1970. 13. W. Rudin, ‘‘Principles of Mathematical Analysis,’’ McGraw-Hill, New York, 1976.