- Email: [email protected]
On Gap Functions and Duality of Variational Inequality Problems
JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS ARTICLE NO.
214, 658]673 Ž1997.
AY975608
On Gap Functions and Duality of Variational Inequality ProblemsU G. Y. Chen Institute of System Sciences, Academic Sinica, Beijing 100080, People’s Republic of China
C. J. Goh and X. Q. Yang† Department of Mathematics, Uni¨ ersity of Western Australia, Nedlands, Western Australia 6907, Australia Submitted by E. S. Lee Received June 10, 1996
We extend the definition of the gap function defined by Auslender for a more general class of variational inequality problems involving some convex function. A study of the duality of the extended variational inequality problem and its dual sheds new light on the meaning of gap functions. Convexity and differentiability of the gap function are also studied and sufficient conditions are derived. We also show how the gap functions for the primal and the dual are related by dual Fenchel optimization problems. Q 1997 Academic Press
1. INTRODUCTION In Hearn w4x, the gap function of a convex optimization problem is discussed, and the meaning of ‘‘gap’’ is interpreted as the difference between the cost function and the maximum of its Wolfe dual. In some operation research problems, optimization models are sometimes inadequate and problems such as asymmetric traffic equilibrium models are often formulated as variational inequality problems, which include convex optimization problems as a special case. In the operations research literature, the common variational inequality problem under studied is given as U
This research is supported by a grant from the Australian Research Council. Address for corresponding author: E-mail: [email protected].
†
658 0022-247Xr97 $25.00 Copyright Q 1997 by Academic Press All rights of reproduction in any form reserved.
659
GAP FUNCTIONS
follows: Problem VI. Given a closed convex set K ; R n and a vector-valued function F: R n ª R n, find a point x 0 g K such that T
F Žx 0 . Žx y x0 . G 0
;x g K .
Ž 1.1.
Existence, uniqueness, and solution methods for this type of problem have been extensively studied Žsee Narguney w6x.. In particular, Auslender w1x defined the following gap function for Problem VI: DEFINITION 1.1 ŽAuslender’s Gap Function.. T
f Ž x . s max F Ž x . Ž x y y . .
Ž 1.2.
ygK
It is not difficult to show that Property 1.1. Ži. f Žx. G 0 ;x g K, and Žii. f Žx 0 . s 0 if and only if x 0 solves Problem VI. These properties can also serve to be the definition of a gap function. In general, this gap function is not differentiable, but Auslender w1x shows that it is differentiable if the ground set K is strongly convex. The nondifferentiability of the gap function poses a major difficulty for minimizing the gap function as a method for solving Problem VI. Fukushima w3x subsequently proposed a differentiable optimization problem for solving Problem VI. To the best of our knowledge, there has been no interpretation of the meaning of ‘‘gap’’ as was done by Hearn Žfor convex optimization problems. in the context of variational inequality. In this paper, we seek to provide a meaningful interpretation to the gap function, and show that it is intimately related to the duality of variational inequality. We also show that Ži. and Žii. can basically be interpreted as weak duality and strong duality, respectively. Lastly, we show that the gap functions of a pair of primal]dual variational inequality corresponds to a pair of primal]dual Fenchel optimization problems.
2. DUALITY OF VARIATIONAL INEQUALITY To understand the duality of Problem VI in its full generality, it is more expedient to study the following class of extended variational inequality problems: Problem EVI. Given a vector-valued injective function F: R n ª R n and a scalar-valued lower semicontinuous proper convex function f : R n ª
660
CHEN, GOH, AND YANG
R, where R s R j q`4 , find a point x 0 g R n such that T
F Ž x 0 . Ž x y x 0 . G f Ž x 0 . y f Ž x.
;x g R n .
Ž 2.1.
Problem EVI was first studied in the context of partial differential equations by Stampacchia w8x. Note that Problem EVI reduces to Problem VI if we let f be the indicator function for the convex ground set K, that is, f Žx. s 0 if x g K and f Žx. s q` if x f K. In general, the effective domain of the convex function f implicitly defines the convex ground set K in the context of Problem VI, since if x f K, that is, f Žx. s q`, then the inequality Ž2.1. is always true, whilst if x 0 f K, then the inequality Ž2.1. is never true. The assumption that F is injective allows one to construct the following function: G: yRange Ž F . ª R n
s.t. G Ž u . s yFy1 Ž yu . .
Ž 2.2.
Consequently, F Žx. s yGy1 Žyx.. Let g: R n ª R be the Fenchel conjugate of f defined by: DEFINITION 2.1 ŽFenchel Conjugate.. f U Ž u . s maxn x T u y f Ž x . 4 .
Ž 2.3.
xgR
In the case where f is the indicator function for some convex set K, its Fenchel conjugate is the support function of K, that is, f U Žu. s max x g K uT x. It is well known ŽRockefellar w7x. that f U is convex, and that if f is closed Žor lower semicontinuous., then the Fenchel conjugate of f U is f. From the definition Ž2.3. we have: LEMMA 2.1 ŽYoung’s Inequality .. f Ž x. q f U Ž u. y x T u G 0
;x, u g R n .
Ž 2.4.
Define the dual of Problem EVI to be ŽMosco w5x.: Problem DEVI. Given G: yRangeŽ F . ª R n, as defined by Ž2.2., and g: R n ª R, as defined by Ž2.3., find a point u 0 g yRangeŽ F . such that T
G Ž u 0 . Ž u y u 0 . G f U Ž u 0 . y f U Ž u.
;u g R n .
Ž 2.5.
Problem EVI and its dual DEVI are related by the duality theorem ŽMosco w5x.: THEOREM 2.1. Ži. x 0 sol¨ es EVI if and only if u 0 s yF Žx 0 . sol¨ es DEVI.
661
GAP FUNCTIONS
Žii. The ¨ ariational inequalities Ž2.1. and Ž2.5. hold if and only if u 0 s yF Žx 0 . or x 0 s yGŽu 0 ., and T
f Ž x 0 . q f U Ž u 0 . y Ž x 0 . u 0 s 0.
Ž 2.6.
Proof. The original proof of Mosco requires the use of subdifferential and other results in abstract notions. The following proof is a simpler one which requires only the definition we have laid down so far. Say x 0 solves Problem EVI. Then T
F Ž x 0 . Ž x y x 0 . G f Ž x 0 . y f Ž x.
;x g R n ,
or T
T
yF Ž x 0 . x 0 y f Ž x 0 . G yF Ž x 0 . x y f Ž x .
;x g R n ,
implying that T
T
yF Ž x 0 . x 0 y f Ž x 0 . s maxn yF Ž x 0 . x y f Ž x . xgR
½
s f U Ž yF Ž x 0 . . s f U Ž u 0 . .
5 Ž 2.7.
If u 0 s yF Žx 0 . does not solve Problem DEVI, then there exists a u g R n such that T
G Ž u 0 . Ž u y u 0 . - f U Ž u 0 . y f U Ž u. and by Ž2.7. we have T
Ž u 0 . x 0 - yf Ž x 0 . y f U Ž u. or T
f Ž x 0 . q f U Ž u . y Ž x 0 . u - 0, contradicting Young’s inequality in Ž2.4.. To prove u 0 solves Problem DEVI implies x 0 solves Problem EVI, we proceed as before with the role of f, F, x, x 0 interchanged with g, G, u, u 0 , respectively, and by noting that the biconjugate of f is itself, since f is a lower semicontinuous, or closed, function. The proof of Žii. follows directly from Ž2.7.. Auslender’s definition of the gap function may now be extended for Problem EVI.
662
CHEN, GOH, AND YANG
DEFINITION 2.2 ŽExtended Gap Function.. T
f Ž x . s maxn F Ž x . Ž x y y . q f Ž x . y f Ž y . 4 .
Ž 2.8.
ygR
This form of definition readily leads to an alternative form involving the Fenchel conjugate of f :
f Ž x . s f Ž x . q f U Ž yF Ž x . . q x T F Ž x . .
Ž 2.9.
As with Auslender’s gap function, this extended gap function also has the following properties: Property 2.1. Ži. f Žx. G 0 ;x g R n, and Žii. f Žx 0 . s 0 if and only if x 0 solves Problem EVI. Clearly, Ži. follows directly from Young’s inequality and can be interpreted as a form of weak duality. Žii. follows from Theorem 1Žii. and can be interpreted as a form of strong duality. The meaning of a ‘‘gap’’ is now apparent. To complete the discussion of VI duality, we may further define the gap function for Problem DEVI as follows: DEFINITION 2.3 ŽDual Gap Function..
c Ž u . s maxn G Ž u . Ž u y v . q f U Ž u . y f U Ž v . 4 , T
vgR
u g yRange Ž F . .
Ž 2.10. Clearly, c satisfies the following gap properties: Property 2.2. Ži. c Žu. G 0 ;u g yRangeŽ F ., and Žii. c Žu 0 . s 0 if and only if u 0 solves Problem DEVI. The primal gap function and dual gap function are related as follows: THEOREM 2.2.
Let x g R n and u g yRangeŽ F .. Then
Ži. f Žx. s 0 if and only if c ŽyF Žx.. s 0 if and only if f Žx. q f U ŽyF Žx.. y x T ŽyF Žx.. s 0, and Žii. c Žu. s 0 if and only if f ŽyGŽu.. s 0 if and only if f ŽyGŽu.. q f U Žu. y uT ŽyGŽu.. s 0. Proof. Ži. f Žx. s 0 if and only if x solves Problem EVI if and only if yF Žx. solves DEVI if and only if c ŽyF Žx.. s 0 if and only if max yFy1 Ž F Ž x . . Ž u y v . y f U Ž v . 4 q f U Ž yF Ž x . . s 0, T
vgR n
GAP FUNCTIONS
663
that is, max x T v y f U Ž v . 4 q f U Ž yF Ž x . . y x T Ž yF Ž x . . s 0
vgR n
if and only if f Ž x . q f U Ž yF Ž x . . y x T Ž yF Ž x . . s 0. The proof for Žii. is similar.
3. PROPERTIES OF GAP FUNCTIONS Gap functions furnish a natural method for solving Problem EVI, since the solution of EVI is also the global minimum of the gap function, and furthermore this solution yields the known optimum value of zero. To be able to solve this optimization problem effectively, we need to understand a bit more about the convexity and smoothness of gap functions. DEFINITION 3.1. A function F: R n ª R n is monotone if, ;x, y g R n, Ž F Žx. y F Žy..T Žx y y. G 0. THEOREM 3.1. con¨ ex.
If F is affine and monotone, then the gap function f is
Proof. It suffices to show that each term of Ž2.9. is convex. f is by definition convex; x T F Žx. is convex if F is monotone. It remains to show that f U ŽyF Žx.. is convex. Now f U is the Fenchel conjugate of a convex function and hence is convex ŽRockefellar w7x.. If F is affine, so is yF. It is well known that the composition of a convex function with an affine function f U (ŽyF . is also convex. Remark 3.1. To have f strictly convex, it suffices to require either f to be strictly convex or F to be strictly monotone. Remark 3.2. Note that in the context of convex optimization, Hearn w4x requires that x T F Žx. be convex, K be polyhedral, and F be concave in order for the gap function to be convex. The last concavity requirement stems from the fact that, without f, the function f U is monotone; hence if F is concave, f U (ŽyF . is convex. Clearly, concavity of F is a weaker condition than linearity of F as required by Theorem 3.1. Unfortunately, with the presence of a convex function f, the conjugate function f U is, in general, no longer monotone. Hence requiring F to be concave would not
664
CHEN, GOH, AND YANG
have made a difference. Another sufficient condition requiring only concavity of F is as follows: THEOREM 3.2. function, and
If F Žx.T x is a con¨ ex function of x, F Žx. is a conca¨ e
f Ž x. s
½
c T x q d, q`,
if x g K , otherwise,
where K s x: Ax s b, x P 04 is a polyhedral set, then the gap function f is con¨ ex. Proof. By expression Ž2.9. and given conditions, it is sufficient to prove that f U is monotone. By the definition of Fenchel conjugate dual and the duality of linear programming, f U Ž y . s maxn x T y y f Ž x . 4 xgR
s max x T y y c T x y d 4 xgK
s yd q
min
lTAPy Tyc T
lT b 4 .
If y1 P y2 , then f U Ž y1 . G f U Ž y 2 . . Thus f U is monotone. Before investigating the differentiability of the gap function f , let us recall some definitions of a proper convex function w7x. DEFINITION 3.2. A proper convex function f : R n ª R is said to be essentially differentiable if the following three conditions hold: Ži. intŽdomŽ f .. is not empty; Žii. f is differentiable on intŽdomŽ f ..; Žiii. lim i ª q` 5 =f Žx i .5 s q` whenever x i 4 is a sequence in intŽdomŽ f .. converging to a boundary point of intŽdomŽ f ... A proper convex function f is said to be essentially strictly convex if f is strictly convex on every convex subset of x < f Žx. / B4 s domŽ f .. It follows from Rockefellar w7x that a closed proper convex function is essentially strictly convex if and only if its conjugate is essentially differentiable.
665
GAP FUNCTIONS
THEOREM 3.3. If F is differentiable on R n and f is essentially differentiable and essentially strictly con¨ ex Ž hence =f is monotone and injecti¨ e ., then the gap function f is differentiable on intŽdomŽ f .. l Fy1 Žint ŽydomŽ f U .... Furthermore, the gradient of f can be explicitly computed to be =f Ž x . s =f Ž x . y Ž =f .
y1
Ž yF Ž x . .
T
=F Ž x . q =F Ž x .
T
q x T =F Ž x . .
Ž 3.1. Proof. Let x g intŽdomŽ f .. l Fy1 ŽintŽydomŽ f U .... Then F is differentiable at x and f is differentiable at x. Thus the gradient of the first and last term of f in Ž2.9. is obvious. We now turn to the differentiability of the Fenchel conjugate f U . It is well known ŽRockefellar w7x. that u g f Žx. if and only if x g f U Žu., where f and f U are the subdifferentials of f and f U , respectively. Since f is essentially strictly convex and yF Žx. g intŽdomŽ f U .., f U is differentiable at yF Žx.. Then f Žx. is a singleton, and f Žx. s =f Žx.4 . Since f is strictly convex, then =f is strictly monotone and hence f is injective. Thus the derivative of f U Žu. is =f U Žu. s x, where =f Žx. s u or x s Ž =f .y1 Žu. s =f U Žu.. Thus the gradient of the second term in the gap function of Ž2.9. can be obtained by the chain rule to be =f U Ž yF Ž x . . s y Ž =f .
y1
Ž yF Ž x . .
T
=F Ž x . .
Since the preceding gradient formula is explicit, we may use it readily to find the minimum of the gap function, and hence to solve Problem EVI using any standard descent method, such as Newton, quasi-Newton, or conjugate gradient methods. Note that general descent methods for solving Problem VI have already been discussed in the literature; see Fukushima w3x and Zhu and Marcotte w10x. The former cast Problem VI into another differentiable optimization problem, and the latter modifies the Auslender gap function to include an extra convex term Žjust like the V term to be discussed in the next section.. Note that in both cases ŽFukushima w3x and Zhu and Marcotte w10x., the gradient formula requires the solution of another optimization problem. In the preceding gradient formula, no optimization problem needs to be solved. EXAMPLE 3.1. Consider the following asymmetric EVI problem in R 3. Let K be a polyhedral set K s x g R 3 : Ax O b, 0 O x O u 4 , where As
ž
1 y1
2 3
1 , 4
/
bs
7 , 5
ž/
2 us 3 . 4
ž/
666
CHEN, GOH, AND YANG
Let F Ž x. s
ž
5 7 y5
1 10 5
1 y10 y1 x q 15 4 y20
/ ž /
and
f Ž x . s 0.1 5x 5 2 .
Note that in this case F is linear and monotone, and by Theorem 3.1 the gap function is convex. Using the unconstrained optimization function fminu from MATLAB, and with an arbitrary initial starting point, the solution converges to xU s Ž0.3636, 0, 0.9060.T almost instantly after a few iterations on a workstation.
4. FURTHER DUALITY RESULTS It appears that if we extend the preceding extended gap function Žas defined in w2x. even further, it is possible to produce more interesting results. Let V: R n = R n ª R be a function satisfying the following assumptions: Assumption 4.1. Ži. V Žx, y. G 0 ;Žx, y. g R n = R n and V Žx, x. s 0 ;x g R n; Žii. for all x g R n, V Žx, y. is convex in the second argument; Žiii. 0 g 2 V Žx, x. ;x g R n, where 2 V Žx, x. is the subdifferential of V Žx, y. with respect to the second argument, and evaluated at y s x. Consider the extended variational inequality problem EVI Ž2.1. together with another variant of the gap function defined as follows. ŽNote that the inclusion of the V term was also discussed in Zhu and Marcotte w10x, in the context of the less general variational inequality problem VI and with some subtle difference.. T
f Ž x . s maxn F Ž x . Ž x y y . q f Ž x . y f Ž y . y V Ž x, y . 4 .
Ž 4.1.
ygR
We will establish that this is also a gap function in the following theorem: THEOREM 4.1. Property 2.1.
f is a gap function of Problem EVI; that is, f Žx. satisfies
Proof. Let T
F Ž x, y . s F Ž x . Ž y y x . q f Ž y . y f Ž x . q V Ž x, y . . Then
f Ž x . s maxn yF Ž x, y . s y minn F Ž x, y . . ygR
ygR
Ž 4.2.
667
GAP FUNCTIONS
Since F Žx, x. s 0 ;x g R n, then min y g R n F Žx, y. F 0 ;x g R n, and hence f Žx. G 0 ;x g R n. Next, suppose that x solves EVI. Then by Assumption 4.1Ži. we have T
F Ž x . Ž y y x . q f Ž y . y f Ž x . q V Ž x, y . T
G F Ž x. Ž y y x. q f Ž y. y f Ž x. G 0
;y g R n ,
that is, min y g R n F Žx, y. G 0, or f Žx. s ymin y g R n F Žx, y. F 0. Thus f Žx. s 0 since f Žy. G 0 ;y g R n. Conversely, suppose f Žx. s 0, that is, max y g R n wyF Žx, y.x s 0, which implies that yF Žx, y. F 0 ;y g R n, or F Žx, y. G 0 ;y g R n. Observe that F Žx, x. s 0, which implies that x is a solution of the following optimization problem: min F Ž x, y . .
ygR n
Since by Assumption 4.1Žii. and the assumption that f is convex, it is clear that F Žx, y. is convex in y. Consequently, the solution x of the convex optimization problem also solves the following variational inequality: Find x g R n such that qT Ž y y x . G 0
;y g R n ,
where q g 2 F Žx, x.. Hence for all z g f Žx. and for all w g 2 V Žx, x. Žwhere f is the subdifferential of f and 2 V is the subdifferential of V with respect to the second argument., we have T
F Ž x. Ž y y x. q zT Ž y y x. q w T Ž y y x. G 0
;y g R n .
Since f is proper convex, f Žy. y f Žx. G z T Žy y x. ;y g R n, where z g f Žx.. Thus T
F Ž x. Ž y y x. q f Ž y. y f Ž x. q w T Ž y y x. G 0 ;y g R n and ;w g 2 V Ž x, x . . Finally, since 0 g 2 V Žx, x. by Assumption 4.1Žiii., we have T
F Ž x. Ž y y x. q f Ž y. y f Ž x. G 0
;y g R n ;
that is, x solves Problem EVI. The gap function as defined in Ž4.1. has nice smoothness properties similar to the smooth optimization problem proposed by Fukushima w3x. Furthermore, it represents an explicit Fenchel dual optimization problem to the gap function of the corresponding dual variational inequality prob-
668
CHEN, GOH, AND YANG
lem. First, we need to define the meaning of dual optimization problems ŽBarbu w2x.. Consider the Žprimal. optimization problem: DEFINITION 4.1 ŽPrimal Fenchel Optimization Problem Pp .. min a Ž y . y b Ž y . ,
Ž 4.3.
ygR n
where a : R n ª R is a convex function and b : R n ª R is a concave function. The Fenchel dual optimization problem is defined by: ŽDual Fenchel Optimization Problem Pd .. max b U Ž v . y a U Ž v . ,
Ž 4.4.
vgR n
where a U : R n ª R is the convex conjugate function of a , that is,
a U Ž v . s maxn v T y y a Ž y . , ygR
and b U : R n ª R is the concave conjugate function of b , that is,
b U Ž v . s minn v T y y b Ž y . . ygR
As inspired by Fukushima, we specialize the gap function as defined in Ž4.1. to the following form: T
½
f Ž x . s maxn F Ž x . Ž x y y . q f Ž x . y f Ž y . y ygR
1 2
T Ž y y x. Q Ž y y x. ,
5
Ž 4.5. where Q g R n = R n is positive definite, and hence the quadratic term in Ž4.5. satisfies Assumption 4.1. This gap function is identified with Problem EVI as defined in Ž2.1.. Similarly, the following can be shown to be a gap function for the dual extended variational inequality defined in Ž2.5.:
c Ž u . s maxn G Ž u . Ž u y v . q f U Ž u . y f U Ž v . vgR
½
T
y
1 2
T Ž v y u. Qy1 Ž v y u. .
5
Ž 4.6.
Let the primal gap function f and the dual gap function c be identified, respectively, with the pair of optimization problems: Problem P1.
½
T
minn F Ž x . Ž y y x . q f Ž y . y f Ž x . q
ygR
1 2
T Ž y y x . Q Ž y y x . . Ž 4.7.
5
669
GAP FUNCTIONS
Problem P2 . maxn y G Ž u . Ž v y u . q f U Ž v . y f U Ž u . q
vgR
½
T
1 2
5
T Ž v y u . Qy1 Ž v y u .
.
Ž 4.8. THEOREM 4.2. If x sol¨ es EVI and u sol¨ es DEVI, then Problem P1 is the Fenchel dual optimization problem of Problem P2 . Proof. Let the functions a and b be defined by T
a Ž y. s F Ž x. Ž y y x. q
1 2
T Ž y y x. Q Ž y y x.
Ž 4.9.
and
b Ž y. s f Ž x. y f Ž y. . Clearly, a is convex and b is concave Žin y.. Thus the objective of Problem P1 is given by a Žy. y b Žy.. We need only to show that maxw b U Žv. y a U Žv.x is equivalent to Problem P2 : 1
a U Ž v . s maxn v T y y F Ž x . Ž y y x . y ygR
s maxn ygR
½ ½Ž
T
T
2
T
v y F Ž x. . y q F Ž x. x y
Using the fact that the Fenchel dual of u s y y x to get
a U Ž v . s maxn Ž v y F Ž x . . u y ugR
s
1 2
½
T Ž y y x. Q Ž y y x.
T
1 2
5
1 2
1 2
5
T Ž y y x. Q Ž y y x. .
x T Qx is
5
1 2
uT Qy1 u, we let
T
T
uT Qu q Ž v y F Ž x . . x q F Ž x . x
T T Ž v y F Ž x . . Qy1 Ž v y F Ž x . . q Ž v y F Ž x . . x q F Ž x . T x.
Remembering that if x solves EVI and u solves DEVI, then u s yF Žx., x s yGŽu., and x T u s f Žx. q f U Žu., consequently,
a U Ž v. s
1 2
T T Ž v q u . Qy1 Ž v q u . y Ž v q u . G Ž u . y f Ž x . y f U Ž u .
or
a U Ž yv . s
1 2
T T Ž v y u. Qy1 Ž v y u. q Ž v y u. G Ž u. y f Ž x . y f U Ž u. .
670
CHEN, GOH, AND YANG
Next,
b U Ž yv . s minn yv T y y f Ž x . q f Ž y . 4 ygR
s y maxn v T y y f Ž y . 4 y f Ž x . ygR
s yf U Ž v . y f Ž x . . Add ya U Žyv. and b U Žyv. to get
b U Ž yv . y a U Ž yv . s y G Ž u. Ž v y u. q f U Ž v . y f U Ž u. q T
1 2
T Ž v y u . Qy1 Ž v y u . .
Finally, since we are maximizing over the whole of R n, max b U Ž yv . y a U Ž yv . 4
ygR n
is the same as max b U Ž v . y a U Ž v . 4 .
ygR n
This completes the proof.
5. CONVEX LOWER BOUND TO THE GAP FUNCTION Under some further assumptions, it is also possible to establish a convex lower bound to the gap function. The following result is an extension of Theorem 3.3 of Zhu and Marcotte w10x to Problem EVI. DEFINITION 5.1. Let a scalar-valued lower-semicontinuous proper convex function f : R n ª R be given. F is said to be strongly pseudo-monotone with respect to f and with modulus m , if there exists a positive constant m such that T
F Ž x. Ž y y x. G f Ž x. y f Ž y. implies that F Ž y . Ž y y x . G m 5x y y 5 2 q f Ž x . y f Ž y . . T
671
GAP FUNCTIONS
In addition to Assumption 4.1, we further assume that:
THEOREM 5.1. Ži. ment; Žii. Žiii. Lipschitz Živ. m.
V is continuously differentiable with respect to the second argu=2 V Žx, x. s 0 ;x; the gradient of V with respect to the second argument, =2 V, is continuous in the second argument with constant L V ; F is strongly pseudo-monotone with respect to f and with modulus
Let x 0 be a solution to EVI. If Ži. ] Živ. hold, then there exists a positi¨ e constant a such that
f Ž x . G a 5x y x 0 5 2
; x g R n.
Proof. Since x 0 solves Problem EVI, we have F Žx 0 .
T
Žy y x 0 . G f Ž x 0 . y f Ž y.
;y.
Ž 5.1.
By assumption Živ., this implies that F Ž y.
T
Ž y y x 0 . G m 5x 0 y y 5 2 q f Ž x 0 . y f Ž y.
;y.
Ž 5.2.
Let x t s x q t Ž x 0 y x. ,
t g Ž 0, 1 . .
Ž 5.3.
Then, by the convexity of f, we have f Ž x t . F tf Ž x 0 . q Ž 1 y t . f Ž x . .
Ž 5.4.
By the convexity of V with respect to the second argument ŽAssumption 4.1Žii.., we have V Ž x, x t . y V Ž x, x . T
F =2 V Ž x, x t . Ž x t y x . T
s Ž =2 V Ž x, x t . y =2 V Ž x, x . . Ž x t y x . F L V 5x t y x 5 2
Ž Assumption Ž ii . .
Ž Assumption Ž iii . . .
Ž 5.5.
By definition of the gap function Žsee Ž4.1.., we have, ; t g Ž0, 1., T
f Ž x . G F Ž x . Ž x y x t . q f Ž x . y f Ž x t . y V Ž x, x t . T
G F Ž x . Ž x y x t . q t f Ž x . q f Ž x 0 . y Ž V Ž x, x t . y V Ž x, x . .
Ž from Ž ii . and Ž 5.4. .
672
CHEN, GOH, AND YANG
G t F Ž x . Ž x y x 0 . q f Ž x . y f Ž x 0 . y L V 5x t y x 5 2 T
Ž from Ž 5.3. and Ž 5.5. . G t m 5x y x 0 5 2 y t 2 L V 5x y x 0 5 2
Ž from Ž 5.1. .
s Ž t m y t 2 L V . 5x y x 0 5 2 . Since the unconstrained maximum of Ž t m y t 2 L V . occurs at mrŽ2 L V ., we choose
½
t s min 1,
m 2 LV
5
,
to obtain
f Ž x . G a 5x y x 0 5 2 , where
¡m y L 0 - a s~ m ¢4 L ,
V,
if m G 2 L V ,
2
otherwise.
V
6. CONCLUDING REMARKS The interpretation of gap functions as Young’s inequality can be generalized to the case of vector variational inequality. The duality of vector variational inequality was previously studied by Yang w9x in Banach space. Like the present case, the gap function for vector variational inequality is intimately related to duality. The problem here is that the duality can only go in one direction. Current work is underway.
REFERENCES 1. A. Auslender, ‘‘Optimisation: Methodes Numeriques,’’ Masson, Paris, 1976. ´ ´ 2. V. Barbu, ‘‘Convexity and Optimization in Banach Spaces,’’ Reidel, Dordrecht, 1986. 3. M. Fukushima, Equivalent differentiable optimization problems and descent methods for asymmetric variational inequality problems, Math. Programming 53 Ž1992., 99]110. 4. D. W. Hearn, The gap function of a convex program, Oper. Res. Lett. 1 Ž1982., 67]71. 5. U. Mosco, Dual variational inequalities, J. Math. Anal. Appl. 40 Ž1972., 202]206. 6. A. Narguney, ‘‘Network Economics}A Variational Inequality Approach,’’ Kluwer Academic, Dordrecht, 1993.
GAP FUNCTIONS
673
7. R. T. Rockefellar, ‘‘Convex Analysis,’’ Princeton University Press, 1970. 8. G. Stampacchia, Formes bilineaires coercitives sur les ensembles convexes, C. R. Acad. ´ Sci. Paris 258 Ž1964., 4413]4416. 9. X. Q. Yang, Vector variational inequality and its duality, Nonlinear Anal. 21 Ž1993., 869]877. 10. D. L. Zhu and P. Marcotte, An extended descent framework for variational inequalities, J. Optim. Theory Appl. 80 Ž1994., 349]366.
214, 658]673 Ž1997.
AY975608
On Gap Functions and Duality of Variational Inequality ProblemsU G. Y. Chen Institute of System Sciences, Academic Sinica, Beijing 100080, People’s Republic of China
C. J. Goh and X. Q. Yang† Department of Mathematics, Uni¨ ersity of Western Australia, Nedlands, Western Australia 6907, Australia Submitted by E. S. Lee Received June 10, 1996
We extend the definition of the gap function defined by Auslender for a more general class of variational inequality problems involving some convex function. A study of the duality of the extended variational inequality problem and its dual sheds new light on the meaning of gap functions. Convexity and differentiability of the gap function are also studied and sufficient conditions are derived. We also show how the gap functions for the primal and the dual are related by dual Fenchel optimization problems. Q 1997 Academic Press
1. INTRODUCTION In Hearn w4x, the gap function of a convex optimization problem is discussed, and the meaning of ‘‘gap’’ is interpreted as the difference between the cost function and the maximum of its Wolfe dual. In some operation research problems, optimization models are sometimes inadequate and problems such as asymmetric traffic equilibrium models are often formulated as variational inequality problems, which include convex optimization problems as a special case. In the operations research literature, the common variational inequality problem under studied is given as U
This research is supported by a grant from the Australian Research Council. Address for corresponding author: E-mail: [email protected].
†
658 0022-247Xr97 $25.00 Copyright Q 1997 by Academic Press All rights of reproduction in any form reserved.
659
GAP FUNCTIONS
follows: Problem VI. Given a closed convex set K ; R n and a vector-valued function F: R n ª R n, find a point x 0 g K such that T
F Žx 0 . Žx y x0 . G 0
;x g K .
Ž 1.1.
Existence, uniqueness, and solution methods for this type of problem have been extensively studied Žsee Narguney w6x.. In particular, Auslender w1x defined the following gap function for Problem VI: DEFINITION 1.1 ŽAuslender’s Gap Function.. T
f Ž x . s max F Ž x . Ž x y y . .
Ž 1.2.
ygK
It is not difficult to show that Property 1.1. Ži. f Žx. G 0 ;x g K, and Žii. f Žx 0 . s 0 if and only if x 0 solves Problem VI. These properties can also serve to be the definition of a gap function. In general, this gap function is not differentiable, but Auslender w1x shows that it is differentiable if the ground set K is strongly convex. The nondifferentiability of the gap function poses a major difficulty for minimizing the gap function as a method for solving Problem VI. Fukushima w3x subsequently proposed a differentiable optimization problem for solving Problem VI. To the best of our knowledge, there has been no interpretation of the meaning of ‘‘gap’’ as was done by Hearn Žfor convex optimization problems. in the context of variational inequality. In this paper, we seek to provide a meaningful interpretation to the gap function, and show that it is intimately related to the duality of variational inequality. We also show that Ži. and Žii. can basically be interpreted as weak duality and strong duality, respectively. Lastly, we show that the gap functions of a pair of primal]dual variational inequality corresponds to a pair of primal]dual Fenchel optimization problems.
2. DUALITY OF VARIATIONAL INEQUALITY To understand the duality of Problem VI in its full generality, it is more expedient to study the following class of extended variational inequality problems: Problem EVI. Given a vector-valued injective function F: R n ª R n and a scalar-valued lower semicontinuous proper convex function f : R n ª
660
CHEN, GOH, AND YANG
R, where R s R j q`4 , find a point x 0 g R n such that T
F Ž x 0 . Ž x y x 0 . G f Ž x 0 . y f Ž x.
;x g R n .
Ž 2.1.
Problem EVI was first studied in the context of partial differential equations by Stampacchia w8x. Note that Problem EVI reduces to Problem VI if we let f be the indicator function for the convex ground set K, that is, f Žx. s 0 if x g K and f Žx. s q` if x f K. In general, the effective domain of the convex function f implicitly defines the convex ground set K in the context of Problem VI, since if x f K, that is, f Žx. s q`, then the inequality Ž2.1. is always true, whilst if x 0 f K, then the inequality Ž2.1. is never true. The assumption that F is injective allows one to construct the following function: G: yRange Ž F . ª R n
s.t. G Ž u . s yFy1 Ž yu . .
Ž 2.2.
Consequently, F Žx. s yGy1 Žyx.. Let g: R n ª R be the Fenchel conjugate of f defined by: DEFINITION 2.1 ŽFenchel Conjugate.. f U Ž u . s maxn x T u y f Ž x . 4 .
Ž 2.3.
xgR
In the case where f is the indicator function for some convex set K, its Fenchel conjugate is the support function of K, that is, f U Žu. s max x g K uT x. It is well known ŽRockefellar w7x. that f U is convex, and that if f is closed Žor lower semicontinuous., then the Fenchel conjugate of f U is f. From the definition Ž2.3. we have: LEMMA 2.1 ŽYoung’s Inequality .. f Ž x. q f U Ž u. y x T u G 0
;x, u g R n .
Ž 2.4.
Define the dual of Problem EVI to be ŽMosco w5x.: Problem DEVI. Given G: yRangeŽ F . ª R n, as defined by Ž2.2., and g: R n ª R, as defined by Ž2.3., find a point u 0 g yRangeŽ F . such that T
G Ž u 0 . Ž u y u 0 . G f U Ž u 0 . y f U Ž u.
;u g R n .
Ž 2.5.
Problem EVI and its dual DEVI are related by the duality theorem ŽMosco w5x.: THEOREM 2.1. Ži. x 0 sol¨ es EVI if and only if u 0 s yF Žx 0 . sol¨ es DEVI.
661
GAP FUNCTIONS
Žii. The ¨ ariational inequalities Ž2.1. and Ž2.5. hold if and only if u 0 s yF Žx 0 . or x 0 s yGŽu 0 ., and T
f Ž x 0 . q f U Ž u 0 . y Ž x 0 . u 0 s 0.
Ž 2.6.
Proof. The original proof of Mosco requires the use of subdifferential and other results in abstract notions. The following proof is a simpler one which requires only the definition we have laid down so far. Say x 0 solves Problem EVI. Then T
F Ž x 0 . Ž x y x 0 . G f Ž x 0 . y f Ž x.
;x g R n ,
or T
T
yF Ž x 0 . x 0 y f Ž x 0 . G yF Ž x 0 . x y f Ž x .
;x g R n ,
implying that T
T
yF Ž x 0 . x 0 y f Ž x 0 . s maxn yF Ž x 0 . x y f Ž x . xgR
½
s f U Ž yF Ž x 0 . . s f U Ž u 0 . .
5 Ž 2.7.
If u 0 s yF Žx 0 . does not solve Problem DEVI, then there exists a u g R n such that T
G Ž u 0 . Ž u y u 0 . - f U Ž u 0 . y f U Ž u. and by Ž2.7. we have T
Ž u 0 . x 0 - yf Ž x 0 . y f U Ž u. or T
f Ž x 0 . q f U Ž u . y Ž x 0 . u - 0, contradicting Young’s inequality in Ž2.4.. To prove u 0 solves Problem DEVI implies x 0 solves Problem EVI, we proceed as before with the role of f, F, x, x 0 interchanged with g, G, u, u 0 , respectively, and by noting that the biconjugate of f is itself, since f is a lower semicontinuous, or closed, function. The proof of Žii. follows directly from Ž2.7.. Auslender’s definition of the gap function may now be extended for Problem EVI.
662
CHEN, GOH, AND YANG
DEFINITION 2.2 ŽExtended Gap Function.. T
f Ž x . s maxn F Ž x . Ž x y y . q f Ž x . y f Ž y . 4 .
Ž 2.8.
ygR
This form of definition readily leads to an alternative form involving the Fenchel conjugate of f :
f Ž x . s f Ž x . q f U Ž yF Ž x . . q x T F Ž x . .
Ž 2.9.
As with Auslender’s gap function, this extended gap function also has the following properties: Property 2.1. Ži. f Žx. G 0 ;x g R n, and Žii. f Žx 0 . s 0 if and only if x 0 solves Problem EVI. Clearly, Ži. follows directly from Young’s inequality and can be interpreted as a form of weak duality. Žii. follows from Theorem 1Žii. and can be interpreted as a form of strong duality. The meaning of a ‘‘gap’’ is now apparent. To complete the discussion of VI duality, we may further define the gap function for Problem DEVI as follows: DEFINITION 2.3 ŽDual Gap Function..
c Ž u . s maxn G Ž u . Ž u y v . q f U Ž u . y f U Ž v . 4 , T
vgR
u g yRange Ž F . .
Ž 2.10. Clearly, c satisfies the following gap properties: Property 2.2. Ži. c Žu. G 0 ;u g yRangeŽ F ., and Žii. c Žu 0 . s 0 if and only if u 0 solves Problem DEVI. The primal gap function and dual gap function are related as follows: THEOREM 2.2.
Let x g R n and u g yRangeŽ F .. Then
Ži. f Žx. s 0 if and only if c ŽyF Žx.. s 0 if and only if f Žx. q f U ŽyF Žx.. y x T ŽyF Žx.. s 0, and Žii. c Žu. s 0 if and only if f ŽyGŽu.. s 0 if and only if f ŽyGŽu.. q f U Žu. y uT ŽyGŽu.. s 0. Proof. Ži. f Žx. s 0 if and only if x solves Problem EVI if and only if yF Žx. solves DEVI if and only if c ŽyF Žx.. s 0 if and only if max yFy1 Ž F Ž x . . Ž u y v . y f U Ž v . 4 q f U Ž yF Ž x . . s 0, T
vgR n
GAP FUNCTIONS
663
that is, max x T v y f U Ž v . 4 q f U Ž yF Ž x . . y x T Ž yF Ž x . . s 0
vgR n
if and only if f Ž x . q f U Ž yF Ž x . . y x T Ž yF Ž x . . s 0. The proof for Žii. is similar.
3. PROPERTIES OF GAP FUNCTIONS Gap functions furnish a natural method for solving Problem EVI, since the solution of EVI is also the global minimum of the gap function, and furthermore this solution yields the known optimum value of zero. To be able to solve this optimization problem effectively, we need to understand a bit more about the convexity and smoothness of gap functions. DEFINITION 3.1. A function F: R n ª R n is monotone if, ;x, y g R n, Ž F Žx. y F Žy..T Žx y y. G 0. THEOREM 3.1. con¨ ex.
If F is affine and monotone, then the gap function f is
Proof. It suffices to show that each term of Ž2.9. is convex. f is by definition convex; x T F Žx. is convex if F is monotone. It remains to show that f U ŽyF Žx.. is convex. Now f U is the Fenchel conjugate of a convex function and hence is convex ŽRockefellar w7x.. If F is affine, so is yF. It is well known that the composition of a convex function with an affine function f U (ŽyF . is also convex. Remark 3.1. To have f strictly convex, it suffices to require either f to be strictly convex or F to be strictly monotone. Remark 3.2. Note that in the context of convex optimization, Hearn w4x requires that x T F Žx. be convex, K be polyhedral, and F be concave in order for the gap function to be convex. The last concavity requirement stems from the fact that, without f, the function f U is monotone; hence if F is concave, f U (ŽyF . is convex. Clearly, concavity of F is a weaker condition than linearity of F as required by Theorem 3.1. Unfortunately, with the presence of a convex function f, the conjugate function f U is, in general, no longer monotone. Hence requiring F to be concave would not
664
CHEN, GOH, AND YANG
have made a difference. Another sufficient condition requiring only concavity of F is as follows: THEOREM 3.2. function, and
If F Žx.T x is a con¨ ex function of x, F Žx. is a conca¨ e
f Ž x. s
½
c T x q d, q`,
if x g K , otherwise,
where K s x: Ax s b, x P 04 is a polyhedral set, then the gap function f is con¨ ex. Proof. By expression Ž2.9. and given conditions, it is sufficient to prove that f U is monotone. By the definition of Fenchel conjugate dual and the duality of linear programming, f U Ž y . s maxn x T y y f Ž x . 4 xgR
s max x T y y c T x y d 4 xgK
s yd q
min
lTAPy Tyc T
lT b 4 .
If y1 P y2 , then f U Ž y1 . G f U Ž y 2 . . Thus f U is monotone. Before investigating the differentiability of the gap function f , let us recall some definitions of a proper convex function w7x. DEFINITION 3.2. A proper convex function f : R n ª R is said to be essentially differentiable if the following three conditions hold: Ži. intŽdomŽ f .. is not empty; Žii. f is differentiable on intŽdomŽ f ..; Žiii. lim i ª q` 5 =f Žx i .5 s q` whenever x i 4 is a sequence in intŽdomŽ f .. converging to a boundary point of intŽdomŽ f ... A proper convex function f is said to be essentially strictly convex if f is strictly convex on every convex subset of x < f Žx. / B4 s domŽ f .. It follows from Rockefellar w7x that a closed proper convex function is essentially strictly convex if and only if its conjugate is essentially differentiable.
665
GAP FUNCTIONS
THEOREM 3.3. If F is differentiable on R n and f is essentially differentiable and essentially strictly con¨ ex Ž hence =f is monotone and injecti¨ e ., then the gap function f is differentiable on intŽdomŽ f .. l Fy1 Žint ŽydomŽ f U .... Furthermore, the gradient of f can be explicitly computed to be =f Ž x . s =f Ž x . y Ž =f .
y1
Ž yF Ž x . .
T
=F Ž x . q =F Ž x .
T
q x T =F Ž x . .
Ž 3.1. Proof. Let x g intŽdomŽ f .. l Fy1 ŽintŽydomŽ f U .... Then F is differentiable at x and f is differentiable at x. Thus the gradient of the first and last term of f in Ž2.9. is obvious. We now turn to the differentiability of the Fenchel conjugate f U . It is well known ŽRockefellar w7x. that u g f Žx. if and only if x g f U Žu., where f and f U are the subdifferentials of f and f U , respectively. Since f is essentially strictly convex and yF Žx. g intŽdomŽ f U .., f U is differentiable at yF Žx.. Then f Žx. is a singleton, and f Žx. s =f Žx.4 . Since f is strictly convex, then =f is strictly monotone and hence f is injective. Thus the derivative of f U Žu. is =f U Žu. s x, where =f Žx. s u or x s Ž =f .y1 Žu. s =f U Žu.. Thus the gradient of the second term in the gap function of Ž2.9. can be obtained by the chain rule to be =f U Ž yF Ž x . . s y Ž =f .
y1
Ž yF Ž x . .
T
=F Ž x . .
Since the preceding gradient formula is explicit, we may use it readily to find the minimum of the gap function, and hence to solve Problem EVI using any standard descent method, such as Newton, quasi-Newton, or conjugate gradient methods. Note that general descent methods for solving Problem VI have already been discussed in the literature; see Fukushima w3x and Zhu and Marcotte w10x. The former cast Problem VI into another differentiable optimization problem, and the latter modifies the Auslender gap function to include an extra convex term Žjust like the V term to be discussed in the next section.. Note that in both cases ŽFukushima w3x and Zhu and Marcotte w10x., the gradient formula requires the solution of another optimization problem. In the preceding gradient formula, no optimization problem needs to be solved. EXAMPLE 3.1. Consider the following asymmetric EVI problem in R 3. Let K be a polyhedral set K s x g R 3 : Ax O b, 0 O x O u 4 , where As
ž
1 y1
2 3
1 , 4
/
bs
7 , 5
ž/
2 us 3 . 4
ž/
666
CHEN, GOH, AND YANG
Let F Ž x. s
ž
5 7 y5
1 10 5
1 y10 y1 x q 15 4 y20
/ ž /
and
f Ž x . s 0.1 5x 5 2 .
Note that in this case F is linear and monotone, and by Theorem 3.1 the gap function is convex. Using the unconstrained optimization function fminu from MATLAB, and with an arbitrary initial starting point, the solution converges to xU s Ž0.3636, 0, 0.9060.T almost instantly after a few iterations on a workstation.
4. FURTHER DUALITY RESULTS It appears that if we extend the preceding extended gap function Žas defined in w2x. even further, it is possible to produce more interesting results. Let V: R n = R n ª R be a function satisfying the following assumptions: Assumption 4.1. Ži. V Žx, y. G 0 ;Žx, y. g R n = R n and V Žx, x. s 0 ;x g R n; Žii. for all x g R n, V Žx, y. is convex in the second argument; Žiii. 0 g 2 V Žx, x. ;x g R n, where 2 V Žx, x. is the subdifferential of V Žx, y. with respect to the second argument, and evaluated at y s x. Consider the extended variational inequality problem EVI Ž2.1. together with another variant of the gap function defined as follows. ŽNote that the inclusion of the V term was also discussed in Zhu and Marcotte w10x, in the context of the less general variational inequality problem VI and with some subtle difference.. T
f Ž x . s maxn F Ž x . Ž x y y . q f Ž x . y f Ž y . y V Ž x, y . 4 .
Ž 4.1.
ygR
We will establish that this is also a gap function in the following theorem: THEOREM 4.1. Property 2.1.
f is a gap function of Problem EVI; that is, f Žx. satisfies
Proof. Let T
F Ž x, y . s F Ž x . Ž y y x . q f Ž y . y f Ž x . q V Ž x, y . . Then
f Ž x . s maxn yF Ž x, y . s y minn F Ž x, y . . ygR
ygR
Ž 4.2.
667
GAP FUNCTIONS
Since F Žx, x. s 0 ;x g R n, then min y g R n F Žx, y. F 0 ;x g R n, and hence f Žx. G 0 ;x g R n. Next, suppose that x solves EVI. Then by Assumption 4.1Ži. we have T
F Ž x . Ž y y x . q f Ž y . y f Ž x . q V Ž x, y . T
G F Ž x. Ž y y x. q f Ž y. y f Ž x. G 0
;y g R n ,
that is, min y g R n F Žx, y. G 0, or f Žx. s ymin y g R n F Žx, y. F 0. Thus f Žx. s 0 since f Žy. G 0 ;y g R n. Conversely, suppose f Žx. s 0, that is, max y g R n wyF Žx, y.x s 0, which implies that yF Žx, y. F 0 ;y g R n, or F Žx, y. G 0 ;y g R n. Observe that F Žx, x. s 0, which implies that x is a solution of the following optimization problem: min F Ž x, y . .
ygR n
Since by Assumption 4.1Žii. and the assumption that f is convex, it is clear that F Žx, y. is convex in y. Consequently, the solution x of the convex optimization problem also solves the following variational inequality: Find x g R n such that qT Ž y y x . G 0
;y g R n ,
where q g 2 F Žx, x.. Hence for all z g f Žx. and for all w g 2 V Žx, x. Žwhere f is the subdifferential of f and 2 V is the subdifferential of V with respect to the second argument., we have T
F Ž x. Ž y y x. q zT Ž y y x. q w T Ž y y x. G 0
;y g R n .
Since f is proper convex, f Žy. y f Žx. G z T Žy y x. ;y g R n, where z g f Žx.. Thus T
F Ž x. Ž y y x. q f Ž y. y f Ž x. q w T Ž y y x. G 0 ;y g R n and ;w g 2 V Ž x, x . . Finally, since 0 g 2 V Žx, x. by Assumption 4.1Žiii., we have T
F Ž x. Ž y y x. q f Ž y. y f Ž x. G 0
;y g R n ;
that is, x solves Problem EVI. The gap function as defined in Ž4.1. has nice smoothness properties similar to the smooth optimization problem proposed by Fukushima w3x. Furthermore, it represents an explicit Fenchel dual optimization problem to the gap function of the corresponding dual variational inequality prob-
668
CHEN, GOH, AND YANG
lem. First, we need to define the meaning of dual optimization problems ŽBarbu w2x.. Consider the Žprimal. optimization problem: DEFINITION 4.1 ŽPrimal Fenchel Optimization Problem Pp .. min a Ž y . y b Ž y . ,
Ž 4.3.
ygR n
where a : R n ª R is a convex function and b : R n ª R is a concave function. The Fenchel dual optimization problem is defined by: ŽDual Fenchel Optimization Problem Pd .. max b U Ž v . y a U Ž v . ,
Ž 4.4.
vgR n
where a U : R n ª R is the convex conjugate function of a , that is,
a U Ž v . s maxn v T y y a Ž y . , ygR
and b U : R n ª R is the concave conjugate function of b , that is,
b U Ž v . s minn v T y y b Ž y . . ygR
As inspired by Fukushima, we specialize the gap function as defined in Ž4.1. to the following form: T
½
f Ž x . s maxn F Ž x . Ž x y y . q f Ž x . y f Ž y . y ygR
1 2
T Ž y y x. Q Ž y y x. ,
5
Ž 4.5. where Q g R n = R n is positive definite, and hence the quadratic term in Ž4.5. satisfies Assumption 4.1. This gap function is identified with Problem EVI as defined in Ž2.1.. Similarly, the following can be shown to be a gap function for the dual extended variational inequality defined in Ž2.5.:
c Ž u . s maxn G Ž u . Ž u y v . q f U Ž u . y f U Ž v . vgR
½
T
y
1 2
T Ž v y u. Qy1 Ž v y u. .
5
Ž 4.6.
Let the primal gap function f and the dual gap function c be identified, respectively, with the pair of optimization problems: Problem P1.
½
T
minn F Ž x . Ž y y x . q f Ž y . y f Ž x . q
ygR
1 2
T Ž y y x . Q Ž y y x . . Ž 4.7.
5
669
GAP FUNCTIONS
Problem P2 . maxn y G Ž u . Ž v y u . q f U Ž v . y f U Ž u . q
vgR
½
T
1 2
5
T Ž v y u . Qy1 Ž v y u .
.
Ž 4.8. THEOREM 4.2. If x sol¨ es EVI and u sol¨ es DEVI, then Problem P1 is the Fenchel dual optimization problem of Problem P2 . Proof. Let the functions a and b be defined by T
a Ž y. s F Ž x. Ž y y x. q
1 2
T Ž y y x. Q Ž y y x.
Ž 4.9.
and
b Ž y. s f Ž x. y f Ž y. . Clearly, a is convex and b is concave Žin y.. Thus the objective of Problem P1 is given by a Žy. y b Žy.. We need only to show that maxw b U Žv. y a U Žv.x is equivalent to Problem P2 : 1
a U Ž v . s maxn v T y y F Ž x . Ž y y x . y ygR
s maxn ygR
½ ½Ž
T
T
2
T
v y F Ž x. . y q F Ž x. x y
Using the fact that the Fenchel dual of u s y y x to get
a U Ž v . s maxn Ž v y F Ž x . . u y ugR
s
1 2
½
T Ž y y x. Q Ž y y x.
T
1 2
5
1 2
1 2
5
T Ž y y x. Q Ž y y x. .
x T Qx is
5
1 2
uT Qy1 u, we let
T
T
uT Qu q Ž v y F Ž x . . x q F Ž x . x
T T Ž v y F Ž x . . Qy1 Ž v y F Ž x . . q Ž v y F Ž x . . x q F Ž x . T x.
Remembering that if x solves EVI and u solves DEVI, then u s yF Žx., x s yGŽu., and x T u s f Žx. q f U Žu., consequently,
a U Ž v. s
1 2
T T Ž v q u . Qy1 Ž v q u . y Ž v q u . G Ž u . y f Ž x . y f U Ž u .
or
a U Ž yv . s
1 2
T T Ž v y u. Qy1 Ž v y u. q Ž v y u. G Ž u. y f Ž x . y f U Ž u. .
670
CHEN, GOH, AND YANG
Next,
b U Ž yv . s minn yv T y y f Ž x . q f Ž y . 4 ygR
s y maxn v T y y f Ž y . 4 y f Ž x . ygR
s yf U Ž v . y f Ž x . . Add ya U Žyv. and b U Žyv. to get
b U Ž yv . y a U Ž yv . s y G Ž u. Ž v y u. q f U Ž v . y f U Ž u. q T
1 2
T Ž v y u . Qy1 Ž v y u . .
Finally, since we are maximizing over the whole of R n, max b U Ž yv . y a U Ž yv . 4
ygR n
is the same as max b U Ž v . y a U Ž v . 4 .
ygR n
This completes the proof.
5. CONVEX LOWER BOUND TO THE GAP FUNCTION Under some further assumptions, it is also possible to establish a convex lower bound to the gap function. The following result is an extension of Theorem 3.3 of Zhu and Marcotte w10x to Problem EVI. DEFINITION 5.1. Let a scalar-valued lower-semicontinuous proper convex function f : R n ª R be given. F is said to be strongly pseudo-monotone with respect to f and with modulus m , if there exists a positive constant m such that T
F Ž x. Ž y y x. G f Ž x. y f Ž y. implies that F Ž y . Ž y y x . G m 5x y y 5 2 q f Ž x . y f Ž y . . T
671
GAP FUNCTIONS
In addition to Assumption 4.1, we further assume that:
THEOREM 5.1. Ži. ment; Žii. Žiii. Lipschitz Živ. m.
V is continuously differentiable with respect to the second argu=2 V Žx, x. s 0 ;x; the gradient of V with respect to the second argument, =2 V, is continuous in the second argument with constant L V ; F is strongly pseudo-monotone with respect to f and with modulus
Let x 0 be a solution to EVI. If Ži. ] Živ. hold, then there exists a positi¨ e constant a such that
f Ž x . G a 5x y x 0 5 2
; x g R n.
Proof. Since x 0 solves Problem EVI, we have F Žx 0 .
T
Žy y x 0 . G f Ž x 0 . y f Ž y.
;y.
Ž 5.1.
By assumption Živ., this implies that F Ž y.
T
Ž y y x 0 . G m 5x 0 y y 5 2 q f Ž x 0 . y f Ž y.
;y.
Ž 5.2.
Let x t s x q t Ž x 0 y x. ,
t g Ž 0, 1 . .
Ž 5.3.
Then, by the convexity of f, we have f Ž x t . F tf Ž x 0 . q Ž 1 y t . f Ž x . .
Ž 5.4.
By the convexity of V with respect to the second argument ŽAssumption 4.1Žii.., we have V Ž x, x t . y V Ž x, x . T
F =2 V Ž x, x t . Ž x t y x . T
s Ž =2 V Ž x, x t . y =2 V Ž x, x . . Ž x t y x . F L V 5x t y x 5 2
Ž Assumption Ž ii . .
Ž Assumption Ž iii . . .
Ž 5.5.
By definition of the gap function Žsee Ž4.1.., we have, ; t g Ž0, 1., T
f Ž x . G F Ž x . Ž x y x t . q f Ž x . y f Ž x t . y V Ž x, x t . T
G F Ž x . Ž x y x t . q t f Ž x . q f Ž x 0 . y Ž V Ž x, x t . y V Ž x, x . .
Ž from Ž ii . and Ž 5.4. .
672
CHEN, GOH, AND YANG
G t F Ž x . Ž x y x 0 . q f Ž x . y f Ž x 0 . y L V 5x t y x 5 2 T
Ž from Ž 5.3. and Ž 5.5. . G t m 5x y x 0 5 2 y t 2 L V 5x y x 0 5 2
Ž from Ž 5.1. .
s Ž t m y t 2 L V . 5x y x 0 5 2 . Since the unconstrained maximum of Ž t m y t 2 L V . occurs at mrŽ2 L V ., we choose
½
t s min 1,
m 2 LV
5
,
to obtain
f Ž x . G a 5x y x 0 5 2 , where
¡m y L 0 - a s~ m ¢4 L ,
V,
if m G 2 L V ,
2
otherwise.
V
6. CONCLUDING REMARKS The interpretation of gap functions as Young’s inequality can be generalized to the case of vector variational inequality. The duality of vector variational inequality was previously studied by Yang w9x in Banach space. Like the present case, the gap function for vector variational inequality is intimately related to duality. The problem here is that the duality can only go in one direction. Current work is underway.
REFERENCES 1. A. Auslender, ‘‘Optimisation: Methodes Numeriques,’’ Masson, Paris, 1976. ´ ´ 2. V. Barbu, ‘‘Convexity and Optimization in Banach Spaces,’’ Reidel, Dordrecht, 1986. 3. M. Fukushima, Equivalent differentiable optimization problems and descent methods for asymmetric variational inequality problems, Math. Programming 53 Ž1992., 99]110. 4. D. W. Hearn, The gap function of a convex program, Oper. Res. Lett. 1 Ž1982., 67]71. 5. U. Mosco, Dual variational inequalities, J. Math. Anal. Appl. 40 Ž1972., 202]206. 6. A. Narguney, ‘‘Network Economics}A Variational Inequality Approach,’’ Kluwer Academic, Dordrecht, 1993.
GAP FUNCTIONS
673
7. R. T. Rockefellar, ‘‘Convex Analysis,’’ Princeton University Press, 1970. 8. G. Stampacchia, Formes bilineaires coercitives sur les ensembles convexes, C. R. Acad. ´ Sci. Paris 258 Ž1964., 4413]4416. 9. X. Q. Yang, Vector variational inequality and its duality, Nonlinear Anal. 21 Ž1993., 869]877. 10. D. L. Zhu and P. Marcotte, An extended descent framework for variational inequalities, J. Optim. Theory Appl. 80 Ž1994., 349]366.