- Email: [email protected]
Study of M-stationarity and strong stationarity for a class of SMPCC problems via SAA Method
Accepted Manuscript Study of M-stationarity and strong stationarity for a class of SMPCC problems via SAA Method Arnab Sur PII: DOI: Reference:
S0167-6377(16)30010-4 http://dx.doi.org/10.1016/j.orl.2016.04.001 OPERES 6081
To appear in:
Operations Research Letters
Received date: 11 August 2015 Revised date: 31 March 2016 Accepted date: 1 April 2016 Please cite this article as: A. Sur, Study of M-stationarity and strong stationarity for a class of SMPCC problems via SAA Method, Operations Research Letters (2016), http://dx.doi.org/10.1016/j.orl.2016.04.001 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
*Manuscript Click here to view linked References
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
Study of M-stationarity and strong stationarity for a Class of SMPCC Problems via SAA Method Arnab Sur∗
Abstract In this article our main aim is to analyze the relative importance of M-stationarity concept over strong stationarity of one stage stochastic mathematical programming problems with complementarity constraints (SMPCC, in short). We establish the consistency of M-stationary multipliers when the SMPCC problem is approximated through the sample average approximation (SAA, for short) method, under the assumption of SMPCC-NNAMCQ. On contrary, strong stationary multipliers are not consistent even in presence of SMPCC-LICQ.
Key Words: Stochastic mathematical programming problem with complementarity constraints (SMPCC), Constraint Qualifications, Strong and M-stationarity, Sample average approximation (SAA) method.
∗ The University of Chicago Booth School of Business, Chicago, IL 60637, USA. email: [email protected]
1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
2
1
Introduction
The study of the stochastic mathematical programming problems with complementarity constraints (SMPCC, for short) is quite a recent activity and there is a great current interest among the optimizers to analyze this class of problems. Stochastic mathematical programming problems with complementarity constraints (SMPCC) is a subclass of the stochastic mathematical programming problems under equilibrium constraints (SMPEC, for short). In SMPEC problems the constraints include stochastic variational inequalities or complementarities. The SMPEC problem with stochastic variational inequality was first studied by Patriksson and Wynter [13]. Later, it was studied in a more detailed fashion by Ye [21], Evgrafov and Patriksson [4], Shapiro [17], Fukushima and Lin [6], Ralph and Xu [14]. In the above mentioned articles the authors considered stochastic variational inequalities (VI, for short) and then complementarities as a special case of stochastic VI. Moreover, SMPEC problems have many applications in economics, electricity markets, traffic networks, management sciences; see, for example Christiansen, Patriksson and Wynter [3], Werner [18], Werner and Wang [19], Chen, Wets and Zhang [2], Henrion and R¨ omisch [8] and the references therein. In this article we shall consider the following model of one-stage SMPCC problem: min
E[f (x, ξ)]
0 ≤ E[G(x, ξ)] ⊥ E[H(x, ξ)] ≥ 0;
(1.1)
where f : Rn × Ξ → R, H, G : Rn × Ξ → Rl and ξ : Ω → Ξ, where Ξ ⊂ Rm is the support set of
the random variable and E[.] denotes the expected value with respect to the probability distribution of ξ and a ⊥ b means that vector a is perpendicular to vector b. This model can be viewed as stochastic
counterpart of the deterministic model considered by Flegel and Kanzow in [5]. A similar one-stage SMPCC model was introduced by Birbil, G¨ urkan and Listes [1] with an application related to toll pricing in a
transportation network. They compared this model with other stochastic mathematical programs with equilibrium constraints from the literature.
2
Definitions and Assumptions
In this section we shall recall necessary preliminaries available in the literature. x∗ is a feasible solution to the SMPCC problem (1.1) and splitting the complementarity term we obtain the following sets of indices corresponding to x∗ which are essential to define stationarity concepts and SMPCC tailored constraint qualifications. α := α(x∗ ) := {i : E[Gi (x∗ , ξ)] = 0, E[Hi (x∗ , ξ)] > 0}, β := β(x∗ ) := {i : E[Gi (x∗ , ξ)] = 0, E[Hi (x∗ , ξ)] = 0}, γ := γ(x∗ ) := {i : E[Gi (x∗ , ξ)] > 0, E[Hi (x∗ , ξ)] = 0}.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
3 Now observe that, Card(α)+ Card(β)+ Card(γ)=l. Further if β = φ, then x∗ will be called a strict complementarity solution. First we would like to write down the definitions of strong and M-stationary conditions, based on the sets of indices. Definition 2.1. Let us consider the SMPCC problem (1.1) with f (., ξ(ω)), Gi (., ξ(ω)) and Hi (., ξ(ω)), for i = 1, ..., l are continuously differentiable ω-a.e. and x∗ is a feasible point of (1.1). Then x∗ is said to satisfy ¯0 ≥ 0, λ ¯ G ∈ R and λ ¯H ∈ R, for i = 1, ....l, Fritz John type strong stationary conditions if there exist scalars λ i i not all zero, such that:
¯0 E[▽x f (x∗ , ξ)] = i) λ
l X
¯G E[▽x Gi (x∗ , ξ)] + λ i
¯ H E[▽x Hi (x∗ , ξ)]; λ i
i=1
i=1
¯ G ∈ R, λ ¯H ∈ R, λ ¯ G = 0, λ ¯H = 0; ii) λ α γ γ α iii)
l X
¯G ≥ 0 and λ ¯ H ≥ 0, ∀ i ∈ β. λ i i
¯0 = If x∗ is a normal Fritz John multiplier, i.e., λ 6 0, then x∗ is a strong stationary point. x∗ is said to be an
¯ G > 0 and M-stationary point if condition iii) in the above definition is replaced by the following: either (λ i ¯ H > 0) or λ ¯G λ ¯H λ i i i = 0, ∀ i ∈ β.
Various stationarity concepts exist in the literature due to different reformulation of MPCC problem. The above condition is termed as M-stationarity as derivation involves calculation of Mordukhovich coderivatives, see [21]. Definition 2.2. SMPCC-No Non-zero Abnormal Multiplier constraint qualification (SMPCC-NNAMCQ for short) is said to be satisfied at a feasible point x∗ if there exist no non-zero scalars λG ∈ Rl and λH ∈ Rl ,
such that
i) 0 =
l X
∗ λG i E[▽x Gi (x , ξ)] +
ii) iii)
= 0, if i ∈ γ and
∗ λH i E[▽x Hi (x , ξ)];
i=1
i=1
λG i
l X
λH i
= 0, if i ∈ α;
H G H either (λG i > 0 and λi > 0) or λi λi = 0, ∀ i ∈ β.
It is clear from the above definition that SMPCC-NNAMCQ is weaker than SMPCC-LICQ. This constraint qualification in the literature of deterministic MPCC problems was introduced by Ye in [21] and later, used it to derive M-stationarity of MPCC problems in [22]; its stochastic version can be found in [20]. Moreover, the following assumptions on the objective and constraint functions are necessary to execute the SAA analysis of SMPCC problem (1.1). They are important to exchange the differential and expectation operations on random functions.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
4 Assumption 2.1. Assume that the functions f (x, .), Gi (x, .) and Hi (x, .) are measurable and bounded by integrable functions Kf (ω), KiG (ω) and KiH (ω) such that k f (x, ξ(ω)) k≤ Kf (ω), k Gi (x, ξ(ω)) k≤ KiG (ω),
and k Hi (x, ξ(ω)) k≤ KiH (ω) ω-a.e, for i = 1, ....l. Further assume that there exist positive valued random variables Cf (ω), CiG (ω) and CiH (ω) such that E[Cf (ω)], E[CiG (ω)] and E[CiH (ω)] for all i = 1, .....l are finite
and let there exist a neighbourhood U of x∗ such that, for every x1 , x2 in U , we have a) |f (x1 , ξ(ω)) − f (x2 , ξ(ω))| ≤ Cf (ω)||x1 − x2 ||
ω-a.e.,
b) |Gi (x1 , ξ(ω)) − Gi (x2 , ξ(ω))| ≤ CiG (ω)||x1 − x2 ||
ω-a.e.,
c) |Hi (x1 , ξ(ω)) − Hi (x2 , ξ(ω))| ≤ CiH (ω)||x1 − x2 ||
ω-a.e.
We shall study strong and M-stationarity conditions using the sample average approximation method (SAA for short) under certain constraint qualifications. The SAA method is a well known technique to solve stochastic programming models with expectation functionals. The consistency analysis of strong and M-stationary points corresponding to a sequence of SAA problems will help us to determine the relative importance of Mstationarity over strong stationarity under weaker conditions than SMPCC-linear independence constraint qualification (SMPCC-LICQ). We shall show that an accumulation point of a sequence of M-stationary points of SAA problems will be an M-stationary point of the SMPCC problem. In general, this is not true for strong stationary points even under the assumption of SMPCC-LICQ.
3
Analysis of Sample Average Approximation Method for SMPCC Problems
In this section we shall discuss the Sample Average Approximation (SAA, for short) method to establish the relative advantage of M-stationarity conditions over strong stationarity in the absence of SMPCC-LICQ. To solve stochastic optimization problems through the SAA method one needs to discretize the underlying probability distribution associated with the random variable. Monte Carlo sampling is an approach to such a discretization. Here a random (or pseudo-random) sample ξ 1 , ξ 2 ,.....ξ N of the random variable ξ is generated and we assume that the sample comprises N independent and identically distributed realizations of ξ. The expected value function E[f (x, ξ)] is approximated by the corresponding sample average function N X 1 f (x, ξ k ) and consequently the SMPCC problem (1.1) will be approximated through the following N k=1
sequence of SAA problems (dependent on N ): min
N 1 X f (x, ξ k ) N k=1
(3.1)
N N 1 X 1 X G(x, ξ k )⊥ H(x, ξ k ) ≥ 0; 0≤ N N k=1
k=1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
5 where f : Rn × Ξ → R and H, G : Rn × Ξ → Rl , and N = 1, 2, .....
¨ The idea was developed under different names: Robinson [15] and G¨ urkan, Ozge and Robinson [7] named it sample path optimization, whereas in Kleywegt, Shapiro and Homem-de-Mello [11] it is named as sample average approximation method. In 2006, Shapiro [17] introduced the SAA method to the literature on SMPEC problems. Now we shall study the convergence analysis of the Fritz John type M-stationary points of a sequence of SAA problems corresponding to the SMPCC problem (1.1) as the sample size N increases. We use the uniform law of large numbers for a random function (for example, see Proposition 7 in [16]) to establish the following convergence result on M-stationarity. Theorem 3.1. Let us consider the SMPCC problem (1.1) with f (., ξ(ω)), Gi (., ξ(ω)) and Hi (., ξ(ω)), i =
1, ....l are continuously differentiable ω-a.e. and Assumption 2.1 holds. Now let us consider a sequence of SAA problems corresponding to the SMPCC problem and let xN be a Fritz John type M-stationary point of the SAA problem (3.1). If x∗ is a cluster point of the sequence {xN }, ω-a.e. Then x∗ is an M-stationary point of the SMPCC problem (1.1), provided SMPCC-NNAMCQ holds at x∗ .
Proof. Consider the SAA problem (3.1) and xN be a Fritz John type M-stationary point of the SAA problem ¯ 0 )N ≥ 0, (3.1). Then by the definition of Fritz John type M-stationarity conditions we have scalars (λ
¯ G )N ∈ R, i = 1, ....l and (λ ¯ H )N ∈ R, i = 1, ....l such that (λ i i i)
N l N l N X X 1 X ¯G N 1 X ¯H N 1 X ¯ N [ [ (λ0 ) ▽x f (xN , ξ k ) = (λi ) ▽x Gi (xN , ξ k )] + (λi ) ▽x Hi (xN , ξ k )]. N N N i=1 i=1 k=1
k=1
k=1
¯ 0 )N , (λ ¯ G )N , (λ ¯ H )N ) 6= 0, ii) ((λ
¯ H )N = 0, where αN := α(xN ) and γN := γ(xN ), ¯ G )N = 0, (λ ¯ H )N ∈ R, (λ ¯ G )N ∈ R, (λ iii) (λ αN γN γN αN iv)
¯ G )N > 0 and (λ ¯ H )N > 0) or (λ ¯ G )N (λ ¯ H )N = 0, ∀ i ∈ βN , where βN := β(xN ). ((λ i i i i
Let
¯ 0 )N , (λ ¯ G )N ), ....., (λ ¯ G )N , (λ ¯ H )N , ......(λ ¯ H )N ) ((λ 1 1 l l λN = q . G G H N 2 N 2 N 2 N 2 ¯ ¯ ¯ ¯ ¯ H )N }2 {(λ0 ) } + {(λ1 ) } + .......{(λl ) } + {(λ1 ) } + ......{(λ l
So, k λN k= 1 for all N ∈ N, a bounded sequence which implies that there exists a convergent subsequence; hence, without any loss of generality, let {λN } be such that λN → λ∗ , ω-a.e. and by the continuity of the
norm k λ∗ k= 1, ω-a.e. Moreover, given that, xN → x∗ , ω-a.e. (without any loss of generality). Let us now
denote that
G N G N H N H N λN = (λN 0 , (λ1 ) , ....(λl ) , (λ1 ) , ......(λl ) ). ∗ G ∗ H ∗ H ∗ λ∗ = (λ∗0 , (λG 1 ) , ....(λl ) , (λ1 ) , ......(λl ) ).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
6
Now dividing both sides of i) by
q ¯ H )N }2 + ...{(λ ¯ H )N }2 , we get ¯ 0 )N }2 + {(λ ¯ G )N }2 + ...{(λ ¯ G )N }2 + {(λ {(λ 1 1 i l
N l N l N X X 1 X G N 1 X H N 1 X N [ [ (λ0 )▽x f (xN , ξ k ) = (λi ) ▽x Gi (xN , ξ k )] + (λi ) ▽x Hi (xN , ξ k )]. N N N i=1 i=1 k=1
k=1
(3.2)
k=1
And from ii), iii) and iv) we obtain ((λ0 )N , (λG )N , (λH )N ) 6= 0,
v)
H N G N N H N (λG αN ) ∈ R, (λγN ) ∈ R, (λγN ) = 0, (λαN ) = 0,
vi) vii)
N H N G N H N ((λG i ) > 0 and (λi ) > 0) or (λi ) (λi ) = 0, ∀ i ∈ βN .
Then the above equation (3.2) implies that (λN 0 )[
N l N l N X X X X 1 X N 1 N 1 (λG (λH ▽x f (xN , ξ k )] = ▽x Gi (xN , ξ k )] + ▽x Hi (xN , ξ k )]. i ) [ i ) [ N N N i=1 i=1 k=1
k=1
(3.3)
k=1
Now for i = 1, ....l ▽x f (., ξ k ), ▽x Gi (., ξ k ) and ▽x Hi (., ξ k ) are continuous on a unit closed ball B1N around N N N 1 X 1 X 1 X xN , for k = 1, ....N. Hence ▽x f (., ξ k ), ▽x Gi (., ξ k ) and ▽x Hi (., ξ k ) are continuous on the N N N k=1
k=1
unit closed ball B1N , for i = 1, ....l.
k=1
We know that xN → x∗ ω-a.e. Hence, for some given ε ∈ (0, 12 ) there exists m ∈ N, such that
k xN −x∗ k< ε, ∀ N > m and ω-a.e. Now consider any N > m and as we have seen above k xN −x∗ k< ω-a.e.; hence, x ∈ ∗
ball
B1M
around x
B1N
M
ω-a.e., as
B1N
1 2
< 1
is the closed unit ball with center x . So if we take the closed unit N
for some M > m, then {x∗ } ⊆ B1M , ω-a.e. and note that for any N > m we have
k xN − xM k≤k xN − x∗ k + k xM − x∗ k<
1 2
+
1 2
= 1 ω-a.e. This implies xN ∈ B1M ω-a.e., for all N > m
and for some M > m. This is true for any arbitrary M > m; hence, the sequence {xN }N >m ⊂ B1M , ω-a.e. \ for any M > m. Thus x∗ ∈ B and {xN }N >m ⊂ B ω-a.e., where B = B1N . N >m
Now using a) in assumption 2.1 we have for x ∈ B, k ▽x f (x, ξ(ω)) k≤ Cf (ω)
ω − a.e.
Further noting that ▽x f (., ξ(ω)) is continuous on B, ω − a.e. and the fact that the sample is i.i.d. allows us to conclude by using Proposition 7 in [16] that the expected value function E[▽x f (x, ξ)] of the SMPCC
problem (1.1) is finite valued and continuous and the expected value function of the SAA problem (3.1) N 1 X ▽x f (x, ξ k ) converges to E[▽x f (x, ξ)], ω-a.e. uniformly on B. N k=1
So we have for N > m, FN (.) =
N 1 X ▽x f (., ξ k ) is a sequence of continuous functions on a compact set N k=1
B such that FN (.) → F (.) = E[▽x f (., ξ)] uniformly on B, ω-a.e. and {xN }N >m is a sequence in B with N 1 X xN → x∗ , ω-a.e. Hence we have FN (xN ) → F (x∗ ) i.e. ▽x f (xN , ξ k ) converges to E[▽x f (x∗ , ξ)], ω-a.e. N k=1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
7
Similarly, we can show that for i = 1, ....l,
N N 1 X 1 X ▽x Gi (xN , ξ k ) converges to E[▽x Gi (x∗ , ξ)] and ▽x Hi (xN , ξ k ) N N k=1
converges to E[▽x Hi (x∗ , ξ)], ω-a.e. Now we know that λN → λ∗ , ω-a.e. and
k=1
N 1 X ▽x f (xN , ξ k ) converges to E[▽x f (x∗ , ξ)], ω-a.e. Hence we have N k=1
N N X 1 X N 1 ▽x f (xN , ξ k )] converges to λ∗0 E[▽x f (x∗ , ξ)], ω-a.e. Similarly, for i = 1, ....l, (λG ▽x Gi (xN , ξ k )] λN i ) [ 0 [ N N k=1
k=1
converges to
∗ ∗ (λG i ) E[▽x Gi (x , ξ)]
and
N (λH i ) [
N 1 X ∗ ∗ ▽x Hi (xN , ξ k )] converges to (λH i ) E[▽x Hi (x , ξ)], ω-a.e. N k=1
l N l N X X X X N 1 ∗ ∗ N 1 (λH (λG ▽x Gi (xN , ξ k )] converges to ▽x Hi (xN , ξ k )] (λG This implies that, i ) [ i ) E[▽x Gi (x , ξ)] and i ) [ N N i=1 i=1 i=1
converges to
l X
l X
k=1
k=1
∗ ∗ (λH i ) E[▽x Hi (x , ξ)], ω-a.e.
i=1
Now from equation (3.3) we have (λN 0 )[
N l N l N X X X X 1 X N 1 N 1 (λG (λH ▽x f (xN , ξ k )] = ▽x Gi (xN , ξ k )] + ▽x Hi (xN , ξ k )]. i ) [ i ) [ N N N i=1 i=1 k=1
k=1
(3.4)
k=1
Now passing to the limit as N → ∞ in equation (3.4) we have λ∗0 E[▽x f (x∗ , ξ)] =
l X
∗ ∗ (λG i ) E[▽x Gi (x , ξ)] +
l X
∗ ∗ (λH i ) E[▽x Hi (x , ξ)].
i=1
i=1
Now using Assumption 2.1 we have that x ∈ B and for i = 1, ....l k Gi (x, ξ(ω)) k≤ KiG (ω)
ω − a.e.
Further, for i = 1, ....l, Gi (., ξ(ω)) is continuous on B, ω-a.e. and as the sample is i.i.d. we can conclude by using the Proposition 7 in [16] that for i = 1, ....l, the expected value function E[Gi (x, ξ)] of the SMPCC problem (1.1) is finite valued and continuous and the expected value function of the SAA probN 1 X lem (3.1) Gi (x, ξ k ) converges to E[Gi (x, ξ)], ω-a.e. uniformly on B. Hence for i = 1, ....l we have N k=1
N 1 X Gi (xN , ξ k ) converges to E[Gi (x∗ , ξ)], ω-a.e. N k=1
Similarly, we can show that for i = 1, ....l,
N 1 X Hi (xN , ξ k ) converges to E[Hi (x∗ , ξ)], ω-a.e. N k=1
So we have for i = 1, ....l, E[Hi (x∗ , ξ)], ω-a.e.
N N 1 X 1 X Gi (xN , ξ k ) converges to E[Gi (x∗ , ξ)] and Hi (xN , ξ k ) converges to N N k=1
k=1
Further, we assume that i ∈ β, for some fixed i. Then we know that E[Gi (x∗ , ξ)] = 0 and E[Hi (x∗ , ξ)] = 0, i.e. N N N 1 X 1 X 1 X [ Gi (xN , ξ k )] → 0 and [ Hi (xN , ξ k )] → 0, ω-a.e. So for each i ∈ β, we have [ Gi (xN , ξ k )] → 0 N N N k=1
k=1
k=1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
8 N
1 X [ Hi (xN , ξ k )] → 0, ω-a.e. as E[Gi (x∗ , ξ)] = 0 and E[Hi (x∗ , ξ)] = 0 respectively. Now we will study N k=1 certain cases to see what restriction can be imposed on the multipliers if i ∈ β. We would again like to stress and
that we have considered a fixed i ∈ β.
N
Case 1: Let us assume that there exists m ∈ N such that for N > m, N
1 X [ Gi (xN , ξ k )] > 0, ω-a.e. and N k=1
1 X N [ Hi (xN , ξ k )] = 0, ω-a.e. i.e. i ∈ γN , ω-a.e. for N > m. Hence (λG = 0, ω-a.e. for N > m, this i ) N k=1
∗ N ∗ G ∗ H ∗ implies (λG i ) = 0 (as λ → λ , ω-a.e.), i.e. (λi ) (λi ) = 0.
N
Case 2: Let us assume that there exists m ∈ N such that for N > m, N
1 X [ Gi (xN , ξ k )] = 0, ω-a.e. and N k=1
1 X N Hi (xN , ξ k )] > 0, ω-a.e. i.e. i ∈ αN , ω-a.e. for N > m. Hence (λH = 0, ω-a.e. for N > m, this [ i ) N k=1
∗ G ∗ H ∗ implies (λH i ) = 0, i.e. (λi ) (λi ) = 0.
N
Case 3: Let us assume that there exists m ∈ N such that for N > m, N
1 X Gi (xN , ξ k )] = 0, ω-a.e. and [ N k=1
1 X [ Hi (xN , ξ k )] = 0, ω-a.e. i.e. i ∈ βN , ω-a.e. for N > m. Noting that xN is an M-type Fritz John point N k=1
N H N G N H N of the SAA problem (3.1), ω-a.e. Hence (λG i ) > 0 and (λi ) > 0) or (λi ) (λi ) = 0, ω-a.e. for N > m. N N N H N Let without any loss of generality (λG > 0 and (λH > 0) or (λG = 0, ω-a.e. for N ∈ N. This i ) i ) i ) (λi )
leads to the following possibilities.
N H N G ∗ i) If for some m1 ∈ N and for any N > m1 , (λG i ) > 0 and (λi ) > 0, ω-a.e. then we have either (λi ) > 0
∗ G ∗ H ∗ and (λH i ) > 0 or (λi ) (λi ) = 0.
N H N G ∗ H ∗ ii) If there exists m1 ∈ N such that for N > m1 , (λG i ) (λi ) = 0, ω-a.e. then (λi ) (λi ) = 0. N H N N N = 0, ω-a.e. for > 0, ω-a.e. for infinitely many N > m and b) (λG > 0 and (λH iii) If a) (λG i ) (λi ) i ) i ) N H N infinitely many N . Thus we can construct two subsequences such that {N > m : (λG i ) > 0 and (λi ) > 0, N H N ω-a.e.} and {N > m : (λG i ) (λi ) = 0, ω-a.e.}.
G ∗ H ∗ G ∗ H ∗ ∗ H ∗ Then from a) we get, (λG i ) > 0 and (λi ) > 0 or (λi ) (λi ) = 0, and from b) we get, (λi ) (λi ) = 0. ∗ H ∗ Thus from a) and b) we get, (λG i ) (λi ) = 0.
Case 4: Let us assume that, a)
N
N
k=1
k=1
1 X 1 X [ Gi (xN , ξ k )] > 0, ω-a.e. and [ Hi (xN , ξ k )] = 0, ω-a.e. for N N
infinitely many N . N N 1 X 1 X b) [ Gi (xN , ξ k )] = 0, ω-a.e. and [ Hi (xN , ξ k )] > 0, ω-a.e. for infinitely many N . N N k=1 N
k=1 N
k=1
k=1
1 X 1 X Gi (xN , ξ k )] = 0, ω-a.e. and [ Hi (xN , ξ k )] = 0, ω-a.e. for infinitely many N . c) [ N N
∗ H ∗ G ∗ H ∗ G ∗ H ∗ From a) and b) we get, (λG i ) (λi ) = 0 and from c) we get, (λi ) > 0 and (λi ) > 0 or (λi ) (λi ) = 0.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
9 ∗ H ∗ Hence from a), b) and c) we get, (λG i ) (λi ) = 0. Among the three possibilities mentioned above a), b) and
c) all can occur, or only a) or b) or c) can occur or they can occur even in pairs a), b) or b), c) or a), c). ∗ H ∗ This will also lead us to the fact (λG i ) (λi ) = 0. ∗ H ∗ G ∗ H ∗ Hence if i ∈ β, then ((λG i ) > 0 and (λi ) > 0) or (λi ) (λi ) = 0. N
Now if i ∈ α, then E[Hi (x∗ , ξ)] > 0. This implies,
1 X [ Hi (xN , ξ k )] > 0, ω-a.e. except for finitely many N k=1
N H ∗ N . Hence (λH i ) = 0, ω-a.e. except for finitely many N . So we can conclude that (λi ) = 0, when i ∈ α.
∗ Similarly, can be shown if i ∈ γ, then E[Gi (x∗ , ξ)] > 0 which implies (λG i ) = 0.
Hence we have,
a) λ∗0 E[▽x f (x∗ , ξ k )] =
l X
∗ ∗ k (λG i ) E[▽x Gi (x , ξ )] +
i=1
l X
∗ ∗ k (λH i ) E[▽x Hi (x , ξ )].
i=1
b)
(λ∗0 , (λG )∗ , (λH )∗ )
6= 0.
c)
∗ H ∗ G ∗ H ∗ (λG α ) ∈ R, (λγ ) ∈ R, (λγ ) = 0, (λα ) = 0.
d)
∗ H ∗ G ∗ H ∗ ((λG i ) > 0 and (λi ) > 0) or (λi ) (λi ) = 0, ∀ i ∈ β.
Now if we put λ∗0 = 0, then by SMPCC-NNAMCQ we shall get λ∗ = 0, which leads us to a contradiction as k λ∗ k= 1. Hence x∗ is an M-stationary point of the SMPCC problem (1.1). Recently, in [12], the authors did the convergence analysis of strong and M-stationary points under the assumptions of a natural type of error bound, the S-type error bound on the complementarity constraints, metric regularity, and with some other assumptions. Moreover, the authors mentioned that the convergence result holds for strong stationary points, but unfortunately this is not true. The following result on consistency of strong stationary points of SMPCC problem (1.1) shows that an accumulation point of a sequence of strong stationary points of SAA problems might not be a strong stationary point. Proposition 3.1. Suppose all assumptions in Theorem 3.1 are true and x∗ is an accumulation point of a sequence {xN } of Fritz John type strong stationary points, ω-a.e. Then, in general, we cannot conclude that x∗ is a strong stationary point even under the assumption of SMPCC-LICQ at x∗ .
Proof. First assume that i ∈ β, for some i = 1, ....l, and i is fixed here. Further assume that there exists N N 1 X 1 X Gi (xN , ξ k )] > 0, ω-a.e. and [ Hi (xN , ξ k )] = 0, ω-a.e. i.e. i ∈ γN , [ m ∈ N such that for N > m, N i=1 N i=1
N ∗ N ω-a.e. for N > m. Hence (λH ∈ R, ω-a.e. for N > m, this does not imply (λH → λ∗ , i ) i ) ≥ 0 (though λ
ω-a.e.). So, in general, we cannot conclude that x∗ is a strong stationary point.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
10 Two aspects are crucial in this article: the consistency of M-stationary points which is not available for strong stationary points and consistency is achievable under the assumption of SMPCC-NNAMCQ. The consistency analysis shows that an accumulation point of a sequence of M-stationary points of SAA problems corresponding to the SMPCC problem (1.1) will be an M-stationary point, but such a convergence result is not true for strong stationary points even under the assumption of SMPCC-LICQ. That is, an accumulation point of a sequence of M-stationary points of SAA problems will be an M-stationary point and could be a local or global minimum of the SMPCC problem in the absence of SMPCC-LICQ. Moreover, recent articles such as in [9] and [10], have shown that there are certain relaxation problems which converge to M-stationary points. We can treat SAA problems in this framework to solve them using the relaxation schemes mentioned in those articles, then we shall obtain a sequence of M-stationary points and consistency of M-stationary points will play a key role at that time. Hence, this result is important from the algorithmic perspective and establishes the relative importance of M-stationarity conditions over strong stationarity in the absence of SMPCC-LICQ.
References [1] S. I. Birbil, G. G¨ urkan and O. Listes (2006). Solving stochastic mathematical programs with complementarity constraints using simulation. Math. Oper. Res. Vol. 31, pp. 739-760. [2] X. Chen, R. J.-B. Wets and Y. Zhang(2012). Stochastic variational inequalities: residual minimization smoothing sample average approximations. SIAM Journal on Optimization Vol. 21, pp. 361-371. [3] S. Christiansen, M. Patriksson and L. Wynter (2001). Stochastic bilevel programming in structural optimization. Structual Multidisciplinary Optimization. Vol. 22, pp. 649-673. [4] A. Evgrafov and M. Patriksson (2004). On the Existence of Solutions to Stochastic Mathematical Programs with Equilibrium Constraints. Journal of Optimization Theory and Applications. Vol. 121, pp. 65-76. [5] M. Flegel and C.Kanzow (2003). A Fritz John approach to first order optimality conditions for mathematical programs with equilibrium constraints. Optimization. Vol. 52, pp. 277-286. [6] M. Fukushima and G-H. Lin (2010). Stochastic Equilibrium Problems and Stochastic Mathematical Programs with Equilibrium Constraints: A Survey. Pacific Journal of Optimization. Vol. 6, pp. 455-482. ¨ [7] G. G¨ urkan, A. Y. Ozge, and S. M. Robinson (1999). Sample-path solutions of stochastic variational inequalities. Math. Program. Vol. 84, pp. 313-334. [8] R. Henrion, W. R¨ omisch (2007). On M-stationary points for a stochastic equilibrium problem under equilibrium constraints in electricity spot market modeling. Appl. Math. Vol. 52, pp. 473-494.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
11 [9] T. Hoheisel, C. Kanzow and A. Schwartz (2013). Theoretical and numerical comparison of relaxation methods for mathematical programs with complementarity constraints. Math. Program. Vol. 1-2, Ser. A, 257-288. [10] C. Kanzow and A. Schwartz (2015). The price of inexactness: convergence properties of relaxation methods for mathematical programs with complementarity constraints revisited. Math. Oper. Res. Vol. 40, pp. 253-275. [11] A. Kleywegt, A. Shapiro, and T. Homem-de-Mello (2002). The sample average approximation method for stochastic discrete optimization. SIAM J. Optim. Vol. 12, pp. 479-502. [12] Y. Liu, H. Xu and G-H. Lin (2012). Stability analysis of one stage stochastic mathematical programs with complementarity constraints. J. Optim. Theory Appl. Vol. 152, pp. 537-555. [13] M. Patriksson and L. Wynter (1999). Stochastic Mathematical Programs with Equilibrium Constraints. Operations Research Letters. Vol. 25, pp. 159-167. [14] D. Ralph and H. Xu (2011). Convergence of stationary points of sample average two-stage stochastic programs: a generalized equation approach. Math. Oper. Res. Vol. 36, pp. 568-592. [15] S. M. Robinson(1996). Analysis of sample-path optimization. Math. Oper. Res. Vol. 21, pp. 513-528. [16] A. Shapiro (2003). Monte Carlo Sampling Methods. Stochastic programming. Edited by A. Ruszczy´ nski and A. Shapiro. pp. 353-425, Handbooks Oper. Res. Management Sci., vol.10, Elsevier Sci. B. V., Amsterdam. [17] A. Shapiro (2006). Stochastic programming with equilibrium constraints. Journal of Optimization Theory and Applications. Vol. 128, pp. 223-243. [18] A. S. Werner (2004). Bilevel stochastic programming problems: Analysis and application to telecommunications. Doctoral dissertation, Norwegian University of Science and Technology, Trondheim, Norway. [19] A. S. Werner and Q. Wang (2009). Resale in vertically separated markets: Profit and consumer surplus implications. Proc. 20th Internat. Sympos. Math. Programming, Chicago. [20] H. Xu and J. J. Ye (2010). Necessary optimality conditions for two-stage stochastic programs with equilibrium constraints. SIAM Journal on Optimization. Vol. 20, pp. 1685-1715. [21] J. J. Ye (2000). Constraint qualifications and necessary optimality conditions for optimization problems with variational inequality constraints. SIAM J. Optim. Vol. 10, pp. 943-962. [22] J. J. Ye (2005). Necessary and sufficient optimality conditions for mathematical programs with equilibrium constraints. J. Math. Anal. Appl. Vol. 307, pp.350-369.
S0167-6377(16)30010-4 http://dx.doi.org/10.1016/j.orl.2016.04.001 OPERES 6081
To appear in:
Operations Research Letters
Received date: 11 August 2015 Revised date: 31 March 2016 Accepted date: 1 April 2016 Please cite this article as: A. Sur, Study of M-stationarity and strong stationarity for a class of SMPCC problems via SAA Method, Operations Research Letters (2016), http://dx.doi.org/10.1016/j.orl.2016.04.001 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
*Manuscript Click here to view linked References
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
Study of M-stationarity and strong stationarity for a Class of SMPCC Problems via SAA Method Arnab Sur∗
Abstract In this article our main aim is to analyze the relative importance of M-stationarity concept over strong stationarity of one stage stochastic mathematical programming problems with complementarity constraints (SMPCC, in short). We establish the consistency of M-stationary multipliers when the SMPCC problem is approximated through the sample average approximation (SAA, for short) method, under the assumption of SMPCC-NNAMCQ. On contrary, strong stationary multipliers are not consistent even in presence of SMPCC-LICQ.
Key Words: Stochastic mathematical programming problem with complementarity constraints (SMPCC), Constraint Qualifications, Strong and M-stationarity, Sample average approximation (SAA) method.
∗ The University of Chicago Booth School of Business, Chicago, IL 60637, USA. email: [email protected]
1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
2
1
Introduction
The study of the stochastic mathematical programming problems with complementarity constraints (SMPCC, for short) is quite a recent activity and there is a great current interest among the optimizers to analyze this class of problems. Stochastic mathematical programming problems with complementarity constraints (SMPCC) is a subclass of the stochastic mathematical programming problems under equilibrium constraints (SMPEC, for short). In SMPEC problems the constraints include stochastic variational inequalities or complementarities. The SMPEC problem with stochastic variational inequality was first studied by Patriksson and Wynter [13]. Later, it was studied in a more detailed fashion by Ye [21], Evgrafov and Patriksson [4], Shapiro [17], Fukushima and Lin [6], Ralph and Xu [14]. In the above mentioned articles the authors considered stochastic variational inequalities (VI, for short) and then complementarities as a special case of stochastic VI. Moreover, SMPEC problems have many applications in economics, electricity markets, traffic networks, management sciences; see, for example Christiansen, Patriksson and Wynter [3], Werner [18], Werner and Wang [19], Chen, Wets and Zhang [2], Henrion and R¨ omisch [8] and the references therein. In this article we shall consider the following model of one-stage SMPCC problem: min
E[f (x, ξ)]
0 ≤ E[G(x, ξ)] ⊥ E[H(x, ξ)] ≥ 0;
(1.1)
where f : Rn × Ξ → R, H, G : Rn × Ξ → Rl and ξ : Ω → Ξ, where Ξ ⊂ Rm is the support set of
the random variable and E[.] denotes the expected value with respect to the probability distribution of ξ and a ⊥ b means that vector a is perpendicular to vector b. This model can be viewed as stochastic
counterpart of the deterministic model considered by Flegel and Kanzow in [5]. A similar one-stage SMPCC model was introduced by Birbil, G¨ urkan and Listes [1] with an application related to toll pricing in a
transportation network. They compared this model with other stochastic mathematical programs with equilibrium constraints from the literature.
2
Definitions and Assumptions
In this section we shall recall necessary preliminaries available in the literature. x∗ is a feasible solution to the SMPCC problem (1.1) and splitting the complementarity term we obtain the following sets of indices corresponding to x∗ which are essential to define stationarity concepts and SMPCC tailored constraint qualifications. α := α(x∗ ) := {i : E[Gi (x∗ , ξ)] = 0, E[Hi (x∗ , ξ)] > 0}, β := β(x∗ ) := {i : E[Gi (x∗ , ξ)] = 0, E[Hi (x∗ , ξ)] = 0}, γ := γ(x∗ ) := {i : E[Gi (x∗ , ξ)] > 0, E[Hi (x∗ , ξ)] = 0}.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
3 Now observe that, Card(α)+ Card(β)+ Card(γ)=l. Further if β = φ, then x∗ will be called a strict complementarity solution. First we would like to write down the definitions of strong and M-stationary conditions, based on the sets of indices. Definition 2.1. Let us consider the SMPCC problem (1.1) with f (., ξ(ω)), Gi (., ξ(ω)) and Hi (., ξ(ω)), for i = 1, ..., l are continuously differentiable ω-a.e. and x∗ is a feasible point of (1.1). Then x∗ is said to satisfy ¯0 ≥ 0, λ ¯ G ∈ R and λ ¯H ∈ R, for i = 1, ....l, Fritz John type strong stationary conditions if there exist scalars λ i i not all zero, such that:
¯0 E[▽x f (x∗ , ξ)] = i) λ
l X
¯G E[▽x Gi (x∗ , ξ)] + λ i
¯ H E[▽x Hi (x∗ , ξ)]; λ i
i=1
i=1
¯ G ∈ R, λ ¯H ∈ R, λ ¯ G = 0, λ ¯H = 0; ii) λ α γ γ α iii)
l X
¯G ≥ 0 and λ ¯ H ≥ 0, ∀ i ∈ β. λ i i
¯0 = If x∗ is a normal Fritz John multiplier, i.e., λ 6 0, then x∗ is a strong stationary point. x∗ is said to be an
¯ G > 0 and M-stationary point if condition iii) in the above definition is replaced by the following: either (λ i ¯ H > 0) or λ ¯G λ ¯H λ i i i = 0, ∀ i ∈ β.
Various stationarity concepts exist in the literature due to different reformulation of MPCC problem. The above condition is termed as M-stationarity as derivation involves calculation of Mordukhovich coderivatives, see [21]. Definition 2.2. SMPCC-No Non-zero Abnormal Multiplier constraint qualification (SMPCC-NNAMCQ for short) is said to be satisfied at a feasible point x∗ if there exist no non-zero scalars λG ∈ Rl and λH ∈ Rl ,
such that
i) 0 =
l X
∗ λG i E[▽x Gi (x , ξ)] +
ii) iii)
= 0, if i ∈ γ and
∗ λH i E[▽x Hi (x , ξ)];
i=1
i=1
λG i
l X
λH i
= 0, if i ∈ α;
H G H either (λG i > 0 and λi > 0) or λi λi = 0, ∀ i ∈ β.
It is clear from the above definition that SMPCC-NNAMCQ is weaker than SMPCC-LICQ. This constraint qualification in the literature of deterministic MPCC problems was introduced by Ye in [21] and later, used it to derive M-stationarity of MPCC problems in [22]; its stochastic version can be found in [20]. Moreover, the following assumptions on the objective and constraint functions are necessary to execute the SAA analysis of SMPCC problem (1.1). They are important to exchange the differential and expectation operations on random functions.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
4 Assumption 2.1. Assume that the functions f (x, .), Gi (x, .) and Hi (x, .) are measurable and bounded by integrable functions Kf (ω), KiG (ω) and KiH (ω) such that k f (x, ξ(ω)) k≤ Kf (ω), k Gi (x, ξ(ω)) k≤ KiG (ω),
and k Hi (x, ξ(ω)) k≤ KiH (ω) ω-a.e, for i = 1, ....l. Further assume that there exist positive valued random variables Cf (ω), CiG (ω) and CiH (ω) such that E[Cf (ω)], E[CiG (ω)] and E[CiH (ω)] for all i = 1, .....l are finite
and let there exist a neighbourhood U of x∗ such that, for every x1 , x2 in U , we have a) |f (x1 , ξ(ω)) − f (x2 , ξ(ω))| ≤ Cf (ω)||x1 − x2 ||
ω-a.e.,
b) |Gi (x1 , ξ(ω)) − Gi (x2 , ξ(ω))| ≤ CiG (ω)||x1 − x2 ||
ω-a.e.,
c) |Hi (x1 , ξ(ω)) − Hi (x2 , ξ(ω))| ≤ CiH (ω)||x1 − x2 ||
ω-a.e.
We shall study strong and M-stationarity conditions using the sample average approximation method (SAA for short) under certain constraint qualifications. The SAA method is a well known technique to solve stochastic programming models with expectation functionals. The consistency analysis of strong and M-stationary points corresponding to a sequence of SAA problems will help us to determine the relative importance of Mstationarity over strong stationarity under weaker conditions than SMPCC-linear independence constraint qualification (SMPCC-LICQ). We shall show that an accumulation point of a sequence of M-stationary points of SAA problems will be an M-stationary point of the SMPCC problem. In general, this is not true for strong stationary points even under the assumption of SMPCC-LICQ.
3
Analysis of Sample Average Approximation Method for SMPCC Problems
In this section we shall discuss the Sample Average Approximation (SAA, for short) method to establish the relative advantage of M-stationarity conditions over strong stationarity in the absence of SMPCC-LICQ. To solve stochastic optimization problems through the SAA method one needs to discretize the underlying probability distribution associated with the random variable. Monte Carlo sampling is an approach to such a discretization. Here a random (or pseudo-random) sample ξ 1 , ξ 2 ,.....ξ N of the random variable ξ is generated and we assume that the sample comprises N independent and identically distributed realizations of ξ. The expected value function E[f (x, ξ)] is approximated by the corresponding sample average function N X 1 f (x, ξ k ) and consequently the SMPCC problem (1.1) will be approximated through the following N k=1
sequence of SAA problems (dependent on N ): min
N 1 X f (x, ξ k ) N k=1
(3.1)
N N 1 X 1 X G(x, ξ k )⊥ H(x, ξ k ) ≥ 0; 0≤ N N k=1
k=1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
5 where f : Rn × Ξ → R and H, G : Rn × Ξ → Rl , and N = 1, 2, .....
¨ The idea was developed under different names: Robinson [15] and G¨ urkan, Ozge and Robinson [7] named it sample path optimization, whereas in Kleywegt, Shapiro and Homem-de-Mello [11] it is named as sample average approximation method. In 2006, Shapiro [17] introduced the SAA method to the literature on SMPEC problems. Now we shall study the convergence analysis of the Fritz John type M-stationary points of a sequence of SAA problems corresponding to the SMPCC problem (1.1) as the sample size N increases. We use the uniform law of large numbers for a random function (for example, see Proposition 7 in [16]) to establish the following convergence result on M-stationarity. Theorem 3.1. Let us consider the SMPCC problem (1.1) with f (., ξ(ω)), Gi (., ξ(ω)) and Hi (., ξ(ω)), i =
1, ....l are continuously differentiable ω-a.e. and Assumption 2.1 holds. Now let us consider a sequence of SAA problems corresponding to the SMPCC problem and let xN be a Fritz John type M-stationary point of the SAA problem (3.1). If x∗ is a cluster point of the sequence {xN }, ω-a.e. Then x∗ is an M-stationary point of the SMPCC problem (1.1), provided SMPCC-NNAMCQ holds at x∗ .
Proof. Consider the SAA problem (3.1) and xN be a Fritz John type M-stationary point of the SAA problem ¯ 0 )N ≥ 0, (3.1). Then by the definition of Fritz John type M-stationarity conditions we have scalars (λ
¯ G )N ∈ R, i = 1, ....l and (λ ¯ H )N ∈ R, i = 1, ....l such that (λ i i i)
N l N l N X X 1 X ¯G N 1 X ¯H N 1 X ¯ N [ [ (λ0 ) ▽x f (xN , ξ k ) = (λi ) ▽x Gi (xN , ξ k )] + (λi ) ▽x Hi (xN , ξ k )]. N N N i=1 i=1 k=1
k=1
k=1
¯ 0 )N , (λ ¯ G )N , (λ ¯ H )N ) 6= 0, ii) ((λ
¯ H )N = 0, where αN := α(xN ) and γN := γ(xN ), ¯ G )N = 0, (λ ¯ H )N ∈ R, (λ ¯ G )N ∈ R, (λ iii) (λ αN γN γN αN iv)
¯ G )N > 0 and (λ ¯ H )N > 0) or (λ ¯ G )N (λ ¯ H )N = 0, ∀ i ∈ βN , where βN := β(xN ). ((λ i i i i
Let
¯ 0 )N , (λ ¯ G )N ), ....., (λ ¯ G )N , (λ ¯ H )N , ......(λ ¯ H )N ) ((λ 1 1 l l λN = q . G G H N 2 N 2 N 2 N 2 ¯ ¯ ¯ ¯ ¯ H )N }2 {(λ0 ) } + {(λ1 ) } + .......{(λl ) } + {(λ1 ) } + ......{(λ l
So, k λN k= 1 for all N ∈ N, a bounded sequence which implies that there exists a convergent subsequence; hence, without any loss of generality, let {λN } be such that λN → λ∗ , ω-a.e. and by the continuity of the
norm k λ∗ k= 1, ω-a.e. Moreover, given that, xN → x∗ , ω-a.e. (without any loss of generality). Let us now
denote that
G N G N H N H N λN = (λN 0 , (λ1 ) , ....(λl ) , (λ1 ) , ......(λl ) ). ∗ G ∗ H ∗ H ∗ λ∗ = (λ∗0 , (λG 1 ) , ....(λl ) , (λ1 ) , ......(λl ) ).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
6
Now dividing both sides of i) by
q ¯ H )N }2 + ...{(λ ¯ H )N }2 , we get ¯ 0 )N }2 + {(λ ¯ G )N }2 + ...{(λ ¯ G )N }2 + {(λ {(λ 1 1 i l
N l N l N X X 1 X G N 1 X H N 1 X N [ [ (λ0 )▽x f (xN , ξ k ) = (λi ) ▽x Gi (xN , ξ k )] + (λi ) ▽x Hi (xN , ξ k )]. N N N i=1 i=1 k=1
k=1
(3.2)
k=1
And from ii), iii) and iv) we obtain ((λ0 )N , (λG )N , (λH )N ) 6= 0,
v)
H N G N N H N (λG αN ) ∈ R, (λγN ) ∈ R, (λγN ) = 0, (λαN ) = 0,
vi) vii)
N H N G N H N ((λG i ) > 0 and (λi ) > 0) or (λi ) (λi ) = 0, ∀ i ∈ βN .
Then the above equation (3.2) implies that (λN 0 )[
N l N l N X X X X 1 X N 1 N 1 (λG (λH ▽x f (xN , ξ k )] = ▽x Gi (xN , ξ k )] + ▽x Hi (xN , ξ k )]. i ) [ i ) [ N N N i=1 i=1 k=1
k=1
(3.3)
k=1
Now for i = 1, ....l ▽x f (., ξ k ), ▽x Gi (., ξ k ) and ▽x Hi (., ξ k ) are continuous on a unit closed ball B1N around N N N 1 X 1 X 1 X xN , for k = 1, ....N. Hence ▽x f (., ξ k ), ▽x Gi (., ξ k ) and ▽x Hi (., ξ k ) are continuous on the N N N k=1
k=1
unit closed ball B1N , for i = 1, ....l.
k=1
We know that xN → x∗ ω-a.e. Hence, for some given ε ∈ (0, 12 ) there exists m ∈ N, such that
k xN −x∗ k< ε, ∀ N > m and ω-a.e. Now consider any N > m and as we have seen above k xN −x∗ k< ω-a.e.; hence, x ∈ ∗
ball
B1M
around x
B1N
M
ω-a.e., as
B1N
1 2
< 1
is the closed unit ball with center x . So if we take the closed unit N
for some M > m, then {x∗ } ⊆ B1M , ω-a.e. and note that for any N > m we have
k xN − xM k≤k xN − x∗ k + k xM − x∗ k<
1 2
+
1 2
= 1 ω-a.e. This implies xN ∈ B1M ω-a.e., for all N > m
and for some M > m. This is true for any arbitrary M > m; hence, the sequence {xN }N >m ⊂ B1M , ω-a.e. \ for any M > m. Thus x∗ ∈ B and {xN }N >m ⊂ B ω-a.e., where B = B1N . N >m
Now using a) in assumption 2.1 we have for x ∈ B, k ▽x f (x, ξ(ω)) k≤ Cf (ω)
ω − a.e.
Further noting that ▽x f (., ξ(ω)) is continuous on B, ω − a.e. and the fact that the sample is i.i.d. allows us to conclude by using Proposition 7 in [16] that the expected value function E[▽x f (x, ξ)] of the SMPCC
problem (1.1) is finite valued and continuous and the expected value function of the SAA problem (3.1) N 1 X ▽x f (x, ξ k ) converges to E[▽x f (x, ξ)], ω-a.e. uniformly on B. N k=1
So we have for N > m, FN (.) =
N 1 X ▽x f (., ξ k ) is a sequence of continuous functions on a compact set N k=1
B such that FN (.) → F (.) = E[▽x f (., ξ)] uniformly on B, ω-a.e. and {xN }N >m is a sequence in B with N 1 X xN → x∗ , ω-a.e. Hence we have FN (xN ) → F (x∗ ) i.e. ▽x f (xN , ξ k ) converges to E[▽x f (x∗ , ξ)], ω-a.e. N k=1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
7
Similarly, we can show that for i = 1, ....l,
N N 1 X 1 X ▽x Gi (xN , ξ k ) converges to E[▽x Gi (x∗ , ξ)] and ▽x Hi (xN , ξ k ) N N k=1
converges to E[▽x Hi (x∗ , ξ)], ω-a.e. Now we know that λN → λ∗ , ω-a.e. and
k=1
N 1 X ▽x f (xN , ξ k ) converges to E[▽x f (x∗ , ξ)], ω-a.e. Hence we have N k=1
N N X 1 X N 1 ▽x f (xN , ξ k )] converges to λ∗0 E[▽x f (x∗ , ξ)], ω-a.e. Similarly, for i = 1, ....l, (λG ▽x Gi (xN , ξ k )] λN i ) [ 0 [ N N k=1
k=1
converges to
∗ ∗ (λG i ) E[▽x Gi (x , ξ)]
and
N (λH i ) [
N 1 X ∗ ∗ ▽x Hi (xN , ξ k )] converges to (λH i ) E[▽x Hi (x , ξ)], ω-a.e. N k=1
l N l N X X X X N 1 ∗ ∗ N 1 (λH (λG ▽x Gi (xN , ξ k )] converges to ▽x Hi (xN , ξ k )] (λG This implies that, i ) [ i ) E[▽x Gi (x , ξ)] and i ) [ N N i=1 i=1 i=1
converges to
l X
l X
k=1
k=1
∗ ∗ (λH i ) E[▽x Hi (x , ξ)], ω-a.e.
i=1
Now from equation (3.3) we have (λN 0 )[
N l N l N X X X X 1 X N 1 N 1 (λG (λH ▽x f (xN , ξ k )] = ▽x Gi (xN , ξ k )] + ▽x Hi (xN , ξ k )]. i ) [ i ) [ N N N i=1 i=1 k=1
k=1
(3.4)
k=1
Now passing to the limit as N → ∞ in equation (3.4) we have λ∗0 E[▽x f (x∗ , ξ)] =
l X
∗ ∗ (λG i ) E[▽x Gi (x , ξ)] +
l X
∗ ∗ (λH i ) E[▽x Hi (x , ξ)].
i=1
i=1
Now using Assumption 2.1 we have that x ∈ B and for i = 1, ....l k Gi (x, ξ(ω)) k≤ KiG (ω)
ω − a.e.
Further, for i = 1, ....l, Gi (., ξ(ω)) is continuous on B, ω-a.e. and as the sample is i.i.d. we can conclude by using the Proposition 7 in [16] that for i = 1, ....l, the expected value function E[Gi (x, ξ)] of the SMPCC problem (1.1) is finite valued and continuous and the expected value function of the SAA probN 1 X lem (3.1) Gi (x, ξ k ) converges to E[Gi (x, ξ)], ω-a.e. uniformly on B. Hence for i = 1, ....l we have N k=1
N 1 X Gi (xN , ξ k ) converges to E[Gi (x∗ , ξ)], ω-a.e. N k=1
Similarly, we can show that for i = 1, ....l,
N 1 X Hi (xN , ξ k ) converges to E[Hi (x∗ , ξ)], ω-a.e. N k=1
So we have for i = 1, ....l, E[Hi (x∗ , ξ)], ω-a.e.
N N 1 X 1 X Gi (xN , ξ k ) converges to E[Gi (x∗ , ξ)] and Hi (xN , ξ k ) converges to N N k=1
k=1
Further, we assume that i ∈ β, for some fixed i. Then we know that E[Gi (x∗ , ξ)] = 0 and E[Hi (x∗ , ξ)] = 0, i.e. N N N 1 X 1 X 1 X [ Gi (xN , ξ k )] → 0 and [ Hi (xN , ξ k )] → 0, ω-a.e. So for each i ∈ β, we have [ Gi (xN , ξ k )] → 0 N N N k=1
k=1
k=1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
8 N
1 X [ Hi (xN , ξ k )] → 0, ω-a.e. as E[Gi (x∗ , ξ)] = 0 and E[Hi (x∗ , ξ)] = 0 respectively. Now we will study N k=1 certain cases to see what restriction can be imposed on the multipliers if i ∈ β. We would again like to stress and
that we have considered a fixed i ∈ β.
N
Case 1: Let us assume that there exists m ∈ N such that for N > m, N
1 X [ Gi (xN , ξ k )] > 0, ω-a.e. and N k=1
1 X N [ Hi (xN , ξ k )] = 0, ω-a.e. i.e. i ∈ γN , ω-a.e. for N > m. Hence (λG = 0, ω-a.e. for N > m, this i ) N k=1
∗ N ∗ G ∗ H ∗ implies (λG i ) = 0 (as λ → λ , ω-a.e.), i.e. (λi ) (λi ) = 0.
N
Case 2: Let us assume that there exists m ∈ N such that for N > m, N
1 X [ Gi (xN , ξ k )] = 0, ω-a.e. and N k=1
1 X N Hi (xN , ξ k )] > 0, ω-a.e. i.e. i ∈ αN , ω-a.e. for N > m. Hence (λH = 0, ω-a.e. for N > m, this [ i ) N k=1
∗ G ∗ H ∗ implies (λH i ) = 0, i.e. (λi ) (λi ) = 0.
N
Case 3: Let us assume that there exists m ∈ N such that for N > m, N
1 X Gi (xN , ξ k )] = 0, ω-a.e. and [ N k=1
1 X [ Hi (xN , ξ k )] = 0, ω-a.e. i.e. i ∈ βN , ω-a.e. for N > m. Noting that xN is an M-type Fritz John point N k=1
N H N G N H N of the SAA problem (3.1), ω-a.e. Hence (λG i ) > 0 and (λi ) > 0) or (λi ) (λi ) = 0, ω-a.e. for N > m. N N N H N Let without any loss of generality (λG > 0 and (λH > 0) or (λG = 0, ω-a.e. for N ∈ N. This i ) i ) i ) (λi )
leads to the following possibilities.
N H N G ∗ i) If for some m1 ∈ N and for any N > m1 , (λG i ) > 0 and (λi ) > 0, ω-a.e. then we have either (λi ) > 0
∗ G ∗ H ∗ and (λH i ) > 0 or (λi ) (λi ) = 0.
N H N G ∗ H ∗ ii) If there exists m1 ∈ N such that for N > m1 , (λG i ) (λi ) = 0, ω-a.e. then (λi ) (λi ) = 0. N H N N N = 0, ω-a.e. for > 0, ω-a.e. for infinitely many N > m and b) (λG > 0 and (λH iii) If a) (λG i ) (λi ) i ) i ) N H N infinitely many N . Thus we can construct two subsequences such that {N > m : (λG i ) > 0 and (λi ) > 0, N H N ω-a.e.} and {N > m : (λG i ) (λi ) = 0, ω-a.e.}.
G ∗ H ∗ G ∗ H ∗ ∗ H ∗ Then from a) we get, (λG i ) > 0 and (λi ) > 0 or (λi ) (λi ) = 0, and from b) we get, (λi ) (λi ) = 0. ∗ H ∗ Thus from a) and b) we get, (λG i ) (λi ) = 0.
Case 4: Let us assume that, a)
N
N
k=1
k=1
1 X 1 X [ Gi (xN , ξ k )] > 0, ω-a.e. and [ Hi (xN , ξ k )] = 0, ω-a.e. for N N
infinitely many N . N N 1 X 1 X b) [ Gi (xN , ξ k )] = 0, ω-a.e. and [ Hi (xN , ξ k )] > 0, ω-a.e. for infinitely many N . N N k=1 N
k=1 N
k=1
k=1
1 X 1 X Gi (xN , ξ k )] = 0, ω-a.e. and [ Hi (xN , ξ k )] = 0, ω-a.e. for infinitely many N . c) [ N N
∗ H ∗ G ∗ H ∗ G ∗ H ∗ From a) and b) we get, (λG i ) (λi ) = 0 and from c) we get, (λi ) > 0 and (λi ) > 0 or (λi ) (λi ) = 0.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
9 ∗ H ∗ Hence from a), b) and c) we get, (λG i ) (λi ) = 0. Among the three possibilities mentioned above a), b) and
c) all can occur, or only a) or b) or c) can occur or they can occur even in pairs a), b) or b), c) or a), c). ∗ H ∗ This will also lead us to the fact (λG i ) (λi ) = 0. ∗ H ∗ G ∗ H ∗ Hence if i ∈ β, then ((λG i ) > 0 and (λi ) > 0) or (λi ) (λi ) = 0. N
Now if i ∈ α, then E[Hi (x∗ , ξ)] > 0. This implies,
1 X [ Hi (xN , ξ k )] > 0, ω-a.e. except for finitely many N k=1
N H ∗ N . Hence (λH i ) = 0, ω-a.e. except for finitely many N . So we can conclude that (λi ) = 0, when i ∈ α.
∗ Similarly, can be shown if i ∈ γ, then E[Gi (x∗ , ξ)] > 0 which implies (λG i ) = 0.
Hence we have,
a) λ∗0 E[▽x f (x∗ , ξ k )] =
l X
∗ ∗ k (λG i ) E[▽x Gi (x , ξ )] +
i=1
l X
∗ ∗ k (λH i ) E[▽x Hi (x , ξ )].
i=1
b)
(λ∗0 , (λG )∗ , (λH )∗ )
6= 0.
c)
∗ H ∗ G ∗ H ∗ (λG α ) ∈ R, (λγ ) ∈ R, (λγ ) = 0, (λα ) = 0.
d)
∗ H ∗ G ∗ H ∗ ((λG i ) > 0 and (λi ) > 0) or (λi ) (λi ) = 0, ∀ i ∈ β.
Now if we put λ∗0 = 0, then by SMPCC-NNAMCQ we shall get λ∗ = 0, which leads us to a contradiction as k λ∗ k= 1. Hence x∗ is an M-stationary point of the SMPCC problem (1.1). Recently, in [12], the authors did the convergence analysis of strong and M-stationary points under the assumptions of a natural type of error bound, the S-type error bound on the complementarity constraints, metric regularity, and with some other assumptions. Moreover, the authors mentioned that the convergence result holds for strong stationary points, but unfortunately this is not true. The following result on consistency of strong stationary points of SMPCC problem (1.1) shows that an accumulation point of a sequence of strong stationary points of SAA problems might not be a strong stationary point. Proposition 3.1. Suppose all assumptions in Theorem 3.1 are true and x∗ is an accumulation point of a sequence {xN } of Fritz John type strong stationary points, ω-a.e. Then, in general, we cannot conclude that x∗ is a strong stationary point even under the assumption of SMPCC-LICQ at x∗ .
Proof. First assume that i ∈ β, for some i = 1, ....l, and i is fixed here. Further assume that there exists N N 1 X 1 X Gi (xN , ξ k )] > 0, ω-a.e. and [ Hi (xN , ξ k )] = 0, ω-a.e. i.e. i ∈ γN , [ m ∈ N such that for N > m, N i=1 N i=1
N ∗ N ω-a.e. for N > m. Hence (λH ∈ R, ω-a.e. for N > m, this does not imply (λH → λ∗ , i ) i ) ≥ 0 (though λ
ω-a.e.). So, in general, we cannot conclude that x∗ is a strong stationary point.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
10 Two aspects are crucial in this article: the consistency of M-stationary points which is not available for strong stationary points and consistency is achievable under the assumption of SMPCC-NNAMCQ. The consistency analysis shows that an accumulation point of a sequence of M-stationary points of SAA problems corresponding to the SMPCC problem (1.1) will be an M-stationary point, but such a convergence result is not true for strong stationary points even under the assumption of SMPCC-LICQ. That is, an accumulation point of a sequence of M-stationary points of SAA problems will be an M-stationary point and could be a local or global minimum of the SMPCC problem in the absence of SMPCC-LICQ. Moreover, recent articles such as in [9] and [10], have shown that there are certain relaxation problems which converge to M-stationary points. We can treat SAA problems in this framework to solve them using the relaxation schemes mentioned in those articles, then we shall obtain a sequence of M-stationary points and consistency of M-stationary points will play a key role at that time. Hence, this result is important from the algorithmic perspective and establishes the relative importance of M-stationarity conditions over strong stationarity in the absence of SMPCC-LICQ.
References [1] S. I. Birbil, G. G¨ urkan and O. Listes (2006). Solving stochastic mathematical programs with complementarity constraints using simulation. Math. Oper. Res. Vol. 31, pp. 739-760. [2] X. Chen, R. J.-B. Wets and Y. Zhang(2012). Stochastic variational inequalities: residual minimization smoothing sample average approximations. SIAM Journal on Optimization Vol. 21, pp. 361-371. [3] S. Christiansen, M. Patriksson and L. Wynter (2001). Stochastic bilevel programming in structural optimization. Structual Multidisciplinary Optimization. Vol. 22, pp. 649-673. [4] A. Evgrafov and M. Patriksson (2004). On the Existence of Solutions to Stochastic Mathematical Programs with Equilibrium Constraints. Journal of Optimization Theory and Applications. Vol. 121, pp. 65-76. [5] M. Flegel and C.Kanzow (2003). A Fritz John approach to first order optimality conditions for mathematical programs with equilibrium constraints. Optimization. Vol. 52, pp. 277-286. [6] M. Fukushima and G-H. Lin (2010). Stochastic Equilibrium Problems and Stochastic Mathematical Programs with Equilibrium Constraints: A Survey. Pacific Journal of Optimization. Vol. 6, pp. 455-482. ¨ [7] G. G¨ urkan, A. Y. Ozge, and S. M. Robinson (1999). Sample-path solutions of stochastic variational inequalities. Math. Program. Vol. 84, pp. 313-334. [8] R. Henrion, W. R¨ omisch (2007). On M-stationary points for a stochastic equilibrium problem under equilibrium constraints in electricity spot market modeling. Appl. Math. Vol. 52, pp. 473-494.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
11 [9] T. Hoheisel, C. Kanzow and A. Schwartz (2013). Theoretical and numerical comparison of relaxation methods for mathematical programs with complementarity constraints. Math. Program. Vol. 1-2, Ser. A, 257-288. [10] C. Kanzow and A. Schwartz (2015). The price of inexactness: convergence properties of relaxation methods for mathematical programs with complementarity constraints revisited. Math. Oper. Res. Vol. 40, pp. 253-275. [11] A. Kleywegt, A. Shapiro, and T. Homem-de-Mello (2002). The sample average approximation method for stochastic discrete optimization. SIAM J. Optim. Vol. 12, pp. 479-502. [12] Y. Liu, H. Xu and G-H. Lin (2012). Stability analysis of one stage stochastic mathematical programs with complementarity constraints. J. Optim. Theory Appl. Vol. 152, pp. 537-555. [13] M. Patriksson and L. Wynter (1999). Stochastic Mathematical Programs with Equilibrium Constraints. Operations Research Letters. Vol. 25, pp. 159-167. [14] D. Ralph and H. Xu (2011). Convergence of stationary points of sample average two-stage stochastic programs: a generalized equation approach. Math. Oper. Res. Vol. 36, pp. 568-592. [15] S. M. Robinson(1996). Analysis of sample-path optimization. Math. Oper. Res. Vol. 21, pp. 513-528. [16] A. Shapiro (2003). Monte Carlo Sampling Methods. Stochastic programming. Edited by A. Ruszczy´ nski and A. Shapiro. pp. 353-425, Handbooks Oper. Res. Management Sci., vol.10, Elsevier Sci. B. V., Amsterdam. [17] A. Shapiro (2006). Stochastic programming with equilibrium constraints. Journal of Optimization Theory and Applications. Vol. 128, pp. 223-243. [18] A. S. Werner (2004). Bilevel stochastic programming problems: Analysis and application to telecommunications. Doctoral dissertation, Norwegian University of Science and Technology, Trondheim, Norway. [19] A. S. Werner and Q. Wang (2009). Resale in vertically separated markets: Profit and consumer surplus implications. Proc. 20th Internat. Sympos. Math. Programming, Chicago. [20] H. Xu and J. J. Ye (2010). Necessary optimality conditions for two-stage stochastic programs with equilibrium constraints. SIAM Journal on Optimization. Vol. 20, pp. 1685-1715. [21] J. J. Ye (2000). Constraint qualifications and necessary optimality conditions for optimization problems with variational inequality constraints. SIAM J. Optim. Vol. 10, pp. 943-962. [22] J. J. Ye (2005). Necessary and sufficient optimality conditions for mathematical programs with equilibrium constraints. J. Math. Anal. Appl. Vol. 307, pp.350-369.