- Email: [email protected]
A global linear and local quadratic single—step noninterior continuation method for monotone semidefinite complementarity problems*
Acta Mathematica Scientia 2007,27B(2):243–253 http://actams.wipm.ac.cn
A GLOBAL LINEAR AND LOCAL QUADRATIC SINGLE–STEP NONINTERIOR CONTINUATION METHOD FOR MONOTONE SEMIDEFINITE COMPLEMENTARITY PROBLEMS∗ Zhang Liping (
)
Department of Mathematical Sciences, Tsinghua University, Beijing 100084, China E-mail: [email protected]
Abstract A noninterior continuation method is proposed for semidefinite complementarity problem (SDCP). This method improves the noninterior continuation methods recently developed for SDCP by Chen and Tseng. The main properties of our method are: (i) it is well defined for the monotones SDCP; (ii) it has to solve just one linear system of equations at each step; (iii) it is shown to be both globally linearly convergent and locally quadratically convergent under suitable assumptions. Key words Semidefinite complementarity problem, noninterior continuation method, global convergence, local quadratic convergence 2000 MR Subject Classification
1
65K10, 90C30, 90C33
Introduction
Let X denote the space of n × n block-diagonal real matrices with m blocks of sizes n1 , · · · , nm , respectively (the blocks are fixed). Thus, X is closed under matrix addition x + y, multiplication xy, transposition xT , and inversion x−1 , where x, y ∈ X . We endow X with the inner product and norm x, y := tr[xT y], x := x, x, where x, y ∈ X and tr[·] denote the matrix trace, that is, tr[x] =
n i=1
xii . (x is the Frobenius-
norm of x and “:=” means “define”.) Let S denote the subspace comprising those x ∈ X that are symmetric, that is, xT = x. Let S+ (respectively, S++ ) denotes the convex cone comprising those x ∈ S that are positive semidefinite (respectively, positive definite). The semidefinite complementarity problem is to find, for a given mapping F : S → S, an x ∈ S satisfying x ∈ S+ , ∗ Received
F (x) ∈ S+ ,
x, F (x) = 0.
(1.1)
December 9, 2004; revised September 1, 2005. This work was supported by the National Natural Science Foundation of China (10201001, 70471008)
244
ACTA MATHEMATICA SCIENTIA
Vol.27 Ser.B
We will assume that F is a continuously differentiable monotone function throughout this article. A function F is said to be monotone, if F (x) − F (y), x − y ≥ 0 ∀x, y ∈ S. We write x y (respectively, x y) to mean x−y is positive semidefinite (respectively, positive definite). By ∇F (x) we denote the Jacobian of F at each x ∈ S viewed as a linear mapping from S to S. For any λ1 , · · · , λn ∈ R, by diag[λi ]ni=1 , we denote the n × n diagonal matrix with diagonal entries λ1 , · · · , λn . We will freely use the following facts about trace [7]: for any x, y ∈ X and any p ∈ O, tr[x] = tr[xT ] = tr[pxpT ], tr[xy] = tr[yx], and tr[x + y] = tr[x] + tr[y]. Also · is a norm on X and, in particular, the triangle inequality and the Cauchy–Schwarz inequality hold for · . For x ∈ S, by [x]+ , we denote the orthogonal projection onto S+ , that is, [x]+ = arg min x − y. For (x, y) ∈ S × S, we define (x, y) = x2 + y2 . Also, R+ y∈S+
and R++ denote the nonnegative and positive reals. Recently, there was much interest in semidefinite linear programs (SDLP) and more generally, semidefinite complementarity programs (SDLCP), which are the extensions of LP and LCP, respectively, with the cone of nonnegative real vectors replaced by the cone of symmetric positive semidefinite real matrices. Accordingly, there was considerable effort to extend solution approaches for LP and LCP to SDLP and SDLCP. The main focus was on the extensions of the interior-point approach to SDLP (see [1, 10, 11, 12] and references therein) to monotone SDLCP [10, 16]. Recently, extensions of the merit function approach [17] and the noninterior continuation/smoothing methods [5] were also considered. To achieve the global linear and local superlinear convergence of the noninterior continuation algorithm proposed in [5], one needs to solve two linear systems of equations at each iteration. In this article, we propose a noninterior continuation method for the monotones SDCP, at each step we only solve one linear system of equations (so we call it as one-step noninterior continuation method). Chen and Chen [2], Qi and Sun and Zhou [14] proposed one-step smoothing Newton method for NCP. Thus, our method improves Chen and Tseng’s method presented in [5]. Under analogous assumptions, we can establish both global linear convergence and local quadratic convergence of our method, and at each step, we only solve a linear system of equations. Whereas the article [5] only got the local superlinear convergence. This article is organized as follows. In Section 2, we list some important properties of the smoothing matrix function which based on the Chen–Harker–Kanzow–Smale smoothing function. We propose an single-step noninterior continuation/smoothing Newton method for the monotone SDCP in Section 3. In Section 4, we establish the global convergence and the global linear convergence of this method under the same assumptions as in [5]. And its local quadratic convergence is shown under analogous assumptions with [5]. Some conclusions are given in Section 5.
2
Smoothing Matrix Function
Our noninterior continuation method is based on the following smoothing matrix function φµ : S × S → S 1 φµ (x, y) := x + y − ((x − y)2 + 4µ2 I) 2 , (2.1)
No.2 Zhang: NONINTERIOR CONTINUATION METHOD FOR COMPLEMENTARITY PROBLEMS 245
where I denotes the n×n identity matrix and µ > 0 is a smoothing parameter. The vector form of the function (2.1) is independently proposed by Chen–Harker [4], Kanzow [8], and Smale [15] in the CP case. This function was also studied by Chen and Tseng in [5]. For the sake of convenience, we list some of its important properties. For any (x, y) ∈ S × S, let ⎡ ⎤ F (x) − y ⎦, (2.2) Hµ (z) := Hµ (x, y) := ⎣ φµ (x, y) where z := (x, y) and φµ (x, y) is a continuous differentiable function defined by (2.1). Obviously, φ0 (x, y) = 2(x − [x − y]+ ) which satisfies that (see [17]) φ0 (x, y) = 0 ⇐⇒ x 0, y 0, x, y = 0. Therefore, we have φµ (x, y) → 0 and (x, y, µ) → (x∗ , y ∗ , 0) =⇒ x∗ 0, y ∗ 0, x∗ , y ∗ = 0
(2.3)
and H0 (x, y) = 0 ⇐⇒ (x, y) solves SDCP(1.1). Then, SDCP(1.1) is approximated by the smoothing equation Hµ (z) = Hµ (x, y) = 0. In our method, we solve the above smoothing equation at each iteration and refine the approximation by reducing the smoothing parameter µ to zero. In interior-point methods and noninterior continuation/smoothing Newton methods, a convergence analysis requires the iterates (z, µ) to lie in a neighborhood of the ”path” defined by Hµ (z) = 0. We will use the following neighborhood: N (β) := {(z, µ) ∈ S × S × R++ : Hµ (z) ≤ βµ, 0 < µ ≤ µ0 },
(2.4)
where β, µ0 ∈ R++ are constants. In what follows we list some important properties of φµ . The following results may be found in [5]. Let φµ : S × S → S be defined by (2.1). Then φµ has the property that φµ is homogeneous of degree 1 in µ, that is, φµ (x, y) = µφ1 (x/µ, y/µ) ∀µ > 0,
∀x, y ∈ S.
The following lemma shows that φµ is Lipschitz continuous in µ. Lemma 2.1 For any x, y ∈ S and any µ, ν ∈ R++ , we have √ φµ (x, y) − φν (x, y) ≤ 2 n|µ − ν|, √ φ0 (x, y) − φµ (x, y) ≤ 2 nµ.
(2.5)
246
ACTA MATHEMATICA SCIENTIA
Vol.27 Ser.B
Now we study the differentiable properties of φµ given by (2.1). For any c ∈ S++ , define the linear mapping Lc : S → S by Lc [x] := cx + xc. It can be seen that Lc is strongly monotonice (that is, x, Lc [x] = 2tr[cx2 ] > 0, whenever −1 x = 0) and so has an inverse L−1 c , that is, for any x ∈ S, Lc [x] is the unique d ∈ S satisfying cd + dc = x. Moreover, L−1 c [x] is continuous in (x, c). Lemma 2.2 (i) Fix any µ ∈ R++ and any x, y, u, v ∈ S, we have that φµ is continuous and differentiable with ∇φµ (x, y)(u, v) = u + v − L−1 c [(x − y)(u − v) + (u − v)(x − y)],
(2.6)
1
where c := ((x − y)2 + 4µ2 I) 2 . (ii) ∇φ1 is defined and Lipschitz continuous. Let Hµ (z) be given by (2.2). We show that the monotonicity of F is sufficient for ∇Hµ (z) to be nonsingular for all z and µ ∈ R++ . Lemma 2.3 If F is a monotone and Hµ (z) is defined by (2.2), then ∇Hµ (z) is nonsingular for all z ∈ S × S and µ ∈ R++ .
3
Algorithm Description
In this section, we describe the single-step noninterior continuation method for solving the monotone SDCP (1.1). Additionally, we show that the neighborhood N (β) defined by (2.4) is bounded under a common condition. Now we state the algorithm, which is an extension of some non–interior smoothing Newton methods for the CP case (see [2, 14]). Algorithm 3.1 Step 0 Choose τ, γ, σ ∈ (0, 1). Take any (x0 , y 0 ) ∈ S × S and µ0 ∈ R++ . Choose β such √ that β ≥ 2 n and (x0 , y 0 , µ0 ) ∈ N (β). Set k := 0. Step 1 If µk = 0, STOP. (xk , y k ) solves SDCP(1.1). Otherwise, go to Step 2. Step 2 If Hµk (xk , y k ) = 0, then set (xk+1 , y k+1 ) := (xk , y k ) and tk := 1, go to Step 4. Otherwise, let (∆xk , ∆y k ) be a solution of the following linear system: ∇Hµk (xk , y k )(∆xk , ∆y k ) = −Hµk (xk , y k ). Step 3
(3.1)
Let tk be the largest t ∈ {1, δ, δ 2, · · ·} such that Hµk (xk + tk ∆xk , y k + tk ∆y k ) ≤ (1 − σtk )Hµk (xk , y k ).
(3.2)
Set (xk+1 , y k+1 ) := (xk + tk ∆xk , y k + tk ∆y k ). Step 4 Set 1 µˆk := (1 − σtk )µk . 2
(3.3)
Let γk be the smallest γ ∈ {1, τ, τ 2 , · · ·} satisfying Hγk µˆk (xk+1 , y k+1 ) ≤ βγk µˆk .
(3.4)
No.2 Zhang: NONINTERIOR CONTINUATION METHOD FOR COMPLEMENTARITY PROBLEMS 247
Set µk+1 := γk µˆk and k := k + 1. Go to Step 1. We have the following remarks about Algorithm 3.1: It is very easy to initialize Algorithm 3.1. One may simply take any (x0 , y 0 ) ∈ S ×S, µ0 > 0 √ as the starting point of the above continuation method and let β > max{2 n, Hµ0 (x0 , y 0 )/µ0 }. Algorithm 3.1 uses µk = 0 as the termination criterion. In practice, we use µk ≤ ε (where ε > 0 is a permissible error) as a termination criterion. At iteration k of Algorithm 3.1, we should only solve one linear system (3.1) and perform one line search (3.2) if Hµk (z k ) = 0. Otherwise, the algorithm needs not to solve (3.1) and performs the line search (3.2). Additionally, Algorithm 3.1 can ensure (z k , µk ) ∈ N (β) and it seems to be simpler than that used in [5]. The updating rule (3.4) for the smoothing parameter µ fastly can reduce µ and ensure the global linear convergence and local quadratic convergence of Algorithm 3.1. Proposition 3.1 If F is a continuous differentiable monotone function and N (β) is √ defined by (2.4). Then, for any β ≥ 2 n, Algorithm 3.1 is well-defined and (z k , µk ) ∈ N (β) for all k. Proof By the description of Algorithm 3.1, without loss of generality, we may assume that µk > 0 at iteration k for all k. It follows from Lemma 2.3 that ∇Hµk (z k ) is nonsingular for all k. Hence Step 2 is well-defined. To show that Step 3 is well-defined, it is sufficient to consider the case Hµk (z k ) = 0. For any θ ∈ (0, 1], set rk := Hµk (z k + θ∆z k ) − Hµk (z k ) − θ∇Hµk (z k )∆z k .
(3.5)
Thus, we have from (3.1) that Hµk (z k + θ∆z k ) ≤ (1 − θ)Hµk (z k ) + rk .
(3.6)
As µk > 0, it follows from Lemma 2.2 that Hµk (z) is continuous differentiable at z k , which implies rk = ◦(θ) by (3.5). Therefore, it follows from (3.6) that there exists θ¯ ∈ (0, 1] such that Hµk (z k + θ∆z k ) ≤ (1 − σθ)Hµk (z k )
(3.7)
¯ (3.7) implies that Step 3 is well-defined. holds for any 0 < θ ≤ θ. We consider the following two cases to show that Step 4 is well-defined. We first consider the case Hµk (z k ) = 0. Then we have from Lemma 2.1, (3.2), and (3.3) that Hµˆk (z k+1 ) ≤ Hµk (z k+1 ) + Hµk (z k+1 ) − Hµˆk (z k+1 ) √ ≤ (1 − σtk )Hµk (z k ) + 2 n|ˆ µk − µk | √ ˆk , (3.8) ≤ (1 − σtk )βµk + nσtk µk ≤ β µ √ where the last inequality uses the fact that β ≥ 2 n and 0 < σtk ≤ 1. We next consider the case Hµk (z k ) = 0. In this case, z k+1 = z k , tk = 1. It follows from Lemma 2.1 and (3.3) that √ Hµˆk (z k+1 ) = Hµk (z k ) − Hµˆk (z k ) ≤ nσtk µk ≤ β µ ˆk , (3.9) √ where the last inequality uses the fact that β ≥ 2 n and 0 < σtk ≤ 1. It follows from (3.8) and (3.9) that there exists γk = min{1, τ, τ 2 , · · ·} such that (3.4) holds. That is, Step 4 is well-defined. By (3.2) and the definition of Step 4, we know that (z k , µk ) ∈ N (β) for all k.
248
4
ACTA MATHEMATICA SCIENTIA
Vol.27 Ser.B
Convergence Analysis
In this section, we establish the global linear convergence and local quadratic convergence of Algorithm 3.1 under suitable assumptions. Without loss of generality, we assume that Algorithm 3.1 generates an infinite sequence {(z k , µk )}. We first discuss the global convergence of Algorithm 3.1. We need the following assumption. A1 The neighborhood N (β) given by (2.4) is bounded for any β > 0 and µ0 > 0. The assumption A1 can ensure the iteration sequence {(x(µ), y(µ))} generated by Algorithm 3.1 to converge as µ reduces to zero. It is also a fundamental assumption for the noninterior continuation methods for the CP case (see [2, 3, 6, 8, 13, 18]). And it was also used in [5]. Theorem 4.1 Assume that F is a continuous differentiable monotone function and the sequence {(z k , µk )} is generated by Algorithm 3.1. If the assumption A1 holds, then lim µk = k→∞
0 and every accumulation point of {z k } is a solution of SDCP(1.1). Proof By A1 and Proposition 3.1, {(z k , µk )} is bounded. Hence, we may assume that ∗ (z , µ∗ ) ∈ S × S × R++ is an accumulation point of {(z k , µk )}. Then there exists some subsequence K ⊂ {0, 1, 2, · · ·} such that lim z k = z ∗ . As {µk } is monotonically decreasing, we k∈K,k→∞
have lim µk = µ∗ ≥ 0. We assume that µ∗ > 0. In this case, by Lemma 2.2, we have k→∞
lim
k∈K,k→∞
∇Hµk (z k ) = ∇Hµ∗ (z ∗ ).
ˆ = {k1 , · · · , ki , ki+1 , · · ·} of K such that Hµ (z k ) = 0 for all If there exists a subsequence K k ˆ Then by Algorithm 3.1, tki = 1 (∀i) and k ∈ K. 1 µki+1 ≤ µki +1 = γki µ ˆki ≤ (1 − σ)µki . 2 Let i → ∞, it follows from the above inequality that 1 0 < µ∗ ≤ (1 − σ)µ∗ , 2 which is a contradiction with 0 < σ < 1. Therefore, without loss of generality, we may assume that Hµk (z k ) = 0 for all k ∈ K. Let tˆk := tk /δ, we have from Step 3 that Hµk (z k + tˆk ∆z k ) − Hµk (z k ) > σHµk (z k ) tˆk
(4.1)
holds for all k ∈ K. (3.3) and lim µk = µ∗ > 0 imply that tk → 0 and tˆk → 0 as k → ∞. k→∞
Hence let k ∈ K, k → ∞, it follows from (3.1) and (4.1) that (1 − σ)Hµ∗ (z ∗ ) ≤ 0,
Hµ∗ (z ∗ ) = 0.
(4.2)
As 0 < σ < 1, (4.2) implies that the assumption µ∗ > 0 is a contradiction. Therefore, we obtain µ∗ = 0. Thus, we have (z k , µk ) → (z ∗ , 0) and Hµk (z k ) → 0 as k ∈ K, k → ∞. As F is continuous and φµ satisfies (2.3), we obtain that z ∗ = (x∗ , y ∗ ) satisfies (1.1), so z ∗ is a solution of SDCP. If F is sufficiently smooth and the following assumption is satisfied, then the global linear convergence of Algorithm 3.1 is achieved.
No.2 Zhang: NONINTERIOR CONTINUATION METHOD FOR COMPLEMENTARITY PROBLEMS 249
A2 There exists ω > 0 such that ∇Hµk (z k )−1 ≤ ω for all k with (z k , µk ) ∈ N (β). Theorem 4.2 Assume that F is monotone and continuous differentiable and the sequence k {(z , µk )} is generated by Algorithm 3.1. If the assumptions A1 and A2 hold and ∇F is Lipschitz continuous with Lipschitz constant κ, then there exists 0 < ρ < 1 such that µk+1 ≤ ρµk for all k. Proof To achieve its proof, by (3.3), it is sufficient to show that there exists t∗ ∈ (0, 1] such that tk ≥ t∗ for all k. If Hµk (z k ) = 0, then tk = 1, which implies that the conclusion holds. Now we assume that Hµk (z k ) = 0. Then we have from (3.1) and A2 that ∆z k ≤ ∇Hµk (z k )−1 Hµk (z k ) ≤ ωHµk (z k ).
(4.3)
Suppose that there exists a subsequence K of {0, 1, 2, · · ·} such that {tk }k∈K → 0 as k → ∞. In what follows, we consider the iterations in the subsequence K. Then, it follows from (3.2) that Hµk (z k + (tk /δ)∆z k ) > (1 − σ(tk /δ))Hµk (z k ).
(4.4)
Set rk := Hµk (z k + (tk /δ)∆z k ) − Hµk (z k ) − (tk /δ)∇Hµk (z k )∆z k . We have from (3.2) that Hµk (z k + (tk /δ)∆z k ) ≤ rk + (1 − tk /δ)Hµk (z k ).
(4.5)
Thus, (4.4)–(4.5) imply rk > (1 − σ)(tk /δ)Hµk (z k ).
(4.6)
Let pk := z k /µk , q k := (tk /δ)∆z k /µk , uk := (tk /δ)∆xk , then we have q k 2 = (tk /δ)2 ∆z k 2 /µ2k ,
Hµk (z k )/µk ≤ β.
(4.7)
As ∇φ1 is Lipschitz continuous with Lipschitz constant κ ˆ := 2 + 8n by Lemma 2.2 (ii) and so is ∇F . Thus, combining (2.5), (4.3) and (4.7), we obtain rk = (−µk (φ1 (pk + q k ) − φ1 (pk ) − ∇φ1 (pk )q k ), F (xk + uk ) − F (xk ) − ∇F (xk )uk ) ≤ µk φ1 (pk + q k ) − φ1 (pk ) − ∇φ1 (pk )q k + F (xk + uk ) − F (xk ) − ∇F (xk )uk 1 1 [∇φ1 (pk + tq k ) − ∇φ1 (pk )]q k dt + [∇F (xk + tuk ) − ∇F (xk )]uk dt = µk 0
≤ µk q k
0
0
1
∇φ1 (pk + tq k ) − ∇φ1 (pk )dt + uk
1
0
∇F (xk + tuk ) − ∇F (xk )dt
1 1 1 ≤ κ ˆ µk q k 2 + κuk 2 ≤ (tk /δ)2 [ˆ κ/µk + κ]∆z k 2 2 2 2 ≤
ω2 ω2 (tk /δ)2 [ˆ β(tk /δ)2 [ˆ κ/µk + κ]Hµk (z k )2 ≤ κ + κµ0 ]Hµk (z k ). 2 2
(4.8)
Therefore, (4.6)–(4.8) imply that κ + κµ0 ]tk . 2δ(1 − σ) < βω 2 [ˆ
(4.9)
250
ACTA MATHEMATICA SCIENTIA
Vol.27 Ser.B
Let k ∈ K, k → ∞ in (4.9), we have from tk → 0 that σ ≥ 1. This is a contradiction with σ ∈ (0, 1). As tk ∈ (0, 1] for all k, there exists t∗ ∈ (0, 1] such that tk ≥ t∗ for all k. Then by (3.3) we get for all k σ σ µk+1 ≤ (1 − tk )µk ≤ (1 − t∗ )µk . 2 2 σ ∗ Take ρ := 1 − 2 t , the above inequality implies that the proof of this theorem is completed. Remark 4.1 Under the same condition as in [5], Theorem 4.2 shows that the one-step noninterior continuation method is globally linearly convergent, whereas our method only need to solve one linear system of equations to obtain the search direction ∆z k . In what follows, we study the the local quadratic convergence for Algorithm 3.1. For our analysis, by Ω we denote the solution set of SDCP (1.1) and we will need the following assumption: A3 The strict complementarity holds for any (x∗ , y ∗ ) ∈ Ω, that is, x∗ + y ∗ 0. The assumption was introduced in the cases of SDLP, monotone SDLCP, and SDCP by Kojima [5, 9] for the local superlinear convergence analysis of interior-point path-following methods and noninterior continuation methods. In addition, [5, 9] needed the assumptions A3 and nondegeneracy to obtain the superlinear convergence of their methods. However, the nondegeneracy assumption in is not required our analysis. The proof of our result is partially based in part on the following lemmas. Lemma 4.3 If A3 holds for any (x∗ , y ∗ ) ∈ Ω, then x∗ − y ∗ is nonsingular and ∇φµ (x, y) is Lipschitz continuous in (x, y) for any (x, y) near (x∗ , y ∗ ). Moreover, for any µ > ν ≥ 0, there exists > 0 such that φµ (x, y) − φν (x, y) ≤ (µ2 − ν 2 )
(4.10)
holds for any (x, y) near (x∗ , y ∗ ). Proof The first part of the result may be found in [5]. Now we prove the second part. As x∗ − y ∗ is nonsingular and hence, (x∗ − y ∗ )2 0. Fix (x, y) near (x∗ , y ∗ ), let λ1 , · · · , λn denote the eigenvalues of (x − y)2 . Then λi ≥ 0 (∀i) and there exists p ∈ O such that (x−y)2 = pT diag[λi ]ni=1 p. From the first part of this lemma, we have Λ := max{λ1 , · · · , λn } > 0. Therefore, we can get φµ (x, y) − φν (x, y) = pT diag[ λi + 4ν 2 − λi + 4µ2 ]ni=1 p n = ( λi + 4ν 2 − λi + 4µ2 )2 i=1
√ ≤ 4 n(µ2 − ν 2 )/Λ.
(4.11)
√ Take := 4 n/Λ, (4.11) implies that (4.10) holds. Lemma 4.4 Suppose that F is monotone and continuous differentiable, the sequence k {(z , µk )} is generated by Algorithm 3.1 and (z ∗ , 0) is an accumulation point of it. If ∇F is Lipschitz continuous and A1, A2, A3 hold and there exists an integer k0 such that Hµk (z k ) = 0 for all k > k0 , then there exists an integer k ∞ > k0 such that z k+1 = z k + ∆z k and µk = O(z k − z ∗ ) for all k > k ∞ . Proof Without loss of generality, we assume that {(z k , µk )} converges to (z ∗ , 0). By Theorem 4.1, we have z ∗ := (x∗ , y ∗ ) ∈ Ω. By Lemma 4.3, for any ε > 0, there exists k1 such
No.2 Zhang: NONINTERIOR CONTINUATION METHOD FOR COMPLEMENTARITY PROBLEMS 251
that z k ∈ N (z ∗ , ε) := {z ∈ S × S : z − z ∗ ≤ ε} and ∇φµk (z) is Lipschitz continuous at z k for all k > k1 . AS ∇F is Lipschitz continuous, then there exists L > 0 such that Hµk (z k + ∆z k ) − Hµk (z k ) − ∇Hµk (z k )∆z k ≤ L∆z k 2
(4.12)
holds for all k > k1 . It follows from (3.1) and A2 that ∆z k ≤ ωHµk (z k ).
(4.13)
Hence, (3.1), (4.12) and (4.13) imply Hµk (z k + ∆z k ) = Hµk (z k + ∆z k ) − Hµk (z k ) − ∇Hµk (z k )∆z k ≤ L∆z k 2 ≤ Lω 2 Hµk (z k )2 .
(4.14)
As Hµk (z k ) → 0, as k → ∞, there exists an integer k2 > 0 such that Lω 2 Hµk (z k ) < (1 − σ) for all k > k2 . Take k ∗ := max{k0 , k1 , k2 } + 1. Then, it follows from (4.14) that for all k > k ∗ , we have Hµk (z k + ∆z k ) ≤ (1 − σ)Hµk (z k ), which implies that z k+1 = z k + ∆z k for all k > k ∗ . Below we want to show that µk = O(z k − z ∗ ) for all sufficiently large k > k ∗ . By A3, H0 (z) is continuous differentiable in N (z ∗ , ε) and there exists L∗ > 0 such that H0 (z k ) − H0 (z ∗ ) ≤ L∗ z k − z ∗
(4.15)
holds for all k > k ∗ . Hence, combining (3.4), (4.10) and (4.15), we get from H0 (z ∗ ) = 0 that βτ µk < Hτ µk (z k ) ≤ Hτ µk (z k ) − H0 (z k ) + H0 (z k ) − H0 (z ∗ ) ≤ τ 2 µ2k + L∗ z k − z ∗
(4.16)
holds for all k > k ∗ . As µk → 0 as k → ∞, there are k o and Lo > 0 such that βτ − τ 2 µk > Lo for all k > k o . Therefore, (4.16) implies that for all k > max{k ∗ , k o }, we have µk ≤ (L∗ /Lo )z k − z ∗ . That is, µk = O(z k − z ∗ )
∀k > max{k ∗ , k o }.
Take k ∞ := max{k ∗ , k o }, then we complete the proof of this lemma. The following theorem discusses the local quadratic convergence of Algorithm 3.1. Theorem 4.5 Suppose that F is monotone and continuous differentiable, the sequence k {(z , µk )} is generated by Algorithm 3.1 and (z ∗ , 0) is an accumulation point. If ∇F is Lipschitz continuous and A1, A2, A3 hold, then there exists k¯ such that µk+1 = O(µ2k )
¯ k¯ + 1, · · · . ∀k = k,
252
ACTA MATHEMATICA SCIENTIA
Vol.27 Ser.B
Proof Without loss of generality, we assume that {(z k , µk )} converges to (z ∗ , 0). Then ˆ Let k¯ := max{k ∞ , k}. ˆ Then for any k ≥ k, ¯ there exists kˆ such that z k ∈ N (z ∗ , ε) for all k > k. we consider the following two cases. If Hµk (z k ) = 0, then by Lemma 4.3 and the definition of Algorithm 3.1, in this case, we have µ ˆk = (1 − σ2 )µk and β(τ γk )ˆ µk < H(τ γk )ˆµk (xk+1 , y k+1 ) = H(τ γk )ˆµk (xk+1 , y k+1 ) − Hµk (xk+1 , y k+1 ) = φ(τ γk )ˆµk (xk+1 , y k+1 ) − φµk (xk+1 , y k+1 ) ˆ2k − µ2k | ≤ µ2k . ≤ |(τ γk )2 µ
(4.17)
By (4.17), ˆk ≤ (/(βτ ))µ2k , µk+1 = γk µ that is, µk+1 = O(µ2k ).
(4.18)
On the other hand, if Hµk (z k ) = 0, then by A2, Lemma 4.3, Lemma 4.4, and (4.12), we get z k+1 − z ∗ = z k + ∆z k − z ∗ = z k − z ∗ − ∇Hµk (z k )−1 Hµk (z k ) ≤ ∇Hµk (z k )−1 [Hµk (z k ) − Hµk (z ∗ ) − ∇Hµk (z k )(z k − z ∗ ) +Hµk (z ∗ ) − H0 (z ∗ )] ≤ ω[Lz k − z ∗ 2 + Hµk (z ∗ ) − H0 (z ∗ )].
(4.19)
By A3, we have (x∗ − y ∗ )2 0 and so its eigenvalues λ∗i > 0, i = 1, · · · , n. Let Λ∗ := maxi λ∗i . Then, there exists p ∈ O such that Hµk (z ∗ ) − H0 (z ∗ ) = φµk (z ∗ ) − φ0 (z ∗ ) = pT diag[ λ∗i − λ∗i + 4µ2k ]ni=1 p n
12 = ( λ∗i − λ∗i + 4µ2k )2 i=1
≤
n
(4µ2k )2 /Λ∗
12
≤4
n/Λ∗ µ2k .
(4.20)
i=1
Therefore, by Lemma 4.4 and (4.19)–(4.20) we have z k+1 − z ∗ = O(z k − z ∗ 2 ),
(4.21)
which implies that there is an η, 0 < η < 1 such that z k+1 − z ∗ ≤ ηz k − z ∗ . Thus, we have z k − z ∗ ≤ z k+1 − z k + z k+1 − z ∗ ≤ ωHµk (z k ) + ηz k − z ∗ ≤ ωβµk + ηz k − z ∗ .
(4.22)
Combining (4.20)–(4.21) and Lemma 4.4, we have µk+1 = O(µ2k ), which implies that we complete the proof.
No.2 Zhang: NONINTERIOR CONTINUATION METHOD FOR COMPLEMENTARITY PROBLEMS 253
5
Conclusions
In this article, based on the known Chen–Harker–Kanzow–Smale smoothing function introduced independently by Chen and Harker, Kanzow, Smale for the CP case, we study the noninterior continuation method for the monotone SDCP, which needs to solve only one linear system of equations at each iteration. Compared to some previous literatures, our algorithm has global linear and local quadratic convergence under weaker assumptions. References 1 Alizadeh F. Interior point methods in semidefinite propgraming with application to combinatorial optimization. SIAM Journal on Optimization, 1995, 5: 13–51 2 Chen B, Chen X. A global linear and local quadratic continuation smoothing method for variational inequalityies with box constrains. Computational Optimization and Application, 2000, 13: 131–158 3 Chen B, Chen X. A global and local superlinear continuation smoothing method for P0 +R0 and monotone NCP. SIAM Journal on Optimization, 1999, 9: 624–645 4 Chen B, Harker P T. A non-interior-point continuation method for linear complementarity problems. SIAM Journal on Matrix Analalysis and Applications, 1993, 14: 1168–1190 5 Chen X, Tseng P. Noninterior continuation methods for solving semidefinite complementarity problems. Mathematical Programming, 2003, 95: 431–474 6 Chen B, Xiu N. A global linear and local quadratic non-interior continuation method for nonlinear complementarity problens based on Chen-Mangasarian smoothing functions. SIAM Journal on Optimization, 1999, 9: 605–623 7 Horn R A, Johnson C R. Matrix Analysis. Cambridge: Cambridge University Press, 1985 8 Kanzow C. Some noninterior continuation methods for linear complementarity problems. SIAM Journal on Matrix Analalysis and Applications, 1996, 17: 851–868 9 Kojima M, Shia M, Shindoh S. Local convergence of predictor-corrector infeasible interior-point algorithms for SDPs and SDLCPs. Mathematical Programming, 1998, 80: 129–160 10 Kojima M, Shindoh S,Hara S. Interior-point methods for the monotone semidefinite linear complementarity problems. SIAM Journal on Optimization, 1997, 7: 85–125 11 Kojima M, Shinda M, Shindoh S. A predictor-corrector interior-point algorithms for the semidefinite linear complementarity problem using the Alizadeh–Haeberly–Overton search direction. SIAM Journal on Optimization, 1999, 9: 446–465 12 Nesterov Y E, Todd M J. Primal-dual interiorpoint methods for self–scaled cones. SIAM Journal on Optimization, 1998, 8: 324–346 13 Qi L, Sun D. Improving the convergence of non-interior point algorithm for nonlinear complementarity problems. Mathematics of Computation, 2000, 69: 283–304 14 Qi L, Sun D, Zhou G. A new look at smoothing Newton methods for nonlinear complementarity problemss and box constrianed variational inequalities. Mathematical Programming, 2000, 87: 1–35 15 Smale S. Algorithms for solving equations. In: Gleason A M ed. Proceedings of International Congress of Mathematicians. Providence, Rhod Island: American Mathematical Society, 1987. 172–195 16 Tseng P. Search directions and convergence analysis of some infeasible path-following methods for the semidefinite LCP. Optimization Methods and Software, 1998, 9: 245–368 17 Tseng P. Merit functions for semidefinite complementarity problems. Mathematical Programming, 1998, 83: 159–185 18 Xu S. The global linear convergence of an infeasible non-interior path-following algorithm for complementarity problems with uniform P -functions. Mathematical Programming, 2000, 87: 501–517
A GLOBAL LINEAR AND LOCAL QUADRATIC SINGLE–STEP NONINTERIOR CONTINUATION METHOD FOR MONOTONE SEMIDEFINITE COMPLEMENTARITY PROBLEMS∗ Zhang Liping (
)
Department of Mathematical Sciences, Tsinghua University, Beijing 100084, China E-mail: [email protected]
Abstract A noninterior continuation method is proposed for semidefinite complementarity problem (SDCP). This method improves the noninterior continuation methods recently developed for SDCP by Chen and Tseng. The main properties of our method are: (i) it is well defined for the monotones SDCP; (ii) it has to solve just one linear system of equations at each step; (iii) it is shown to be both globally linearly convergent and locally quadratically convergent under suitable assumptions. Key words Semidefinite complementarity problem, noninterior continuation method, global convergence, local quadratic convergence 2000 MR Subject Classification
1
65K10, 90C30, 90C33
Introduction
Let X denote the space of n × n block-diagonal real matrices with m blocks of sizes n1 , · · · , nm , respectively (the blocks are fixed). Thus, X is closed under matrix addition x + y, multiplication xy, transposition xT , and inversion x−1 , where x, y ∈ X . We endow X with the inner product and norm x, y := tr[xT y], x := x, x, where x, y ∈ X and tr[·] denote the matrix trace, that is, tr[x] =
n i=1
xii . (x is the Frobenius-
norm of x and “:=” means “define”.) Let S denote the subspace comprising those x ∈ X that are symmetric, that is, xT = x. Let S+ (respectively, S++ ) denotes the convex cone comprising those x ∈ S that are positive semidefinite (respectively, positive definite). The semidefinite complementarity problem is to find, for a given mapping F : S → S, an x ∈ S satisfying x ∈ S+ , ∗ Received
F (x) ∈ S+ ,
x, F (x) = 0.
(1.1)
December 9, 2004; revised September 1, 2005. This work was supported by the National Natural Science Foundation of China (10201001, 70471008)
244
ACTA MATHEMATICA SCIENTIA
Vol.27 Ser.B
We will assume that F is a continuously differentiable monotone function throughout this article. A function F is said to be monotone, if F (x) − F (y), x − y ≥ 0 ∀x, y ∈ S. We write x y (respectively, x y) to mean x−y is positive semidefinite (respectively, positive definite). By ∇F (x) we denote the Jacobian of F at each x ∈ S viewed as a linear mapping from S to S. For any λ1 , · · · , λn ∈ R, by diag[λi ]ni=1 , we denote the n × n diagonal matrix with diagonal entries λ1 , · · · , λn . We will freely use the following facts about trace [7]: for any x, y ∈ X and any p ∈ O, tr[x] = tr[xT ] = tr[pxpT ], tr[xy] = tr[yx], and tr[x + y] = tr[x] + tr[y]. Also · is a norm on X and, in particular, the triangle inequality and the Cauchy–Schwarz inequality hold for · . For x ∈ S, by [x]+ , we denote the orthogonal projection onto S+ , that is, [x]+ = arg min x − y. For (x, y) ∈ S × S, we define (x, y) = x2 + y2 . Also, R+ y∈S+
and R++ denote the nonnegative and positive reals. Recently, there was much interest in semidefinite linear programs (SDLP) and more generally, semidefinite complementarity programs (SDLCP), which are the extensions of LP and LCP, respectively, with the cone of nonnegative real vectors replaced by the cone of symmetric positive semidefinite real matrices. Accordingly, there was considerable effort to extend solution approaches for LP and LCP to SDLP and SDLCP. The main focus was on the extensions of the interior-point approach to SDLP (see [1, 10, 11, 12] and references therein) to monotone SDLCP [10, 16]. Recently, extensions of the merit function approach [17] and the noninterior continuation/smoothing methods [5] were also considered. To achieve the global linear and local superlinear convergence of the noninterior continuation algorithm proposed in [5], one needs to solve two linear systems of equations at each iteration. In this article, we propose a noninterior continuation method for the monotones SDCP, at each step we only solve one linear system of equations (so we call it as one-step noninterior continuation method). Chen and Chen [2], Qi and Sun and Zhou [14] proposed one-step smoothing Newton method for NCP. Thus, our method improves Chen and Tseng’s method presented in [5]. Under analogous assumptions, we can establish both global linear convergence and local quadratic convergence of our method, and at each step, we only solve a linear system of equations. Whereas the article [5] only got the local superlinear convergence. This article is organized as follows. In Section 2, we list some important properties of the smoothing matrix function which based on the Chen–Harker–Kanzow–Smale smoothing function. We propose an single-step noninterior continuation/smoothing Newton method for the monotone SDCP in Section 3. In Section 4, we establish the global convergence and the global linear convergence of this method under the same assumptions as in [5]. And its local quadratic convergence is shown under analogous assumptions with [5]. Some conclusions are given in Section 5.
2
Smoothing Matrix Function
Our noninterior continuation method is based on the following smoothing matrix function φµ : S × S → S 1 φµ (x, y) := x + y − ((x − y)2 + 4µ2 I) 2 , (2.1)
No.2 Zhang: NONINTERIOR CONTINUATION METHOD FOR COMPLEMENTARITY PROBLEMS 245
where I denotes the n×n identity matrix and µ > 0 is a smoothing parameter. The vector form of the function (2.1) is independently proposed by Chen–Harker [4], Kanzow [8], and Smale [15] in the CP case. This function was also studied by Chen and Tseng in [5]. For the sake of convenience, we list some of its important properties. For any (x, y) ∈ S × S, let ⎡ ⎤ F (x) − y ⎦, (2.2) Hµ (z) := Hµ (x, y) := ⎣ φµ (x, y) where z := (x, y) and φµ (x, y) is a continuous differentiable function defined by (2.1). Obviously, φ0 (x, y) = 2(x − [x − y]+ ) which satisfies that (see [17]) φ0 (x, y) = 0 ⇐⇒ x 0, y 0, x, y = 0. Therefore, we have φµ (x, y) → 0 and (x, y, µ) → (x∗ , y ∗ , 0) =⇒ x∗ 0, y ∗ 0, x∗ , y ∗ = 0
(2.3)
and H0 (x, y) = 0 ⇐⇒ (x, y) solves SDCP(1.1). Then, SDCP(1.1) is approximated by the smoothing equation Hµ (z) = Hµ (x, y) = 0. In our method, we solve the above smoothing equation at each iteration and refine the approximation by reducing the smoothing parameter µ to zero. In interior-point methods and noninterior continuation/smoothing Newton methods, a convergence analysis requires the iterates (z, µ) to lie in a neighborhood of the ”path” defined by Hµ (z) = 0. We will use the following neighborhood: N (β) := {(z, µ) ∈ S × S × R++ : Hµ (z) ≤ βµ, 0 < µ ≤ µ0 },
(2.4)
where β, µ0 ∈ R++ are constants. In what follows we list some important properties of φµ . The following results may be found in [5]. Let φµ : S × S → S be defined by (2.1). Then φµ has the property that φµ is homogeneous of degree 1 in µ, that is, φµ (x, y) = µφ1 (x/µ, y/µ) ∀µ > 0,
∀x, y ∈ S.
The following lemma shows that φµ is Lipschitz continuous in µ. Lemma 2.1 For any x, y ∈ S and any µ, ν ∈ R++ , we have √ φµ (x, y) − φν (x, y) ≤ 2 n|µ − ν|, √ φ0 (x, y) − φµ (x, y) ≤ 2 nµ.
(2.5)
246
ACTA MATHEMATICA SCIENTIA
Vol.27 Ser.B
Now we study the differentiable properties of φµ given by (2.1). For any c ∈ S++ , define the linear mapping Lc : S → S by Lc [x] := cx + xc. It can be seen that Lc is strongly monotonice (that is, x, Lc [x] = 2tr[cx2 ] > 0, whenever −1 x = 0) and so has an inverse L−1 c , that is, for any x ∈ S, Lc [x] is the unique d ∈ S satisfying cd + dc = x. Moreover, L−1 c [x] is continuous in (x, c). Lemma 2.2 (i) Fix any µ ∈ R++ and any x, y, u, v ∈ S, we have that φµ is continuous and differentiable with ∇φµ (x, y)(u, v) = u + v − L−1 c [(x − y)(u − v) + (u − v)(x − y)],
(2.6)
1
where c := ((x − y)2 + 4µ2 I) 2 . (ii) ∇φ1 is defined and Lipschitz continuous. Let Hµ (z) be given by (2.2). We show that the monotonicity of F is sufficient for ∇Hµ (z) to be nonsingular for all z and µ ∈ R++ . Lemma 2.3 If F is a monotone and Hµ (z) is defined by (2.2), then ∇Hµ (z) is nonsingular for all z ∈ S × S and µ ∈ R++ .
3
Algorithm Description
In this section, we describe the single-step noninterior continuation method for solving the monotone SDCP (1.1). Additionally, we show that the neighborhood N (β) defined by (2.4) is bounded under a common condition. Now we state the algorithm, which is an extension of some non–interior smoothing Newton methods for the CP case (see [2, 14]). Algorithm 3.1 Step 0 Choose τ, γ, σ ∈ (0, 1). Take any (x0 , y 0 ) ∈ S × S and µ0 ∈ R++ . Choose β such √ that β ≥ 2 n and (x0 , y 0 , µ0 ) ∈ N (β). Set k := 0. Step 1 If µk = 0, STOP. (xk , y k ) solves SDCP(1.1). Otherwise, go to Step 2. Step 2 If Hµk (xk , y k ) = 0, then set (xk+1 , y k+1 ) := (xk , y k ) and tk := 1, go to Step 4. Otherwise, let (∆xk , ∆y k ) be a solution of the following linear system: ∇Hµk (xk , y k )(∆xk , ∆y k ) = −Hµk (xk , y k ). Step 3
(3.1)
Let tk be the largest t ∈ {1, δ, δ 2, · · ·} such that Hµk (xk + tk ∆xk , y k + tk ∆y k ) ≤ (1 − σtk )Hµk (xk , y k ).
(3.2)
Set (xk+1 , y k+1 ) := (xk + tk ∆xk , y k + tk ∆y k ). Step 4 Set 1 µˆk := (1 − σtk )µk . 2
(3.3)
Let γk be the smallest γ ∈ {1, τ, τ 2 , · · ·} satisfying Hγk µˆk (xk+1 , y k+1 ) ≤ βγk µˆk .
(3.4)
No.2 Zhang: NONINTERIOR CONTINUATION METHOD FOR COMPLEMENTARITY PROBLEMS 247
Set µk+1 := γk µˆk and k := k + 1. Go to Step 1. We have the following remarks about Algorithm 3.1: It is very easy to initialize Algorithm 3.1. One may simply take any (x0 , y 0 ) ∈ S ×S, µ0 > 0 √ as the starting point of the above continuation method and let β > max{2 n, Hµ0 (x0 , y 0 )/µ0 }. Algorithm 3.1 uses µk = 0 as the termination criterion. In practice, we use µk ≤ ε (where ε > 0 is a permissible error) as a termination criterion. At iteration k of Algorithm 3.1, we should only solve one linear system (3.1) and perform one line search (3.2) if Hµk (z k ) = 0. Otherwise, the algorithm needs not to solve (3.1) and performs the line search (3.2). Additionally, Algorithm 3.1 can ensure (z k , µk ) ∈ N (β) and it seems to be simpler than that used in [5]. The updating rule (3.4) for the smoothing parameter µ fastly can reduce µ and ensure the global linear convergence and local quadratic convergence of Algorithm 3.1. Proposition 3.1 If F is a continuous differentiable monotone function and N (β) is √ defined by (2.4). Then, for any β ≥ 2 n, Algorithm 3.1 is well-defined and (z k , µk ) ∈ N (β) for all k. Proof By the description of Algorithm 3.1, without loss of generality, we may assume that µk > 0 at iteration k for all k. It follows from Lemma 2.3 that ∇Hµk (z k ) is nonsingular for all k. Hence Step 2 is well-defined. To show that Step 3 is well-defined, it is sufficient to consider the case Hµk (z k ) = 0. For any θ ∈ (0, 1], set rk := Hµk (z k + θ∆z k ) − Hµk (z k ) − θ∇Hµk (z k )∆z k .
(3.5)
Thus, we have from (3.1) that Hµk (z k + θ∆z k ) ≤ (1 − θ)Hµk (z k ) + rk .
(3.6)
As µk > 0, it follows from Lemma 2.2 that Hµk (z) is continuous differentiable at z k , which implies rk = ◦(θ) by (3.5). Therefore, it follows from (3.6) that there exists θ¯ ∈ (0, 1] such that Hµk (z k + θ∆z k ) ≤ (1 − σθ)Hµk (z k )
(3.7)
¯ (3.7) implies that Step 3 is well-defined. holds for any 0 < θ ≤ θ. We consider the following two cases to show that Step 4 is well-defined. We first consider the case Hµk (z k ) = 0. Then we have from Lemma 2.1, (3.2), and (3.3) that Hµˆk (z k+1 ) ≤ Hµk (z k+1 ) + Hµk (z k+1 ) − Hµˆk (z k+1 ) √ ≤ (1 − σtk )Hµk (z k ) + 2 n|ˆ µk − µk | √ ˆk , (3.8) ≤ (1 − σtk )βµk + nσtk µk ≤ β µ √ where the last inequality uses the fact that β ≥ 2 n and 0 < σtk ≤ 1. We next consider the case Hµk (z k ) = 0. In this case, z k+1 = z k , tk = 1. It follows from Lemma 2.1 and (3.3) that √ Hµˆk (z k+1 ) = Hµk (z k ) − Hµˆk (z k ) ≤ nσtk µk ≤ β µ ˆk , (3.9) √ where the last inequality uses the fact that β ≥ 2 n and 0 < σtk ≤ 1. It follows from (3.8) and (3.9) that there exists γk = min{1, τ, τ 2 , · · ·} such that (3.4) holds. That is, Step 4 is well-defined. By (3.2) and the definition of Step 4, we know that (z k , µk ) ∈ N (β) for all k.
248
4
ACTA MATHEMATICA SCIENTIA
Vol.27 Ser.B
Convergence Analysis
In this section, we establish the global linear convergence and local quadratic convergence of Algorithm 3.1 under suitable assumptions. Without loss of generality, we assume that Algorithm 3.1 generates an infinite sequence {(z k , µk )}. We first discuss the global convergence of Algorithm 3.1. We need the following assumption. A1 The neighborhood N (β) given by (2.4) is bounded for any β > 0 and µ0 > 0. The assumption A1 can ensure the iteration sequence {(x(µ), y(µ))} generated by Algorithm 3.1 to converge as µ reduces to zero. It is also a fundamental assumption for the noninterior continuation methods for the CP case (see [2, 3, 6, 8, 13, 18]). And it was also used in [5]. Theorem 4.1 Assume that F is a continuous differentiable monotone function and the sequence {(z k , µk )} is generated by Algorithm 3.1. If the assumption A1 holds, then lim µk = k→∞
0 and every accumulation point of {z k } is a solution of SDCP(1.1). Proof By A1 and Proposition 3.1, {(z k , µk )} is bounded. Hence, we may assume that ∗ (z , µ∗ ) ∈ S × S × R++ is an accumulation point of {(z k , µk )}. Then there exists some subsequence K ⊂ {0, 1, 2, · · ·} such that lim z k = z ∗ . As {µk } is monotonically decreasing, we k∈K,k→∞
have lim µk = µ∗ ≥ 0. We assume that µ∗ > 0. In this case, by Lemma 2.2, we have k→∞
lim
k∈K,k→∞
∇Hµk (z k ) = ∇Hµ∗ (z ∗ ).
ˆ = {k1 , · · · , ki , ki+1 , · · ·} of K such that Hµ (z k ) = 0 for all If there exists a subsequence K k ˆ Then by Algorithm 3.1, tki = 1 (∀i) and k ∈ K. 1 µki+1 ≤ µki +1 = γki µ ˆki ≤ (1 − σ)µki . 2 Let i → ∞, it follows from the above inequality that 1 0 < µ∗ ≤ (1 − σ)µ∗ , 2 which is a contradiction with 0 < σ < 1. Therefore, without loss of generality, we may assume that Hµk (z k ) = 0 for all k ∈ K. Let tˆk := tk /δ, we have from Step 3 that Hµk (z k + tˆk ∆z k ) − Hµk (z k ) > σHµk (z k ) tˆk
(4.1)
holds for all k ∈ K. (3.3) and lim µk = µ∗ > 0 imply that tk → 0 and tˆk → 0 as k → ∞. k→∞
Hence let k ∈ K, k → ∞, it follows from (3.1) and (4.1) that (1 − σ)Hµ∗ (z ∗ ) ≤ 0,
Hµ∗ (z ∗ ) = 0.
(4.2)
As 0 < σ < 1, (4.2) implies that the assumption µ∗ > 0 is a contradiction. Therefore, we obtain µ∗ = 0. Thus, we have (z k , µk ) → (z ∗ , 0) and Hµk (z k ) → 0 as k ∈ K, k → ∞. As F is continuous and φµ satisfies (2.3), we obtain that z ∗ = (x∗ , y ∗ ) satisfies (1.1), so z ∗ is a solution of SDCP. If F is sufficiently smooth and the following assumption is satisfied, then the global linear convergence of Algorithm 3.1 is achieved.
No.2 Zhang: NONINTERIOR CONTINUATION METHOD FOR COMPLEMENTARITY PROBLEMS 249
A2 There exists ω > 0 such that ∇Hµk (z k )−1 ≤ ω for all k with (z k , µk ) ∈ N (β). Theorem 4.2 Assume that F is monotone and continuous differentiable and the sequence k {(z , µk )} is generated by Algorithm 3.1. If the assumptions A1 and A2 hold and ∇F is Lipschitz continuous with Lipschitz constant κ, then there exists 0 < ρ < 1 such that µk+1 ≤ ρµk for all k. Proof To achieve its proof, by (3.3), it is sufficient to show that there exists t∗ ∈ (0, 1] such that tk ≥ t∗ for all k. If Hµk (z k ) = 0, then tk = 1, which implies that the conclusion holds. Now we assume that Hµk (z k ) = 0. Then we have from (3.1) and A2 that ∆z k ≤ ∇Hµk (z k )−1 Hµk (z k ) ≤ ωHµk (z k ).
(4.3)
Suppose that there exists a subsequence K of {0, 1, 2, · · ·} such that {tk }k∈K → 0 as k → ∞. In what follows, we consider the iterations in the subsequence K. Then, it follows from (3.2) that Hµk (z k + (tk /δ)∆z k ) > (1 − σ(tk /δ))Hµk (z k ).
(4.4)
Set rk := Hµk (z k + (tk /δ)∆z k ) − Hµk (z k ) − (tk /δ)∇Hµk (z k )∆z k . We have from (3.2) that Hµk (z k + (tk /δ)∆z k ) ≤ rk + (1 − tk /δ)Hµk (z k ).
(4.5)
Thus, (4.4)–(4.5) imply rk > (1 − σ)(tk /δ)Hµk (z k ).
(4.6)
Let pk := z k /µk , q k := (tk /δ)∆z k /µk , uk := (tk /δ)∆xk , then we have q k 2 = (tk /δ)2 ∆z k 2 /µ2k ,
Hµk (z k )/µk ≤ β.
(4.7)
As ∇φ1 is Lipschitz continuous with Lipschitz constant κ ˆ := 2 + 8n by Lemma 2.2 (ii) and so is ∇F . Thus, combining (2.5), (4.3) and (4.7), we obtain rk = (−µk (φ1 (pk + q k ) − φ1 (pk ) − ∇φ1 (pk )q k ), F (xk + uk ) − F (xk ) − ∇F (xk )uk ) ≤ µk φ1 (pk + q k ) − φ1 (pk ) − ∇φ1 (pk )q k + F (xk + uk ) − F (xk ) − ∇F (xk )uk 1 1 [∇φ1 (pk + tq k ) − ∇φ1 (pk )]q k dt + [∇F (xk + tuk ) − ∇F (xk )]uk dt = µk 0
≤ µk q k
0
0
1
∇φ1 (pk + tq k ) − ∇φ1 (pk )dt + uk
1
0
∇F (xk + tuk ) − ∇F (xk )dt
1 1 1 ≤ κ ˆ µk q k 2 + κuk 2 ≤ (tk /δ)2 [ˆ κ/µk + κ]∆z k 2 2 2 2 ≤
ω2 ω2 (tk /δ)2 [ˆ β(tk /δ)2 [ˆ κ/µk + κ]Hµk (z k )2 ≤ κ + κµ0 ]Hµk (z k ). 2 2
(4.8)
Therefore, (4.6)–(4.8) imply that κ + κµ0 ]tk . 2δ(1 − σ) < βω 2 [ˆ
(4.9)
250
ACTA MATHEMATICA SCIENTIA
Vol.27 Ser.B
Let k ∈ K, k → ∞ in (4.9), we have from tk → 0 that σ ≥ 1. This is a contradiction with σ ∈ (0, 1). As tk ∈ (0, 1] for all k, there exists t∗ ∈ (0, 1] such that tk ≥ t∗ for all k. Then by (3.3) we get for all k σ σ µk+1 ≤ (1 − tk )µk ≤ (1 − t∗ )µk . 2 2 σ ∗ Take ρ := 1 − 2 t , the above inequality implies that the proof of this theorem is completed. Remark 4.1 Under the same condition as in [5], Theorem 4.2 shows that the one-step noninterior continuation method is globally linearly convergent, whereas our method only need to solve one linear system of equations to obtain the search direction ∆z k . In what follows, we study the the local quadratic convergence for Algorithm 3.1. For our analysis, by Ω we denote the solution set of SDCP (1.1) and we will need the following assumption: A3 The strict complementarity holds for any (x∗ , y ∗ ) ∈ Ω, that is, x∗ + y ∗ 0. The assumption was introduced in the cases of SDLP, monotone SDLCP, and SDCP by Kojima [5, 9] for the local superlinear convergence analysis of interior-point path-following methods and noninterior continuation methods. In addition, [5, 9] needed the assumptions A3 and nondegeneracy to obtain the superlinear convergence of their methods. However, the nondegeneracy assumption in is not required our analysis. The proof of our result is partially based in part on the following lemmas. Lemma 4.3 If A3 holds for any (x∗ , y ∗ ) ∈ Ω, then x∗ − y ∗ is nonsingular and ∇φµ (x, y) is Lipschitz continuous in (x, y) for any (x, y) near (x∗ , y ∗ ). Moreover, for any µ > ν ≥ 0, there exists > 0 such that φµ (x, y) − φν (x, y) ≤ (µ2 − ν 2 )
(4.10)
holds for any (x, y) near (x∗ , y ∗ ). Proof The first part of the result may be found in [5]. Now we prove the second part. As x∗ − y ∗ is nonsingular and hence, (x∗ − y ∗ )2 0. Fix (x, y) near (x∗ , y ∗ ), let λ1 , · · · , λn denote the eigenvalues of (x − y)2 . Then λi ≥ 0 (∀i) and there exists p ∈ O such that (x−y)2 = pT diag[λi ]ni=1 p. From the first part of this lemma, we have Λ := max{λ1 , · · · , λn } > 0. Therefore, we can get φµ (x, y) − φν (x, y) = pT diag[ λi + 4ν 2 − λi + 4µ2 ]ni=1 p n = ( λi + 4ν 2 − λi + 4µ2 )2 i=1
√ ≤ 4 n(µ2 − ν 2 )/Λ.
(4.11)
√ Take := 4 n/Λ, (4.11) implies that (4.10) holds. Lemma 4.4 Suppose that F is monotone and continuous differentiable, the sequence k {(z , µk )} is generated by Algorithm 3.1 and (z ∗ , 0) is an accumulation point of it. If ∇F is Lipschitz continuous and A1, A2, A3 hold and there exists an integer k0 such that Hµk (z k ) = 0 for all k > k0 , then there exists an integer k ∞ > k0 such that z k+1 = z k + ∆z k and µk = O(z k − z ∗ ) for all k > k ∞ . Proof Without loss of generality, we assume that {(z k , µk )} converges to (z ∗ , 0). By Theorem 4.1, we have z ∗ := (x∗ , y ∗ ) ∈ Ω. By Lemma 4.3, for any ε > 0, there exists k1 such
No.2 Zhang: NONINTERIOR CONTINUATION METHOD FOR COMPLEMENTARITY PROBLEMS 251
that z k ∈ N (z ∗ , ε) := {z ∈ S × S : z − z ∗ ≤ ε} and ∇φµk (z) is Lipschitz continuous at z k for all k > k1 . AS ∇F is Lipschitz continuous, then there exists L > 0 such that Hµk (z k + ∆z k ) − Hµk (z k ) − ∇Hµk (z k )∆z k ≤ L∆z k 2
(4.12)
holds for all k > k1 . It follows from (3.1) and A2 that ∆z k ≤ ωHµk (z k ).
(4.13)
Hence, (3.1), (4.12) and (4.13) imply Hµk (z k + ∆z k ) = Hµk (z k + ∆z k ) − Hµk (z k ) − ∇Hµk (z k )∆z k ≤ L∆z k 2 ≤ Lω 2 Hµk (z k )2 .
(4.14)
As Hµk (z k ) → 0, as k → ∞, there exists an integer k2 > 0 such that Lω 2 Hµk (z k ) < (1 − σ) for all k > k2 . Take k ∗ := max{k0 , k1 , k2 } + 1. Then, it follows from (4.14) that for all k > k ∗ , we have Hµk (z k + ∆z k ) ≤ (1 − σ)Hµk (z k ), which implies that z k+1 = z k + ∆z k for all k > k ∗ . Below we want to show that µk = O(z k − z ∗ ) for all sufficiently large k > k ∗ . By A3, H0 (z) is continuous differentiable in N (z ∗ , ε) and there exists L∗ > 0 such that H0 (z k ) − H0 (z ∗ ) ≤ L∗ z k − z ∗
(4.15)
holds for all k > k ∗ . Hence, combining (3.4), (4.10) and (4.15), we get from H0 (z ∗ ) = 0 that βτ µk < Hτ µk (z k ) ≤ Hτ µk (z k ) − H0 (z k ) + H0 (z k ) − H0 (z ∗ ) ≤ τ 2 µ2k + L∗ z k − z ∗
(4.16)
holds for all k > k ∗ . As µk → 0 as k → ∞, there are k o and Lo > 0 such that βτ − τ 2 µk > Lo for all k > k o . Therefore, (4.16) implies that for all k > max{k ∗ , k o }, we have µk ≤ (L∗ /Lo )z k − z ∗ . That is, µk = O(z k − z ∗ )
∀k > max{k ∗ , k o }.
Take k ∞ := max{k ∗ , k o }, then we complete the proof of this lemma. The following theorem discusses the local quadratic convergence of Algorithm 3.1. Theorem 4.5 Suppose that F is monotone and continuous differentiable, the sequence k {(z , µk )} is generated by Algorithm 3.1 and (z ∗ , 0) is an accumulation point. If ∇F is Lipschitz continuous and A1, A2, A3 hold, then there exists k¯ such that µk+1 = O(µ2k )
¯ k¯ + 1, · · · . ∀k = k,
252
ACTA MATHEMATICA SCIENTIA
Vol.27 Ser.B
Proof Without loss of generality, we assume that {(z k , µk )} converges to (z ∗ , 0). Then ˆ Let k¯ := max{k ∞ , k}. ˆ Then for any k ≥ k, ¯ there exists kˆ such that z k ∈ N (z ∗ , ε) for all k > k. we consider the following two cases. If Hµk (z k ) = 0, then by Lemma 4.3 and the definition of Algorithm 3.1, in this case, we have µ ˆk = (1 − σ2 )µk and β(τ γk )ˆ µk < H(τ γk )ˆµk (xk+1 , y k+1 ) = H(τ γk )ˆµk (xk+1 , y k+1 ) − Hµk (xk+1 , y k+1 ) = φ(τ γk )ˆµk (xk+1 , y k+1 ) − φµk (xk+1 , y k+1 ) ˆ2k − µ2k | ≤ µ2k . ≤ |(τ γk )2 µ
(4.17)
By (4.17), ˆk ≤ (/(βτ ))µ2k , µk+1 = γk µ that is, µk+1 = O(µ2k ).
(4.18)
On the other hand, if Hµk (z k ) = 0, then by A2, Lemma 4.3, Lemma 4.4, and (4.12), we get z k+1 − z ∗ = z k + ∆z k − z ∗ = z k − z ∗ − ∇Hµk (z k )−1 Hµk (z k ) ≤ ∇Hµk (z k )−1 [Hµk (z k ) − Hµk (z ∗ ) − ∇Hµk (z k )(z k − z ∗ ) +Hµk (z ∗ ) − H0 (z ∗ )] ≤ ω[Lz k − z ∗ 2 + Hµk (z ∗ ) − H0 (z ∗ )].
(4.19)
By A3, we have (x∗ − y ∗ )2 0 and so its eigenvalues λ∗i > 0, i = 1, · · · , n. Let Λ∗ := maxi λ∗i . Then, there exists p ∈ O such that Hµk (z ∗ ) − H0 (z ∗ ) = φµk (z ∗ ) − φ0 (z ∗ ) = pT diag[ λ∗i − λ∗i + 4µ2k ]ni=1 p n
12 = ( λ∗i − λ∗i + 4µ2k )2 i=1
≤
n
(4µ2k )2 /Λ∗
12
≤4
n/Λ∗ µ2k .
(4.20)
i=1
Therefore, by Lemma 4.4 and (4.19)–(4.20) we have z k+1 − z ∗ = O(z k − z ∗ 2 ),
(4.21)
which implies that there is an η, 0 < η < 1 such that z k+1 − z ∗ ≤ ηz k − z ∗ . Thus, we have z k − z ∗ ≤ z k+1 − z k + z k+1 − z ∗ ≤ ωHµk (z k ) + ηz k − z ∗ ≤ ωβµk + ηz k − z ∗ .
(4.22)
Combining (4.20)–(4.21) and Lemma 4.4, we have µk+1 = O(µ2k ), which implies that we complete the proof.
No.2 Zhang: NONINTERIOR CONTINUATION METHOD FOR COMPLEMENTARITY PROBLEMS 253
5
Conclusions
In this article, based on the known Chen–Harker–Kanzow–Smale smoothing function introduced independently by Chen and Harker, Kanzow, Smale for the CP case, we study the noninterior continuation method for the monotone SDCP, which needs to solve only one linear system of equations at each iteration. Compared to some previous literatures, our algorithm has global linear and local quadratic convergence under weaker assumptions. References 1 Alizadeh F. Interior point methods in semidefinite propgraming with application to combinatorial optimization. SIAM Journal on Optimization, 1995, 5: 13–51 2 Chen B, Chen X. A global linear and local quadratic continuation smoothing method for variational inequalityies with box constrains. Computational Optimization and Application, 2000, 13: 131–158 3 Chen B, Chen X. A global and local superlinear continuation smoothing method for P0 +R0 and monotone NCP. SIAM Journal on Optimization, 1999, 9: 624–645 4 Chen B, Harker P T. A non-interior-point continuation method for linear complementarity problems. SIAM Journal on Matrix Analalysis and Applications, 1993, 14: 1168–1190 5 Chen X, Tseng P. Noninterior continuation methods for solving semidefinite complementarity problems. Mathematical Programming, 2003, 95: 431–474 6 Chen B, Xiu N. A global linear and local quadratic non-interior continuation method for nonlinear complementarity problens based on Chen-Mangasarian smoothing functions. SIAM Journal on Optimization, 1999, 9: 605–623 7 Horn R A, Johnson C R. Matrix Analysis. Cambridge: Cambridge University Press, 1985 8 Kanzow C. Some noninterior continuation methods for linear complementarity problems. SIAM Journal on Matrix Analalysis and Applications, 1996, 17: 851–868 9 Kojima M, Shia M, Shindoh S. Local convergence of predictor-corrector infeasible interior-point algorithms for SDPs and SDLCPs. Mathematical Programming, 1998, 80: 129–160 10 Kojima M, Shindoh S,Hara S. Interior-point methods for the monotone semidefinite linear complementarity problems. SIAM Journal on Optimization, 1997, 7: 85–125 11 Kojima M, Shinda M, Shindoh S. A predictor-corrector interior-point algorithms for the semidefinite linear complementarity problem using the Alizadeh–Haeberly–Overton search direction. SIAM Journal on Optimization, 1999, 9: 446–465 12 Nesterov Y E, Todd M J. Primal-dual interiorpoint methods for self–scaled cones. SIAM Journal on Optimization, 1998, 8: 324–346 13 Qi L, Sun D. Improving the convergence of non-interior point algorithm for nonlinear complementarity problems. Mathematics of Computation, 2000, 69: 283–304 14 Qi L, Sun D, Zhou G. A new look at smoothing Newton methods for nonlinear complementarity problemss and box constrianed variational inequalities. Mathematical Programming, 2000, 87: 1–35 15 Smale S. Algorithms for solving equations. In: Gleason A M ed. Proceedings of International Congress of Mathematicians. Providence, Rhod Island: American Mathematical Society, 1987. 172–195 16 Tseng P. Search directions and convergence analysis of some infeasible path-following methods for the semidefinite LCP. Optimization Methods and Software, 1998, 9: 245–368 17 Tseng P. Merit functions for semidefinite complementarity problems. Mathematical Programming, 1998, 83: 159–185 18 Xu S. The global linear convergence of an infeasible non-interior path-following algorithm for complementarity problems with uniform P -functions. Mathematical Programming, 2000, 87: 501–517