- Email: [email protected]
A full-NT-step infeasible interior-point algorithm for SDP based on kernel functions
Applied Mathematics and Computation 217 (2011) 4990–4999
Contents lists available at ScienceDirect
Applied Mathematics and Computation journal homepage: www.elsevier.com/locate/amc
A full-NT-step infeasible interior-point algorithm for SDP based on kernel functions q Zhongyi Liu a,⇑, Wenyu Sun b a b
College of Science, Hohai University, Nanjing 210098, China School of Mathematics Science, Nanjing Normal University, Nanjing 210097, China
a r t i c l e
i n f o
a b s t r a c t This paper proposes an infeasible interior-point algorithm with full Nesterov–Todd (NT) steps for semidefinite programming (SDP). The main iteration consists of a feasibility step and several centrality steps. First we present a full NT step infeasible interior-point algorithm based on the classic logarithmical barrier function. After that a specific kernel function is introduced. The feasibility step is induced by this kernel function instead of the classic logarithmical barrier function. This kernel function has a finite value on the boundary. The result of polynomial complexity, Oðn log n=eÞ, coincides with the best known one for infeasible interior-point methods. Ó 2010 Elsevier Inc. All rights reserved.
Keywords: Semidefinite programming Full Nesterov–Todd steps Infeasible interior-point methods Polynomial complexity Kernel functions
1. Introduction We are concerned with SDP problems, whose primal and dual forms are
ðPÞ
minfTrðCXÞ : TrðAi XÞ ¼ bi ; i ¼ 1; 2; . . . ; m; X 0g
ðDÞ
max b y :
and
( T
m X
) yi Ai þ S ¼ C; S 0 ;
i¼1 n
where Ai ; C; X; S 2 S , and b; y 2 Rm . The matrices Ai ði ¼ 1; 2; . . . ; mÞ are assumed to be linearly independent. By introducing
AX :¼ ðTrðA1 XÞ; TrðA2 XÞ; . . . ; TrðAm XÞÞT and
A y :¼
m X
y i Ai ;
i¼1
ðPÞ and ðDÞ can be expressed into the following equivalent forms
ðPÞ
minfTrðCXÞ : AX ¼ b; X 0g
and q
This work was supported in part by the Fundamental Funds for the Central Universities in China (No. 2009B27314).
⇑ Corresponding author.
E-mail addresses: [email protected] (Z. Liu), [email protected] (W. Sun). 0096-3003/$ - see front matter Ó 2010 Elsevier Inc. All rights reserved. doi:10.1016/j.amc.2010.11.049
Z. Liu, W. Sun / Applied Mathematics and Computation 217 (2011) 4990–4999
4991
T
maxfb y : A y þ S ¼ C; S 0g:
ðDÞ
Primal–dual interior-point methods (IPMs) for SDP have been widely studied, the reader is referred to Klerk [2] and Wolkowicz et al. [9]. Recently a full-Newton step infeasible interior-point algorithm for linear programming (LP) was presented by Roos [8]. The result of polynomial complexity coincides with the best known one for infeasible IPMs. Some extensions were carried out by Mansouri and Roos [4], Liu and Sun [3]. Mansouri and Roos [5] extended this algorithm to SDP by using a specific feasibility step. Peng et al. [7] introduced a new class of primal–dual IPMs based on self-regular proximities. These methods did n’t use the classic Newton directions. Instead they used a direction that can be characterized as a negative gradient direction (in a scaled space) for a so-called self-regular barrier function. The barrier function is determined by a simple univariate function, called its kernel function. Bai et al. [1] introduced a new barrier function which is n’t a barrier function in the usual sense. In the current paper, we first propose an infeasible interior-point algorithm with full-NT steps. The main iteration in the algorithm consists of a feasibility step and several centrality steps, whose feasibility step is induced by the classic logarithm barrier function. And then we propose a new infeasible interior-point algorithm, whose feasibility step is induced by a specific kernel function. For X 2 Sn ; kðXÞ denotes the vector of eigenvalues. Two different forms of norm will be used,
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u n uX 2 kXk2 :¼ maxfjki ðXÞjg and kXkF ¼ t ki ðXÞ: i
i¼1
2. The statement of the algorithm As usual we assume that the initial iterates X 0 ; y0 and S0 are as follows
X 0 ¼ S0 ¼ fI;
l0 ¼ f2 ;
y0 ¼ 0;
where I is the n n identity matrix,
l0 is the initial dual gap and f > 0 is such that
kX þ S k2 6 f for some optimal solution ðX ; y ; S Þ of (P) and (D). The perturbed KKT condition for (P) and (D) is
AX ¼ b;
X 0;
A y þ S ¼ C;
S 0;
XS ¼ lI: Newton’s methods yield the following system of equations
ADX ¼ b AX; A Dy þ DS ¼ C A y S;
DXS þ X DS ¼ lI XS: If X is primal feasible and ðy; SÞ dual feasible, the above system reduces to
ADX ¼ 0; A Dy þ DS ¼ 0;
ð2:1Þ
DXS þ X DS ¼ lI XS: We consider the symmetrization scheme that yields the NT direction. Let us define the matrix
1 1 12 1 1 1 12 1 1 1 P ¼ X 2 X 2 SX 2 X 2 ¼ S2 S2 XS2 S2
ð2:2Þ
1
and D ¼ P 2 . The matrix D can be used to rescale X and S to be the same matrix V, defined by
1 1 V :¼ pffiffiffiffi D1 XD1 ¼ pffiffiffiffi DSD:
l
ð2:3Þ
l
Obviously D and V are symmetric and positive definite. After defining
1 DX :¼ pffiffiffiffi D1 DXD1 ;
l
1 DS :¼ pffiffiffiffi DDSD;
l
ð2:4Þ
the complementary condition in (2.1) reduces to
DX þ DS ¼ V 1 V:
ð2:5Þ
4992
Z. Liu, W. Sun / Applied Mathematics and Computation 217 (2011) 4990–4999
2.1. Infeasible IPMs based on logarithm barrier functions For any mð0 < m 6 1Þ, we consider the perturbed problems ðPm Þ and ðDm Þ, defined by
ðPm Þ
n o min Tr C mR0C X : AX ¼ b mr0b ; X 0
ðDm Þ
max
and
n
o T b mr0b y : A y þ S ¼ C mR0C ; S 0 ;
where
r 0b ¼ b AX 0 ; R0C ¼ C A y0 S0 : If m ¼ 1, then ðP m Þ and ðDm Þ both have a strictly feasible solution. This means that both perturbed problems satisfy the Slater’s regularity condition. Lemma 2.1 [5, Lemma 4.1]. Let the original problems, (P) and (D), be feasible. Then for each m such that 0 < m 6 1 the perturbed problems, ðP m Þ and ðDm Þ, are strictly feasible. Assuming that (P) and (D) are feasible, it follows from Lemma 2.1 that ðP m Þ and ðDm Þ satisfy the Slater’s regularity condition, for m 2 ð0; 1. Then their central path exists. The central path for ðPm Þ and ðDm Þ is defined by the solution sets fðXðlÞ; yðlÞ; SðlÞÞ; l > 0g of the following system
AX ¼ b mr 0b ;
X 0;
A y þ S ¼ C mR0C ;
S 0;
XS ¼ lI: If m 2 ð0; 1 and l ¼ mf2 , we denote this unique solution as ðXðmÞ; yðmÞ; SðmÞÞ. XðmÞ is the l-center of ðPm Þ, and ðyðmÞ; SðlÞÞ the lcenter of ðDm Þ. By taking m ¼ 1, one has ðXð1Þ; yð1Þ; Sð1ÞÞ ¼ ðX 0 ; y0 ; S0 Þ ¼ ðfI; 0; fIÞ. We measure proximity of iterates ðX; y; SÞ to the l-center of the perturbed problems ðPm Þ and ðDm Þ by the quantity
dðX; S; lÞ :¼ dðVÞ :¼
1 1 kV VkF : 2
ð2:6Þ
Initially one has X ¼ S ¼ fI and l ¼ f2 , whence V ¼ I and dðX; S; lÞ ¼ 0. In the sequel we assume that, at the start of each iteration, dðX; S; lÞ is smaller than or equal to a (small) positive threshold value s. Of course this is true at the start of the first iteration. Before describing the algorithm, we give the definition of a kernel function. Definition 2.2. A twice differentiable function wðtÞ : ð0; 1Þ ! ½0; 1Þ is called a kernel function if
wð1Þ ¼ 0; w0 ð1Þ ¼ 0; w00 ðtÞ > 0;
8t > 0:
The right-hand side of (2.5) is the negative gradient direction of the following function
WðVÞ :¼ TrðwðVÞÞ; whose kernel function is the logarithmic barrier function
wðtÞ ¼
1 2 ðt 1Þ log t: 2
Using (2.2)–(2.5), the complementary condition now becomes
D1 DXSD þ DDSXD1 ¼ lI D1 XSD: We use the following system to define the feasibility step ðDf X; Df y; Df SÞ, denoted by
ADf X ¼ hmr 0b ; A Df y þ Df S ¼ hmR0C ; 1
f
f
ð2:7Þ 1
D X D SD þ DD SXD
1
¼ lI D XSD;
where h 2 ð0; 1Þ. The algorithm begins with an infeasible interior point ðX; y; SÞ such that ðX; y; SÞ is feasible for the perturbed problems, TrðXSÞ ¼ nl and dðX; S; lÞ 6 s. First we find a new point ðX f ; yf ; Sf Þ which is feasible for the perturbed problems with mþ :¼ ð1 hÞm. Then l is decreased to lþ :¼ ð1 hÞl. Generally dðX f ; Sf ; lþ Þ 6 s can be satisfied no longer. So a limited
Z. Liu, W. Sun / Applied Mathematics and Computation 217 (2011) 4990–4999
4993
centering (centrality) steps are applied to produce new points ðX þ ; yþ ; Sþ Þ such that dðX þ ; Sþ ; lþ Þ 6 s. This process is repeated until the algorithm terminates. After a feasibility step, the new iterates
X f ¼ X þ Df X;
yf ¼ y þ Df y;
Sf ¼ S þ Df S;
are still interior points (i.e., X f 0 and Sf 0) since h is small enough. In the centrality steps, starting at the iterates ðX f ; yf ; Sf Þ and targeting at the l-center, the search direction ðDX; Dy; DSÞ are the usual NT direction, defined by
ADf X ¼ 0; A Df y þ Df S ¼ 0; D1 Df SD þ DDf SXD1 ¼ lI D1 XSD: Now we give a more formal description of the algorithm in Table 1. 2.2. Infeasible IPMs based on a new kernel function System (2.7) can be reduced to the following system
ADfX ¼ hmr 0b ; A DfX where
Df y
l þ
1 þ DfS ¼ pffiffiffiffi hmDR0C D;
l
DfS
DfX ; DfS
¼V
1
V;
are defined similar to (2.4) and
AðÞ ¼ ðTrðA1 ðÞÞ; TrðA2 ðÞÞ; . . . ; TrðAm ðÞÞÞT ;
Ai ¼
pffiffiffiffi lDAi D:
Now the new feasibility step is defined by a different system
ADfX ¼ hmr 0b ; A DfX
Df y
l þ
1 þ DfS ¼ pffiffiffiffi hmDR0C D;
l
DfS
¼ rUðVÞ;
where the kernel function of UðVÞ is
/ðtÞ ¼
1 ðt 1Þ2 : 2
Since /0 ðtÞ ¼ t 1, the third equation in the system can be written as
DfX þ DfS ¼ I V: It is worth pointing out that all known kernel functions (denoted by wðtÞ here) have the properties
Table 1 Algorithm. Primal–Dual Infeasible IPMs for SDP Input: Accuracy parameter e > 0; barrier update parameter h; 0 < h < 1; threshold parameter s > 0. begin X :¼ fI; y :¼ 0; S :¼ fI; m ¼ 1; while maxfTrðXSÞ; kb AXk; kC A y SkF g P e do begin feasibility step: ðX; y; SÞ :¼ ðX; y; SÞ þ ðDf X; Df y; Df SÞ;
l-update: l :¼ ð1 hÞl; centering steps: while dðX; S; lÞ P s do ðX; y; SÞ :¼ ðX; y; SÞ þ ðDX; Dy; DSÞ; end while end end while end
ð2:8Þ
4994
Z. Liu, W. Sun / Applied Mathematics and Computation 217 (2011) 4990–4999
lim wðtÞ ¼ 1; lim wðtÞ ¼ 1: t!1
t#0
The new kernel function in this paper satisfies the second property but it fails to have the first, that is,
lim /ðtÞ ¼ /ð0Þ ¼ t#0
1 6 1: 2
This means that if either X or S approaches the boundary of the feasible region then UðVÞ converges to a finite value. Similar to (2.6), we define
1 2
rðX; S; lÞ :¼ rðVÞ :¼ kI VkF : It is obvious that
rðVÞ ¼ 0 if and only if V ¼ I.
3. Technical results Denoting the iterates after a centrality step as X þ ; yþ ; Sþ , we recall the following results from Klerk [2]. Lemma 3.1. Let X; S satisfy the Slater’s regularity condition and feasible.
l > 0. If d :¼ dðX; S; lÞ < 1, then the full-NT step is strictly
Corollary 3.2. Let X; S satisfy the Slater’s regularity condition and
l > 0 such that dðX; S; lÞ < 1. One has TrðX þ Sþ Þ ¼ nl.
Lemma 3.3. After a feasible full-NT step the proximity function satisfies
dþ :¼ dðX þ ; Sþ ; lÞ 6
d2 2ð1 d2 Þ
:
pffiffiffi Lemma 3.4. If d :¼ dðX; S; lÞ < 1= 2, then dðX þ ; Sþ ; lÞ < d2 . By using Lemma 3.4, the required number of centrality steps can easily be obtained. After the pffiffiffi dðX; S; lþ Þ 6 1= 2, and hence after k centrality steps the iterates ðX; y; SÞ satisfy
l-update, one has
2k 1 dðX; S; lþ Þ 6 pffiffiffi : 2 One can easily deduce that no more than
1 log2 log2 2 ;
ð3:1Þ
s
centrality steps are needed. Now we give some lemmas, which will be used in the analysis. Lemma 3.5. For any t > 0, one has
1 j1 tj 6 t ; t
1 1 2 t t 2 6 jt 1 tj:
Proof. The proof of both results above is easy.
h
Lemma p 3.6. Let ffiffiffiffiffiffiffiffiffiffiffi ffi ðX; SÞ be a primal–dual NT pair and e :¼ V 12 = 1 h. Then V
e Þ2 6 ð1 hÞdðVÞ2 þ dð V
h2 n : 4ð1 hÞ
Proof. By kVk2F ¼ n, one has 1 2
kV k2F ¼
n X i¼1
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u n uX pffiffiffi ki ðVÞ 6 t ki ðVÞ2 n ¼ n: i¼1
l > 0 such that TrðXSÞ ¼ nl. Moreover let dðVÞ ¼ dðX; S; lÞ and
Z. Liu, W. Sun / Applied Mathematics and Computation 217 (2011) 4990–4999
4995
Hence
2
2 pffiffiffiffiffiffiffiffiffiffiffiffi 1 pffiffiffiffiffiffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffiffiffiffiffi
1
1 1
1 1
1
1 hV 12 pffiffiffiffiffiffiffiffiffiffiffi
1 h V 12 V 12 þ
2 ¼ 2 pffiffiffiffiffiffiffiffiffiffiffiffi V 2 ffi 1 h V V
4 4 1h 1h F F
1 2 2 2 2 1 2 1 1 1 1 1 h 12 h 1 h n 1
2
2 ¼ h n V 2
V V 2 þ
V Tr V 2 V 2 V 2 6 ð1 hÞdðVÞ þ F F F 4 2 4ð1 hÞ 2 4ð1 hÞ
e Þ2 ¼ dð V
6 ð1 hÞdðVÞ2 þ
h2 n ; 4ð1 hÞ
where the first inequality is due to Lemma 3.5.
h
, then Xða Þ 0; Sða Þ 0. Lemma 3.7 [2, Lemma 6.1]. If one has detðXðaÞSðaÞÞ > 0; 80 6 a 6 a Let Q be an n n real symmetric matrix and M be an n n real skew-symmetric matrix, we recall the following result. Lemma 3.8 [6, Lemma 2.1]. If Q is positive definite, then TrððQ þ MÞ1 Þ 6 TrðQ 1 Þ. Lemma 3.9 [2, Lemma 6.3]. If Q is positive definite, then detðQ þ MÞ > 0. Lemma 3.10. Let A; B 2 Sn ; A þ B 0 and kmin ðAÞ þ kmin ðBÞ > 0. Then n X i¼1
n X ðki ðAÞ þ ki ðBÞÞ; i¼1
n X i¼1
ki ðA þ BÞ ¼
n X 1 1 ¼ : ki ðA þ BÞ ki ðAÞ þ ki ðBÞ i¼1
Proof. The first equality is well-known. For the second equality, let
g1 :¼ kmin ðA þ BÞ; g2 :¼ kmin ðAÞ þ kmin ðBÞ: Thus g1 > 0 and g2 > 0. Now one has
X X n n n n X 1 1 ki ðAÞ þ ki ðBÞ ki ðA þ BÞ 1 X ¼ 6 ðk ðAÞ þ ki ðBÞ ki ðA þ BÞÞ ¼ 0; i¼1 ki ðAÞ þ ki ðBÞ i¼1 ki ðA þ BÞ i¼1 ki ðA þ BÞðki ðAÞ þ ki ðBÞÞ g1 g2 i¼1 i which implies the second result in the lemma follows.
h
In the sequel we denotes e as the all-one vector. Lemma 3.11 [4, Lemma A.1]. For i ¼ 1; . . . ; m, let fi : R ! R denote convex functions. Then, for any nonzero vector z 2 R, the following inequality n X i¼1
n X 1 X fi ðzi Þ 6 T zj fj ðeT zÞ þ fi ð0Þ e z j¼1 i–j
!
holds.
4. Analysis of the feasibility step 4.1. Effect of the feasibility step From Section 2, we know that the feasibility step generates new iterates X f ; yf and Sf that satisfy the feasibility conditions for ðP mþ Þ and ðDmþ Þ (i.e., primal feasible and dual feasible), except possibly the positive semidefinite conditions. A crucial element in the analysis is to show that after the feasibility step dðX f ; Sf ; lþ Þ 6 1 such that the new iterates are positive definite and within the region where the Newton process targeting at the lþ -center of ðPmþ Þ and ðDmþ Þ is quadratically convergent. Let X; y and S denote the iterates at the start of an iteration and assume that dðX; S; lÞ 6 s. Recall that at the start of the first iteration this is true since dðX 0 ; S0 ; l0 Þ ¼ 0. Defining DfX ; DfS as in (2.4) and V as in (2.3). We may write
4996
Z. Liu, W. Sun / Applied Mathematics and Computation 217 (2011) 4990–4999
D1 X f Sf D ¼ D1 ðX þ Df XÞðS þ Df SÞD ¼ D1 ðXS þ Df XS þ X Df S þ Df X Df SÞD ¼ D1 XSD þ D1 Df XSD þ D1 X Df SD þ D1 Df X Df SD: Since D1 X Df SD DDf SXD1 and (2.8), one has
X f Sf lV þ D1 Df X Df SD:
ð4:1Þ
Using (2.3) and (2.4), we may also write
pffiffiffiffi X f ¼ X þ Df X ¼ lDðV þ DfX ÞD; pffiffiffiffi Sf ¼ S þ Df S ¼ lD1 ðV þ DfS ÞD1 :
ð4:2Þ ð4:3Þ
Denoting
DfXS :¼
1 f f DX DS þ DfS DfX ; 2
we can get the following result. Lemma 4.1. The new iterates are certainly strictly feasible if and only if V þ DfXS 0. Proof. Let
X f ðaÞ ¼ X þ aDf X;
Sf ðaÞ ¼ S þ aDf S:
From (4.1), one can get
1 X f ðaÞSf ðaÞ lðV þ a2 DfX DfS Þ ¼ l V þ a2 DfXS þ a2 DfX DfS DfS DfX : 2 The matrix DfX DfS DfS DfX is skew-symmetric. Lemma 3.9 implies that the determinant of ½X f ðaÞSf ðaÞ will be positive if the matrix V þ a2 DfXS . Using Lemma 3.7, the ‘if’ part is completed. For the ‘only if’ part, we can use the associated NT scaling (2.2) and (2.3) to rescale the left-hand side to ðV f ðaÞÞ2 , thus
V þ a2 DfXS ðV f ðaÞÞ2
1 f f DX DS DfS DfX : 2
Following the similar way as the proof of the ‘if’ part, the ‘only if’ part is obvious.
h
Lemma 4.2. The iterates X f ; yf and Sf are strictly feasible if
kDfX kF <
1
qðdÞ
where
qðdÞ :¼ d þ
and kDfS kF <
1
qðdÞ
;
qffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 þ d2 :
ð4:4Þ
Proof. It is clear from (4.2) that X f is strictly feasible if and only if V þ DfX 0. This certainly holds if kkðDfX Þk < mini fki ðVÞg. Since
2d ¼ kV V 1 kF ¼ kkðVÞ kðV 1 Þk; the minimal value t that an entry of the vector kðVÞ can attain will satisfy t 6 1 and 1=t t ¼ 2d. The last equation here impffiffiffiffiffiffiffiffiffiffiffiffiffi ffi plies t2 þ 2dt 1 ¼ 0, which gives t ¼ d þ 1 þ d2 ¼ 1=qðdÞ. This proves the first inequality in the lemma. The second inequality is obtained in the similar way. h The proof of this lemma makes clear that the eigenvalues of V satisfy
1
qðdÞ
6 ki ðVÞ2 6 qðdÞ;
i ¼ 1; 2; . . . ; n:
In the sequel we denote
x :¼
1 2
This implies
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi kDfX k2F þ kDfS k2F :
ð4:5Þ
Z. Liu, W. Sun / Applied Mathematics and Computation 217 (2011) 4990–4999
4997
1 TrðDfXS Þ ¼ TrðDfX DfS Þ 6 kDfX k2 kDfS kF 6 kDfX kF kDfS kF 6 ðkDfX k2F þ kDfS k2F Þ 6 2x2 ; 2
!
f f f f Df Df þ Df Df
D D þ D D
X S f f f X S S X S X
2xi :¼ jki DXS j ¼ ki 6
6 kDX kF kDS kF 6 2x2 :
2 2 F
Lemma 4.3. Assuming V þ DfXS 0, one has
4dðV f Þ2 6 4ð1 hÞdðVÞ2 þ
h2 n 2x2 2ð1 hÞqðdÞ2 x2 þ : þ 1h 1h 1 2qðdÞx2
Proof. From (2.6), one has
dðX f ; Sf ; lþ Þ :¼ dðV f Þ :¼
1 f kV ðV f Þ1 kF ; 2
where ðV f Þ
1
lþ
D1 X f Sf D:
Using (4.1), we obtain
D1 X f Sf D lV þ D1 Df X Df SD: Furthermore, we get
ðV f Þ2
V þ DfX DfS V þ DfXS DfX DfS DfS DfX l l V þ þ DfX DfS ¼ ¼ þ : lþ l 1h 1h 2ð1 hÞ
Then
4dðV f Þ2 ¼ TrððV f Þ2 Þ þ TrððV f Þ2 Þ 2n 6 Tr
V þ DfXS 1h
!
þ Tr ð1 hÞðV þ DfXS Þ1 2n
n n n n X X ki ðVÞ þ ki ðDfXS Þ X ki ðVÞ þ ki ðDfXS Þ X 1h 1h 2n ¼ 2n þ þ f 1 h 1 h ðV þ D Þ ðVÞ þ ki ðDfXS Þ k k i i i¼1 i¼1 i¼1 i¼1 XS n n X ki ðVÞ þ 2xi X 1h 6 2n; þ k 1 h ðVÞ 2xi i i¼1 i¼1
¼
where we apply Lemma 3.8 for the first inequality and Lemma 3.10 for the third equality. For each i, we define
fi ðzi Þ :¼
ki ðVÞ þ zi 1h 2; þ ki ðVÞ zi 1h
i ¼ 1; 2; . . . ; n:
One can easily verify that if ki ðVÞ zi > 0 then fi ðzi Þ is convex in zi . Taking zi ¼ 2x2i , we can require
ki ðVÞ 2x2i > 0: By using (4.5), this certainly holds if
2x2 <
1
qðdÞ
ð4:6Þ
:
We may use Lemma 3.11 and give
4dðV f Þ2 6
n X
fj ðxj Þ 6
n 1 X
2x2j fj ð2x2 Þ þ
X
! fi ð0Þ
2x2 j¼1 i–j X !# n 2 1 X k ðVÞ þ 2 x 1 h ki ðVÞ 1 h j 2 2xj 2 þ þ þ 2 : ¼ 2x2 j¼1 kj ðVÞ 2x2 1 h ki ðVÞ 1h i–j j¼1
"
Using Lemma 3.6, we obtain
X ki ðVÞ i–j
Hence
1h
þ
X n 1h ki ðVÞ 1 h kj ðVÞ 1 h þ þ 2 ¼ 2 2 ki ðVÞ 1 h ki ðVÞ 1 h kj ðVÞ i¼1 kj ðVÞ 1 h h2 n þ 2 : 6 4ð1 hÞdðVÞ þ 1 h kj ðVÞ 1h
4998
Z. Liu, W. Sun / Applied Mathematics and Computation 217 (2011) 4990–4999
4dðV f Þ2 6 4ð1 hÞdðVÞ þ
n 2 h2 n 1 X 1h kj ðVÞ 1 h 2 kj ðVÞ þ 2x 2 x 2 þ þ þ 2 j 1 h 2x2 j¼1 kj ðVÞ 2x2 1 h kj ðVÞ 1h
¼ 4ð1 hÞdðVÞ þ
n h2 n 2x2 1 X 2ð1 hÞx2 þ 2x2j þ 2 1 h 1 h 2x j¼1 kj ðVÞðkj ðVÞ 2x2 Þ
6 4ð1 hÞdðVÞ þ
h2 n 2x2 2ð1 hÞx2 h2 n 2x2 2ð1 hÞqðdÞ2 x2 þ þ : þ 1 1 þ ¼ 4ð1 hÞdðVÞ þ 1 h 1 h qðdÞ ðqðdÞ 2x2 Þ 1h 1h 1 2qðdÞx2
The lemma follows.
h
pffiffiffi Because we need to have dðV f Þ 6 1 2, it follows from this lemma that it suffices if
4ð1 hÞdðVÞ þ
h2 n 2x2 2ð1 hÞqðdÞ2 x2 þ 6 2: þ 1h 1h 1 2qðdÞx2
ð4:7Þ
At this stage we decide to choose
1 a s ¼ ; h ¼ pffiffiffi ; a 6 1: 8
ð4:8Þ
2 n
The left-hand side of (4.7) is monotonically increasing with respect to x2 , then for n P 1 and dðVÞ 6 s, together with (4.6), one can verify that
1
1
x 6 pffiffiffi ) dðV f Þ 6 pffiffiffi : 2 2
ð4:9Þ
2
4.2. Upper bound for xðVÞ Let
L ¼ fN 2 Sn : AN ¼ 0g: Thus the affine space fN 2 Sn : AN ¼ hmr 0b g equals to DfX þ L and implies DfS 2 p1ffiffilffi hmDR0C D þ L? . We can get the following result. Lemma 4.4. Let Q be the (unique) point in the intersection of the affine spaces DfX þ L and DfS þ L? . Then
2x 6
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi kQ k2F þ ðkQkF þ rðVÞÞ2 :
Proof. We can get the proof by using I V instead of V 1 V. For details, see Roos [8].
h
pffiffiffi pffiffiffi From (4.9) we know that x 6 1= 2 2 is needed in order that dðV f Þ 6 1= 2. Due to Lemma 4.4 this will hold if
kQ k2F þ ðkQkF þ rðVÞÞ2 6
1 : 2
ð4:10Þ
In fact, according to Mansouri and Roos [5], one has
kQ kF 6 nhqðdÞð1 þ qðdÞ2 Þ: Since d 6 s ¼ 18, we have
kQ kF 6 nhqðdÞð1 þ qðdÞ2 Þ ¼ 2:586nh: By using h ¼ 2pa ffiffin, the above inequality reads
kQ kF 6
pffiffiffi 2:586 na : 2
ð4:11Þ
2 Note that (4.10) and Lemma 3.5 imply that it suffices if kQ k2F þ kQ kF þ 14 6 1=2. The latter holds if kQ kF 6 0:359. Hence, combining this with (4.11), we obtain
pffiffiffi 2:586 na 6 0:359: 2
This means if we take
a¼
5 pffiffiffi ; 18 n
pffiffiffi it is guaranteed that dðV f Þ 6 1= 2.
ð4:12Þ
Z. Liu, W. Sun / Applied Mathematics and Computation 217 (2011) 4990–4999
4999
4.3. Complexity In the previous sections we have found that if at the start of an iteration the iterates satisfy dðX; S; lÞ 6 s, withpsffiffiffiand h as defined in (4.8), and with taking a as in (4.12), then after the feasibility step the iterates satisfy dðX; S; lþ Þ 6 1= 2. According to (3.1), at most
1 log2 log2 2 ¼ log2 ðlog2 64Þ;
s
centrality steps suffice to get iterates that satisfy dðX; S; lþ Þ 6 s. So each main iteration consists of one feasibility step and at most 3 centrality steps. In each main iteration both the duality gap and the norms of the residual vectors are reduced by the factor 1 h. Using TrðX 0 S0 Þ ¼ nf2 , the total number of iterations is bounded above by
maxfnf2 ; kr 0b k; kR0C kF g 1 log : h e Since
a 5 h ¼ pffiffiffi ¼ ; 2 n 36n the total number of inner iterations is therefore bounded above by
22n log
maxfnf2 ; kr 0b k; kR0C kF g
e
:
5. Concluding remarks In the paper we extended the full-Newton step infeasible interior-point algorithm to SDP. The feasibility step was induced by the classic logarithm barrier function. After that we used a new kernel function to induce the feasibility step and we analyzed the algorithm based on this kernel function. The same result of complexity can be obtained. New kernel-functionsbased methods may be studied later. References [1] Y.Q. Bai, M. El Ghami, C. Roos, A new efficient large-update primal-dual interior-point method based a finite barrier, SIAM J. Optim. 13 (3) (2002) 766– 782. [2] E. de Klerk, Aspects of Semidefinite Programming: Interior Point Methods and Selected Applications, Kluwer Academic Publishers, Dordrecht, The Netherlands, 2002. [3] Z. Liu, W. Sun, An infeasible interior-point algorithm with full-Newton step for linear programming, Numer. Algorithms 46 (2) (2007) 173–188. [4] H. Mansouri, C. Roos, Simplified OðnLÞ infeasible interior-point algorithm for linear optimization using full Newton steps, Optim. Methods Softw. 22 (3) (2007) 519–530. [5] H. Mansouri, C. Roos, A new full-Newton step OðnÞ infeasible interior-point algorithm for semidefinite optimization, Numer. Algorithms 52 (2) (2009) 225–255. [6] J. Peng, C. Roos, T. Terlaky, New complexity analysis of the primal-dual method for semidefinite optimization based on NT-direction, J. Optim. Theor. Appl. 109 (2) (2001) 327–343. [7] J. Peng, C. Roos, T. Terlaky, Self-regular functions and new search directions for linear and semidefinite optimization, Math. Prog. 93 (1) (2002) 129–171. [8] C. Roos, A full-Newton step OðnÞ infeasible interior-point algorithm for linear optimization, SIAM J. Optim. 16 (4) (2006) 1110–1136. [9] H. Wolkowicz, R. Saigal, L. Vandenberghe, Handbook of semidefinite programming: theory, Algorithms and Applications, Kluwer, Norwell, MA, 1999.
Contents lists available at ScienceDirect
Applied Mathematics and Computation journal homepage: www.elsevier.com/locate/amc
A full-NT-step infeasible interior-point algorithm for SDP based on kernel functions q Zhongyi Liu a,⇑, Wenyu Sun b a b
College of Science, Hohai University, Nanjing 210098, China School of Mathematics Science, Nanjing Normal University, Nanjing 210097, China
a r t i c l e
i n f o
a b s t r a c t This paper proposes an infeasible interior-point algorithm with full Nesterov–Todd (NT) steps for semidefinite programming (SDP). The main iteration consists of a feasibility step and several centrality steps. First we present a full NT step infeasible interior-point algorithm based on the classic logarithmical barrier function. After that a specific kernel function is introduced. The feasibility step is induced by this kernel function instead of the classic logarithmical barrier function. This kernel function has a finite value on the boundary. The result of polynomial complexity, Oðn log n=eÞ, coincides with the best known one for infeasible interior-point methods. Ó 2010 Elsevier Inc. All rights reserved.
Keywords: Semidefinite programming Full Nesterov–Todd steps Infeasible interior-point methods Polynomial complexity Kernel functions
1. Introduction We are concerned with SDP problems, whose primal and dual forms are
ðPÞ
minfTrðCXÞ : TrðAi XÞ ¼ bi ; i ¼ 1; 2; . . . ; m; X 0g
ðDÞ
max b y :
and
( T
m X
) yi Ai þ S ¼ C; S 0 ;
i¼1 n
where Ai ; C; X; S 2 S , and b; y 2 Rm . The matrices Ai ði ¼ 1; 2; . . . ; mÞ are assumed to be linearly independent. By introducing
AX :¼ ðTrðA1 XÞ; TrðA2 XÞ; . . . ; TrðAm XÞÞT and
A y :¼
m X
y i Ai ;
i¼1
ðPÞ and ðDÞ can be expressed into the following equivalent forms
ðPÞ
minfTrðCXÞ : AX ¼ b; X 0g
and q
This work was supported in part by the Fundamental Funds for the Central Universities in China (No. 2009B27314).
⇑ Corresponding author.
E-mail addresses: [email protected] (Z. Liu), [email protected] (W. Sun). 0096-3003/$ - see front matter Ó 2010 Elsevier Inc. All rights reserved. doi:10.1016/j.amc.2010.11.049
Z. Liu, W. Sun / Applied Mathematics and Computation 217 (2011) 4990–4999
4991
T
maxfb y : A y þ S ¼ C; S 0g:
ðDÞ
Primal–dual interior-point methods (IPMs) for SDP have been widely studied, the reader is referred to Klerk [2] and Wolkowicz et al. [9]. Recently a full-Newton step infeasible interior-point algorithm for linear programming (LP) was presented by Roos [8]. The result of polynomial complexity coincides with the best known one for infeasible IPMs. Some extensions were carried out by Mansouri and Roos [4], Liu and Sun [3]. Mansouri and Roos [5] extended this algorithm to SDP by using a specific feasibility step. Peng et al. [7] introduced a new class of primal–dual IPMs based on self-regular proximities. These methods did n’t use the classic Newton directions. Instead they used a direction that can be characterized as a negative gradient direction (in a scaled space) for a so-called self-regular barrier function. The barrier function is determined by a simple univariate function, called its kernel function. Bai et al. [1] introduced a new barrier function which is n’t a barrier function in the usual sense. In the current paper, we first propose an infeasible interior-point algorithm with full-NT steps. The main iteration in the algorithm consists of a feasibility step and several centrality steps, whose feasibility step is induced by the classic logarithm barrier function. And then we propose a new infeasible interior-point algorithm, whose feasibility step is induced by a specific kernel function. For X 2 Sn ; kðXÞ denotes the vector of eigenvalues. Two different forms of norm will be used,
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u n uX 2 kXk2 :¼ maxfjki ðXÞjg and kXkF ¼ t ki ðXÞ: i
i¼1
2. The statement of the algorithm As usual we assume that the initial iterates X 0 ; y0 and S0 are as follows
X 0 ¼ S0 ¼ fI;
l0 ¼ f2 ;
y0 ¼ 0;
where I is the n n identity matrix,
l0 is the initial dual gap and f > 0 is such that
kX þ S k2 6 f for some optimal solution ðX ; y ; S Þ of (P) and (D). The perturbed KKT condition for (P) and (D) is
AX ¼ b;
X 0;
A y þ S ¼ C;
S 0;
XS ¼ lI: Newton’s methods yield the following system of equations
ADX ¼ b AX; A Dy þ DS ¼ C A y S;
DXS þ X DS ¼ lI XS: If X is primal feasible and ðy; SÞ dual feasible, the above system reduces to
ADX ¼ 0; A Dy þ DS ¼ 0;
ð2:1Þ
DXS þ X DS ¼ lI XS: We consider the symmetrization scheme that yields the NT direction. Let us define the matrix
1 1 12 1 1 1 12 1 1 1 P ¼ X 2 X 2 SX 2 X 2 ¼ S2 S2 XS2 S2
ð2:2Þ
1
and D ¼ P 2 . The matrix D can be used to rescale X and S to be the same matrix V, defined by
1 1 V :¼ pffiffiffiffi D1 XD1 ¼ pffiffiffiffi DSD:
l
ð2:3Þ
l
Obviously D and V are symmetric and positive definite. After defining
1 DX :¼ pffiffiffiffi D1 DXD1 ;
l
1 DS :¼ pffiffiffiffi DDSD;
l
ð2:4Þ
the complementary condition in (2.1) reduces to
DX þ DS ¼ V 1 V:
ð2:5Þ
4992
Z. Liu, W. Sun / Applied Mathematics and Computation 217 (2011) 4990–4999
2.1. Infeasible IPMs based on logarithm barrier functions For any mð0 < m 6 1Þ, we consider the perturbed problems ðPm Þ and ðDm Þ, defined by
ðPm Þ
n o min Tr C mR0C X : AX ¼ b mr0b ; X 0
ðDm Þ
max
and
n
o T b mr0b y : A y þ S ¼ C mR0C ; S 0 ;
where
r 0b ¼ b AX 0 ; R0C ¼ C A y0 S0 : If m ¼ 1, then ðP m Þ and ðDm Þ both have a strictly feasible solution. This means that both perturbed problems satisfy the Slater’s regularity condition. Lemma 2.1 [5, Lemma 4.1]. Let the original problems, (P) and (D), be feasible. Then for each m such that 0 < m 6 1 the perturbed problems, ðP m Þ and ðDm Þ, are strictly feasible. Assuming that (P) and (D) are feasible, it follows from Lemma 2.1 that ðP m Þ and ðDm Þ satisfy the Slater’s regularity condition, for m 2 ð0; 1. Then their central path exists. The central path for ðPm Þ and ðDm Þ is defined by the solution sets fðXðlÞ; yðlÞ; SðlÞÞ; l > 0g of the following system
AX ¼ b mr 0b ;
X 0;
A y þ S ¼ C mR0C ;
S 0;
XS ¼ lI: If m 2 ð0; 1 and l ¼ mf2 , we denote this unique solution as ðXðmÞ; yðmÞ; SðmÞÞ. XðmÞ is the l-center of ðPm Þ, and ðyðmÞ; SðlÞÞ the lcenter of ðDm Þ. By taking m ¼ 1, one has ðXð1Þ; yð1Þ; Sð1ÞÞ ¼ ðX 0 ; y0 ; S0 Þ ¼ ðfI; 0; fIÞ. We measure proximity of iterates ðX; y; SÞ to the l-center of the perturbed problems ðPm Þ and ðDm Þ by the quantity
dðX; S; lÞ :¼ dðVÞ :¼
1 1 kV VkF : 2
ð2:6Þ
Initially one has X ¼ S ¼ fI and l ¼ f2 , whence V ¼ I and dðX; S; lÞ ¼ 0. In the sequel we assume that, at the start of each iteration, dðX; S; lÞ is smaller than or equal to a (small) positive threshold value s. Of course this is true at the start of the first iteration. Before describing the algorithm, we give the definition of a kernel function. Definition 2.2. A twice differentiable function wðtÞ : ð0; 1Þ ! ½0; 1Þ is called a kernel function if
wð1Þ ¼ 0; w0 ð1Þ ¼ 0; w00 ðtÞ > 0;
8t > 0:
The right-hand side of (2.5) is the negative gradient direction of the following function
WðVÞ :¼ TrðwðVÞÞ; whose kernel function is the logarithmic barrier function
wðtÞ ¼
1 2 ðt 1Þ log t: 2
Using (2.2)–(2.5), the complementary condition now becomes
D1 DXSD þ DDSXD1 ¼ lI D1 XSD: We use the following system to define the feasibility step ðDf X; Df y; Df SÞ, denoted by
ADf X ¼ hmr 0b ; A Df y þ Df S ¼ hmR0C ; 1
f
f
ð2:7Þ 1
D X D SD þ DD SXD
1
¼ lI D XSD;
where h 2 ð0; 1Þ. The algorithm begins with an infeasible interior point ðX; y; SÞ such that ðX; y; SÞ is feasible for the perturbed problems, TrðXSÞ ¼ nl and dðX; S; lÞ 6 s. First we find a new point ðX f ; yf ; Sf Þ which is feasible for the perturbed problems with mþ :¼ ð1 hÞm. Then l is decreased to lþ :¼ ð1 hÞl. Generally dðX f ; Sf ; lþ Þ 6 s can be satisfied no longer. So a limited
Z. Liu, W. Sun / Applied Mathematics and Computation 217 (2011) 4990–4999
4993
centering (centrality) steps are applied to produce new points ðX þ ; yþ ; Sþ Þ such that dðX þ ; Sþ ; lþ Þ 6 s. This process is repeated until the algorithm terminates. After a feasibility step, the new iterates
X f ¼ X þ Df X;
yf ¼ y þ Df y;
Sf ¼ S þ Df S;
are still interior points (i.e., X f 0 and Sf 0) since h is small enough. In the centrality steps, starting at the iterates ðX f ; yf ; Sf Þ and targeting at the l-center, the search direction ðDX; Dy; DSÞ are the usual NT direction, defined by
ADf X ¼ 0; A Df y þ Df S ¼ 0; D1 Df SD þ DDf SXD1 ¼ lI D1 XSD: Now we give a more formal description of the algorithm in Table 1. 2.2. Infeasible IPMs based on a new kernel function System (2.7) can be reduced to the following system
ADfX ¼ hmr 0b ; A DfX where
Df y
l þ
1 þ DfS ¼ pffiffiffiffi hmDR0C D;
l
DfS
DfX ; DfS
¼V
1
V;
are defined similar to (2.4) and
AðÞ ¼ ðTrðA1 ðÞÞ; TrðA2 ðÞÞ; . . . ; TrðAm ðÞÞÞT ;
Ai ¼
pffiffiffiffi lDAi D:
Now the new feasibility step is defined by a different system
ADfX ¼ hmr 0b ; A DfX
Df y
l þ
1 þ DfS ¼ pffiffiffiffi hmDR0C D;
l
DfS
¼ rUðVÞ;
where the kernel function of UðVÞ is
/ðtÞ ¼
1 ðt 1Þ2 : 2
Since /0 ðtÞ ¼ t 1, the third equation in the system can be written as
DfX þ DfS ¼ I V: It is worth pointing out that all known kernel functions (denoted by wðtÞ here) have the properties
Table 1 Algorithm. Primal–Dual Infeasible IPMs for SDP Input: Accuracy parameter e > 0; barrier update parameter h; 0 < h < 1; threshold parameter s > 0. begin X :¼ fI; y :¼ 0; S :¼ fI; m ¼ 1; while maxfTrðXSÞ; kb AXk; kC A y SkF g P e do begin feasibility step: ðX; y; SÞ :¼ ðX; y; SÞ þ ðDf X; Df y; Df SÞ;
l-update: l :¼ ð1 hÞl; centering steps: while dðX; S; lÞ P s do ðX; y; SÞ :¼ ðX; y; SÞ þ ðDX; Dy; DSÞ; end while end end while end
ð2:8Þ
4994
Z. Liu, W. Sun / Applied Mathematics and Computation 217 (2011) 4990–4999
lim wðtÞ ¼ 1; lim wðtÞ ¼ 1: t!1
t#0
The new kernel function in this paper satisfies the second property but it fails to have the first, that is,
lim /ðtÞ ¼ /ð0Þ ¼ t#0
1 6 1: 2
This means that if either X or S approaches the boundary of the feasible region then UðVÞ converges to a finite value. Similar to (2.6), we define
1 2
rðX; S; lÞ :¼ rðVÞ :¼ kI VkF : It is obvious that
rðVÞ ¼ 0 if and only if V ¼ I.
3. Technical results Denoting the iterates after a centrality step as X þ ; yþ ; Sþ , we recall the following results from Klerk [2]. Lemma 3.1. Let X; S satisfy the Slater’s regularity condition and feasible.
l > 0. If d :¼ dðX; S; lÞ < 1, then the full-NT step is strictly
Corollary 3.2. Let X; S satisfy the Slater’s regularity condition and
l > 0 such that dðX; S; lÞ < 1. One has TrðX þ Sþ Þ ¼ nl.
Lemma 3.3. After a feasible full-NT step the proximity function satisfies
dþ :¼ dðX þ ; Sþ ; lÞ 6
d2 2ð1 d2 Þ
:
pffiffiffi Lemma 3.4. If d :¼ dðX; S; lÞ < 1= 2, then dðX þ ; Sþ ; lÞ < d2 . By using Lemma 3.4, the required number of centrality steps can easily be obtained. After the pffiffiffi dðX; S; lþ Þ 6 1= 2, and hence after k centrality steps the iterates ðX; y; SÞ satisfy
l-update, one has
2k 1 dðX; S; lþ Þ 6 pffiffiffi : 2 One can easily deduce that no more than
1 log2 log2 2 ;
ð3:1Þ
s
centrality steps are needed. Now we give some lemmas, which will be used in the analysis. Lemma 3.5. For any t > 0, one has
1 j1 tj 6 t ; t
1 1 2 t t 2 6 jt 1 tj:
Proof. The proof of both results above is easy.
h
Lemma p 3.6. Let ffiffiffiffiffiffiffiffiffiffiffi ffi ðX; SÞ be a primal–dual NT pair and e :¼ V 12 = 1 h. Then V
e Þ2 6 ð1 hÞdðVÞ2 þ dð V
h2 n : 4ð1 hÞ
Proof. By kVk2F ¼ n, one has 1 2
kV k2F ¼
n X i¼1
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u n uX pffiffiffi ki ðVÞ 6 t ki ðVÞ2 n ¼ n: i¼1
l > 0 such that TrðXSÞ ¼ nl. Moreover let dðVÞ ¼ dðX; S; lÞ and
Z. Liu, W. Sun / Applied Mathematics and Computation 217 (2011) 4990–4999
4995
Hence
2
2 pffiffiffiffiffiffiffiffiffiffiffiffi 1 pffiffiffiffiffiffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffiffiffiffiffi
1
1 1
1 1
1
1 hV 12 pffiffiffiffiffiffiffiffiffiffiffi
1 h V 12 V 12 þ
2 ¼ 2 pffiffiffiffiffiffiffiffiffiffiffiffi V 2 ffi 1 h V V
4 4 1h 1h F F
1 2 2 2 2 1 2 1 1 1 1 1 h 12 h 1 h n 1
2
2 ¼ h n V 2
V V 2 þ
V Tr V 2 V 2 V 2 6 ð1 hÞdðVÞ þ F F F 4 2 4ð1 hÞ 2 4ð1 hÞ
e Þ2 ¼ dð V
6 ð1 hÞdðVÞ2 þ
h2 n ; 4ð1 hÞ
where the first inequality is due to Lemma 3.5.
h
, then Xða Þ 0; Sða Þ 0. Lemma 3.7 [2, Lemma 6.1]. If one has detðXðaÞSðaÞÞ > 0; 80 6 a 6 a Let Q be an n n real symmetric matrix and M be an n n real skew-symmetric matrix, we recall the following result. Lemma 3.8 [6, Lemma 2.1]. If Q is positive definite, then TrððQ þ MÞ1 Þ 6 TrðQ 1 Þ. Lemma 3.9 [2, Lemma 6.3]. If Q is positive definite, then detðQ þ MÞ > 0. Lemma 3.10. Let A; B 2 Sn ; A þ B 0 and kmin ðAÞ þ kmin ðBÞ > 0. Then n X i¼1
n X ðki ðAÞ þ ki ðBÞÞ; i¼1
n X i¼1
ki ðA þ BÞ ¼
n X 1 1 ¼ : ki ðA þ BÞ ki ðAÞ þ ki ðBÞ i¼1
Proof. The first equality is well-known. For the second equality, let
g1 :¼ kmin ðA þ BÞ; g2 :¼ kmin ðAÞ þ kmin ðBÞ: Thus g1 > 0 and g2 > 0. Now one has
X X n n n n X 1 1 ki ðAÞ þ ki ðBÞ ki ðA þ BÞ 1 X ¼ 6 ðk ðAÞ þ ki ðBÞ ki ðA þ BÞÞ ¼ 0; i¼1 ki ðAÞ þ ki ðBÞ i¼1 ki ðA þ BÞ i¼1 ki ðA þ BÞðki ðAÞ þ ki ðBÞÞ g1 g2 i¼1 i which implies the second result in the lemma follows.
h
In the sequel we denotes e as the all-one vector. Lemma 3.11 [4, Lemma A.1]. For i ¼ 1; . . . ; m, let fi : R ! R denote convex functions. Then, for any nonzero vector z 2 R, the following inequality n X i¼1
n X 1 X fi ðzi Þ 6 T zj fj ðeT zÞ þ fi ð0Þ e z j¼1 i–j
!
holds.
4. Analysis of the feasibility step 4.1. Effect of the feasibility step From Section 2, we know that the feasibility step generates new iterates X f ; yf and Sf that satisfy the feasibility conditions for ðP mþ Þ and ðDmþ Þ (i.e., primal feasible and dual feasible), except possibly the positive semidefinite conditions. A crucial element in the analysis is to show that after the feasibility step dðX f ; Sf ; lþ Þ 6 1 such that the new iterates are positive definite and within the region where the Newton process targeting at the lþ -center of ðPmþ Þ and ðDmþ Þ is quadratically convergent. Let X; y and S denote the iterates at the start of an iteration and assume that dðX; S; lÞ 6 s. Recall that at the start of the first iteration this is true since dðX 0 ; S0 ; l0 Þ ¼ 0. Defining DfX ; DfS as in (2.4) and V as in (2.3). We may write
4996
Z. Liu, W. Sun / Applied Mathematics and Computation 217 (2011) 4990–4999
D1 X f Sf D ¼ D1 ðX þ Df XÞðS þ Df SÞD ¼ D1 ðXS þ Df XS þ X Df S þ Df X Df SÞD ¼ D1 XSD þ D1 Df XSD þ D1 X Df SD þ D1 Df X Df SD: Since D1 X Df SD DDf SXD1 and (2.8), one has
X f Sf lV þ D1 Df X Df SD:
ð4:1Þ
Using (2.3) and (2.4), we may also write
pffiffiffiffi X f ¼ X þ Df X ¼ lDðV þ DfX ÞD; pffiffiffiffi Sf ¼ S þ Df S ¼ lD1 ðV þ DfS ÞD1 :
ð4:2Þ ð4:3Þ
Denoting
DfXS :¼
1 f f DX DS þ DfS DfX ; 2
we can get the following result. Lemma 4.1. The new iterates are certainly strictly feasible if and only if V þ DfXS 0. Proof. Let
X f ðaÞ ¼ X þ aDf X;
Sf ðaÞ ¼ S þ aDf S:
From (4.1), one can get
1 X f ðaÞSf ðaÞ lðV þ a2 DfX DfS Þ ¼ l V þ a2 DfXS þ a2 DfX DfS DfS DfX : 2 The matrix DfX DfS DfS DfX is skew-symmetric. Lemma 3.9 implies that the determinant of ½X f ðaÞSf ðaÞ will be positive if the matrix V þ a2 DfXS . Using Lemma 3.7, the ‘if’ part is completed. For the ‘only if’ part, we can use the associated NT scaling (2.2) and (2.3) to rescale the left-hand side to ðV f ðaÞÞ2 , thus
V þ a2 DfXS ðV f ðaÞÞ2
1 f f DX DS DfS DfX : 2
Following the similar way as the proof of the ‘if’ part, the ‘only if’ part is obvious.
h
Lemma 4.2. The iterates X f ; yf and Sf are strictly feasible if
kDfX kF <
1
qðdÞ
where
qðdÞ :¼ d þ
and kDfS kF <
1
qðdÞ
;
qffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 þ d2 :
ð4:4Þ
Proof. It is clear from (4.2) that X f is strictly feasible if and only if V þ DfX 0. This certainly holds if kkðDfX Þk < mini fki ðVÞg. Since
2d ¼ kV V 1 kF ¼ kkðVÞ kðV 1 Þk; the minimal value t that an entry of the vector kðVÞ can attain will satisfy t 6 1 and 1=t t ¼ 2d. The last equation here impffiffiffiffiffiffiffiffiffiffiffiffiffi ffi plies t2 þ 2dt 1 ¼ 0, which gives t ¼ d þ 1 þ d2 ¼ 1=qðdÞ. This proves the first inequality in the lemma. The second inequality is obtained in the similar way. h The proof of this lemma makes clear that the eigenvalues of V satisfy
1
qðdÞ
6 ki ðVÞ2 6 qðdÞ;
i ¼ 1; 2; . . . ; n:
In the sequel we denote
x :¼
1 2
This implies
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi kDfX k2F þ kDfS k2F :
ð4:5Þ
Z. Liu, W. Sun / Applied Mathematics and Computation 217 (2011) 4990–4999
4997
1 TrðDfXS Þ ¼ TrðDfX DfS Þ 6 kDfX k2 kDfS kF 6 kDfX kF kDfS kF 6 ðkDfX k2F þ kDfS k2F Þ 6 2x2 ; 2
!
f f f f Df Df þ Df Df
D D þ D D
X S f f f X S S X S X
2xi :¼ jki DXS j ¼ ki 6
6 kDX kF kDS kF 6 2x2 :
2 2 F
Lemma 4.3. Assuming V þ DfXS 0, one has
4dðV f Þ2 6 4ð1 hÞdðVÞ2 þ
h2 n 2x2 2ð1 hÞqðdÞ2 x2 þ : þ 1h 1h 1 2qðdÞx2
Proof. From (2.6), one has
dðX f ; Sf ; lþ Þ :¼ dðV f Þ :¼
1 f kV ðV f Þ1 kF ; 2
where ðV f Þ
1
lþ
D1 X f Sf D:
Using (4.1), we obtain
D1 X f Sf D lV þ D1 Df X Df SD: Furthermore, we get
ðV f Þ2
V þ DfX DfS V þ DfXS DfX DfS DfS DfX l l V þ þ DfX DfS ¼ ¼ þ : lþ l 1h 1h 2ð1 hÞ
Then
4dðV f Þ2 ¼ TrððV f Þ2 Þ þ TrððV f Þ2 Þ 2n 6 Tr
V þ DfXS 1h
!
þ Tr ð1 hÞðV þ DfXS Þ1 2n
n n n n X X ki ðVÞ þ ki ðDfXS Þ X ki ðVÞ þ ki ðDfXS Þ X 1h 1h 2n ¼ 2n þ þ f 1 h 1 h ðV þ D Þ ðVÞ þ ki ðDfXS Þ k k i i i¼1 i¼1 i¼1 i¼1 XS n n X ki ðVÞ þ 2xi X 1h 6 2n; þ k 1 h ðVÞ 2xi i i¼1 i¼1
¼
where we apply Lemma 3.8 for the first inequality and Lemma 3.10 for the third equality. For each i, we define
fi ðzi Þ :¼
ki ðVÞ þ zi 1h 2; þ ki ðVÞ zi 1h
i ¼ 1; 2; . . . ; n:
One can easily verify that if ki ðVÞ zi > 0 then fi ðzi Þ is convex in zi . Taking zi ¼ 2x2i , we can require
ki ðVÞ 2x2i > 0: By using (4.5), this certainly holds if
2x2 <
1
qðdÞ
ð4:6Þ
:
We may use Lemma 3.11 and give
4dðV f Þ2 6
n X
fj ðxj Þ 6
n 1 X
2x2j fj ð2x2 Þ þ
X
! fi ð0Þ
2x2 j¼1 i–j X !# n 2 1 X k ðVÞ þ 2 x 1 h ki ðVÞ 1 h j 2 2xj 2 þ þ þ 2 : ¼ 2x2 j¼1 kj ðVÞ 2x2 1 h ki ðVÞ 1h i–j j¼1
"
Using Lemma 3.6, we obtain
X ki ðVÞ i–j
Hence
1h
þ
X n 1h ki ðVÞ 1 h kj ðVÞ 1 h þ þ 2 ¼ 2 2 ki ðVÞ 1 h ki ðVÞ 1 h kj ðVÞ i¼1 kj ðVÞ 1 h h2 n þ 2 : 6 4ð1 hÞdðVÞ þ 1 h kj ðVÞ 1h
4998
Z. Liu, W. Sun / Applied Mathematics and Computation 217 (2011) 4990–4999
4dðV f Þ2 6 4ð1 hÞdðVÞ þ
n 2 h2 n 1 X 1h kj ðVÞ 1 h 2 kj ðVÞ þ 2x 2 x 2 þ þ þ 2 j 1 h 2x2 j¼1 kj ðVÞ 2x2 1 h kj ðVÞ 1h
¼ 4ð1 hÞdðVÞ þ
n h2 n 2x2 1 X 2ð1 hÞx2 þ 2x2j þ 2 1 h 1 h 2x j¼1 kj ðVÞðkj ðVÞ 2x2 Þ
6 4ð1 hÞdðVÞ þ
h2 n 2x2 2ð1 hÞx2 h2 n 2x2 2ð1 hÞqðdÞ2 x2 þ þ : þ 1 1 þ ¼ 4ð1 hÞdðVÞ þ 1 h 1 h qðdÞ ðqðdÞ 2x2 Þ 1h 1h 1 2qðdÞx2
The lemma follows.
h
pffiffiffi Because we need to have dðV f Þ 6 1 2, it follows from this lemma that it suffices if
4ð1 hÞdðVÞ þ
h2 n 2x2 2ð1 hÞqðdÞ2 x2 þ 6 2: þ 1h 1h 1 2qðdÞx2
ð4:7Þ
At this stage we decide to choose
1 a s ¼ ; h ¼ pffiffiffi ; a 6 1: 8
ð4:8Þ
2 n
The left-hand side of (4.7) is monotonically increasing with respect to x2 , then for n P 1 and dðVÞ 6 s, together with (4.6), one can verify that
1
1
x 6 pffiffiffi ) dðV f Þ 6 pffiffiffi : 2 2
ð4:9Þ
2
4.2. Upper bound for xðVÞ Let
L ¼ fN 2 Sn : AN ¼ 0g: Thus the affine space fN 2 Sn : AN ¼ hmr 0b g equals to DfX þ L and implies DfS 2 p1ffiffilffi hmDR0C D þ L? . We can get the following result. Lemma 4.4. Let Q be the (unique) point in the intersection of the affine spaces DfX þ L and DfS þ L? . Then
2x 6
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi kQ k2F þ ðkQkF þ rðVÞÞ2 :
Proof. We can get the proof by using I V instead of V 1 V. For details, see Roos [8].
h
pffiffiffi pffiffiffi From (4.9) we know that x 6 1= 2 2 is needed in order that dðV f Þ 6 1= 2. Due to Lemma 4.4 this will hold if
kQ k2F þ ðkQkF þ rðVÞÞ2 6
1 : 2
ð4:10Þ
In fact, according to Mansouri and Roos [5], one has
kQ kF 6 nhqðdÞð1 þ qðdÞ2 Þ: Since d 6 s ¼ 18, we have
kQ kF 6 nhqðdÞð1 þ qðdÞ2 Þ ¼ 2:586nh: By using h ¼ 2pa ffiffin, the above inequality reads
kQ kF 6
pffiffiffi 2:586 na : 2
ð4:11Þ
2 Note that (4.10) and Lemma 3.5 imply that it suffices if kQ k2F þ kQ kF þ 14 6 1=2. The latter holds if kQ kF 6 0:359. Hence, combining this with (4.11), we obtain
pffiffiffi 2:586 na 6 0:359: 2
This means if we take
a¼
5 pffiffiffi ; 18 n
pffiffiffi it is guaranteed that dðV f Þ 6 1= 2.
ð4:12Þ
Z. Liu, W. Sun / Applied Mathematics and Computation 217 (2011) 4990–4999
4999
4.3. Complexity In the previous sections we have found that if at the start of an iteration the iterates satisfy dðX; S; lÞ 6 s, withpsffiffiffiand h as defined in (4.8), and with taking a as in (4.12), then after the feasibility step the iterates satisfy dðX; S; lþ Þ 6 1= 2. According to (3.1), at most
1 log2 log2 2 ¼ log2 ðlog2 64Þ;
s
centrality steps suffice to get iterates that satisfy dðX; S; lþ Þ 6 s. So each main iteration consists of one feasibility step and at most 3 centrality steps. In each main iteration both the duality gap and the norms of the residual vectors are reduced by the factor 1 h. Using TrðX 0 S0 Þ ¼ nf2 , the total number of iterations is bounded above by
maxfnf2 ; kr 0b k; kR0C kF g 1 log : h e Since
a 5 h ¼ pffiffiffi ¼ ; 2 n 36n the total number of inner iterations is therefore bounded above by
22n log
maxfnf2 ; kr 0b k; kR0C kF g
e
:
5. Concluding remarks In the paper we extended the full-Newton step infeasible interior-point algorithm to SDP. The feasibility step was induced by the classic logarithm barrier function. After that we used a new kernel function to induce the feasibility step and we analyzed the algorithm based on this kernel function. The same result of complexity can be obtained. New kernel-functionsbased methods may be studied later. References [1] Y.Q. Bai, M. El Ghami, C. Roos, A new efficient large-update primal-dual interior-point method based a finite barrier, SIAM J. Optim. 13 (3) (2002) 766– 782. [2] E. de Klerk, Aspects of Semidefinite Programming: Interior Point Methods and Selected Applications, Kluwer Academic Publishers, Dordrecht, The Netherlands, 2002. [3] Z. Liu, W. Sun, An infeasible interior-point algorithm with full-Newton step for linear programming, Numer. Algorithms 46 (2) (2007) 173–188. [4] H. Mansouri, C. Roos, Simplified OðnLÞ infeasible interior-point algorithm for linear optimization using full Newton steps, Optim. Methods Softw. 22 (3) (2007) 519–530. [5] H. Mansouri, C. Roos, A new full-Newton step OðnÞ infeasible interior-point algorithm for semidefinite optimization, Numer. Algorithms 52 (2) (2009) 225–255. [6] J. Peng, C. Roos, T. Terlaky, New complexity analysis of the primal-dual method for semidefinite optimization based on NT-direction, J. Optim. Theor. Appl. 109 (2) (2001) 327–343. [7] J. Peng, C. Roos, T. Terlaky, Self-regular functions and new search directions for linear and semidefinite optimization, Math. Prog. 93 (1) (2002) 129–171. [8] C. Roos, A full-Newton step OðnÞ infeasible interior-point algorithm for linear optimization, SIAM J. Optim. 16 (4) (2006) 1110–1136. [9] H. Wolkowicz, R. Saigal, L. Vandenberghe, Handbook of semidefinite programming: theory, Algorithms and Applications, Kluwer, Norwell, MA, 1999.