Convergence of the homotopy path for a full-Newton step infeasible interior-point method

Operations Research Letters 38 (2010) 147–151

Contents lists available at ScienceDirect

Operations Research Letters journal homepage: www.elsevier.com/locate/orl

Convergence of the homotopy path for a full-Newton step infeasible interior-point method A. Asadi, G. Gu ∗ , C. Roos Faculty of Electrical Engineering, Mathematics and Computer Science, Delft University of Technology, The Netherlands

article

info

Article history: Received 22 July 2009 Accepted 8 November 2009 Available online 26 November 2009 Keywords: Linear optimization Infeasible interior-point method Full-Newton step Homotopy path

abstract Roos [C. Roos, A full-Newton step O(n) infeasible interior-point algorithm for linear optimization. SIAM J. Optim. 16 (4) (2006) 1110–1136 (electronic)] proposed a new primal–dual infeasible interior-point method for linear optimization. This new method can be viewed as a homotopy method. In this work, we show that the homotopy path has precisely one accumulation point in the optimal set. Moreover, this accumulation point is the analytic center of a subset of the optimal set and depends on the starting point of the infeasible interior-point method. © 2009 Elsevier B.V. All rights reserved.

1. Introduction

2. Convergence properties

We consider the Linear Optimization (LO) problem in the standard form

As has become usual for IIPMs, we start the algorithm with a triple (x0 , y0 , s0 ) such that

(P)

min{c T x : Ax = b, x ≥ 0},

with its dual problem (D)

max{bT y : AT y + s = c , s ≥ 0}.

Here A ∈ Rm×n , b, y ∈ Rm , and c, x, s ∈ Rn . Without loss of generality we assume that rank(A) = m. The vectors x, y, and s are the vectors of variables. In [6] a new primal–dual Infeasible Interior-Point Method (IIPM) is proposed for solving the above LO problems. This new method differs from the usual IIPMs (e.g. [2–5,9]) in that the new method uses only full-Newton steps (instead of damped steps), which has the advantage that no line searches are needed. The motivation for the use of full steps is that, though such methods are less greedy, the best iteration bounds for IIPMs (namely, O(nL)) are obtained for such methods. The new IIPM can be viewed as a homotopy method, which approximately solves a sequence of perturbed problems. In this work we investigate its convergence properties. We prove that the homotopy path converges to a strictly complementary (optimal) solution, which is the analytic center of a subset of the optimal set of (P) and (D). It might be worth mentioning that our result differs from those of [2,8], where the global convergence and some analyticity properties for usual IIPMs are presented.

∗ Corresponding address: Faculty of Electrical Engineering, Mathematics and Computer Science, Delft University of Technology, P.O. Box 5031, 2600 GA Delft, The Netherlands. E-mail address: [email protected] (G. Gu). 0167-6377/$ – see front matter © 2009 Elsevier B.V. All rights reserved. doi:10.1016/j.orl.2009.11.006

x0 = ζ e,

y0 = 0,

s0 = ζ e,

where ζ (starting parameter) denotes an arbitrary positive number. So the initial values of the primal and dual residual vectors are b − Aζ e and c −ζ e, respectively. For any 0 < ν ≤ 1 we consider the perturbed problem defined by

(Pν )

min{(c − ν(c − ζ e))T x : Ax = b − ν(b − Aζ e), x ≥ 0},

with its dual problem

(Dν )

max{(b − ν(b − Aζ e))T y : AT y + s = c − ν(c − ζ e),

s ≥ 0}. Note that if ν = 1, then x = x0 yields a strictly feasible solution of (Pν ) and (y, s) = (y0 , s0 ) a strictly feasible solution of (Dν ). We assume that both problems (P) and (D) are feasible; then we know (see e.g. [6]) that the system b − Ax = ν(b − Aζ e), c − AT y − s = ν(c − ζ e), xs = νζ 2 e,

x ≥ 0, s ≥ 0, 0≤ν≤1

(1)

has a unique solution for every ν ∈ (0, 1] (here, we denote by e the all-one vector of appropriate size; moreover, if x, s ∈ Rn , then xs denotes the componentwise, or Hadamard product of the vectors x and s). This solution is the µ-center (µ = νζ 2 ) of the corresponding perturbed problems (Pν ) and (Dν ), which we denoted as (x(ν), y(ν), s(ν)). By applying the implicit function theorem, we may easily see that (x(ν), y(ν), s(ν)) depends

148

A. Asadi et al. / Operations Research Letters 38 (2010) 147–151

analytically on ν and forms a continuous path, which is the homotopy path. In the sequel, we will investigate the convergence properties of the homotopy path (as ν goes to 0). We denote the support of any nonnegative vector x as σ (x). So, if x ∈ Rn+ then

σ (x) = {i : xi > 0} . To simplify notation we use the notation x = x(ν), y = y(ν), and s = s(ν) in the next lemma. Lemma 2.1. Let (x∗ , y∗ , s∗ ) denote an arbitrary optimal solution of (P) and (D). Then we have, for any ν ∈ (0, 1],

" (1 − ν)ζ

X x∗ i

i∈σ (x∗ )

xi

+

X s∗ i i∈σ (s∗ )

#

si

= (1 − ν)eT (x∗ + s∗ ) − eT (x + s) + (1 + ν)ζ n.

(2)

Proof. Since b = Ax∗ and c = AT y∗ + s∗ , the system (1) can be rewritten as A(x − x) = ν A(x − ζ e), AT (y∗ − y) + (s∗ − s) = ν(AT y∗ + s∗ − ζ e), xs = νζ 2 e, ∗

x ≥ 0, s ≥ 0, 0 ≤ ν ≤ 1.

∗

the set of optimal solutions of (P) and (D). Moreover, any such accumulation point (˜x, y˜ , s˜) is strictly complementary and satisfies

Using that the row space of A and its null space are orthogonal, we obtain

(1 − ν)(s x + x s ) = ν(1 − ν)ζ e (x + s ) − νζ e (x + s) + xT s + ν 2 ζ 2 eT e. T

∗

∗

T

eT (x(ν) + s(ν)) ≤ max(eT (x∗ + s∗ ) + ζ n, 2ζ n).

By the definition of the sets σ (x∗ ) and σ (s∗ ) we have x∗i = 0 if i 6∈ σ (x∗ ) and s∗i = 0 if i 6∈ σ (s∗ ). Hence it follows that

"

# X

(1 − ν)

∗

si xi +

i∈σ (x∗ )

X

∗

xi si

i∈σ (s∗ )

= ν(1 − ν)ζ eT (x∗ + s∗ ) − νζ eT (x + s) + xT s + ν 2 ζ 2 n. Using xs = νζ 2 e, from the third equation of (1), we get

" (1 − ν)

X νζ 2

i∈σ (x∗ )

xi

∗

xi +

X νζ 2 i∈σ (s∗ )

si

# ∗

si

= ν(1 − ν)ζ eT (x∗ + s∗ ) − νζ eT (x + s) + νζ 2 n + ν 2 ζ 2 n. After dividing both sides by νζ we obtain (2), thus completing the proof of the lemma. 

Hence the homotopy path, i.e. the set {(x(ν), y(ν), s(ν)) : 0 < ν ≤ 1}, lies in the compact set eT (x(ν) + s(ν)) ≤ max(eT (x∗ + s∗ ) + ζ n, 2ζ n), where x(ν) ≥ 0 and s(ν) ≥ 0. Now let ν1 = 1 and {νk }∞ k=1 be a strictly decreasing sequence converging to 0 if k → ∞, and let xk = x(νk ), yk = y(νk ) and sk = s(νk ). Since the sequence (xk , sk ) lies in a compact set, it has an accumulation point (˜x, s˜). It follows that a subsequence of the sequence (xk , sk ) converges to (˜x, s˜). Without loss of generality we assume below that the sequence (xk , sk ) itself converges to ∞ (˜x, s˜). Since (xk )T sk = νk ζ 2 n, the sequence (xk )T sk k=1 is strictly

decreasing, and converges to 0. Thus it follows that x˜ T s˜ = 0. Since A has full rank, s˜ determines y˜ uniquely such that (˜x, y˜ , s˜) is an optimal solution of (P) and (D). Putting (x∗ , y∗ , s∗ ) = (ˆx, yˆ , sˆ), ν = νk and (x, y, s) = (xk , yk , sk ) in (2), while also using (4), we get

 Since the left-hand side of the identity in (2) is nonnegative, the following corollary follows trivially.

(1 − νk )ζ

X xˆ i  i∈Bopt

Corollary 2.2. For any optimal solution (x , y , s ) of (P) and (D) and for any ν ∈ (0, 1] one has ∗

∗

∗

eT (x(ν) + s(ν)) ≤ (1 − ν)eT (x∗ + s∗ ) + (1 + ν)ζ n.

(5)

opt

Proof. Since the right-hand side in (3) depends linearly on ν and 0 ≤ ν ≤ 1, we have

Since (x∗ )T s∗ = 0 we derive from this that T ∗

 X sˆi X xˆ i  = eT (ˆx + sˆ) − eT (˜x + s˜) + ζ n. + ζ ˜ ˜ x s i i i∈N i∈B 

opt

 T (1 − ν)x∗ − x + νζ e [(1 − ν)s∗ − s + νζ e] = 0. T ∗

Fig. 1. Feasible region of the dual problem for α = 1 and β = 2.

(3)

By the well known theorem of Goldman and Tucker (cf. [1,7]) the problems (P) and (D) have a strictly complementary (optimal) solution (ˆx, yˆ , sˆ). Hence, when denoting the classes in the optimal partition of (P) and (D) as Bopt and Nopt , one has for each optimal solution (x∗ , y∗ , s∗ ) of (P) and (D) that

xki

 X sˆi  = (1 − νk )eT (ˆx + sˆ) + k i∈Nopt

si

− eT (xk + sk ) + (1 + νk )ζ n,

k = 1, 2, . . . .

Now letting k go to ∞, we have that νk goes to 0, xk goes to x˜ and sk to s˜. Thus we obtain the relation (5). Since the right-hand side expression in (5) is a real number, the left-hand side expression must be well-defined. Thus it follows that if i ∈ Bopt then x˜ i > 0, and if i ∈ Nopt then s˜i > 0. Hence it follows that σ (˜x) = Bopt and σ (˜s) = Nopt , proving that (˜x, y˜ , s˜) is strictly complementary. This completes the proof of the lemma. 

(4)

The following lemma makes clear that the homotopy path has only one accumulation point, which implies that it converges.

Lemma 2.3. Let (ˆx, yˆ , sˆ) be any strictly complementary solution of (P) and (D). The homotopy path has an accumulation point in

Lemma 2.4. The homotopy path has precisely one accumulation point in the optimal set.

Bopt = σ (ˆx) ⊇ σ (x∗ ),

Nopt = σ (ˆs) ⊇ σ (s∗ ).

A. Asadi et al. / Operations Research Letters 38 (2010) 147–151

149

Fig. 2. Homotopy path for α = 1 and β = 2, and several values of ζ .

Proof. By Lemma 2.3 the homotopy path has an accumulation point (˜x, y˜ , s˜) in the optimal set. Suppose we have another accumulation point (¯x, y¯ , s¯) of the homotopy path in the optimal set. By applying Lemma 2.3 twice, the first time with (˜x, y˜ , s˜) = (˜x, y˜ , s˜) and (ˆx, yˆ , sˆ) = (¯x, y¯ , s¯) and the second time with (˜x, y˜ , s˜) = (¯x, y¯ , s¯) and (ˆx, yˆ , sˆ) = (˜x, y˜ , s˜), we obtain

We finally prove that the limit of the homotopy path is the analytic center of a subset of the set of optimal solutions.

 X s¯i X x¯ i  = eT (¯x + s¯) − eT (˜x + s˜) + ζ n, + ζ x˜ s˜ i∈Bopt i i∈Nopt i   X x˜ i X s˜i  = eT (˜x + s˜) − eT (¯x + s¯) + ζ n. ζ + ¯ ¯ x s i i i∈N i∈B

Proof. Let (˜x, y˜ , s˜) be a strictly complementary solution of (P) and (D) that is an accumulation point of the homotopy path. Let S (˜x, s˜) denote the set of optimal solutions of (P) and (D) such that eT (x∗ + s∗ ) ≤ eT (˜x + s˜), and (x∗ , y∗ , s∗ ) ∈ S (˜x, s˜). Using arguments similar to those in the proof of Lemma 2.3, replacing (ˆx, yˆ , sˆ) by (x∗ , y∗ , s∗ ) and using (4), one proves that



opt

opt



 X x∗ X s∗ i i  ζ + = eT (x∗ + s∗ ) − eT (˜x + s˜) + ζ n. ˜ ˜ x s i i i∈N i∈B

By adding these relations, while defining

 zi =

x¯ i /˜xi , s¯i /˜si ,

if i ∈ Bopt , if i ∈ Nopt ,

(7)

opt

opt

Using eT (x∗ + s∗ ) ≤ eT (˜x + s˜), and upon dividing both sides by ζ , this implies

we obtain

ζ

Lemma 2.5. Let (˜x, y˜ , s˜) be the limit point of the homotopy path in the optimal set. Then it is the analytic center of the set of optimal solutions (x∗ , y∗ , s∗ ) of (P) and (D) satisfying eT (x∗ + s∗ ) ≤ eT (˜x +˜s).

n X (zi + zi−1 ) = 2ζ n.

(6)

i=1

X x∗ X s∗ i i + ≤ n. x˜ i s˜i i∈B i∈N opt

Since each zi is the quotient of two positive numbers, we have zi > 0. Therefore, 1 2

− 21

zi + zi−1 = (zi − zi

)2 + 2 ≥ 2,

with equality if and only if zi = 1. Thus it follows from (6) that zi = 1 for each i, which means that x¯ = x˜ and s¯ = s˜. This proves the lemma. 

opt

The left-hand side expression is a sum of (at most) n nonnegative numbers. Using the arithmetic–geometric mean inequality we obtain

1/n   Y x∗ Y s∗ X s∗ 1 X x∗i i i i   ≤   ≤ 1. + x˜ s˜ n i∈B x˜ i s˜ i∈Bopt i i∈Nopt i i∈Nopt i opt 

150

A. Asadi et al. / Operations Research Letters 38 (2010) 147–151

Fig. 3. Homotopy path for α = 200 and β = 100, and several values of ζ .

Thus we have

We consider the case where

" # α c= β ,

Y x∗ Y s∗ i i ≤ 1. ˜ ˜ x s i i∈Bopt i∈Nopt i

A=

It will be convenient to define the function

where α and β are positive numbers. Then (P) and (D) are respectively given by

f (x∗ , s∗ ) :=

Y

Y

x∗i

i∈Bopt

s∗i

(8)

i∈Nopt

 α 1



−β

0 , 1

1

0

  b=

0 , 3

min {α x1 + β x2 : α x1 − β x2 = 0, x1 + x2 + x3 = 3, x = (x1 ; x2 ; x3 ) ≥ 0} ,

on the set of optimal solutions of (P) and (D). Then we have

max {3y2 : α y1 + y2 ≤ α, − β y1 + y2 ≤ β, y2 ≤ 0} .

f (x∗ , s∗ ) ≤ f (˜x, s˜),

The feasible region of the dual problem is depicted in Fig. 1. One may easily verify that the set of optimal solutions is given by

∀(x∗ , y∗ , s∗ ) ∈ S (˜x, s˜).

This means that (˜x, y˜ , s˜) maximizes the product

Q

i∈Bopt

∗

xi

Q

i∈Nopt

s∗i on the set S (˜x, s˜). Note that f (˜x, s˜) is positive, because the pair (˜x, s˜) is strictly complementary. On the other hand, for optimal solutions that are not strictly complementary we have f (x∗ , s∗ ) = 0. Hence the maximum of f (x∗ , s∗ ) occurs in a strictly complementary solution. The logarithmic function being strictly monotonically increasing, we can equally well maximize log f (x∗ , s∗ ), which has the same maximizer(s) on the set of strictly complementary solutions in S (˜x, s˜). However, when the pair (x∗ , s∗ ) is strictly complementary, one has log f (x∗ , s∗ ) :=

X i∈Bopt

log x∗i +

X

log s∗i .

i∈Nopt

Since the set S (˜x, s˜) is convex, by definition (see, e.g., [7]) the maximizer of f (x∗ , s∗ ) on S (˜x, s˜) is the analytic center of S (˜x, s˜).  A question that arises is that of whether the limit point of the homotopy path depends on the starting parameter ζ , or not. We answer this question by using the following example.

(

" #   " #! ) 0 α(1 − y1 ) y1 0 , (x, y, s) = , β(1 + y1 ) , −1 ≤ y1 ≤ 1 . 0

3

(9)

0

We conclude from (9) that the classes in the optimal partition are given by Bopt = {3} ,

Nopt = {1, 2} .

As a consequence we have

Y i∈Bopt

x∗i

Y

s∗i = 3 · α(1 − y1 ) · β(1 + y1 ) = 3αβ(1 − y21 ).

i∈Nopt

The last expression is maximal for y1 = 0. Hence, putting y1 = 0 in (9), we get that the analytic center is given by

" #   " #! 0 α 0 0 , ( x, y , s) = , β . 3

0

0

Now we turn to the homotopy path. We proceed by taking α = 1 and β = 2. For that case we computed numerically the

A. Asadi et al. / Operations Research Letters 38 (2010) 147–151

homotopy path for several values of ζ . The results are shown in Fig. 2. The starting point of the homotopy path is the zero vector, which is drawn as a ‘+’. The limit point is drawn as a ‘×’. The figure clearly demonstrates that the limit point depends greatly on the value of ζ . It may be noted that in each of the four cases the limit point is such that y˜ 1 ≤ 0. This also follows from Lemma 2.5. Because for any optimal solution (x, y, s) we have eT (x + s) = 3 + α + β + (β − α)y1 = 6 + y1 ,

Y i∈Bopt

Y

xi

si = 6(1 − y21 ).

i∈Nopt

Hence, by Lemma 2.5 we should have y1 ≤ y˜ 1

⇒

y21 ≥ y˜ 21 .

This implication can be true only if y˜ 1 ≤ 0. When α > β , one proves in the same way that y˜ 1 ≥ 0. For an illustration we refer the reader to Fig. 3. One might observe that in all cases it is true that the larger the value of ζ is, the closer the limit point is to the analytic center. This holds indeed in general, as we show in the next lemma. Lemma 2.6. Let ζ > 0 and (˜x, y˜ , s˜) be the limit point in the optimal set of the corresponding homotopy path. If ζ goes to infinity then (˜x, y˜ , s˜) converges to the analytic center of the optimal set. Proof. Since x˜ ≥ 0 and s˜ ≥ 0 we have eT (˜x + s˜) ≥ 0. Hence it follows from (7) that



 X x∗ X s∗ i i  ζ + ≤ eT (x∗ + s∗ ) + ζ n, ˜ ˜ x s i i i∈B i∈N opt

opt

where (x , y , s ) denotes some optimal triple. Dividing by ζ on both sides we get ∗

∗

∗

 1 n

151



X s∗ X x∗ i  i + ≤ 1, λ˜ x λ˜ si i i∈Nopt i∈Bopt



λ :=

e T ( x∗ + s∗ )

ζn

+ 1.

Due to the geometric–arithmetic mean inequality this implies

 1n Y x∗ Y s∗ i  i  ≤ 1. λ˜ x λ˜ si i i∈Nopt i∈Bopt 

With f as defined in (8), this implies f (x∗ , s∗ ) ≤ λn f (˜x, s˜). When ζ goes to infinity, then λ approaches 1, making clear that (˜x, y˜ , s˜) converges to the analytic center of the optimal set.  References [1] A.J. Goldman, A.W. Tucker, Theory of linear programming, in: Linear Inequalities and Related Systems, in: Ann. of Math. Studies, vol. 38, Princeton University Press, Princeton, NJ, 1956, pp. 53–97. [2] Masakazu Kojima, Nimrod Megiddo, Shinji Mizuno, A primal–dual infeasibleinterior-point algorithm for linear programming, Math. Program. 61 (3) (1993) 263–280. [3] Irvin J. Lustig, Feasibility issues in a primal–dual interior-point method for linear programming, Math. Program. 49 (2) (1990/91) 145–162. [4] Shinji Mizuno, Polynomiality of infeasible-interior-point algorithms for linear programming, Math. Program. 67 (1) (1994) 109–119. [5] Florian A. Potra, An infeasible-interior-point predictor–corrector algorithm for linear programming, SIAM J. Optim. 6 (1) (1996) 19–32. [6] C. Roos, A full-Newton step O(n) infeasible interior-point algorithm for linear optimization, SIAM J. Optim. 16 (4) (2006) 1110–1136 (electronic). [7] Cornelis Roos, Tamás Terlaky, Jean-Philippe Vial, Interior point methods for linear optimization, in: Theory and Algorithms for Linear Optimization, second edition, Springer, New York, 2006. [8] Josef Stoer, Martin Wechs, On the analyticity properties of infeasible-interiorpoint paths for monotone linear complementarity problems, Numer. Math. 81 (4) (1999) 631–645. [9] Yin Zhang, On the convergence of a class of infeasible interior-point methods for the horizontal linear complementarity problem, SIAM J. Optim. 4 (1) (1994) 208–227.