- Email: [email protected]
Convergent conic linear programming relaxations for cone convex polynomial programs
Accepted Manuscript Convergent conic linear programming relaxations for cone convex polynomial programs T.D. Chuong, V. Jeyakumar PII: DOI: Reference:
S0167-6377(16)30134-1 http://dx.doi.org/10.1016/j.orl.2017.03.003 OPERES 6206
To appear in:
Operations Research Letters
Received date: 17 October 2016 Revised date: 6 March 2017 Accepted date: 6 March 2017 Please cite this article as: T.D. Chuong, V. Jeyakumar, Convergent conic linear programming relaxations for cone convex polynomial programs, Operations Research Letters (2017), http://dx.doi.org/10.1016/j.orl.2017.03.003 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Convergent Conic Linear Programming Relaxations for Cone Convex Polynomial Programs T. D. Chuong∗ and V. Jeyakumar† Revised Version: March 6, 2017
Abstract In this paper we show that a hierarchy of conic linear programming relaxations of a cone-convex polynomial programming problem converges asymptotically under a mild well-posedness condition which can easily be checked numerically for polynomials. We also establish that an additional qualification condition guarantees finite convergence of the hierarchy. Consequently, we derive convergent semi-definite programming relaxations for convex matrix polynomial programs as well as easily tractable conic linear programming relaxations for a class of pth-order cone convex polynomial programs. Key words. Cone-convex polynomial program, conic linear programming relaxation, convergent relaxation, semidefinite programming. AMS subject classifications. 49K99, 65K10, 90C29, 90C46.
1
Introduction
Consider the cone-convex polynomial program: inf {f (x) | G(x) ∈ −K},
x∈Rn
(P)
where f : Rn → R is a convex polynomial, K ⊂ Rm is a closed convex cone with the vertex at the origin, and G : Rn → Rm is a K-convex polynomial in the sense that λG(x) + (1 − λ)G(y) − G(λx + (1 − λ)y) ∈ K for all x, y ∈ Rn , λ ∈ [0, 1], G := (G1 , . . . , Gm ) with Gi , i = 1, . . . , m, being polynomials on Rn . The model problem of the form (P) covers a broad range of convex programming problems, including the standard convex programs with inequality constraints [3, 10], convex semidefinite programs [7, 21] and pth-order cone programs [1, 3]. The problems of the form (P) frequently appear in robust optimization [2]. It is known, for example (cf. [2, Theorem 6.3.2]), that the robust counterpart of a convex quadratic program with a second-order cone constraint under norm-bounded uncertainty can be rewritten as a conic convex quadratic optimization problem with the positive semi-definite cone. Also, many basic control problems, such as static output feedback design problems, are modeled as conic polynomial optimization problems in terms of polynomial matrix inequalities [5, 17]. Recently, an exact conic linear programming relaxation has been established in [6] for a subclass of cone-convex polynomial programs of the form (P) where the map G is K-SOS-convex polynomial. That study has provided a unified treatment for the semidefinite programming approximation scheme of convex polynomial programs as it covers corresponding results for problems, such as matrix SOS-convex polynomial programs [15] and SOS-convex polynomial programs with inequality constraints [8, 9]. It has been derived by first developing a sum of squares polynomial representation of positivity of an SOS-convex polynomial over a conic SOS-convex inequality system with the help of a separation theorem for convex sets under a qualification condition. In this paper we develop a new conic linear programming relaxation scheme for the cone-convex polynomial program (P) and establish its convergence. We define a hierarchy of conic linear programming relaxation problems in terms of a so-called truncated quadratic module and the dual cone of K, where the quadratic module involves only the objective function. It results in sum of squares relaxation problems for the problem (P) where the multiplier associated with the constraint is a constant vector. Consequently, we obtain corresponding convergent relaxations for ∗ School of Mathematics and Statistics, University of New South Wales, Sydney NSW 2052, Australia; email: [email protected]. † School of Mathematics and Statistics, University of New South Wales, Sydney NSW 2052, Australia; email: [email protected].
1
2
t. d. chuong and v. jeyakumar
convex matrix polynomial programs, polynomial pth-order cone convex programs and standard convex polynomial programs with inequality constraints (i.e., K := Rm + ). We establish the convergence of the hierarchy by employing the Putinar’s Positivstellensatz [18] together with the Hahn-Banach strong separation theorem. A convergent hierarchy of semidefinite programming (SDP) relaxations has already been given in [10, Theorem 2.1] for the special case of (P), where K := Rm + and G := (G1 , . . . , Gm ). However, the multipliers associated with the constraints Gi , i = 1, 2, . . . , m, of those relaxation problems are sum of squares polynomials. This results in higher degree sum of squares relaxation problems which then induce semidefinite programming relaxations of size that is often large for the present status of SDP solvers. Our result involves the truncated quadratic module (see its definition in (2.4)), where the multipliers associated with the constraints Gi , i = 1, 2, . . . , m, are constants rather than sum of squares polynomials as in [10]. As a result, the present scheme has the potential to simplify the computation of the resulting semidefinite programming relaxation problems compared to the corresponding ones arising from the approach of [10]. The outline of the paper is as follows. We first show, in Section 2, that the optimal values of the conic linear programming relaxations converge asymptotically to the optimal value of the cone-convex polynomial program (P) under a mild well-posedness assumption in the sense that the feasible set of the problem (P) is nonempty and the objective function f is coercive. Consequently, we obtain corresponding results for convex matrix polynomial programs, polynomial pth-order cone convex programs, and convex polynomial programs with inequality constraints. We then show, in Section 3, that an additional qualification condition guarantees finite convergence of the hierarchy. We provide examples to illustrate how our relaxation schemes can be used to find the optimal value of the coneconvex polynomial program (P) by using the Matlab toolboxes such as CVX [4] or YALMIP [12, 13].
2
Asymptotic Convergence of Conic Linear Programming Relaxations
In this section, we establish an asymptotic convergence of a sequence of the conic linear programming relaxations for the cone-convex polynomial program (P). Let us start with the following qualification condition. Well-posedness. We say that the problem (P) is well-posed if the feasible set of (P) is nonempty and the objective polynomial f is coercive, i.e., lim inf f (x) = +∞. ||x||→∞
The first lemma provides a necessary and sufficient optimality criterion for the problem (P) without any constraint qualification. In what follows, we use the dual cone of K ⊂ Rm given by K ∗ := {y ∈ Rm | hy, ki ≥ 0 for all k ∈ K}.
Lemma 2.1. (Asymptotic multiplier characterization of optimality) Let x ˆ ∈ Rn be a feasible point of the well-posed problem (P). Then, x ˆ is an optimal solution of problem (P) if and only if for any > 0, there exists λ ∈ K ∗ such that f (x) + hλ, G(x)i − f (ˆ x) + > 0,
∀x ∈ Rn .
(2.1)
Proof. [=⇒] Assume that x ˆ ∈ Rn is an optimal solution of (P). Let > 0 and let C := {x ∈ Rn | G(x) ∈ −K}. As the problem (P) is well-posed, i.e., f is coercive on Rn , which implies that the convex set is closed.
Ω := {(r, y) ∈ R1+m | ∃x ∈ Rn , f (x) ≤ r, y ∈ G(x) + K}
e := Ω + {( − f (ˆ Then, the convex set Ω x), 0)} is closed as well. Since x ˆ is an optimal solution of (P), we e The strong separation theorem (see, e.g., [14, Theorem 2.2]) guarantees that there exists assert that (0, 0) ∈ / Ω. 0 6= (λ0 , λ) ∈ R × Rm such that n o inf λ0 (r + − f (ˆ x)) + hλ, yi | (r, y) ∈ Ω > 0. This ensures that λ0 ≥ 0 and λ ∈ K ∗ . Moreover, we assert that there exists δ0 > 0 such that λ0 (f (x) − f (ˆ x) + ) + hλ, G(x)i > δ0 , ∀x ∈ Rn
(2.2)
due to (f (x), G(x)) ∈ Ω for each x ∈ R . If λ0 = 0, then we assert by (2.2) that hλ, G(ˆ x)i > δ0 > 0. This contradicts the fact that x ˆ is a feasible point of problem (P), and thus, hλ, G(ˆ x)i ≤ 0. So, we can assume without loss of generality that λ0 = 1 and hence (2.1) holds. n
[⇐=] Let > 0. Assume that there exists λ ∈ K ∗ such that (2.1) holds. Let x ˜ ∈ Rn be an arbitrary feasible point of problem (P). It follows that hλ, G(˜ x)i ≤ 0 due to λ ∈ K ∗ and G(˜ x) ∈ −K. Then, f (˜ x) + hλ, G(˜ x)i ≤ f (˜ x), which together with (2.1) entails that f (˜ x) > f (ˆ x) − .
3
t. d. chuong and v. jeyakumar
Since > 0 was arbitrarily chosen, we conclude that f (ˆ x) ≤ f (˜ x). Consequently, x ˆ is an optimal solution of problem (P), which completes the proof. 2 Denote by R[x] the ring of real polynomials in x := (x1 , . . . , xn ). The polynomial f ∈ R[x] is a sum of squares r P fj2 . The set of all sum
polynomial (see, e.g., [11]) if there exist polynomials fj ∈ R[x], j = 1, . . . , r such that f =
j=1
of squares polynomials on Rn is denoted Σn , while the set of all sum of squares polynomials on Rn with degree at most d is denoted Σn,d . Given polynomials {g1 , . . . , gr } ⊂ R[x], the notation M(g1 , . . . , gr ) stands for the set of polynomials generated by {g1 , . . . , gr }, i.e., M(g1 , . . . , gr ) := {σ0 + σ1 g1 + . . . + σr gr | σj ∈ Σn , j = 0, 1, . . . , r}.
(2.3)
The set M(g1 , . . . , gr ) is archimedean if there exists h ∈ M(g1 , . . . , gr ) such that the set {x ∈ Rn | h(x) ≥ 0} is compact. Lemma 2.2. (Putinar’s Positivstellensatz [18]) Let f, gj ∈ R[x], j = 1, . . . , r. Suppose that M(g1 , . . . , gr ) is archimedean. If f (x) > 0 for all x ∈ {y ∈ Rn | gj (y) ≥ 0, j = 1, . . . , r}, then f ∈ M(g1 , . . . , gr ), i.e., there r P σj gj . exist σj ∈ Σn , j = 0, 1, . . . , r, such that f = σ0 + j=1
Let τ ∈ R be such that τ ≥ f (ˆ x), where x ˆ ∈ Rn is a feasible point of problem (P). Given k ∈ N, we define the truncated quadratic module Mk generated by the polynomial τ − f as Mk := {σ0 + σ1 (τ − f ) |σl ∈ Σn , l = 0, 1, deg(σ0 ) ≤ k, deg(σ1 f ) ≤ k, deg(Gi ) ≤ k, i = 1, . . . , m}.
(2.4)
Conic linear programming relaxation problems. We examine a family of conic linear programming relaxation problems for the cone-convex polynomial program (P). For each k ∈ N, let us consider the conic linear programming relaxation problem of (P): n o sup t |f + hλ, Gi − t ∈ Mk , λ ∈ K ∗ , (Pk ) t∈R,λ∈Rm
where Mk is given by (2.4). The first theorem shows that if the cone-convex polynomial program (P) has an optimal solution, then the optimal values of the conic linear programming relaxation problems (Pk ) (k ∈ N) converge to the optimal value of the problem (P) when the degree bound k tends to infinity. Theorem 2.3. (Asymptotic convergence of relaxations) Let x ¯ ∈ Rn be an optimal solution of the well-posed problem (P). Then, we have lim fk∗ = f (¯ x), k→∞
where fk∗ :=
sup t∈R,λ∈Rm
n o t |f + hλ, Gi − t ∈ Mk , λ ∈ K ∗ , k ∈ N.
Proof. We first prove that ∗ fk∗ ≤ fk+1 ≤ · · · ≤ f (¯ x) for all k ∈ N.
(2.5)
∗ Since it is obvious by definition that fk∗ ≤ fk+1 for all k ∈ N, we only need to verify that
fk∗ ≤ f (¯ x) for all k ∈ N.
(2.6)
To see this, let us consider any k ∈ N. If the feasible set of problem (Pk ) is empty, then fk∗ = −∞, and in this case, (2.6) holds trivially. Now, let (t, λ) ∈ R × Rm be a feasible point of problem (Pk ). Then, there exist σl ∈ Σn , l = 0, 1 such that f + hλ, Gi − t = σ0 + σ1 (τ − f ) with deg(σ0 ) ≤ k, deg(σ1 f ) ≤ k, deg(Gi ) ≤ k, i = 1, . . . , m. We rewrite (2.7) as the following form 1 + σ1 (x) f (x) = σ0 (x) − hλ, G(x)i + t + σ1 (x)f (¯ x) + σ1 (x) τ − f (¯ x) for all x ∈ Rn .
(2.7)
(2.8)
4
t. d. chuong and v. jeyakumar
Note here that τ ≥ f (ˆ x) ≥ f (¯ x), and it holds that hλ, G(¯ x)i ≤ 0 as x ¯ is a feasible point of problem (P). Keeping in mind the nonnegativity of sum of squares polynomials, estimating (2.8) at x ¯, we obtain that 1 + σ1 (¯ x) f (¯ x) ≥ t + σ1 (¯ x)f (¯ x), or equivalently, f (¯ x) ≥ t. It entails that fk∗ ≤ f (¯ x), which concludes that (2.6) is valid, and so is (2.5).
Now, let > 0. Since x ¯ is an optimal solution of problem (P), we invoke Lemma 2.1 to assert that there exists λ ∈ K ∗ such that f (x) + hλ, G(x)i − f (¯ x) + > 0,
(2.9)
∀x ∈ Rn .
Consider the function h : R → R given by h(x) := f (x) + hλ, G(x)i − f (¯ x) + for x ∈ R . We see that h is a convex polynomial and h(x) > 0 for all x ∈ Rn by virtue of (2.9). n
n
˜ := {x ∈ Rn | f (x) ≤ τ } is compact, Note further that the coercivity of f on Rn guarantees that the sublevel set Ω ˜ 6= ∅ due to τ ≥ f (ˆ ˜ Furthermore, since τ − f ∈ M(τ − f ) and and obviously, Ω x) ≥ f (¯ x) and hence being x ¯ ∈ Ω. ˜ = {x ∈ Rn | τ − f (x) ≥ 0}, we conclude that the quadratic module M(τ − f ) is archimedean. As h is positive Ω ˜ we claim by the Putinar’s Positivstellensatz (cf. Lemma 2.2) that there on Rn and in particular, h is positive on Ω, exist σ0 , σ1 ∈ Σn such that h = σ0 + σ1 (τ − f ).
It follows that there exists k ∈ N such that
f + hλ, Gi − f (¯ x) + = σ0 + σ1 (τ − f ) ∈ Mk , which shows that f + hλ, Gi − t ∈ Mk , where t := f (¯ x) − . Hence, (t , λ) is a feasible point of problem (Pk ), which gives us that fk∗ ≥ t = f (¯ x) − . Since > 0 was chosen arbitrarily, we arrive at lim inf fk∗ ≥ f (¯ x), which together with (2.5) confirms that lim fk∗ = f (¯ x). The proof of the theorem is complete.
k→∞
k→∞
2
p Matrix polynomial programs. Let S+ ⊂ S p denote the cone of p × p positive semidefinite matrices, where S p stands for the space of symmetric p × p matrices. We now consider a cone-convex polynomial program with matrix inequality constraints as p inf {f (x) | G(x) ∈ −S+ },
x∈Rn
(MP)
p where f : Rn → R is a convex polynomial and G : Rn → S p is an S+ -convex polynomial.
Recall that for A, B ∈ S p , the inner product is given by hA, Bi := Tr(AB), where Tr(·) refers to the trace operation. In particular, Tr(λG)(x) := Tr(λG(x)) for all x ∈ Rn and all λ ∈ S p . In this case, we obtain an asymptotic convergence of semidefinite linear relaxations for the matrix polynomial program (MP). Corollary 2.4. (Asymptotic convergence of SDP relaxations for (MP)) Let x ¯ ∈ Rn be an optimal solution of the well-posed problem (MP). Then, we have lim fk∗ = f (¯ x),
k→∞
where fk∗ :=
sup t∈R,λ∈S p
o n p t |f + Tr(λG) − t ∈ Mk , λ ∈ S+ , k ∈ N.
, we see that the spaces S p and Rm have the same dimensions, and there exists an Proof. Setting m := p(p+1) 2 invertible linear map L : S p → Rm such that L(A)> L(B) = Tr(AB) for all A, B ∈ S p . So, we can identify S p equipped with the trace inner product as Rm with the Euclidean inner product by associating each symmetric matrix A ∈ S p to L(A) ∈ Rm . Now, it is clear that the problem (MP) can be viewed in the form p p of problem (P) with K := S+ . In this setting, it is true that K ∗ = S+ , and thus, the desired result is followed by Theorem 2.3. 2
5
t. d. chuong and v. jeyakumar Polynomial pth-order cone programs. Let Kp := {y := (y1 , . . . , ym ) ∈ Rm | y1 ≥
s p
m P
i=2
|yi |p } be the pth-order
cone with p > 1, and let f : Rn → R be a convex polynomial. We now consider a pth-order cone convex polynomial program with quadratic constraints as inf {f (x) | G(x) ∈ −Kp },
(POP)
x∈Rn
where G := (G1 , . . . , Gm ) is a quadratic vector-valued function, i.e., Gi : Rn → R, i = 1, . . . , m are quadratic functions given by Gj (x) :=
1 > x Ai x + a> i x + bi , i = 1, . . . , m 2
with bi ∈ R, ai ∈ Rn , Ai ∈ S n , i = 1, . . . , m, where S n stands for the set of symmetric (n × n) matrices as above. Recall that, for a given A ∈ S n , the smallest eigenvalue (resp., largest eigenvalue) of A is defined by λmin (A) := p > > 2 inf xx>Ax (resp., λmax (A) := sup xx>Ax ) and the matrix 2-norm of A is given by ||A||2 := sup ||Ax|| λmax (A> A). ||x||2 = x x
x6=0
x6=0
x6=0
Below, we use q ∈ R to stand for the conjugate number of p, i.e., it satisfies
1 p
+
1 q
= 1.
In this setting, we arrive at an asymptotic convergence of conic linear programming relaxations for the pthorder cone convex polynomial program (POP), where the relaxation problems can be equivalently reformulated as tractable conic problems involving semidefinite linear constraints and norm constraints. Corollary 2.5. (Asymptotic convergence of tractable conic linear programming relaxations) Let x ¯ ∈ Rn s m P be an optimal solution of the well-posed problem (POP). Assume that λmin (A1 ) ≥ p ||Ai ||p2 (cf. [6]). Then, we i=2
have
lim fk∗ = f (¯ x),
k→∞
where fk∗ :=
sup t∈R,(λ1 ,...,λm )∈Rm
t|f+
m X i=1
v um uX q λi Gi − t ∈ Mk , λ1 ≥ t |λi |q , k ∈ N. i=2
Proof. Observe first that the dual cone of Kp , denoted by Kp∗ , is the cone Kq = {λ := (λ1 , . . . , λm ) ∈ R where q ∈ R satisfies
1 p
+
1 q
m
v um uX q | λ1 ≥ t |λi |q }, i=2
= 1. Let us take any λ := (λ1 , . . . , λm ) ∈ Kp∗ . Then, hλ := hλ, Gi =
m P
λi Gi is a
i=1
quadratic function on R . Similar to the proof of [6, Corollary 3.2], we obtain that z [∇ hλ (x)]z ≥ 0 for every x, z ∈ Rn . It shows that hλ is convex. Therefore, G is convex with respect to the cone Kp . Now, we see that the problem (POP) can be viewed in the form of problem (P) with K := Kp . So, the desired result is followed by Theorem 2.3. 2 >
n
2
Convex polynomial programs. Let us now consider a convex polynomial program of the form: inf {f (x) | Gi (x) ≤ 0, i = 1, . . . , m},
x∈Rn
(PO)
where f : Rn → R and Gi : Rn → R, i = 1, . . . , m are convex polynomials. In this framework, we come to an asymptotic convergence of semidefinite linear relaxations for the convex polynomial program (PO). Corollary 2.6. (Asymptotic convergence of SDP relaxations) Let x ¯ ∈ Rn be an optimal solution of the well-posed problem (PO). Then, we have lim fk∗ = f (¯ x), k→∞
where fk∗ :=
sup t∈R,λ∈Rm
m n o X t |f + λi Gi − t ∈ Mk , λ := (λ1 , . . . , λm ) ∈ Rm + , k ∈ N. i=1
6
t. d. chuong and v. jeyakumar
Proof. It is evident that the problem (PO) can be viewed in the form of problem (P) with K := Rm + . In this setting, it is true that K ∗ = Rm 2 + , and thus, the desired result follows from Theorem 2.3. Remark 2.7. In passing, we note that an asymptotic convergence result that is similar to Corollary 2.6 was given in [10, Theorem 2.1] without the well-posedness of the problem (PO) under a different approach. Indeed, the proof of [10, Theorem 2.1] used an orthogonal matrix transform that allows us to turn a convex polynomial that is bounded below into a lower dimensional coercive polynomial. Thus, [10, Theorem 2.1] did not require the wellposedness. However, the multipliers associated with the constraints Gi , i = 1, 2, . . . , m, of those relaxation problems are sum of squares polynomials. This results in higher degree sum of squares relaxation problems which then induce semidefinite programming relaxations of size that is often large for the present status of SDP solvers. Our result involves the truncated quadratic module defined in (2.4), where the multipliers associated with the constraints Gi , i = 1, 2, . . . , m, are constants rather than sum of squares polynomials as in [10, Theorem 2.1]. As a result, our approach, within the well-posed setting, has the potential to simplify the computation of the resulting semidefinite programming relaxation problems compared to the corresponding ones arising from [10, Theorem 2.1]. We now provide a simple example, which illustrates the asymptotic convergence of optimal values of conic linear programming relaxation problem (Pk ) (k ∈ N) to the optimal value of the cone-convex polynomial program (P).
Example 2.8. (Asymptotic convergence) Consider a cone-convex polynomial problem of the form min f (x) := x4 + x − 2 | G(x) ∈ −K2 , x∈R
(EP1)
where the vector polynomial G := (G1 , G2 ) is given by G1 (x) := x4 + x2 , G2 (x) := x2 , x ∈ R, and the cone K2 is given by K2 := {y := (y1 , y2 ) ∈ R2 | y1 ≥ |y2 |} ⊂ R2 . In this setting, the dual cone K2∗ = K2 . Take any λ := (λ1 , λ2 ) ∈ K2∗ . It means that λ1 ≥ |λ2 |, which follows that λ1 ≥ 0 and λ1 +λ2 ≥ 0. Then, it guarantees that hλ := hλ, Gi is a convex function due to hλ (x) = λ1 x4 +(λ1 +λ2 )x2 for x ∈ R. So, G is K2 -convex polynomial. It can be checked that x ¯ := 0 is an optimal solution of problem (EP1) with the corresponding optimal value f (¯ x) = −2. Moreover, the problem (EP1) is well-posed. For each k ∈ N, we consider the conic relaxation problem of (EP1) given by sup
t∈R,(λ1 ,λ2 )∈R2
2 n o X t |f + λi Gi − t ∈ Mk , λ1 ≥ |λ2 | ,
(EP1k )
i=1
where Mk := {σ0 + σ1 (τ − f ) | σl ∈ Σ1 , l = 0, 1, deg(σ0 ) ≤ k, deg(σ1 f ) ≤ k, deg(Gi ) ≤ k, i = 1, 2} with τ ≥ −2 = 2 P f (¯ x). Note that the relation f + λi Gi − t ∈ Mk means that there exist σl ∈ Σ1 , l = 0, 1 such that i=1
f − σ1 (τ − f ) +
2 X i=1
λi Gi − t = σ0 ,
(2.10)
where deg(σ0 ) ≤ k, deg(σ1 f ) ≤ k, deg(Gi ) ≤ k, i = 1, 2. Let us test our problem for the case of τ := −1 and k := 4. In this case, it follows that deg(σl ) ≤ 4 for l = 0, 1, and therefore, we can find symmetric (3 × 3) matrices Bl such σl = X >Bl X and Bl 0, where X := (1, x, x2 ) that B1l B2l B3l (see, e.g., [16, Lemma 3.33]) for l = 0, 1. Letting Bl := B2l B4l B5l , l = 0, 1, we derive from (2.10) that B3l B5l B6l
B10 = −2 − B11 − t, B20 = B61 = 0.
1+B11 1 2 , B2
= 0, 2B30 + B40 = λ1 + λ2 , 2B31 + B41 = 0, B50 = B51 = 0, B60 = 1 + B11 + λ1 and
Putting B11 := b11 , B3l := bl3 ∈ R, l = 0, 1, we see that the problem (EP1k ) with k := 4 becomes the following semi-definite program 1+b11 −2 − b11 − t b03 n 2 1+b11 sup t | 0, λ1 + λ2 − 2b03 0 2 0 1 b3 0 1 + b1 + λ1 ! b11 0 b13 λ1 0 0 1 0 −2b3 0 0, 0 λ1 − λ2 0 0, (EP1-SDP) 0 0 λ1 + λ2 b13 0 0 o t ∈ R, λi ∈ R, i = 1, 2, b11 ∈ R, bl3 ∈ R, l = 0, 1 .
7
t. d. chuong and v. jeyakumar Using the Matlab toolbox CVX (see, e.g., [4]), we solve the problem (EP1-SDP). The solver returns an approximate optimal value −2.00017 for the problem (EP1).
In fact, in this setting, the convergence of optimal values of the relaxation problems (EP1k ) (k ∈ N) to the optimal value of the primal problem (EP1) is asymptotic in the sense that there do not exist k ∈ N and tk ∈ R such that fk∗ = tk = f (¯ x). Assume on the contrary that there exist k ∈ N and λ1 , λ2 ∈ R with λ1 ≥ |λ2 | such that f+
2 X i=1
λi Gi − f (¯ x) ∈ Mk ,
which means that there exist σl ∈ Σ1 , l = 0, 1 with deg(σ0 ) ≤ k, deg(σ1 f ) ≤ k, deg(Gi ) ≤ k, i = 1, 2 such that f − σ1 (τ − f ) +
2 X i=1
λi Gi − f (¯ x) = σ0 ,
where τ ≥ −2. By rearranging this, we obtain that 1+λ1 +σ1 (x) x4 −σ1 (x)(τ +2)+(1+σ1 (x))x−λ1 +λ2 = σ0 (x) ≥ 0 for each x ∈ R. It entails that, for each x ∈ R, 1 + λ1 + σ1 (x) x4 ≥ −(1 + σ1 (x))x − (λ1 + λ2 )x2 . (2.11) Considering x :=
−1 n ,
where n ∈ N, we deduce from (2.11) that −1 −1 1 + λ 1 + σ1 ≥ 1 + σ1 n − (λ1 + λ2 ) n n
(2.12)
for n large enough. Now, letting n → ∞ in (2.12), we arrive at a contradiction.
3
Finite Convergence of Conic Linear Programming Relaxations
In this section, we show that the cone-convex polynomial program (P) admits a finite convergence with attainment of the conic linear programming relaxation problems (Pk ) (k ∈ N) under a qualification condition. Here, the “finite convergence with attainment” means that there exists k¯ ∈ N such that the conic linear programming relaxation problem (Pk¯ ) has an optimal solution and the optimal values of problems (Pk ) are the same as that of problem (P) ¯ for all k ≥ k. To this end, we need the following qualification condition. KKT Conic Qualification. We say that the problem (P) satisfies the KKT Conic Qualification (KKTCQ) if there exist an optimal solution x ¯ of (P), k ∈ N and λ ∈ K ∗ such that f + hλ, Gi − f (¯ x) ∈ Mk ,
(3.13)
where Mk is the truncated quadratic module defined as in (2.4). The following proposition shows that, under the Slater condition and the positiveness of the Hessian of the objective function at the reference solution, the cone-convex polynomial program (P) satisfies the (KKTCQ). Proposition 3.1. (KKTCQ under the Slater condition) Let x ¯ ∈ Rn be an optimal solution of problem (P) such that ∇2 f (¯ x) 0, and let τ ∈ R be such that τ > f (¯ x). Assume that intK 6= ∅ and the Slater condition holds for the problem (P), i.e., there exists x ˜ ∈ Rn such that G(˜ x) ∈ −intK.
(3.14)
Then, the problem (P) satisfies the (KKTCQ). Proof. Put
X := {(r, y) ∈ R1+m | ∃x ∈ Rn , f (x) − f (¯ x) < r, y ∈ G(x) + K}.
We see that X 6= ∅ due to (f (˜ x)−f (¯ x)+, 0) ∈ X for each > 0. Moreover, we can check that X is a convex set and, as x ¯ is an optimal solution of (P), it follows that (0, 0) ∈ / X. Using a separation theorem (see, e.g., [14, Theorem 2.5]), we find (λ0 , λ) ∈ (R × Rm ) \ {0} such that n o inf λ0 r + hλ, yi | (r, y) ∈ X ≥ 0. (3.15)
8
t. d. chuong and v. jeyakumar This ensures that λ0 ≥ 0 and λ ∈ K ∗ . Let > 0. Then, due to (f (x) − f (¯ x) + , G(x)) ∈ X for each x ∈ Rn , we derive from (3.15) that λ0 f (x) − f (¯ x) + + hλ, G(x)i ≥ 0 for all x ∈ Rn .
(3.16)
Due to (3.14), it stems from (3.16) that λ0 6= 0 and therefore, there is no loss of generality in assuming that λ0 = 1. So, we obtain that f (x) − f (¯ x) + + hλ, G(x)i ≥ 0 for all x ∈ Rn . Since > 0 was arbitrarily chosen, we arrive at the conclusion that f (x) − f (¯ x) + hλ, G(x)i ≥ 0 for all x ∈ Rn . Let the function h : Rn → R be given by h(x) := f (x) + hλ, G(x)i − f (¯ x) for x ∈ Rn . We see that h is a convex n polynomial and h(x) ≥ 0 for all x ∈ R . It entails especially that h(¯ x) = hλ, G(¯ x)i ≥ 0. In addition, hλ, G(¯ x)i ≤ 0 as x ¯ is a feasible point of problem (P). So, h(¯ x) = 0 = infn h(x). x∈R
The convexity of hλ, Gi ensures that ∇2 hλ, G(¯ x)i 0. Moreover, ∇2 f (¯ x) 0 by our assumption. Then, it follows that ∇2 h(¯ x) = ∇2 f (¯ x) + ∇2 hλ, G(¯ x)i 0. So, h is strictly convex on Rn (cf. [10, Lemma 3.1]) and we conclude that x ¯ is the unique minimizer of h on Rn . Note further that the condition ∇2 f (¯ x) 0 ensures that f is coercive on Rn (cf. [10, Lemma 3.1]). Then, n ˜ ˜ 6= ∅ due to x ˜ Furthermore, since the sublevel set Ω := {x ∈ R | f (x) ≤ τ } is compact, and obviously, Ω ¯ ∈ Ω. n ˜ τ −f ∈ M(τ −f ) and Ω = {x ∈ R | τ −f (x) ≥ 0}, we conclude that the quadratic module M(τ −f ) is archimedean. Now, using the representation of non-negative polynomials having finitely many zeros (cf. [19, Example 3.18] or [20, Corollary 3.6]) applied to h, we find σ0 , σ1 ∈ Σn such that h = σ0 + σ1 (τ − f ), and thus, there exists k ∈ N such that f + hλ, Gi − f (¯ x) = σ0 + σ1 (τ − f ) ∈ Mk , i.e., (3.13) holds. So, the problem (P) satisfies the (KKTCQ).
2
For the case of the convex polynomial problem (PO), we obtain an optimality condition in terms of sum of squares representations for convex polynomials, which was given in [10, Theorem 3.2]. Corollary 3.2. (Optimality via sum of squares) Let x ¯ ∈ Rn be an optimal solution of problem (PO) such that ∇2 f (¯ x) 0, and let τ ∈ R be such that τ > f (¯ x). Assume that the Slater condition holds for the problem (PO), i.e., there exists x ˜ ∈ Rn such that Gj (˜ x) < 0, i = 1, . . . , m. Then, there exist k ∈ N and λ := (λ1 , . . . , λm ) ∈ Rm + such that f+
m X i=1
λi Gi − f (¯ x) ∈ Mk ,
where Mk is the truncated quadratic module defined as in (2.4). Proof. We see that the problem (PO) can be viewed in the form of problem (P) with K := Rm + . In this setting, 2 it is true that K ∗ = Rm + , and thus, the desired result is followed by Proposition 3.1. We are in position to present a finite convergence with attainment of the conic linear programming relaxation problems (Pk ) (k ∈ N) for the cone-convex polynomial program (P).
Theorem 3.3. (Finite convergence with attainment) Let x ¯ ∈ Rn be an optimal solution of problem (P). Assume that the problem (P) satisfies the (KKTCQ). Then, for any τ ≥ f (¯ x), there exists k¯ ∈ N such that the problem (Pk ) has an optimal solution (t¯, λ) satisfying ¯ fk∗ = t¯ = f (¯ x) for all k ≥ k,
(3.17)
where fk∗ :=
max
t∈R,λ∈Rm
n o t |f + hλ, Gi − t ∈ Mk , λ ∈ K ∗ .
Proof. Similar to the proof of Theorem 2.3, we obtain that ∗ fk∗ ≤ fk+1 ≤ · · · ≤ f (¯ x) for all k ∈ N.
(3.18)
Now, let τ ≥ f (¯ x). Since the problem (P) satisfies the (KKTCQ), there exist an optimal solution x0 , k¯ ∈ N and λ ∈ K ∗ such that f + hλ, Gi − t¯ ∈ Mk¯ ,
9
t. d. chuong and v. jeyakumar
¯ which in turn implies that f ∗ ≥ t¯ for where t¯ := f (x0 ). So, (t¯, λ) is a feasible point of problem (Pk ) for all k ≥ k, k ¯ Note further that f (x0 ) = f (¯ all k ≥ k. x) as x ¯ is an optimal solution of problem (P). Thus, due to (3.18), we arrive ¯ Therefore, we conclude that (t¯, λ) is an optimal solution of problem (Pk ) at f (¯ x) = t¯ ≤ fk∗ ≤ f (¯ x) for all k ≥ k. ¯ for all k ≥ k and (3.17) is valid. 2 Let us now employ the finite convergence with attainment result established above to derive a corresponding one for the convex polynomial problem (PO). Note further that, by this way, analogous results can be obtained for the problems (MP) and (POP). Corollary 3.4. (Finite convergence of SDP relaxations) Let x ¯ ∈ Rn be an optimal solution of problem (PO). Assume that the problem (PO) satisfies the (KKTCQ). Then, for any τ ≥ f (¯ x), there exists k¯ ∈ N such that the problem (Pk ) has an optimal solution (t¯, λ) satisfying ¯ f ∗ = t¯ = f (¯ x) for all k ≥ k, k
where
fk∗ :=
m n o X max m t |f + λi Gi − t ∈ Mk , λ := (λ1 , . . . , λm ) ∈ Rm + .
t∈R,λ∈R
i=1
Proof. It is evident that the problem (PO) can be viewed in the form of problem (P) with K := Rm + . In this setting, it is true that K ∗ = Rm 2 + , and thus, the desired result is followed by Theorem 3.3. We close this paper with a simple example, which shows how our relaxation schemes can be applied to find the optimal value of a cone-convex polynomial program. Example 3.5. Consider a convex optimization problem of the form q 4 2 4 2 4x81 + x42 . inf f (x) := x + x + 2 | 1 − 2x − x ≥ 1 2 1 2 2 x:=(x1 ,x2 )∈R
(EP2)
The problem (EP2) can be expressed in terms of problem (P) with G := (G1 , G2 , G3 ) and K := K2 , where the 4 2 2 polynomials Gi , i = 1, 2, 3, are given by G1 (x) := 2x41 + x22 − p1, G2 (x) := 2x1 , G3 (x) := x2 , x := (x1 , x2 ) ∈ R , and the cone K2 is given by K2 := {y := (y1 , y2 , y3 ) ∈ R3 | y1 ≥ y22 + y32 } ⊂ R3 . p In this framework, the dual cone K2∗ = K2 . Take any λ := (λ1 , λ2 , λ3 ) ∈ K2∗ . We see that λ1 ≥ λ22 + λ23 ≥ 4 max {|λ2 |, |λ3 |}, and thus hλ := hλ, Gi is a convex function due to hλ (x) = 2(λ1 + λ2 )x1 + (λ1 + λ3 )x22 − λ1 for x ∈ R2 . So, G is K2 -convex polynomial.
It can be checked that x ¯ := (0, 0) is an optimal solution of problem (EP2) with the corresponding optimal 0 0 2 value f (¯ x) = 2. Moreover, ∇ f (¯ x) = 0 and the Slater condition holds for this case. We assert by 0 2 Proposition 3.1 that the problem (EP2) satisfies (KKTCQ). Now, we use the relaxation problems (Pk ) (k ∈ N) to verify the optimal value f (¯ x) = 2. For each k ∈ N, we consider the relaxation problem of (EP2) as follows: 3 q n o X 2 + λ2 , max t |f + λ G − t ∈ M , λ ≥ λ (EP2k ) i i k 1 2 3 3 t∈R,(λ1 ,λ2 ,λ3 )∈R
i=1
where Mk := {σ0 + σ1 (τ − f ) | σl ∈ Σ2 , l = 0, 1, deg(σ0 ) ≤ k, deg(σ1 f ) ≤ k, deg(Gi ) ≤ k, i = 1, 2, 3} with 3 P τ > 2 = f (¯ x). Note that the relation f + λi Gi − t ∈ Mk means that there exist σl ∈ Σ2 , l = 0, 1 such that i=1
f − σ1 (τ − f ) +
3 X i=1
λi Gi − t = σ0 ,
where deg(σ0 ) ≤ k, deg(σ1 f ) ≤ k, deg(Gi ) ≤ k, i = 1, 2, 3.
Let us test our problem for the case of τ := 3 and k := 8. In this case, the problem (EP2k ) becomes a linear optimization problem with sum of squares constraints and a norm constraint as 3 q o n X (EP2-SP) max t | f − σ1 (τ − f ) + λi Gi − t ∈ Σ2,8 , λ1 ≥ λ22 + λ23 , σ1 ∈ Σ2,8 , t ∈ R, λi ∈ R, i = 1, 2, 3 . i=1
Using the Matlab toolbox YALMIP (see, e.g., [12,13]), we solve the problem (EP2-SP). The solver returns the truly optimal value 2 for the problem (EP2).
Acknowledgements. The authors are grateful to anonymous referees for valuable comments and suggestions. Research of the first author was supported by the UNSW Vice-Chancellor’s Postdoctoral Research Fellowship. Research of the second author was partially supported by a grant from the Australian Research Council.
10
t. d. chuong and v. jeyakumar
References [1] E. D. Andersen, C. Roos, T. Terlaky, Notes on duality in second order and p-order cone optimization, Optimization 51 (2002), no. 4, 627–643. [2] A. Ben-Tal, L. El Ghaoui, A. Nemirovski, Robust Optimization, Princeton Ser. Appl. Math., Princeton University Press, Princeton, NJ, 2009. [3] S. Boyd, L. Vandenberghe, Convex optimization. Cambridge University Press, Cambridge, 2004. [4] M. Grant, S. Boyd, CVX: Matlab software for disciplined convex programming, http://cvxr.com/cvx, March 2014.
version 2.1.
[5] D. Henrion, J.-B. Lasserre, Convergent relaxations of polynomial matrix inequalities and static output feedback, IEEE Trans. Automat. Control 51 (2006), no. 2, 192–202. [6] V. Jeyakumar, G. Li, Exact conic programming relaxations for a class of convex polynomial cone programs, J. Optim. Theory Appl. (2016), DOI: 10.1007/s10957-016-1023-x. [7] V. Jeyakumar, G. Li, Exact SDP relaxations for classes of nonlinear semidefinite programming problems, Oper. Res. Lett. 40 (2012), no. 6, 529–536. [8] V. Jeyakumar, G. Li, J. Vicente-Perez, Robust SOS-Convex polynomial programs: Exact SDP relaxations, Optim. Lett., 9(1) (2015), 1-18. [9] V. Jeyakumar, J. Vicente-Perez, Dual semidefinite programs without duality gaps for a class of convex minimax programs, J. Optim. Theory Appl. 162 (2014), no. 3, 735–753. [10] V. Jeyakumar, T. S. Pham, G. Li, Convergence of the Lasserre hierarchy of SDP relaxations for convex polynomial programs without compactness, Oper. Res. Lett. 42 (2014), no. 1, 34–40. [11] J. B. Lasserre, Moments, Positive Polynomials and Their Applications, Imperial College Press, London (2009). [12] J. Lofberg, Pre- and post-processing sum-of-squares programs in practice, IEEE Trans. Autom. Cont. 54, 1007–1011 (2009). [13] J. Lofberg, YALMIP: A toolbox for modeling and optimization in MATLAB. In: Proceedings of the CACSD Conference, Taipei, Taiwan (2004). [14] B. S. Mordukhovich, N. M. Nam, An easy path to convex analysis and applications, Synthesis Lectures on Mathematics and Statistics, 14. Morgan & Claypool Publishers, Williston, 2014. [15] J. Nie, Polynomial matrix inequality and semidefinite representation, Math. Oper. Res. 36 (2011), no. 3, 398– 415. [16] P. A. Parrilo, Polynomial optimization, sums of squares, and applications. Semidefinite optimization and convex algebraic geometry, 251–291, MOS-SIAM Ser. Optim., 13, SIAM, Philadelphia, PA, 2013. [17] E. Pauwels, D. Henrion, J.-P. Lasserre, Linear conic optimization for inverse optimal control, SIAM J. Control Optim. 54 (2016), no. 3, 1798–1825. [18] M. Putinar, Positive polynomials on compact semi-algebraic sets, Indiana Univ. Math. J. 42 (1993) 203–206. [19] C. Scheiderer, Sums of squares on real algebraic curves, Math. Z. 245 (2003) 725–760. [20] C. Scheiderer, Distinguished representations of non-negative polynomials, J. Algebra 289 (2005), no. 2, 558–573. [21] A. Shapiro, First and second order analysis of nonlinear semidefinite programs. Semidefinite programming, Math. Program. 77 (1997), no. 2, Ser. B, 301–320.
S0167-6377(16)30134-1 http://dx.doi.org/10.1016/j.orl.2017.03.003 OPERES 6206
To appear in:
Operations Research Letters
Received date: 17 October 2016 Revised date: 6 March 2017 Accepted date: 6 March 2017 Please cite this article as: T.D. Chuong, V. Jeyakumar, Convergent conic linear programming relaxations for cone convex polynomial programs, Operations Research Letters (2017), http://dx.doi.org/10.1016/j.orl.2017.03.003 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Convergent Conic Linear Programming Relaxations for Cone Convex Polynomial Programs T. D. Chuong∗ and V. Jeyakumar† Revised Version: March 6, 2017
Abstract In this paper we show that a hierarchy of conic linear programming relaxations of a cone-convex polynomial programming problem converges asymptotically under a mild well-posedness condition which can easily be checked numerically for polynomials. We also establish that an additional qualification condition guarantees finite convergence of the hierarchy. Consequently, we derive convergent semi-definite programming relaxations for convex matrix polynomial programs as well as easily tractable conic linear programming relaxations for a class of pth-order cone convex polynomial programs. Key words. Cone-convex polynomial program, conic linear programming relaxation, convergent relaxation, semidefinite programming. AMS subject classifications. 49K99, 65K10, 90C29, 90C46.
1
Introduction
Consider the cone-convex polynomial program: inf {f (x) | G(x) ∈ −K},
x∈Rn
(P)
where f : Rn → R is a convex polynomial, K ⊂ Rm is a closed convex cone with the vertex at the origin, and G : Rn → Rm is a K-convex polynomial in the sense that λG(x) + (1 − λ)G(y) − G(λx + (1 − λ)y) ∈ K for all x, y ∈ Rn , λ ∈ [0, 1], G := (G1 , . . . , Gm ) with Gi , i = 1, . . . , m, being polynomials on Rn . The model problem of the form (P) covers a broad range of convex programming problems, including the standard convex programs with inequality constraints [3, 10], convex semidefinite programs [7, 21] and pth-order cone programs [1, 3]. The problems of the form (P) frequently appear in robust optimization [2]. It is known, for example (cf. [2, Theorem 6.3.2]), that the robust counterpart of a convex quadratic program with a second-order cone constraint under norm-bounded uncertainty can be rewritten as a conic convex quadratic optimization problem with the positive semi-definite cone. Also, many basic control problems, such as static output feedback design problems, are modeled as conic polynomial optimization problems in terms of polynomial matrix inequalities [5, 17]. Recently, an exact conic linear programming relaxation has been established in [6] for a subclass of cone-convex polynomial programs of the form (P) where the map G is K-SOS-convex polynomial. That study has provided a unified treatment for the semidefinite programming approximation scheme of convex polynomial programs as it covers corresponding results for problems, such as matrix SOS-convex polynomial programs [15] and SOS-convex polynomial programs with inequality constraints [8, 9]. It has been derived by first developing a sum of squares polynomial representation of positivity of an SOS-convex polynomial over a conic SOS-convex inequality system with the help of a separation theorem for convex sets under a qualification condition. In this paper we develop a new conic linear programming relaxation scheme for the cone-convex polynomial program (P) and establish its convergence. We define a hierarchy of conic linear programming relaxation problems in terms of a so-called truncated quadratic module and the dual cone of K, where the quadratic module involves only the objective function. It results in sum of squares relaxation problems for the problem (P) where the multiplier associated with the constraint is a constant vector. Consequently, we obtain corresponding convergent relaxations for ∗ School of Mathematics and Statistics, University of New South Wales, Sydney NSW 2052, Australia; email: [email protected]. † School of Mathematics and Statistics, University of New South Wales, Sydney NSW 2052, Australia; email: [email protected].
1
2
t. d. chuong and v. jeyakumar
convex matrix polynomial programs, polynomial pth-order cone convex programs and standard convex polynomial programs with inequality constraints (i.e., K := Rm + ). We establish the convergence of the hierarchy by employing the Putinar’s Positivstellensatz [18] together with the Hahn-Banach strong separation theorem. A convergent hierarchy of semidefinite programming (SDP) relaxations has already been given in [10, Theorem 2.1] for the special case of (P), where K := Rm + and G := (G1 , . . . , Gm ). However, the multipliers associated with the constraints Gi , i = 1, 2, . . . , m, of those relaxation problems are sum of squares polynomials. This results in higher degree sum of squares relaxation problems which then induce semidefinite programming relaxations of size that is often large for the present status of SDP solvers. Our result involves the truncated quadratic module (see its definition in (2.4)), where the multipliers associated with the constraints Gi , i = 1, 2, . . . , m, are constants rather than sum of squares polynomials as in [10]. As a result, the present scheme has the potential to simplify the computation of the resulting semidefinite programming relaxation problems compared to the corresponding ones arising from the approach of [10]. The outline of the paper is as follows. We first show, in Section 2, that the optimal values of the conic linear programming relaxations converge asymptotically to the optimal value of the cone-convex polynomial program (P) under a mild well-posedness assumption in the sense that the feasible set of the problem (P) is nonempty and the objective function f is coercive. Consequently, we obtain corresponding results for convex matrix polynomial programs, polynomial pth-order cone convex programs, and convex polynomial programs with inequality constraints. We then show, in Section 3, that an additional qualification condition guarantees finite convergence of the hierarchy. We provide examples to illustrate how our relaxation schemes can be used to find the optimal value of the coneconvex polynomial program (P) by using the Matlab toolboxes such as CVX [4] or YALMIP [12, 13].
2
Asymptotic Convergence of Conic Linear Programming Relaxations
In this section, we establish an asymptotic convergence of a sequence of the conic linear programming relaxations for the cone-convex polynomial program (P). Let us start with the following qualification condition. Well-posedness. We say that the problem (P) is well-posed if the feasible set of (P) is nonempty and the objective polynomial f is coercive, i.e., lim inf f (x) = +∞. ||x||→∞
The first lemma provides a necessary and sufficient optimality criterion for the problem (P) without any constraint qualification. In what follows, we use the dual cone of K ⊂ Rm given by K ∗ := {y ∈ Rm | hy, ki ≥ 0 for all k ∈ K}.
Lemma 2.1. (Asymptotic multiplier characterization of optimality) Let x ˆ ∈ Rn be a feasible point of the well-posed problem (P). Then, x ˆ is an optimal solution of problem (P) if and only if for any > 0, there exists λ ∈ K ∗ such that f (x) + hλ, G(x)i − f (ˆ x) + > 0,
∀x ∈ Rn .
(2.1)
Proof. [=⇒] Assume that x ˆ ∈ Rn is an optimal solution of (P). Let > 0 and let C := {x ∈ Rn | G(x) ∈ −K}. As the problem (P) is well-posed, i.e., f is coercive on Rn , which implies that the convex set is closed.
Ω := {(r, y) ∈ R1+m | ∃x ∈ Rn , f (x) ≤ r, y ∈ G(x) + K}
e := Ω + {( − f (ˆ Then, the convex set Ω x), 0)} is closed as well. Since x ˆ is an optimal solution of (P), we e The strong separation theorem (see, e.g., [14, Theorem 2.2]) guarantees that there exists assert that (0, 0) ∈ / Ω. 0 6= (λ0 , λ) ∈ R × Rm such that n o inf λ0 (r + − f (ˆ x)) + hλ, yi | (r, y) ∈ Ω > 0. This ensures that λ0 ≥ 0 and λ ∈ K ∗ . Moreover, we assert that there exists δ0 > 0 such that λ0 (f (x) − f (ˆ x) + ) + hλ, G(x)i > δ0 , ∀x ∈ Rn
(2.2)
due to (f (x), G(x)) ∈ Ω for each x ∈ R . If λ0 = 0, then we assert by (2.2) that hλ, G(ˆ x)i > δ0 > 0. This contradicts the fact that x ˆ is a feasible point of problem (P), and thus, hλ, G(ˆ x)i ≤ 0. So, we can assume without loss of generality that λ0 = 1 and hence (2.1) holds. n
[⇐=] Let > 0. Assume that there exists λ ∈ K ∗ such that (2.1) holds. Let x ˜ ∈ Rn be an arbitrary feasible point of problem (P). It follows that hλ, G(˜ x)i ≤ 0 due to λ ∈ K ∗ and G(˜ x) ∈ −K. Then, f (˜ x) + hλ, G(˜ x)i ≤ f (˜ x), which together with (2.1) entails that f (˜ x) > f (ˆ x) − .
3
t. d. chuong and v. jeyakumar
Since > 0 was arbitrarily chosen, we conclude that f (ˆ x) ≤ f (˜ x). Consequently, x ˆ is an optimal solution of problem (P), which completes the proof. 2 Denote by R[x] the ring of real polynomials in x := (x1 , . . . , xn ). The polynomial f ∈ R[x] is a sum of squares r P fj2 . The set of all sum
polynomial (see, e.g., [11]) if there exist polynomials fj ∈ R[x], j = 1, . . . , r such that f =
j=1
of squares polynomials on Rn is denoted Σn , while the set of all sum of squares polynomials on Rn with degree at most d is denoted Σn,d . Given polynomials {g1 , . . . , gr } ⊂ R[x], the notation M(g1 , . . . , gr ) stands for the set of polynomials generated by {g1 , . . . , gr }, i.e., M(g1 , . . . , gr ) := {σ0 + σ1 g1 + . . . + σr gr | σj ∈ Σn , j = 0, 1, . . . , r}.
(2.3)
The set M(g1 , . . . , gr ) is archimedean if there exists h ∈ M(g1 , . . . , gr ) such that the set {x ∈ Rn | h(x) ≥ 0} is compact. Lemma 2.2. (Putinar’s Positivstellensatz [18]) Let f, gj ∈ R[x], j = 1, . . . , r. Suppose that M(g1 , . . . , gr ) is archimedean. If f (x) > 0 for all x ∈ {y ∈ Rn | gj (y) ≥ 0, j = 1, . . . , r}, then f ∈ M(g1 , . . . , gr ), i.e., there r P σj gj . exist σj ∈ Σn , j = 0, 1, . . . , r, such that f = σ0 + j=1
Let τ ∈ R be such that τ ≥ f (ˆ x), where x ˆ ∈ Rn is a feasible point of problem (P). Given k ∈ N, we define the truncated quadratic module Mk generated by the polynomial τ − f as Mk := {σ0 + σ1 (τ − f ) |σl ∈ Σn , l = 0, 1, deg(σ0 ) ≤ k, deg(σ1 f ) ≤ k, deg(Gi ) ≤ k, i = 1, . . . , m}.
(2.4)
Conic linear programming relaxation problems. We examine a family of conic linear programming relaxation problems for the cone-convex polynomial program (P). For each k ∈ N, let us consider the conic linear programming relaxation problem of (P): n o sup t |f + hλ, Gi − t ∈ Mk , λ ∈ K ∗ , (Pk ) t∈R,λ∈Rm
where Mk is given by (2.4). The first theorem shows that if the cone-convex polynomial program (P) has an optimal solution, then the optimal values of the conic linear programming relaxation problems (Pk ) (k ∈ N) converge to the optimal value of the problem (P) when the degree bound k tends to infinity. Theorem 2.3. (Asymptotic convergence of relaxations) Let x ¯ ∈ Rn be an optimal solution of the well-posed problem (P). Then, we have lim fk∗ = f (¯ x), k→∞
where fk∗ :=
sup t∈R,λ∈Rm
n o t |f + hλ, Gi − t ∈ Mk , λ ∈ K ∗ , k ∈ N.
Proof. We first prove that ∗ fk∗ ≤ fk+1 ≤ · · · ≤ f (¯ x) for all k ∈ N.
(2.5)
∗ Since it is obvious by definition that fk∗ ≤ fk+1 for all k ∈ N, we only need to verify that
fk∗ ≤ f (¯ x) for all k ∈ N.
(2.6)
To see this, let us consider any k ∈ N. If the feasible set of problem (Pk ) is empty, then fk∗ = −∞, and in this case, (2.6) holds trivially. Now, let (t, λ) ∈ R × Rm be a feasible point of problem (Pk ). Then, there exist σl ∈ Σn , l = 0, 1 such that f + hλ, Gi − t = σ0 + σ1 (τ − f ) with deg(σ0 ) ≤ k, deg(σ1 f ) ≤ k, deg(Gi ) ≤ k, i = 1, . . . , m. We rewrite (2.7) as the following form 1 + σ1 (x) f (x) = σ0 (x) − hλ, G(x)i + t + σ1 (x)f (¯ x) + σ1 (x) τ − f (¯ x) for all x ∈ Rn .
(2.7)
(2.8)
4
t. d. chuong and v. jeyakumar
Note here that τ ≥ f (ˆ x) ≥ f (¯ x), and it holds that hλ, G(¯ x)i ≤ 0 as x ¯ is a feasible point of problem (P). Keeping in mind the nonnegativity of sum of squares polynomials, estimating (2.8) at x ¯, we obtain that 1 + σ1 (¯ x) f (¯ x) ≥ t + σ1 (¯ x)f (¯ x), or equivalently, f (¯ x) ≥ t. It entails that fk∗ ≤ f (¯ x), which concludes that (2.6) is valid, and so is (2.5).
Now, let > 0. Since x ¯ is an optimal solution of problem (P), we invoke Lemma 2.1 to assert that there exists λ ∈ K ∗ such that f (x) + hλ, G(x)i − f (¯ x) + > 0,
(2.9)
∀x ∈ Rn .
Consider the function h : R → R given by h(x) := f (x) + hλ, G(x)i − f (¯ x) + for x ∈ R . We see that h is a convex polynomial and h(x) > 0 for all x ∈ Rn by virtue of (2.9). n
n
˜ := {x ∈ Rn | f (x) ≤ τ } is compact, Note further that the coercivity of f on Rn guarantees that the sublevel set Ω ˜ 6= ∅ due to τ ≥ f (ˆ ˜ Furthermore, since τ − f ∈ M(τ − f ) and and obviously, Ω x) ≥ f (¯ x) and hence being x ¯ ∈ Ω. ˜ = {x ∈ Rn | τ − f (x) ≥ 0}, we conclude that the quadratic module M(τ − f ) is archimedean. As h is positive Ω ˜ we claim by the Putinar’s Positivstellensatz (cf. Lemma 2.2) that there on Rn and in particular, h is positive on Ω, exist σ0 , σ1 ∈ Σn such that h = σ0 + σ1 (τ − f ).
It follows that there exists k ∈ N such that
f + hλ, Gi − f (¯ x) + = σ0 + σ1 (τ − f ) ∈ Mk , which shows that f + hλ, Gi − t ∈ Mk , where t := f (¯ x) − . Hence, (t , λ) is a feasible point of problem (Pk ), which gives us that fk∗ ≥ t = f (¯ x) − . Since > 0 was chosen arbitrarily, we arrive at lim inf fk∗ ≥ f (¯ x), which together with (2.5) confirms that lim fk∗ = f (¯ x). The proof of the theorem is complete.
k→∞
k→∞
2
p Matrix polynomial programs. Let S+ ⊂ S p denote the cone of p × p positive semidefinite matrices, where S p stands for the space of symmetric p × p matrices. We now consider a cone-convex polynomial program with matrix inequality constraints as p inf {f (x) | G(x) ∈ −S+ },
x∈Rn
(MP)
p where f : Rn → R is a convex polynomial and G : Rn → S p is an S+ -convex polynomial.
Recall that for A, B ∈ S p , the inner product is given by hA, Bi := Tr(AB), where Tr(·) refers to the trace operation. In particular, Tr(λG)(x) := Tr(λG(x)) for all x ∈ Rn and all λ ∈ S p . In this case, we obtain an asymptotic convergence of semidefinite linear relaxations for the matrix polynomial program (MP). Corollary 2.4. (Asymptotic convergence of SDP relaxations for (MP)) Let x ¯ ∈ Rn be an optimal solution of the well-posed problem (MP). Then, we have lim fk∗ = f (¯ x),
k→∞
where fk∗ :=
sup t∈R,λ∈S p
o n p t |f + Tr(λG) − t ∈ Mk , λ ∈ S+ , k ∈ N.
, we see that the spaces S p and Rm have the same dimensions, and there exists an Proof. Setting m := p(p+1) 2 invertible linear map L : S p → Rm such that L(A)> L(B) = Tr(AB) for all A, B ∈ S p . So, we can identify S p equipped with the trace inner product as Rm with the Euclidean inner product by associating each symmetric matrix A ∈ S p to L(A) ∈ Rm . Now, it is clear that the problem (MP) can be viewed in the form p p of problem (P) with K := S+ . In this setting, it is true that K ∗ = S+ , and thus, the desired result is followed by Theorem 2.3. 2
5
t. d. chuong and v. jeyakumar Polynomial pth-order cone programs. Let Kp := {y := (y1 , . . . , ym ) ∈ Rm | y1 ≥
s p
m P
i=2
|yi |p } be the pth-order
cone with p > 1, and let f : Rn → R be a convex polynomial. We now consider a pth-order cone convex polynomial program with quadratic constraints as inf {f (x) | G(x) ∈ −Kp },
(POP)
x∈Rn
where G := (G1 , . . . , Gm ) is a quadratic vector-valued function, i.e., Gi : Rn → R, i = 1, . . . , m are quadratic functions given by Gj (x) :=
1 > x Ai x + a> i x + bi , i = 1, . . . , m 2
with bi ∈ R, ai ∈ Rn , Ai ∈ S n , i = 1, . . . , m, where S n stands for the set of symmetric (n × n) matrices as above. Recall that, for a given A ∈ S n , the smallest eigenvalue (resp., largest eigenvalue) of A is defined by λmin (A) := p > > 2 inf xx>Ax (resp., λmax (A) := sup xx>Ax ) and the matrix 2-norm of A is given by ||A||2 := sup ||Ax|| λmax (A> A). ||x||2 = x x
x6=0
x6=0
x6=0
Below, we use q ∈ R to stand for the conjugate number of p, i.e., it satisfies
1 p
+
1 q
= 1.
In this setting, we arrive at an asymptotic convergence of conic linear programming relaxations for the pthorder cone convex polynomial program (POP), where the relaxation problems can be equivalently reformulated as tractable conic problems involving semidefinite linear constraints and norm constraints. Corollary 2.5. (Asymptotic convergence of tractable conic linear programming relaxations) Let x ¯ ∈ Rn s m P be an optimal solution of the well-posed problem (POP). Assume that λmin (A1 ) ≥ p ||Ai ||p2 (cf. [6]). Then, we i=2
have
lim fk∗ = f (¯ x),
k→∞
where fk∗ :=
sup t∈R,(λ1 ,...,λm )∈Rm
t|f+
m X i=1
v um uX q λi Gi − t ∈ Mk , λ1 ≥ t |λi |q , k ∈ N. i=2
Proof. Observe first that the dual cone of Kp , denoted by Kp∗ , is the cone Kq = {λ := (λ1 , . . . , λm ) ∈ R where q ∈ R satisfies
1 p
+
1 q
m
v um uX q | λ1 ≥ t |λi |q }, i=2
= 1. Let us take any λ := (λ1 , . . . , λm ) ∈ Kp∗ . Then, hλ := hλ, Gi =
m P
λi Gi is a
i=1
quadratic function on R . Similar to the proof of [6, Corollary 3.2], we obtain that z [∇ hλ (x)]z ≥ 0 for every x, z ∈ Rn . It shows that hλ is convex. Therefore, G is convex with respect to the cone Kp . Now, we see that the problem (POP) can be viewed in the form of problem (P) with K := Kp . So, the desired result is followed by Theorem 2.3. 2 >
n
2
Convex polynomial programs. Let us now consider a convex polynomial program of the form: inf {f (x) | Gi (x) ≤ 0, i = 1, . . . , m},
x∈Rn
(PO)
where f : Rn → R and Gi : Rn → R, i = 1, . . . , m are convex polynomials. In this framework, we come to an asymptotic convergence of semidefinite linear relaxations for the convex polynomial program (PO). Corollary 2.6. (Asymptotic convergence of SDP relaxations) Let x ¯ ∈ Rn be an optimal solution of the well-posed problem (PO). Then, we have lim fk∗ = f (¯ x), k→∞
where fk∗ :=
sup t∈R,λ∈Rm
m n o X t |f + λi Gi − t ∈ Mk , λ := (λ1 , . . . , λm ) ∈ Rm + , k ∈ N. i=1
6
t. d. chuong and v. jeyakumar
Proof. It is evident that the problem (PO) can be viewed in the form of problem (P) with K := Rm + . In this setting, it is true that K ∗ = Rm 2 + , and thus, the desired result follows from Theorem 2.3. Remark 2.7. In passing, we note that an asymptotic convergence result that is similar to Corollary 2.6 was given in [10, Theorem 2.1] without the well-posedness of the problem (PO) under a different approach. Indeed, the proof of [10, Theorem 2.1] used an orthogonal matrix transform that allows us to turn a convex polynomial that is bounded below into a lower dimensional coercive polynomial. Thus, [10, Theorem 2.1] did not require the wellposedness. However, the multipliers associated with the constraints Gi , i = 1, 2, . . . , m, of those relaxation problems are sum of squares polynomials. This results in higher degree sum of squares relaxation problems which then induce semidefinite programming relaxations of size that is often large for the present status of SDP solvers. Our result involves the truncated quadratic module defined in (2.4), where the multipliers associated with the constraints Gi , i = 1, 2, . . . , m, are constants rather than sum of squares polynomials as in [10, Theorem 2.1]. As a result, our approach, within the well-posed setting, has the potential to simplify the computation of the resulting semidefinite programming relaxation problems compared to the corresponding ones arising from [10, Theorem 2.1]. We now provide a simple example, which illustrates the asymptotic convergence of optimal values of conic linear programming relaxation problem (Pk ) (k ∈ N) to the optimal value of the cone-convex polynomial program (P).
Example 2.8. (Asymptotic convergence) Consider a cone-convex polynomial problem of the form min f (x) := x4 + x − 2 | G(x) ∈ −K2 , x∈R
(EP1)
where the vector polynomial G := (G1 , G2 ) is given by G1 (x) := x4 + x2 , G2 (x) := x2 , x ∈ R, and the cone K2 is given by K2 := {y := (y1 , y2 ) ∈ R2 | y1 ≥ |y2 |} ⊂ R2 . In this setting, the dual cone K2∗ = K2 . Take any λ := (λ1 , λ2 ) ∈ K2∗ . It means that λ1 ≥ |λ2 |, which follows that λ1 ≥ 0 and λ1 +λ2 ≥ 0. Then, it guarantees that hλ := hλ, Gi is a convex function due to hλ (x) = λ1 x4 +(λ1 +λ2 )x2 for x ∈ R. So, G is K2 -convex polynomial. It can be checked that x ¯ := 0 is an optimal solution of problem (EP1) with the corresponding optimal value f (¯ x) = −2. Moreover, the problem (EP1) is well-posed. For each k ∈ N, we consider the conic relaxation problem of (EP1) given by sup
t∈R,(λ1 ,λ2 )∈R2
2 n o X t |f + λi Gi − t ∈ Mk , λ1 ≥ |λ2 | ,
(EP1k )
i=1
where Mk := {σ0 + σ1 (τ − f ) | σl ∈ Σ1 , l = 0, 1, deg(σ0 ) ≤ k, deg(σ1 f ) ≤ k, deg(Gi ) ≤ k, i = 1, 2} with τ ≥ −2 = 2 P f (¯ x). Note that the relation f + λi Gi − t ∈ Mk means that there exist σl ∈ Σ1 , l = 0, 1 such that i=1
f − σ1 (τ − f ) +
2 X i=1
λi Gi − t = σ0 ,
(2.10)
where deg(σ0 ) ≤ k, deg(σ1 f ) ≤ k, deg(Gi ) ≤ k, i = 1, 2. Let us test our problem for the case of τ := −1 and k := 4. In this case, it follows that deg(σl ) ≤ 4 for l = 0, 1, and therefore, we can find symmetric (3 × 3) matrices Bl such σl = X >Bl X and Bl 0, where X := (1, x, x2 ) that B1l B2l B3l (see, e.g., [16, Lemma 3.33]) for l = 0, 1. Letting Bl := B2l B4l B5l , l = 0, 1, we derive from (2.10) that B3l B5l B6l
B10 = −2 − B11 − t, B20 = B61 = 0.
1+B11 1 2 , B2
= 0, 2B30 + B40 = λ1 + λ2 , 2B31 + B41 = 0, B50 = B51 = 0, B60 = 1 + B11 + λ1 and
Putting B11 := b11 , B3l := bl3 ∈ R, l = 0, 1, we see that the problem (EP1k ) with k := 4 becomes the following semi-definite program 1+b11 −2 − b11 − t b03 n 2 1+b11 sup t | 0, λ1 + λ2 − 2b03 0 2 0 1 b3 0 1 + b1 + λ1 ! b11 0 b13 λ1 0 0 1 0 −2b3 0 0, 0 λ1 − λ2 0 0, (EP1-SDP) 0 0 λ1 + λ2 b13 0 0 o t ∈ R, λi ∈ R, i = 1, 2, b11 ∈ R, bl3 ∈ R, l = 0, 1 .
7
t. d. chuong and v. jeyakumar Using the Matlab toolbox CVX (see, e.g., [4]), we solve the problem (EP1-SDP). The solver returns an approximate optimal value −2.00017 for the problem (EP1).
In fact, in this setting, the convergence of optimal values of the relaxation problems (EP1k ) (k ∈ N) to the optimal value of the primal problem (EP1) is asymptotic in the sense that there do not exist k ∈ N and tk ∈ R such that fk∗ = tk = f (¯ x). Assume on the contrary that there exist k ∈ N and λ1 , λ2 ∈ R with λ1 ≥ |λ2 | such that f+
2 X i=1
λi Gi − f (¯ x) ∈ Mk ,
which means that there exist σl ∈ Σ1 , l = 0, 1 with deg(σ0 ) ≤ k, deg(σ1 f ) ≤ k, deg(Gi ) ≤ k, i = 1, 2 such that f − σ1 (τ − f ) +
2 X i=1
λi Gi − f (¯ x) = σ0 ,
where τ ≥ −2. By rearranging this, we obtain that 1+λ1 +σ1 (x) x4 −σ1 (x)(τ +2)+(1+σ1 (x))x−λ1 +λ2 = σ0 (x) ≥ 0 for each x ∈ R. It entails that, for each x ∈ R, 1 + λ1 + σ1 (x) x4 ≥ −(1 + σ1 (x))x − (λ1 + λ2 )x2 . (2.11) Considering x :=
−1 n ,
where n ∈ N, we deduce from (2.11) that −1 −1 1 + λ 1 + σ1 ≥ 1 + σ1 n − (λ1 + λ2 ) n n
(2.12)
for n large enough. Now, letting n → ∞ in (2.12), we arrive at a contradiction.
3
Finite Convergence of Conic Linear Programming Relaxations
In this section, we show that the cone-convex polynomial program (P) admits a finite convergence with attainment of the conic linear programming relaxation problems (Pk ) (k ∈ N) under a qualification condition. Here, the “finite convergence with attainment” means that there exists k¯ ∈ N such that the conic linear programming relaxation problem (Pk¯ ) has an optimal solution and the optimal values of problems (Pk ) are the same as that of problem (P) ¯ for all k ≥ k. To this end, we need the following qualification condition. KKT Conic Qualification. We say that the problem (P) satisfies the KKT Conic Qualification (KKTCQ) if there exist an optimal solution x ¯ of (P), k ∈ N and λ ∈ K ∗ such that f + hλ, Gi − f (¯ x) ∈ Mk ,
(3.13)
where Mk is the truncated quadratic module defined as in (2.4). The following proposition shows that, under the Slater condition and the positiveness of the Hessian of the objective function at the reference solution, the cone-convex polynomial program (P) satisfies the (KKTCQ). Proposition 3.1. (KKTCQ under the Slater condition) Let x ¯ ∈ Rn be an optimal solution of problem (P) such that ∇2 f (¯ x) 0, and let τ ∈ R be such that τ > f (¯ x). Assume that intK 6= ∅ and the Slater condition holds for the problem (P), i.e., there exists x ˜ ∈ Rn such that G(˜ x) ∈ −intK.
(3.14)
Then, the problem (P) satisfies the (KKTCQ). Proof. Put
X := {(r, y) ∈ R1+m | ∃x ∈ Rn , f (x) − f (¯ x) < r, y ∈ G(x) + K}.
We see that X 6= ∅ due to (f (˜ x)−f (¯ x)+, 0) ∈ X for each > 0. Moreover, we can check that X is a convex set and, as x ¯ is an optimal solution of (P), it follows that (0, 0) ∈ / X. Using a separation theorem (see, e.g., [14, Theorem 2.5]), we find (λ0 , λ) ∈ (R × Rm ) \ {0} such that n o inf λ0 r + hλ, yi | (r, y) ∈ X ≥ 0. (3.15)
8
t. d. chuong and v. jeyakumar This ensures that λ0 ≥ 0 and λ ∈ K ∗ . Let > 0. Then, due to (f (x) − f (¯ x) + , G(x)) ∈ X for each x ∈ Rn , we derive from (3.15) that λ0 f (x) − f (¯ x) + + hλ, G(x)i ≥ 0 for all x ∈ Rn .
(3.16)
Due to (3.14), it stems from (3.16) that λ0 6= 0 and therefore, there is no loss of generality in assuming that λ0 = 1. So, we obtain that f (x) − f (¯ x) + + hλ, G(x)i ≥ 0 for all x ∈ Rn . Since > 0 was arbitrarily chosen, we arrive at the conclusion that f (x) − f (¯ x) + hλ, G(x)i ≥ 0 for all x ∈ Rn . Let the function h : Rn → R be given by h(x) := f (x) + hλ, G(x)i − f (¯ x) for x ∈ Rn . We see that h is a convex n polynomial and h(x) ≥ 0 for all x ∈ R . It entails especially that h(¯ x) = hλ, G(¯ x)i ≥ 0. In addition, hλ, G(¯ x)i ≤ 0 as x ¯ is a feasible point of problem (P). So, h(¯ x) = 0 = infn h(x). x∈R
The convexity of hλ, Gi ensures that ∇2 hλ, G(¯ x)i 0. Moreover, ∇2 f (¯ x) 0 by our assumption. Then, it follows that ∇2 h(¯ x) = ∇2 f (¯ x) + ∇2 hλ, G(¯ x)i 0. So, h is strictly convex on Rn (cf. [10, Lemma 3.1]) and we conclude that x ¯ is the unique minimizer of h on Rn . Note further that the condition ∇2 f (¯ x) 0 ensures that f is coercive on Rn (cf. [10, Lemma 3.1]). Then, n ˜ ˜ 6= ∅ due to x ˜ Furthermore, since the sublevel set Ω := {x ∈ R | f (x) ≤ τ } is compact, and obviously, Ω ¯ ∈ Ω. n ˜ τ −f ∈ M(τ −f ) and Ω = {x ∈ R | τ −f (x) ≥ 0}, we conclude that the quadratic module M(τ −f ) is archimedean. Now, using the representation of non-negative polynomials having finitely many zeros (cf. [19, Example 3.18] or [20, Corollary 3.6]) applied to h, we find σ0 , σ1 ∈ Σn such that h = σ0 + σ1 (τ − f ), and thus, there exists k ∈ N such that f + hλ, Gi − f (¯ x) = σ0 + σ1 (τ − f ) ∈ Mk , i.e., (3.13) holds. So, the problem (P) satisfies the (KKTCQ).
2
For the case of the convex polynomial problem (PO), we obtain an optimality condition in terms of sum of squares representations for convex polynomials, which was given in [10, Theorem 3.2]. Corollary 3.2. (Optimality via sum of squares) Let x ¯ ∈ Rn be an optimal solution of problem (PO) such that ∇2 f (¯ x) 0, and let τ ∈ R be such that τ > f (¯ x). Assume that the Slater condition holds for the problem (PO), i.e., there exists x ˜ ∈ Rn such that Gj (˜ x) < 0, i = 1, . . . , m. Then, there exist k ∈ N and λ := (λ1 , . . . , λm ) ∈ Rm + such that f+
m X i=1
λi Gi − f (¯ x) ∈ Mk ,
where Mk is the truncated quadratic module defined as in (2.4). Proof. We see that the problem (PO) can be viewed in the form of problem (P) with K := Rm + . In this setting, 2 it is true that K ∗ = Rm + , and thus, the desired result is followed by Proposition 3.1. We are in position to present a finite convergence with attainment of the conic linear programming relaxation problems (Pk ) (k ∈ N) for the cone-convex polynomial program (P).
Theorem 3.3. (Finite convergence with attainment) Let x ¯ ∈ Rn be an optimal solution of problem (P). Assume that the problem (P) satisfies the (KKTCQ). Then, for any τ ≥ f (¯ x), there exists k¯ ∈ N such that the problem (Pk ) has an optimal solution (t¯, λ) satisfying ¯ fk∗ = t¯ = f (¯ x) for all k ≥ k,
(3.17)
where fk∗ :=
max
t∈R,λ∈Rm
n o t |f + hλ, Gi − t ∈ Mk , λ ∈ K ∗ .
Proof. Similar to the proof of Theorem 2.3, we obtain that ∗ fk∗ ≤ fk+1 ≤ · · · ≤ f (¯ x) for all k ∈ N.
(3.18)
Now, let τ ≥ f (¯ x). Since the problem (P) satisfies the (KKTCQ), there exist an optimal solution x0 , k¯ ∈ N and λ ∈ K ∗ such that f + hλ, Gi − t¯ ∈ Mk¯ ,
9
t. d. chuong and v. jeyakumar
¯ which in turn implies that f ∗ ≥ t¯ for where t¯ := f (x0 ). So, (t¯, λ) is a feasible point of problem (Pk ) for all k ≥ k, k ¯ Note further that f (x0 ) = f (¯ all k ≥ k. x) as x ¯ is an optimal solution of problem (P). Thus, due to (3.18), we arrive ¯ Therefore, we conclude that (t¯, λ) is an optimal solution of problem (Pk ) at f (¯ x) = t¯ ≤ fk∗ ≤ f (¯ x) for all k ≥ k. ¯ for all k ≥ k and (3.17) is valid. 2 Let us now employ the finite convergence with attainment result established above to derive a corresponding one for the convex polynomial problem (PO). Note further that, by this way, analogous results can be obtained for the problems (MP) and (POP). Corollary 3.4. (Finite convergence of SDP relaxations) Let x ¯ ∈ Rn be an optimal solution of problem (PO). Assume that the problem (PO) satisfies the (KKTCQ). Then, for any τ ≥ f (¯ x), there exists k¯ ∈ N such that the problem (Pk ) has an optimal solution (t¯, λ) satisfying ¯ f ∗ = t¯ = f (¯ x) for all k ≥ k, k
where
fk∗ :=
m n o X max m t |f + λi Gi − t ∈ Mk , λ := (λ1 , . . . , λm ) ∈ Rm + .
t∈R,λ∈R
i=1
Proof. It is evident that the problem (PO) can be viewed in the form of problem (P) with K := Rm + . In this setting, it is true that K ∗ = Rm 2 + , and thus, the desired result is followed by Theorem 3.3. We close this paper with a simple example, which shows how our relaxation schemes can be applied to find the optimal value of a cone-convex polynomial program. Example 3.5. Consider a convex optimization problem of the form q 4 2 4 2 4x81 + x42 . inf f (x) := x + x + 2 | 1 − 2x − x ≥ 1 2 1 2 2 x:=(x1 ,x2 )∈R
(EP2)
The problem (EP2) can be expressed in terms of problem (P) with G := (G1 , G2 , G3 ) and K := K2 , where the 4 2 2 polynomials Gi , i = 1, 2, 3, are given by G1 (x) := 2x41 + x22 − p1, G2 (x) := 2x1 , G3 (x) := x2 , x := (x1 , x2 ) ∈ R , and the cone K2 is given by K2 := {y := (y1 , y2 , y3 ) ∈ R3 | y1 ≥ y22 + y32 } ⊂ R3 . p In this framework, the dual cone K2∗ = K2 . Take any λ := (λ1 , λ2 , λ3 ) ∈ K2∗ . We see that λ1 ≥ λ22 + λ23 ≥ 4 max {|λ2 |, |λ3 |}, and thus hλ := hλ, Gi is a convex function due to hλ (x) = 2(λ1 + λ2 )x1 + (λ1 + λ3 )x22 − λ1 for x ∈ R2 . So, G is K2 -convex polynomial.
It can be checked that x ¯ := (0, 0) is an optimal solution of problem (EP2) with the corresponding optimal 0 0 2 value f (¯ x) = 2. Moreover, ∇ f (¯ x) = 0 and the Slater condition holds for this case. We assert by 0 2 Proposition 3.1 that the problem (EP2) satisfies (KKTCQ). Now, we use the relaxation problems (Pk ) (k ∈ N) to verify the optimal value f (¯ x) = 2. For each k ∈ N, we consider the relaxation problem of (EP2) as follows: 3 q n o X 2 + λ2 , max t |f + λ G − t ∈ M , λ ≥ λ (EP2k ) i i k 1 2 3 3 t∈R,(λ1 ,λ2 ,λ3 )∈R
i=1
where Mk := {σ0 + σ1 (τ − f ) | σl ∈ Σ2 , l = 0, 1, deg(σ0 ) ≤ k, deg(σ1 f ) ≤ k, deg(Gi ) ≤ k, i = 1, 2, 3} with 3 P τ > 2 = f (¯ x). Note that the relation f + λi Gi − t ∈ Mk means that there exist σl ∈ Σ2 , l = 0, 1 such that i=1
f − σ1 (τ − f ) +
3 X i=1
λi Gi − t = σ0 ,
where deg(σ0 ) ≤ k, deg(σ1 f ) ≤ k, deg(Gi ) ≤ k, i = 1, 2, 3.
Let us test our problem for the case of τ := 3 and k := 8. In this case, the problem (EP2k ) becomes a linear optimization problem with sum of squares constraints and a norm constraint as 3 q o n X (EP2-SP) max t | f − σ1 (τ − f ) + λi Gi − t ∈ Σ2,8 , λ1 ≥ λ22 + λ23 , σ1 ∈ Σ2,8 , t ∈ R, λi ∈ R, i = 1, 2, 3 . i=1
Using the Matlab toolbox YALMIP (see, e.g., [12,13]), we solve the problem (EP2-SP). The solver returns the truly optimal value 2 for the problem (EP2).
Acknowledgements. The authors are grateful to anonymous referees for valuable comments and suggestions. Research of the first author was supported by the UNSW Vice-Chancellor’s Postdoctoral Research Fellowship. Research of the second author was partially supported by a grant from the Australian Research Council.
10
t. d. chuong and v. jeyakumar
References [1] E. D. Andersen, C. Roos, T. Terlaky, Notes on duality in second order and p-order cone optimization, Optimization 51 (2002), no. 4, 627–643. [2] A. Ben-Tal, L. El Ghaoui, A. Nemirovski, Robust Optimization, Princeton Ser. Appl. Math., Princeton University Press, Princeton, NJ, 2009. [3] S. Boyd, L. Vandenberghe, Convex optimization. Cambridge University Press, Cambridge, 2004. [4] M. Grant, S. Boyd, CVX: Matlab software for disciplined convex programming, http://cvxr.com/cvx, March 2014.
version 2.1.
[5] D. Henrion, J.-B. Lasserre, Convergent relaxations of polynomial matrix inequalities and static output feedback, IEEE Trans. Automat. Control 51 (2006), no. 2, 192–202. [6] V. Jeyakumar, G. Li, Exact conic programming relaxations for a class of convex polynomial cone programs, J. Optim. Theory Appl. (2016), DOI: 10.1007/s10957-016-1023-x. [7] V. Jeyakumar, G. Li, Exact SDP relaxations for classes of nonlinear semidefinite programming problems, Oper. Res. Lett. 40 (2012), no. 6, 529–536. [8] V. Jeyakumar, G. Li, J. Vicente-Perez, Robust SOS-Convex polynomial programs: Exact SDP relaxations, Optim. Lett., 9(1) (2015), 1-18. [9] V. Jeyakumar, J. Vicente-Perez, Dual semidefinite programs without duality gaps for a class of convex minimax programs, J. Optim. Theory Appl. 162 (2014), no. 3, 735–753. [10] V. Jeyakumar, T. S. Pham, G. Li, Convergence of the Lasserre hierarchy of SDP relaxations for convex polynomial programs without compactness, Oper. Res. Lett. 42 (2014), no. 1, 34–40. [11] J. B. Lasserre, Moments, Positive Polynomials and Their Applications, Imperial College Press, London (2009). [12] J. Lofberg, Pre- and post-processing sum-of-squares programs in practice, IEEE Trans. Autom. Cont. 54, 1007–1011 (2009). [13] J. Lofberg, YALMIP: A toolbox for modeling and optimization in MATLAB. In: Proceedings of the CACSD Conference, Taipei, Taiwan (2004). [14] B. S. Mordukhovich, N. M. Nam, An easy path to convex analysis and applications, Synthesis Lectures on Mathematics and Statistics, 14. Morgan & Claypool Publishers, Williston, 2014. [15] J. Nie, Polynomial matrix inequality and semidefinite representation, Math. Oper. Res. 36 (2011), no. 3, 398– 415. [16] P. A. Parrilo, Polynomial optimization, sums of squares, and applications. Semidefinite optimization and convex algebraic geometry, 251–291, MOS-SIAM Ser. Optim., 13, SIAM, Philadelphia, PA, 2013. [17] E. Pauwels, D. Henrion, J.-P. Lasserre, Linear conic optimization for inverse optimal control, SIAM J. Control Optim. 54 (2016), no. 3, 1798–1825. [18] M. Putinar, Positive polynomials on compact semi-algebraic sets, Indiana Univ. Math. J. 42 (1993) 203–206. [19] C. Scheiderer, Sums of squares on real algebraic curves, Math. Z. 245 (2003) 725–760. [20] C. Scheiderer, Distinguished representations of non-negative polynomials, J. Algebra 289 (2005), no. 2, 558–573. [21] A. Shapiro, First and second order analysis of nonlinear semidefinite programs. Semidefinite programming, Math. Program. 77 (1997), no. 2, Ser. B, 301–320.