- Email: [email protected]
Checking local optimality in constrained quadratic programming is NP-hard
Volume 7, Number 1
OPERATIONS RESEARCH LETYERS
C H E C K I N G L O C A L O F F I M A L F I ' Y IN C O N S ~ I N E D P R O G R A M M I N G IS N P - H A R D *
February 1988
QUADRATIC
P.M. PARDALOS and G. SCHNITGER Computer Science Department, The Pennsylvania State University, 333 Whitmore l.$'brary, University Parle, PA 16802, USA Received June 1987 Revised November 1987
In this paper we prove that the problem of checking local optimality for a feasible point and the problem of checking if a local minimum is strict, are NP-hard problems. As a consequence the problem of checking whether a function is locally strictly convex is also NP-hard. indefinite quadratic programming • NP-hard
1. Introduction We are concerned here with non-convex programming problems of the form minf(x), X
s.t.
given local 'minimum is also strict. The 3-satisfiability problem is the following: Given a setof Boolean variables xl,..., x,, and given S, a Boolean expression in conjunctive normal form with exactly 3 literals per clause, S ffi ( Z l l -I- Z12 -I- Z13)(Z21 -I- Z22 -I- Z 2 3 ) . . .
Ax~b,
x>0,
where .f(x) is an indefinite quadratic function. Problems of this type appear in many applications [7] and in general are very difficult to solve [3]. Here we obtain insight in the intractability of non-convex Programming by proving-that the problem of checking local optimality for a given feasible point is NP-hard. Moreover, checking if a g/~en local minimum is strict is also NP-hard. This also proves that the problem of checking whether a quadratic function is locally strictly convex is NP-hard. Our results solve also problems 1 and 6 proposed by Murty in [1]. Related results for nonconvex programming problems were obtained in [6] and [8].
× +z.2 + z.3), where each literal Zk~ is either some variable x~ or its negation xi, is there a truth assignment for the variables x~ which makes S TRUE? Cook [2] has shown that 3-satisfiability is NPcomplete. For each instance of 3-satisfiability we construct an instance of an optimization problem in the real variables Xo, xl,..., x,. For each clause in $ we associate a linear inequality. For example for a clause of the form xl + x2 + x3 we write an inequality of the form xl + x2 + (1 - x3) + Xo > 3/2.
2. Checking strict local optimality is NP-hard
Inequalities of that form can be written as a system of linear inequalities,
In this section the 3-satisfiability, problem is reduced to the problem of checking whether a
A~x > 3/2 + c,
* Supported by the National Science Foundation under Contract no. DCR-8407256 and by the Office of Naval Research under Contract no. N0014-80-0517.
where As is a sparse matrix with entries in [0, 1, -1]. Let us consider the set D($) of feasible points satisfying the following linear con-
016%6377/88/$3.$0 0 1988, Elsevier Science Publ~hers B.V. (North-Holland)
33
Volume 7, Number I
OlaF.RATIONSRESEARCHLETTERS
Yo, 1/2 + Yo}, i - 1,..., n. Then the variables xi, i = 1,..., n defined by
straints: A~x ~ 3 / 2 + c, 1/2 - Xo < x~ < 1/2 + Xo, x~>0, iffi0, 1 , . . . , n .
i = 1,..., n,
With a given instance of the 3-satisfiabifity problem we associate the follo~'Lng indefinite quadratic problem: /I
x e D ( S)
/ix) = - E (x,- 0/2i- I
Xo))
x(x,- 0/2 + Xo)). Note that/(x)ffi - E ~ ' . l ( x , - 1/2) 2 + nx 2, i.e., the obJective function is a separable indefinite quadratic function with one convex and n concave variables. l t m a r k 1. (a) f ( x ) > 0 for all feasible points x. Therefore, the feasible point x * ffi (0, 1/2,... ,1/2) is a local (global) minimum o f / ( x ) s i n c e / ( x * ) = 0. (b) f ( x ) - 0 if and only if x i ~ { 1 / 2 - x o , 1/2 + xo}, for i f f i l , . . . , n . A strict local minimum for the above quadratic problem, is a feasible point x* for which there exists an ( • 0 such that f(x*)
forall
February 1988
xGD(S)C~{x:O<
IIx-x*ll <(}.
The following theorem implies that checking strict locad optimality is NP-hard. Therefore, we cannot expect to find a polynomial time algorithm for this problem (assuming P ~ NP). Waeorem 1. S is satisfiable iff x * ffi (0, 1/2,... ,1/2) is not a strict local minimum.
x~(y)
_[0 1
if if
y,-1/2-yo, y~ffil/2+yo
satisfy ~q. Remark Z (a) A continuously differentiable func~'.~.on is said to be locally convex in some small region if its Hessian matrix is positive semidefinite in that region, and locally strictly convex if the Hessian is positive definite in that region. Second order conditions for local minima (see, for example, [4] or [5]) require that the function be locally strictly convex. As a corollary of the above theorem we have that the problem of checking whether a function is locally strictly convex in some region is NP-hard. (b) Fix X o - 1/2 in the above indefinite quadratic problem. Then the objective function f ( x ) is concave with x* as the global minimum. Consider the problem of determining whether a given point is a strict global minimum. Our previous arguments imply that this problem is also NP-hard.
3. Checking local optimality is NiP.hard Consider now the problem of checking local optimality. We prove that this probler~ is NP-hard. Given the 3-satisfiability problem, consider the following separable indefinite quadratic program (over the same feasible domain D($)): n
rain # ( x ) - - ~ ( x , - ( 1 / 2 - Xo))
xGD($)
i-1
x(x,- (1/2 + Xo))
Proof. Let x l , . . . , xn be a truth assignment satisfying S. For any Xo and i - 1,..., n consider
1/2-x 0 x°-1/2+Xo
ff if
xi-0, xlffil.
For x ° - (Xo, x°,..., x°), we have that f ( x °) = 0. Since Xo can be chosen to be arbitrarily close to zero, x* is not a strict local minimum. Suppose now that x* ffi (0, 1/2,..., 1/2) is not a strict local minimum~ that is, there exists y such
that f ( y ) f f ( x * ) = O ; 34
therefore, y~E { 1 / 2 -
i-1
Theorem 2. S is satisfiable iff x* - (0, 1/2,... ,1/2) is not a local minimum.
Proof. Let x l , . . . , x, be a truth assignment satisfying S. Given any Xo .~bitrarily close to zero, define for i-- 1,..., n 1/2-x o x°ffi 1 / 2 + x o
if if
x iffiO, xiffil.
Volume 7, Number I
OPERATIONS RESEARCH LETTERS
Then we can easily see that x°ffi (Xo, x°,..., x °) is feasible and f(x0) _
- Tx02 <0ffif(x*).
Hence, x* is not a local minimum. Suppose now that x* is not a local minimum. Then there exists x ffi ( x l , . . . , xn) such that f ( x ) < 0. We will now show, by contradiction, that we can find in each clause of S one literal of value > 1/2. This implies that S is satisfiable. For instance, consider a constraint (clause) of the form x~ + x2 + x3 + Xo > 3/2. Then there is one literal I of value at least 1 / 2 Xo/3. We only consider the case i - xl. The other cases follow by an analogous argument. We know by assumption that xl < 1/2. Let p ( x ) - F ? . l ( x i - ( 1 / 2 - X o ) ) ( x l - ( 1 / 2 + Xo) be the 'penalty term' in the objective function. Then p ( x ) > - ( x l - (1/2 + xo))(xl - (1/2 - x0) > (2/3)x02. On the other hand, for the 'payoff term' q ( x ) - ( 1 / 2 n ) ~ n . . l ( x i - 1/2) 2 we obtain q(x) > - x 2 / 2 . Hence f ( x ) > (2/3)x02 - (1/2)Xo2 > 0 ~ contradiction. L'3 Remark 3. If the objective function is concave, then its local (global) minima occur at the vertices of the feasible domain. Let v b e a given vertex: We can compute its adjacent vertices v~,..., Vm in polynomial tin~e, It is enough to check local optimality of v over t h e simplex generated b y
February 1988
v, v 1, ..., v,,. This in turn can be checked by considering the m one-dimensional problems of minimizin~ the function over the edges of the simplex. Hence, we can verify local optimality for concave programming in polynomi~ time. Observe that our objective function (in both theorems) contains only one 'convex variable'.
References [1] A. Bachem and H.W. Hamacher, "Applications of combinatorial methods in mathematical programming: Abstracts and open problems", Research Report no. 85-13, Sonderforschungsbereich 303, University of Bonn, 1985. [2] S.A, Cook, "The complexity of theorem proving procedures", Proc. 3rd Ann. ACM Syrup. on Theory of Comput. ing, 151-158 (1971). [3] M.R. Garey and D.S. Johnson, Computers and Intractability: A Guide to the Theory of NP.completeness, Freeman, San Francisco, CA, 1979. [4] D.G. Luenberser, Linear and Noniinear Programming, 2nd ed., Addison-Wesley, Reading, MA. [5] G.P. McCormick, "Second-order conditions for constrained nfir~aa", SIAM 3". of Applied Mathematics 15, 641-652 (1967). K.G. Murty and S.N. Kabadi,-"Some NP-complete problems in quadratic and nonlinear programming", Technical Report 85-23, Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, MI. [7] P.M. Pardalos and J.B. Rosen, Constrained Global Optimization: Algorithms and Applications, Lecture Notes in Computer Science 268, Springer, Berlin, 1987. [8] A. Vergis, K. Steigfi'tz and B. Dickinson, "The complexity of analog computation", Math. and Computers in Simulation 28, 91-113 (1986).
35
OPERATIONS RESEARCH LETYERS
C H E C K I N G L O C A L O F F I M A L F I ' Y IN C O N S ~ I N E D P R O G R A M M I N G IS N P - H A R D *
February 1988
QUADRATIC
P.M. PARDALOS and G. SCHNITGER Computer Science Department, The Pennsylvania State University, 333 Whitmore l.$'brary, University Parle, PA 16802, USA Received June 1987 Revised November 1987
In this paper we prove that the problem of checking local optimality for a feasible point and the problem of checking if a local minimum is strict, are NP-hard problems. As a consequence the problem of checking whether a function is locally strictly convex is also NP-hard. indefinite quadratic programming • NP-hard
1. Introduction We are concerned here with non-convex programming problems of the form minf(x), X
s.t.
given local 'minimum is also strict. The 3-satisfiability problem is the following: Given a setof Boolean variables xl,..., x,, and given S, a Boolean expression in conjunctive normal form with exactly 3 literals per clause, S ffi ( Z l l -I- Z12 -I- Z13)(Z21 -I- Z22 -I- Z 2 3 ) . . .
Ax~b,
x>0,
where .f(x) is an indefinite quadratic function. Problems of this type appear in many applications [7] and in general are very difficult to solve [3]. Here we obtain insight in the intractability of non-convex Programming by proving-that the problem of checking local optimality for a given feasible point is NP-hard. Moreover, checking if a g/~en local minimum is strict is also NP-hard. This also proves that the problem of checking whether a quadratic function is locally strictly convex is NP-hard. Our results solve also problems 1 and 6 proposed by Murty in [1]. Related results for nonconvex programming problems were obtained in [6] and [8].
× +z.2 + z.3), where each literal Zk~ is either some variable x~ or its negation xi, is there a truth assignment for the variables x~ which makes S TRUE? Cook [2] has shown that 3-satisfiability is NPcomplete. For each instance of 3-satisfiability we construct an instance of an optimization problem in the real variables Xo, xl,..., x,. For each clause in $ we associate a linear inequality. For example for a clause of the form xl + x2 + x3 we write an inequality of the form xl + x2 + (1 - x3) + Xo > 3/2.
2. Checking strict local optimality is NP-hard
Inequalities of that form can be written as a system of linear inequalities,
In this section the 3-satisfiability, problem is reduced to the problem of checking whether a
A~x > 3/2 + c,
* Supported by the National Science Foundation under Contract no. DCR-8407256 and by the Office of Naval Research under Contract no. N0014-80-0517.
where As is a sparse matrix with entries in [0, 1, -1]. Let us consider the set D($) of feasible points satisfying the following linear con-
016%6377/88/$3.$0 0 1988, Elsevier Science Publ~hers B.V. (North-Holland)
33
Volume 7, Number I
OlaF.RATIONSRESEARCHLETTERS
Yo, 1/2 + Yo}, i - 1,..., n. Then the variables xi, i = 1,..., n defined by
straints: A~x ~ 3 / 2 + c, 1/2 - Xo < x~ < 1/2 + Xo, x~>0, iffi0, 1 , . . . , n .
i = 1,..., n,
With a given instance of the 3-satisfiabifity problem we associate the follo~'Lng indefinite quadratic problem: /I
x e D ( S)
/ix) = - E (x,- 0/2i- I
Xo))
x(x,- 0/2 + Xo)). Note that/(x)ffi - E ~ ' . l ( x , - 1/2) 2 + nx 2, i.e., the obJective function is a separable indefinite quadratic function with one convex and n concave variables. l t m a r k 1. (a) f ( x ) > 0 for all feasible points x. Therefore, the feasible point x * ffi (0, 1/2,... ,1/2) is a local (global) minimum o f / ( x ) s i n c e / ( x * ) = 0. (b) f ( x ) - 0 if and only if x i ~ { 1 / 2 - x o , 1/2 + xo}, for i f f i l , . . . , n . A strict local minimum for the above quadratic problem, is a feasible point x* for which there exists an ( • 0 such that f(x*)
forall
February 1988
xGD(S)C~{x:O<
IIx-x*ll <(}.
The following theorem implies that checking strict locad optimality is NP-hard. Therefore, we cannot expect to find a polynomial time algorithm for this problem (assuming P ~ NP). Waeorem 1. S is satisfiable iff x * ffi (0, 1/2,... ,1/2) is not a strict local minimum.
x~(y)
_[0 1
if if
y,-1/2-yo, y~ffil/2+yo
satisfy ~q. Remark Z (a) A continuously differentiable func~'.~.on is said to be locally convex in some small region if its Hessian matrix is positive semidefinite in that region, and locally strictly convex if the Hessian is positive definite in that region. Second order conditions for local minima (see, for example, [4] or [5]) require that the function be locally strictly convex. As a corollary of the above theorem we have that the problem of checking whether a function is locally strictly convex in some region is NP-hard. (b) Fix X o - 1/2 in the above indefinite quadratic problem. Then the objective function f ( x ) is concave with x* as the global minimum. Consider the problem of determining whether a given point is a strict global minimum. Our previous arguments imply that this problem is also NP-hard.
3. Checking local optimality is NiP.hard Consider now the problem of checking local optimality. We prove that this probler~ is NP-hard. Given the 3-satisfiability problem, consider the following separable indefinite quadratic program (over the same feasible domain D($)): n
rain # ( x ) - - ~ ( x , - ( 1 / 2 - Xo))
xGD($)
i-1
x(x,- (1/2 + Xo))
Proof. Let x l , . . . , xn be a truth assignment satisfying S. For any Xo and i - 1,..., n consider
1/2-x 0 x°-1/2+Xo
ff if
xi-0, xlffil.
For x ° - (Xo, x°,..., x°), we have that f ( x °) = 0. Since Xo can be chosen to be arbitrarily close to zero, x* is not a strict local minimum. Suppose now that x* ffi (0, 1/2,..., 1/2) is not a strict local minimum~ that is, there exists y such
that f ( y ) f f ( x * ) = O ; 34
therefore, y~E { 1 / 2 -
i-1
Theorem 2. S is satisfiable iff x* - (0, 1/2,... ,1/2) is not a local minimum.
Proof. Let x l , . . . , x, be a truth assignment satisfying S. Given any Xo .~bitrarily close to zero, define for i-- 1,..., n 1/2-x o x°ffi 1 / 2 + x o
if if
x iffiO, xiffil.
Volume 7, Number I
OPERATIONS RESEARCH LETTERS
Then we can easily see that x°ffi (Xo, x°,..., x °) is feasible and f(x0) _
- Tx02 <0ffif(x*).
Hence, x* is not a local minimum. Suppose now that x* is not a local minimum. Then there exists x ffi ( x l , . . . , xn) such that f ( x ) < 0. We will now show, by contradiction, that we can find in each clause of S one literal of value > 1/2. This implies that S is satisfiable. For instance, consider a constraint (clause) of the form x~ + x2 + x3 + Xo > 3/2. Then there is one literal I of value at least 1 / 2 Xo/3. We only consider the case i - xl. The other cases follow by an analogous argument. We know by assumption that xl < 1/2. Let p ( x ) - F ? . l ( x i - ( 1 / 2 - X o ) ) ( x l - ( 1 / 2 + Xo) be the 'penalty term' in the objective function. Then p ( x ) > - ( x l - (1/2 + xo))(xl - (1/2 - x0) > (2/3)x02. On the other hand, for the 'payoff term' q ( x ) - ( 1 / 2 n ) ~ n . . l ( x i - 1/2) 2 we obtain q(x) > - x 2 / 2 . Hence f ( x ) > (2/3)x02 - (1/2)Xo2 > 0 ~ contradiction. L'3 Remark 3. If the objective function is concave, then its local (global) minima occur at the vertices of the feasible domain. Let v b e a given vertex: We can compute its adjacent vertices v~,..., Vm in polynomial tin~e, It is enough to check local optimality of v over t h e simplex generated b y
February 1988
v, v 1, ..., v,,. This in turn can be checked by considering the m one-dimensional problems of minimizin~ the function over the edges of the simplex. Hence, we can verify local optimality for concave programming in polynomi~ time. Observe that our objective function (in both theorems) contains only one 'convex variable'.
References [1] A. Bachem and H.W. Hamacher, "Applications of combinatorial methods in mathematical programming: Abstracts and open problems", Research Report no. 85-13, Sonderforschungsbereich 303, University of Bonn, 1985. [2] S.A, Cook, "The complexity of theorem proving procedures", Proc. 3rd Ann. ACM Syrup. on Theory of Comput. ing, 151-158 (1971). [3] M.R. Garey and D.S. Johnson, Computers and Intractability: A Guide to the Theory of NP.completeness, Freeman, San Francisco, CA, 1979. [4] D.G. Luenberser, Linear and Noniinear Programming, 2nd ed., Addison-Wesley, Reading, MA. [5] G.P. McCormick, "Second-order conditions for constrained nfir~aa", SIAM 3". of Applied Mathematics 15, 641-652 (1967). K.G. Murty and S.N. Kabadi,-"Some NP-complete problems in quadratic and nonlinear programming", Technical Report 85-23, Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, MI. [7] P.M. Pardalos and J.B. Rosen, Constrained Global Optimization: Algorithms and Applications, Lecture Notes in Computer Science 268, Springer, Berlin, 1987. [8] A. Vergis, K. Steigfi'tz and B. Dickinson, "The complexity of analog computation", Math. and Computers in Simulation 28, 91-113 (1986).
35