Generalized exponential penalty function for nonlinear programming

Copyright 0 1994 E&vicr Science Ltd Printedin Great Britain.All kzhta reserved

Pergamon

GENERALIZED EXPONENTIAL PENALTY FUNCTION FOR NONLINEAR PROGRAMMING J. QIN and D. T. NGUYEN Civil Engineering Department, Old Dominion University, Norfolk, VA 23529, U.S.A. (Received 4 November 1992) Abstraet4neralized exponential penalty functions are constructed for the multiplier methods in solving nonlinear programming problems. The nonsmooth extreme constraint g,,, is replaced by a single smooth constraint g, by using the general&d exponential function (base a > 1). The approximation of g, to g,, can be refined by increasing the base a. The KS(x) function is found to be a special case of the proposed g, function (i.e. set o = e). A simple algorithm, using the proposed generalized exponential function g,, is suggested for solving nonlinear programming problems. Both small and large-scale nonlinear programming problems (up to 2000 variables and 2000 nonlinear constraints) are solved to test tbe present algorithm.

1. INTRODUCTION

Consider the (NLP) problem min s.t.

following

nonlinear

f(x) g,(x) do

(i = 1,2,.

f(x) gi(x)aO

(i=1,2

programming

. . ,m)

(1)

generalized exponential function g, is used to obtain a single constraint in the multiplier method to provide a smooth composite objective function. Several nonlinear programming problems [12] are solved by using this generalized exponential function. The largest problem solved involves 2000 variables and 2000 nonlinear constraints.

or min s.t.

,...,

m),

2. DERIVATIONS OF THE GENERALIZED EXPONENTIAL FUNCTION

(2)

wheref(x) and g,(x) are smooth, nonlinear functions of x (x E R”). Although many methods have been developed for solving the above constrained nonlinear programming problems, such as the barrier and penalty

Problem (1) is equivalent to the following nonlinear programming problem

function sequential methods [l] and the multiplier method [2,3], they either have computational difficulties or have to deal with a nonsmooth combined objective function. Recently, the exponential function was used to provide a smooth cumulative constraint function in the combined objective function for the multiplier method [4,5]. The exponential function, known as KS(x) function [6] has already been used in optimal structural synthesis and designs [7-lo]. One disadvantage of using the KS(x) function, however, is that the user has to choose a suitable parameter p. If p is too small then the results

similarly, problem (2) is equivalent to

min s.t.

min s.t.

g,,,(x) = g,,,(x)

(3)

f(x) &Ii”(x ) 2 09

where the single constraint g,,,,(x) defined as the ‘extreme’ constraint expressed as

(4) [or g,,(x)] is which can be

= maxI&)>

(for problem (3) i = 1,2,. . . , m)

will not be accurate, if p is too large then the KS(x) function will become nonsmooth [l 11. However, p

should be positive infinite in its derivation given in [4,5]. In this paper, a generalized exponential function g, is constructed such that p is no longer needed as an accuracy controlling parameter, and the KS(x) function is included as its special form. This proposed

f(x) g/M,(x) G 0

= g,,,(x) = minki (for problem (4), i = 1,2, . . . , m). When the original problem (1) or (2) has at least one active constraint, then the problem (3) or (4) can 509

J. QIN and D. T. NGUYEN

510 Table Iteration

No.

I. Computational

II%+ I - Xk II

1

Note. x,, = {O.OOOl,.

4(x, a) =f(x)

1.0 - 1.0457 x - 1.0357 x -5.3822 x -3.6524 x -2.7202 x

-0.00006 0.52028132 0.51834827 0.51825999 0.51822897 0.51821282

10-3 1o-4 1O-s IO-’ IO-’

. . , 0.0001) (not feasible); reference [12]‘s optimum: f= 0.518163274.

be solved by means of the multiplier penalty method [31 minimize

f(G)

&w

70927.3 6.89 x IO-* 3.092 x 1O-3 1.708 x IO-* 1.019 x IO-2 2.830 x IO-*

3.59349 5.2368 x lo-* 1.9833 x lo-’ 1.5833 x IO-’ 8.6909 x 1o-5

I

for Example

II& II

0 2 3 4 5

results

+ age,,(x) -i

p,,(x)‘,

(5)

property suggests that g,(x) is a function of gi(x) (i=1,2,... , m). In other words, g,(x) are the basic variables of g,(x). Now we need a function that can select the extreme variable by itself. This can be done by first defining a generalized exponential function F = F(gi(x), a), such as

where a is a Lagrange multiplier and c is a penalty factor, a should be updated during the iterations by the formula at + , = ak+ cg,, (xk h where x, is the solution of (5) at step k, and c can be kept constant or can be increased gradually. It should also be noted here that x, in the above formula should contain the updated values upon exiting from a successful unconstrained minimization. However, the use of g,,, can cause difficulties in the solution of(S), since g,,, is not a smooth function. We will try to construct a smooth function g,(x) to replace g,,,(x) in order to avoid those difficulties. It is obvious that g,(x) should have the following properties

F

=

aPBi(X)

f

(a > Lp # 0).

i=l

It can be seen that aP@@)(a> 1) is a monotone increasing function of g,(x) when p > 0, and a monotone decreasing function of g,(x) when p < 0. Thus, if a is large enough then F is a good approximation of aJ’GXf.But we only need the approximation of g,,,(x), so we have to invert F to get g,,,(x). Since F = up&v, this suggests that g,(x) should take the form

=j

log, F

=j

log,

.

1. g,(x) =g,,,(x)

(if m = 1)

g,(x) =g,,,(x)

+ 4x)

2. g,(x) = G(g(x))

It is found that g,(x) has the following property

(if m > 1)

(i = 1,. . . , m).

g~,(x)4g,(x)Cg,,(x)+~loem

The first property of g,(x) assures that g,(x) is a good approximation of g,,,(x), while the second

Table 2. Computational Iteration

No.

0

1 2 3 4 5 6 7 8 9 Nore. x,=(-0.0001 f = 680.6300573.

IlXk,, -

xk 11

5.952827

1.264876 0.683534 0.767874 0.632013 7.137 x 10-Z 4.382 x IO-’ 1.224 x IO-* 4.6328 x 1O-6 ,...,

@ ’ 07&.U= g,,, )

results for Example

II4k II 741.26 7287.57 10506.29 5618.36 2321.53 170.833 128.389 165.360 310.71 I 21.744

2

RmtLv 0.0006 2.7630 x -4.4204 x -1.2526 x -2.1536 x -3.0169 x -6.4009 x 2.9859 x -4.4446 x -4.652 x

f&l 1183.0224 741.596 700.750 686.735 682.340 680.663 680.636 680.635 680.633 680.633

IO-’ 1O-4 IO-’ 1O-4 lO-4 1O-5 lO-5 1O-5 lO-5

-O.OOOl} (not feasible); reference

[12]‘s

optimum:

Generalized exponential penalty function for nonlinear programming

511

Table 3. Computational results for Example 3 Iteration No.

-xkii

ilxk+l

0

i#k

-

I 2 3 4 5 6 7 8

//

8,

5745375.68 2912.1413 1.1212x 10-Z 1.56625 7.9255 33.9941 644691 83.9927 38.4962

37.2981 2.1901 2.3034 x lo-* 1.1488x lO-3 4.4096 x IO-” 1.9303x 10-e 1.2158x lO-4 8.1318 x lO-5

fbk)

29.6 1.1732x IO-’ 1.0324x lO-3 8.7870 x lo-’ 4.5236 x 1O-5 3.1488 x lo-’ 2.4715 x lo-’ 1.4508x lO-5 1.4542x lO-J

566766 386.9546 245.7901 244.959 244.947 244.930 244.924 244.920 244.917

Note. x, = {10, . . . , 10) (not feasible); reference [12]‘soptimum: f= 244.899698. nonlinear programming problem (3) is now under consideration]

QJ< 0,&x,

=

min s-t.

!LIizJ.

If we require the error term (l&l) lo&m less than c, then the base a can be determined

[assume problem

f(x) g,(x) Q 0

(6)

through the solution of the following sequential unconstrained nonlinear programming problem

a > mli@ic.

minimize

f#G a) =f(x)

+ ag,(x) +

f gAx)l,

(7)

The KS(x) function as it is used in [4-lo] is defined as KS(x)

=$In

i @d(x) . ( i-1 )

It is interesting to note that the KS(x) function is a special form of the proposed g,(x) function. One only has to set the special base: a = e. However, the advantage of using the g,(x) function is that we do not have to adjust the controlling parameter p as it is required in the KS(x) function. Furthermore, we can control the smoothness of the g,(x) function by using different bases (a > 1) during the iterations. In fact, the nonzero parameter p is only used as a sign for the extreme constraint (g_, = g,,,_, if p > 0; or gCX, = &li”9 if p < 0). However, p must be positive infinite in the derivation given in [4,5], and the KS(x) function fails if p 6 1. Since higher accuracy in the constraint conditions is desired in later iterations, it is suggested that the base a should be increased during the iterations.

where any standard unconstrained NLP methods (such as BFGS or DEP) [13,14] can be used (the BFGS method is used in this paper). The simple NLP algorithm can be written as follows. 1. Set initial values for base a and c, and x0, let a, = 0, k = 0. 2. Minimize 4(x, c) using any standard unconstrained NLP methods. 3. Convergence check: Ilx, + , - x, jl < C, if satisfied go to 5. 4. Update c, a and c by ak+ 1= ak+ c&(xk)

(xk is the minimi~r)

ok+, = 10’OoDak ck+l =max(l,~,O~,

1.1 ck)

go to 2. 5. stop. 4. NUMERICAL APPLICATIONS

3. A STEP-BY-STEP NONLINEAR PR~RAMM~G ALGORITHM

When the extreme constraint in problem (3) or (4) is replaced by a single constraint g,(x) as defined above, we now can solve the following constrained

Four nonlinear programming problems are solved using the above algorithm, the first three examples are taken from [12], while the last example is constructed by the authors to test the capability of the

Table 4. Computational results for Example 4 Iteration No. 0

1 2

1 --xkil

iixk+

-

402.5014 9.1612 x lo-’

II 1ou 2926.69 26093.2 llcbk

9.345

x

i?muv 99 -1.7541 x 10-s 5.935 x lo-’

fhk)

-2,oOO.oOo - 1998.77 - 1999.997

Note. x0 = { 10,. . . , IO} (not feasible); optimum: J = -2000. This problem is solved on Gray Y-MP running one processor with CPU time = 564.681sec.

J. QIN and D. T. NGU~H

512

present algorithm for solving large-scale nonlinear Example 4 programming problems. For all these examples, p = 1 Large-scale nonlinear programming and the initial values for a and c are a, = IO’@‘, c, = 10,000, and E = 0.0001. The equality constraint g(x) = 0 is replaced by g(x) < 0 and -g(x) ,< 0. min S(X) = - i ~1 ,=I Example 1 Problem number: 66 (see 112, p. 881)

s.t.

g,(x)=

f

x;+nxf-(2n

test problem

-l)
]=I min

f(x) = 0.2x, -0.8x, (i = 1,. . . ,m,j

s.t.

In this example, n = m = 2000, the optimum solution is: x = (1, . . . , 1) and the optimum value is f = -n = -2000. Numerical results are presented in Table 4. It should be noted here than an effective equation solver [15] (using Gaussian elimination) is used in the BFGS method (see step 2 of Sec. 3) in order to reduce the total solution time for this large-scale NLP problem.

exp(x,) - x3 < 0 0
100

06x36

10.

The optimum solution as well as a history of the iterations are presented in Table 1. Example 2

100 (see [ 12, p. 1111)

Problem number: min

f(x)=(x,

- IO)*+ 5(x2 - 12)2+x: + 3(x4 - 1l)* + 10x; + 7x; f x: - 4x,x, - 10x, - 8x,

s.t.

g,(x)=2x:+3x’:+n,+4x:+5x~-127~0 gZ(x)= 7x,+ 3X2-k ~~x~+x,-x~g3(x) = 23x, +x;

282
+ 6x: - 8x, - 196 < 0

g.,(x)=4x;+x;-3x,x1+2x;+5x6-llx,
5. DEXXJSSIONS AND CONCLUSIONS

A generalized exponential penalty function g,(x) is constructed where p is no longer needed as an accuracy controlling parameter as it is in the KS(x) function. Rather, p is only introduced here to facilitate the treatment of the extreme ~nst~ant g,,, (either maximum or minimum). Furthermore, the KS(x) function is considered as the special case of the proposed g,(x) function. Since the nonsmooth extreme constraint g,,,(x) is replaced by a smooth constraint g,(x). it is expected that the present algorithm will lead to better performance during the optimization process. In fact, in all the examples considered in this paper, optimum solutions are essentially obtained with less than five iterations. The proposed g,(x) function can be used in a wide range of optimization problems, such as structural optimization. Acknowledgement-The authors would like to acknowledge the financial support under a NASA grant NAGI-858. REFERENCES

3

Problem number: min

f(x)=

119 (see [12, p. 1271)

1.

16 16 1 ~a,,(x~+xi+l)(x~+xi+l)

2.

i-Ii-1

s.t.

f i).

exp(x, I- x2 Q 0

,~Ebi,xj-ci=O ‘n

(i = 1,. . . ,8)

O<.xi<5

(i=1,...,16)

aU, b,, c,:

cf. Appendix.

The corresponding Table 3.

iterative solutions

3. 4. 5.

are listed in

6.

G. P. McCormick, Nonlinear Programming, Theory, Algorithm, and A~piications. John Wiley, New York (1983). M. J. D. Poweil, A method for non&near constrains in minimization problems. In Optimization (Edited by R. Fletcher), pp. 283-298. Academic Press, London (1969). M. R. Hestenes, Multiplier and gradient methods. J. Optimiz. Theor. Appl. 4, 303-320 (1969). A. B. Templeman and X. S. Li, A maximum entropy approach to constrained non-linear programming. Engng Optimiz. 12, 191-205 (1987). X. S. Li, An aggregate function method for nonlinear programming. Science in China (Series A) 34, 1467-1473 (t991). G. Kreisselmeier and R. Steinhauser, Systematic control design by optimizing a vector performance index. Proc.

513

Generalized exponential penalty function for nonlinear programming IFAC Symp. on Computer Aided Design of Control System, Zurich, Switzerland, August (1979). 7. P. Hajela, Further developments in the controlled growth approach for optimal structural synthesis. ASME Paper 82-DET-62.(1982). 8. P. Haiela. A look at two underutilized methods for optimum structural design. Engng Optimiz. 11, 21-30 (1987). 9. J. Sobieszczanski-Sobieski, B. B. James and A. R. Dovi, Structural optimi~tion by multilevel decomposition. AIAA Jni zj, 17751782 (-1985). 10. J. Sobieszczanski-Sobieski. A. R. Dovi and G. A. Wrenn, A new algorithm’ for general multiobjective optimization. Proc. AIAA/ASME/ASCEIAHS 29th Structural Dynamics and Materials Conference, Wi~iamsburg, VA, Paper 88-2434, April (1988).

1I. J. S. Arora, A. I. Chahande and J. K. Paeng, Multiplier methods for engineering optimization. ht. J. IVumer. Met/t. Engng 3% 1485-i525 (1991). 12. W. Hock and K. Schittkowski. Test Exan&es for Noalineor Progreg Codes. Springer, Berlin (1981). 13. J. S. Arora, Introduction to Optimum Design. McGrawHill (1989). 14. R. T. Haftka, Z. Gurdal and M. P. Kamat, Elements of Structural Optimization, 2nd E&t. Kluwer Academic (1990). 15. T. K. Agarwal, 0. 0. Storaasli and D. T. Nguyen, A parallel-vector algorithm for rapid structural analysis on high-performance computers. Proc. of AIAA/ ASMEIASCEIAHS 31st SDM Conference. Lona Beach,’ CA, ‘April 2-4, 1990, AIAA paper N< 90-l 149.

APPENDIX

j

1

2

3

4

5

6

ajj %

I 0 0

0 1 :,

Q4j

8

0 1 0 0 8

0 0 0 0 1 0

0 0 0 0 1 1 0 0

0

1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0

0.19 -1.3 0 0.95 0 -0.33 0

0.25 1.82 b1.16 -0.54 -1.43 0.31 0

a2j

a5j

a6i %j

arj a91

aloj %j e12j %j aI4j

eIJj au6j

0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

brj 0.22 0.20 b, -1.46 0 &j 1.29 -0.89 b4j -1.10 -1.06 b 51 0 0 2 01.12 -1.72 0 b;

0

0.45

c.

2.5

1.1

0 0 :: 0 0 : 0 :

0.26 -3.1

x 0 0 0 I: :: 0

8 0 : 0 0 0

7

8

9

: t

1 0 0 8

0 0 0 1 11000100 8 0 I 0 0 0010 0 1 10010001 0100010 00101000 0 0 0 0 0 0 0 0 0 0

0 0 1 0 0 0 0 8 0 ::

Z.58

0

-3.5

1.3

2.1

; 0 0 0 0 : 0

0.15 0.11 0.12 -1.15 0 0.80 -0.96 0 -0.49 0 -1.78 -0.41 1.51 0.59 -0.33 1.62 01.24 0.21 1.12

-1.1

;

-1.03 2.3

0.13 0 0 0 -0.43 -0.26 0 0.10 -1.5

1.00 0 0 0 0 -3.6 0 0

10

11 12

13

00 0 00000~ 100010 8::

14

IS

0

0

00 0 0:

16

1 0 0

00~010 0 010 00 000100 000010 00000

IO

0 0

0 0 0 0

0 1.000 0 1.000 0 0 0 1.00 0 0 0 0 1.00 0 0 00 0 0 0 01.00 1.0

0 0 0 0 0 0

0

1

00

00000

110

1 0 0 0