- Email: [email protected]
A new filled function for unconstrained global optimization
Applied Mathematics and Computation 174 (2006) 419–429 www.elsevier.com/locate/amc
A new filled function for unconstrained global optimization Xiaoli Wang *, Guobiao Zhou Department of Mathematics, Shanghai Jiao Tong University, Shanghai 200240, PR China
Abstract Seeking global optima of an unconstrained and multi-modal global optimization problem minx2X f ðxÞ by constructing a filled function is concerned in this paper. On the basis of analyzing filled functions presented before, a new filled function is proposed, and it is proved to meet the properties of a filled function. Moreover, solutions of numerical experiments show that the function is quite effective. Ó 2005 Published by Elsevier Inc. Keywords: Filled function; Global optimization; Local minimizer; Global minimizer
1. Introduction Global optimization is mainly concerned with the characters and algorithms on the global minimum of multi-modal nonlinear functions. More and more practical problems in science, economics, engineering and other regions can be formulated as global optimization problems, which cannot be solved effectively with traditional algorithms of nonlinear programming. Lots of *
Corresponding author. E-mail address: [email protected] (X. Wang).
0096-3003/$ - see front matter Ó 2005 Published by Elsevier Inc. doi:10.1016/j.amc.2005.05.009
420
X. Wang, G. Zhou / Appl. Math. Comput. 174 (2006) 419–429
researchers have been attracted to the field of global optimization. As we know, there are two categories of global optimization approaches: probabilistic [1,4,13] and deterministic [2,3,5–12,14–18]. The filled function method studied in this paper is included by the latter. Compared to the other methods, it has an outstanding advantage that its thought is simple while its realization is relatively easy and effective. The filled function method was introduced by Ge in [5]. The thought was constructing an auxiliary function to transfer from a local minimizer to a lower minimizer and so does the iteration until the global minimum is obtained. Since then, many filled functions [5,7,9–11,15,16] have been proposed successively. Just on the basis of those functions, a new filled function is raised in this paper. Compared to those functions, this one is more effective, and although there are two parameters in this function, they are easy to adjust. The numerical experiment results show that the function is quite satisfying. Consider the following global optimization problem: ðP 0 Þ
minn f ðxÞ; x2R
where f : Rn ! R is coercive, i.e. limkxk!þ1 f ðxÞ ! þ1, then there exists a closed bounded box X 2 Rn containing all global minimizers of (P0) in its interior. Then (P0) is equivalent to the following problem: ðP Þ
min f ðxÞ. x2X
Throughout the paper, x* is a global minimizer of f(x), and xi ði P 1Þ is the isolated minimizer of f(x). Moreover, k Æ k is taken as k Æ k2. Some concepts of filled functions are introduced as follows: Definition 1.1. The basin of f(x) at an isolated minimizer x1 is a connected domain B1 which contains x1 , and in which the steepest descent trajectory of f(x) starting form any point converges to x1 , while the descent trajectory of f(x) starting from any initial point x 62 B1 does not converge to x1 . Conversely, if x1 is an isolated maximizer, the basin of f(x) at x1 is defined as the hill of f(x). A basin of f(x) at x2 , B2 , is higher (lower) than B1 , if f ðx2 Þ P f ðx1 Þ (f ðx2 Þ 6 f ðx1 Þ respectively). Definition 1.2. A connected domain S 1 B1 is defined as the simple basin at x1 if the equality ðx x1 ÞT 5 f ðxÞ > 0 holds for any point x 2 S 1 and x 6¼ x1 . A function P(x) is said to be a filled function of f(x) at the local minimizer x1 if it satisfies the following properties: (p1) x1 is a local maximizer of P(x) and the whole basin B1 of f(x) at x1 is a part of the hill of P(x).
X. Wang, G. Zhou / Appl. Math. Comput. 174 (2006) 419–429
421
(p2) P(x) has no minimizers or saddle points in any basin of f(x) higher than B1 . (p3) If f(x) has a basin B2 lower than B1 , then for any point x 2 B2 , there is a point x0 2 B2 that minimizes p(x) on the line through x and x1 . Ge proposed the filled function in [5]: ! x x 2 1 1 exp P ðx; r; qÞ ¼ ; r þ f ðxÞ q2
ð1:1Þ
where r, q are parameters. In (1.1), when the value of kx x1 k is quite large, P(x, r, q) and the exponenkxx1 k2 tial term exp q2 will tend to zero which means the change of P(x, r, q) and $P(x, r, q) will be undistinguished. According to the determining criterion of the algorithm, the inappropriate point will be got when pursuing the stationary point of P(x, r, q), then the global minimizer of f(x) will probably be lost. In addition, whether the function satisfies the ‘‘filling properties’’ mainly depends on the value of parameters. In case that the value is not appropriate, the efficiency of the algorithm will be influenced, and we even cannot find the global minimizer of the problem possibly, which can be shown in Table 1. Table 1 includes the results of numerical experiments solving the 6-hump back function with the algorithm in [1]. Where the initial point is (1.6, 0.9)T, and the initial given value of r þ f ðx1 Þ is 3, and Kf means the total number of evaluations for the objective function, the filled function and their gradients, and NF means that the global minimizer cannot be found. Consequently, putting forward filled functions whose parameters are easy to adjust is an improving method. So filled functions with one parameter were proposed. For example, Li and Sheng [9] proposed the filled function as follows which were extended to solve the Lipschitz programming: h 2 i Qðx; AÞ ¼ f ðxÞ f x1 exp Ax x1 ; ð1:2Þ we can see it still has the same shortcoming as (1.1). Kong and Zhuang [7] proposed the function with the form as follows: 2 ð1:3Þ Qðx; AÞ ¼ ln 1 þ f ðxÞ f x1 exp Ax x1 . Obviously, the above two filled functions still have the same shortcoming that is be affected by the exponential term as (1.1). Considering such a fact that if f(x) > 0, then f(x) and ln f(x) have the same extreme points [15]. Therefore, to Table 1 q2
1
101
102
103
104
105
106
Kf
NF
358
93
80
561
1029
NF
422
X. Wang, G. Zhou / Appl. Math. Comput. 174 (2006) 419–429
tackle the above handicap, we can take logarithm of (1.1), then the effect of the exponential term will be eliminated. Then a new filled function is got: 2 x x1 P 1 ðx; qÞ ¼ ln f ðxÞ f x1 . ð1:4Þ q2 Similarly, a filled function as follows have been proposed in [11]: 2 Qðx; AÞ ¼ A ln 1 þ f ðxÞ f x1 þ x x1 .
ð1:5Þ
Obviously, x 2 X in (1.3) and (1.5) must satisfy f ðxÞ > f ðx1 Þ 1, similarly, x 2 X in (1.4) must satisfy f ðxÞ > f ðx1 Þ, which restricts the application of this function. Liu proposed successively in [10,11] the following filled functions: 1 2 akx x1 k ; arctan½f ðxÞ f ðx1 Þ a 1=m . LðxÞ ¼ p ½f ðxÞ f ðx1 Þ kx x1 k
LðxÞ ¼
ð1:6Þ ð1:7Þ
We can see from the above two forms that they are not continuous functions, just as mentioned in [10,11] that we need to suppose f ðxÞ > f ðx1 Þ. On the basis of the above filled functions, we propose a new improved one that combines the advantages of these functions. If the new filled function has a local minimizer x1, it must belong to a lower basin than the current one B1 . Importantly, if the parameters are chosen appropriately, we can ensure that the minimizer of the filled function x1 2 S 1 ¼ fx 2 Xjf ðxÞ < f ðx1 Þg, thus the new minimum of f(x) will be lower than f ðx1 Þ evidently, which obviously satisfies the property (p3). This paper is organized as four sections. In Section 2, we propose a new filled function, and prove that the function satisfies the filling properties. In Section 3, we give a solution algorithm and make numerical implementation which testifies the effectiveness of the function. Finally, we draw some conclusions in Section 4. 2. A new filled function The new filled function proposed in this paper is as follows: n o 2 W ðx; a; kÞ ¼ a ln max 0; f ðxÞ f1 þ 1 kx x1 k2 n 2 o 1 k max 0; f1 f ðxÞ ;
ð2:1Þ
where f1 is the value of the function f(x) at the present local minimizer x1 , and a, k > 0 are parameters.
X. Wang, G. Zhou / Appl. Math. Comput. 174 (2006) 419–429
423
It can be evidently observed from the form of (2.1) that to ensure the continuance of W(x, a, k), the term of square is given as the above, and to realize that W(x, a, k) has minimizers in the region S1 ¼ fx 2 Xjf ðxÞ < f1 g when the parameters are chosen properly, the last term is added on the basis of (1.5). When f ðxÞ P f1 , h i 2 2 W ðx; a; kÞ ¼ a ln f ðxÞ f1 þ 1 x x1 ; ð2:2Þ and when f ðxÞ < f1 , 2 h 2 i W ðx; a; kÞ ¼ x x 1 k f f ðxÞ . 1
ð2:3Þ
1
In the following part, we will prove that W(x, a, k) really satisfies the properties of a filled function under some conditions of parameters a, k. Theorem 2.1. Assume that x1 is a local minimizer of f(x) and W(x, a, k) is defined as (2.1), then x1 is a local maximizer of W(x, a, k). Proof. For 8x 2 B1 , and x 6¼ x1 , we have f ðxÞ > f1 . Since parameter a > 0, then W ðx; a; kÞ ¼ a ln½ðf ðxÞ f1 Þ2 þ 1 kx x1 k2 2
2
< a ln½ðf1 f1 Þ þ 1 kx1 x1 k ¼ 0 ¼ W ðx1 ; a; kÞ: Thus x1 is a local maximizer of W(x, a, k).
h
Theorem 2.2. Assume that x1 is a local minimizer of f(x), and x is a point different from x1 and satisfies f ðxÞ P f1 . (i) If the given direction d satisfies d T ðx x1 Þ > 0 and d$f(x) P 0 or d T ðx x1 Þ P 0 and dT$f(x) > 0, then for whatever value of a > 0, d is a descent direction of W(x, a, k) at point x. (ii) If the given direction d is a descent direction at point x, that is dT$f(x) < 0, when i h 2 d T x x1 f ðxÞ f1 þ 1 0
ð2:5Þ
424
X. Wang, G. Zhou / Appl. Math. Comput. 174 (2006) 419–429
(i) Since d T ðx x1 Þ > 0 dT$f(x) > 0, we have d T rW ðx; a; kÞ ¼
d$f(x) P 0
and
2aðf ðxÞ f1 Þ ðf ðxÞ f1 Þ2 þ 1
or
d T ðx x1 Þ P 0
and
d T rf ðxÞ 2d T ðx x1 Þ < 0.
(ii) Since dT$f(x) < 0 and condition (2.4) holds, we also obviously have d T rW ðx; a; kÞ < 0; so d is a descent direction of W(x, a, k).
h
Theorem 2.1 and the first half of Theorem 2.2 show that W(x, a, k) satisfies the filling property (p1) of a filled function. Provided the direction d ¼ x x1 and f ðxÞ P f1 , Theorem 2.2 shows that the point x cannot be the stationary point of W(x, a, k) when a always satisfies condition (2.4). That is to say, if (2.4) holds, any point x of B1 or some basin higher than B1 cannot be the stationary point or saddle point, so the function W(x, a, k) satisfies the filling property (p2). The following theorem shows this property of W(x, a, k) in a further degree. Theorem 2.3. Assume that x1 is an isolated local minimizer of f(x), and point x satisfies f ðxÞ P f1 . When 2D ; ð2:6Þ 0
x2X
W(x, a, k) has not any minimizer or saddle point in the region S 2 ¼ fx 2 Xjf ðxÞ P f1 g. Proof. Suppose that ^x is a minimizer or saddle point of f(x) which is different from x1 , and ^x 2 S 2 ¼ fx 2 Xjf ðxÞ P f1 g. Then we have krW ð^x; a; kÞk ¼ 0. That is aðf ð^xÞ f1 Þ ðf ð^xÞ f1 Þ2 þ 1 then a¼
krf ð^xÞk ¼ kx x1 k;
i h ^x x f ð^xÞ f 2 þ 1 1 1 krf ð^xÞkðf ð^xÞ f1 Þ
.
ð2:8Þ
X. Wang, G. Zhou / Appl. Math. Comput. 174 (2006) 419–429
425
It is obvious that ^x 62 S 1 , then k^x x1 k P D > 0. In addition, we have krf ð^xÞk 6 L. Consider the function 2 f ðxÞ f1 þ 1 ; GðxÞ ¼ f ðxÞ f1 when f ðxÞ f1 P 0, it has minimum at the point x satisfying f ðxÞ f1 ¼ 1, that is 2 f ðxÞ f1 þ 1 GðxÞ ¼ P 2; ð2:9Þ f ðxÞ f1 so i h ^x x f ð^xÞ f 2 þ 1 1 1 2D ; ð2:10Þ P a¼ L krf ð^xÞkðf ð^xÞ f1 Þ this obviously contradicts with (2.6). So any minimizer or saddle point of W(x, a, k) cannot belong to the region S 2 ¼ fx 2 Xjf ðxÞ P f1 g. h Theorem 2.3 shows that there is not any stationary point of W(x, a, k) in the region S 2 ¼ fx 2 Xjf ðxÞ P f1 g when a is properly chosen, which further indicates that W(x, a, k) satisfies the property (p2). Theorem 2.4. Assume that x1 is a local minimizer of f(x), and f ðxÞ P f1 , and the given direction d satisfies dT$f(x) < 0 and d T ðx x1 Þ > 0. When i h 2 d T x x1 f ðxÞ f1 þ 1 a> ; ð2:11Þ ðd T rf ðxÞÞðf ðxÞ f1 Þ d is an ascent direction of W(X, a, k) at point X. 2aðf ðxÞf Þ
Proof. When (2.11) holds, d T rW ðx;a;kÞ¼ ðf ðxÞf Þ21þ1 d T rf ðxÞ2d T ðxx1 Þ>0, 1
then d is an ascent direction of W(x, a, k) at point x. h Theorem 2.5. Assume that x1 is a local minimizer of f(x), and x2 is another local minimizer satisfying f ðx2 Þ < f1 . There exists r > 0, so that for 8x 2 N ðx2 ; rÞ, when a, k respectively satisfy 2D 1 ; k > 2; 0 0, there exists a point x 0 which minimizes W(x, a, k) on the line through x1 and x. Proof. There obviously exists r > 0 so that for 8x 2 N ðx2 ; rÞ, we have f1 > f ðxÞ P f ðx2 Þ, and f1 f ðxÞ P e > 0. Then the form of W(x, a, k) is as (2.3), so
426
X. Wang, G. Zhou / Appl. Math. Comput. 174 (2006) 419–429 2
2
2
W ðx; a; kÞ ¼ kx x1 k ½1 kðf1 f ðxÞÞ P kx x1 k ð1 ke2 Þ > 0; and while W ðx1 ; a; kÞ ¼ 0. According to Theorems 2.2 and 2.3, when 0 < a < 2D L f ðxÞ P f1 , the trajectory starting from x1 is descendent, then there is a point x0 2 B2 certainly which minimizes W(x, a, k) on the line through x1 and x. h Theorems 2.4 and 2.5 indicate that W(x, a, k) satisfies the property (p3) of a filled function when the parameters a, k are chosen appropriately. So we have proved that W(x, a, k) is a filled function. In fact, the parameters a, k are easy to adjust, and if they are adjusted properly, as long as W(x, a, k) has local minimizers, they must be in the region S 1 ¼ fx 2 Xjf ðxÞ < f1 g. 3. Solution algorithm and numerical experiments 3.1. Solution algorithm In this section, we present an algorithm as follows to solve the problem (P). In the algorithm, d > 0, a > 0, 0 < e < 1, and the large enough positive integer Nm are all given constants. Step 0: Set a :¼ 1, k :¼ 105, i :¼ 1, N :¼ 1. Step 1: Starting from an initial point x0, minimize f(x) and obtain the first local minimizer x1 and the value f1 . Step 2: Construct the filled function: W ðx; a; kÞ ¼ a lnf1 þ ½maxð0; f ðxÞ f1 Þ2 g 2
2
kx x1 k f1 k½maxð0; f1 f ðxÞÞ g. Step 3: If i 6 2n, starting from the point x1 :¼ x1 þ dd i , where di is a randomly selected direction whose norm are less than 1, minimize W(x, a, k) and obtain the minimizer x2, turn to Step 4; otherwise, set N :¼ N + 1, turn to Step 6. Step 4: If x2 2 X turn to Step 5; otherwise, set i = i + 1, and turn to Step 3. Step 5: Minimize the objective function f(x) starting from x2, and find a local minimizer x2 . If f ðx2 Þ < f1 . Set x1 :¼ x2 and f1 :¼ f ðx2 Þ, then turn to Step 2; otherwise, turn to Step 6. Step 6: If N 6 Nm, then if a > a, set a :¼ e Æ a, i :¼ 1; otherwise, set a :¼ 0, and turn to Step 3; otherwise, stop the algorithm and take x1 as the global minimizer of the problem (P). According to the feature of W(x, a, k), when the parameter k is large enough and a is appropriately small, W(x, a, k) must be an effective filled function. So in the implement, we can firstly give the parameter k a large enough value, then
X. Wang, G. Zhou / Appl. Math. Comput. 174 (2006) 419–429
427
we only need to adjust the parameter a, just as the above algorithm shows. In addition, since the initial point x1 is randomly chosen around x1 when minimizing the filled function W(x, a, k), if we solve the problem (P) using this algorithm, we may get different global minimizers for the same initial point x0. Thus if we have numerical experiments for proper times, we can get all the global minimizers of (P) just as the examples shown as follows.
3.2. Numerical experiments and results The results of numerical experiments show that the above solution algorithm is effective for solving multi-modal problems like (P). Take the following problems as examples, after running the algorithm procedure for proper times, all the global minimizers of these problems can be found successfully. (1) 6-hump back camel function: f ðxÞ ¼ 4x21 2.1x41 þ x61 =3 x1 x2 4x22 þ 4x42 ;
3 6 x1 ; x2 6 3.
Initial point x0: (1.6, 0.9)T. Two global minimizers x*: (0.0898, 0.7127)T, (0.0898, 0.7127)T. Global minimum: f * = 1.0316. (2) Goldstein and Price function: f ðxÞ ¼ ½1 þ ðx1 þ x2 þ 1Þ2 ð19 14x1 þ 3x21 14x2 þ 6x1 x2 þ 3x22 Þ 2
½30 þ ð2x1 3x2 Þ ð18 32x1 þ 12x21 þ 48x2 36x1 x2 þ 27x22 Þ; 3 6 x1 ; x2 6 3. Initial point x0: (1.6, 0.9)T. One global minimizer x*: (0.0000, 1.0000)T. Global minimum: f * = 3.0000. (3) Two-dimensional Shubert function: ( )( ) 5 5 X X f ðxÞ ¼ i cos½ði þ 1Þx1 þ i i cos½ði þ 1Þx2 þ i ; i¼1
i¼1
10 6 x1 ; x2 6 10. Initial point x0: (1, 1)T. Eighteen global minimizer x*: (0.8003, 1.4251)T, (0.8003, 7.7083)T, (0.8003, 4.8581)T, (4.8581, 5.4829)T, (4.8581, 0.8003)T, (4.8581, 7.0835)T, (5.4829, 7.7083)T, (5.4829, 4.8581)T, (5.4829, 1.4251)T,
428
X. Wang, G. Zhou / Appl. Math. Comput. 174 (2006) 419–429
(7.0835, 7.7083)T, (7.0835, 1.4251)T, (7.0835, 4.8581)T, (7.7083, 5.4829)T, (7.7083, 0.8003)T, (7.7083, 7.0835)T, (1.4251, 0.8003)T, (1.4251, 7.0835)T, (1.4251, 5.4829)T. Global minimum: f * = 186.7309. (4) n-dimension function: ( ) n1 X n 2 2 2 2 10sin ðpx1 Þ þ f ðxÞ ¼ ½ðxi 1Þ ð1 þ 10sin ðpxiþ1 Þ þ ðxn 1Þ ; p i¼1 10 6 xi 6 10. n = 3: Initial point x0: (2, 3, 5.1)T. One global minimizer x*: (1.0000, 1.0000, 1.0000). Global minimum f * = 6.0873e010. n = 6: Initial point x0 (2, 3, 5.1, 6, 4, 3)T. One global minimizer x*: (1.0000, 0.9993, 1.0002, 1.0003, 1.0027, 1.0001)T. Global minimum: f * = 2.6426e009. n = 10: Initial point x0: (8.9, 5.4, 4.7, 7.3, 9.8, 0, 2.6, 6, 1, 0.5)T. One global minimizer x*: (1.0000, 0.9956, 0.9932, 1.0027, 0.9988, 1.0012, 0.9996, 1.0009, 1.0020, 0.9990)T. Global minimum: f * = 1.5992e008.
4. Conclusions The filled function method is to seek the global minimum of some kinds of problems such as (P). The basic thought is leaving from a local minimizer to a lower one by an constructed auxiliary function. The filled function proposed in this paper improves the defect in [5] that the global minimizers may be lost because of the effect of the exponential term, and the parameters of this one are easier to adjust. Especially, if the parameters are properly chosen, provided the filled function has any local minimizer x1, x1 must belong to the region S 1 ¼ fx 2 Xjf ðxÞ < f ðx1 Þg, so we can obtain the minimizer xi , and the value of f(x) at xi must be lower than the present one. So it unquestionably further strengthen the ‘‘filling performance’’ of the filled function. Numerical experiments also show that the filled function W(x, a, k) is effective. We should point out that the filled function proposed in this paper is also be applicable to solve the global optimization problems in nonsmooth programming.
X. Wang, G. Zhou / Appl. Math. Comput. 174 (2006) 419–429
429
References [1] N. Baba, Global optimization of functions by the random optimization method, International Journal of Control 30 (1979) 1061–1065. [2] P. Basso, Iterative methods for the localization of the global maximum, SIAM Journal on Numerical Analysis 19 (1982) 781–792. [3] F. Branin, Widely convergent methods for finding multiple solutions of simultaneous nonlinear equations, IBM Journal of Research Developments 16 (1972) 504–522. [4] C. Dorea, Limiting distribution for random optimization methods, SIAM Journal on Control and Optimization 24 (1986) 76–82. [5] R.P. Ge, A filled function method for finding a global minimizer of a function of several variables, Mathematical Programming 46 (1990) 191–204. [6] R.P. Ge, Y.F. Qin, A class of filled functions for finding global minimizers of a function of several variables, Journal of Optimization Theory and Applications 54 (1987) 241–252. [7] M. Kong, J.-n. Zhuang, A class of new filled functions for pursuing the global minima of a non-smooth function with multi variables, Collegial Transaction of Computational mathematics 18 (2) (1996) 165–174 (in Chinese). [8] A.V. Levy, A. Montalvo, The tunneling algorithm for the global minimization of functions, SIAM Journal on Scientific and Statistical Computing 6 (1985) 15–29. [9] B.-x. Li, Y. Sheng, Filled functions method for the Lipschitz programming, Systems Mathematics and Science 11 (4) (1991) 346–348 (in Chinese). [10] X. Liu, A class of generalized filled functions with improved computability, Journal of Computational and Applied Mathematics 137 (2001) 61–69. [11] X. Liu, Several filled functions with mitigators, Applied Mathematics and Computation 133 (2002) 375–387. [12] J. Snyman, L. Fatti, A multi-start global minimization algorithm with dynamic search trajectories, Journal of Optimization Theory and Applications 54 (1987) 121–141. [13] A. To¨rn, Cluster analysis using seed points and density determined hyperspheres as an aid to global optimization, IEEE Transactions on Systems, Man, and Cybernetics 7 (1977) 610–616. [14] D.J. Wales, H.A. Scheraga, Global optimization of clusters, crystals and biomolecules, Science 285 (1999) 1368–1372. [15] Z. Xu, H.-x. Huang, P.M. Pardalos, C.-x. Xu, Filled functions for unconstrained global optimization, Journal of Global Optimization 20 (2001) 49–65. [16] L.-y. Zhang, S.-y. Liu, Z.-h. Ge, A revised filled function for the global optimzation of the Lipschitz programming, Collegial Transaction of Computational Mathematics 25 (2) (2003) 153–159 (in Chinese). [17] L.-s. Zhang, N.G. Chi-kong, Duan Li, Wei-wen Tian, A new filled function method for global optimization, Journal of Global Optimization 28 (2004) 17–43. [18] Wen-xing Zhu, Qing-xiang Fu, A sequential convexification method (SCM) for continuous global optimization, Journal of Global Optimization 26 (2003) 167–182.
A new filled function for unconstrained global optimization Xiaoli Wang *, Guobiao Zhou Department of Mathematics, Shanghai Jiao Tong University, Shanghai 200240, PR China
Abstract Seeking global optima of an unconstrained and multi-modal global optimization problem minx2X f ðxÞ by constructing a filled function is concerned in this paper. On the basis of analyzing filled functions presented before, a new filled function is proposed, and it is proved to meet the properties of a filled function. Moreover, solutions of numerical experiments show that the function is quite effective. Ó 2005 Published by Elsevier Inc. Keywords: Filled function; Global optimization; Local minimizer; Global minimizer
1. Introduction Global optimization is mainly concerned with the characters and algorithms on the global minimum of multi-modal nonlinear functions. More and more practical problems in science, economics, engineering and other regions can be formulated as global optimization problems, which cannot be solved effectively with traditional algorithms of nonlinear programming. Lots of *
Corresponding author. E-mail address: [email protected] (X. Wang).
0096-3003/$ - see front matter Ó 2005 Published by Elsevier Inc. doi:10.1016/j.amc.2005.05.009
420
X. Wang, G. Zhou / Appl. Math. Comput. 174 (2006) 419–429
researchers have been attracted to the field of global optimization. As we know, there are two categories of global optimization approaches: probabilistic [1,4,13] and deterministic [2,3,5–12,14–18]. The filled function method studied in this paper is included by the latter. Compared to the other methods, it has an outstanding advantage that its thought is simple while its realization is relatively easy and effective. The filled function method was introduced by Ge in [5]. The thought was constructing an auxiliary function to transfer from a local minimizer to a lower minimizer and so does the iteration until the global minimum is obtained. Since then, many filled functions [5,7,9–11,15,16] have been proposed successively. Just on the basis of those functions, a new filled function is raised in this paper. Compared to those functions, this one is more effective, and although there are two parameters in this function, they are easy to adjust. The numerical experiment results show that the function is quite satisfying. Consider the following global optimization problem: ðP 0 Þ
minn f ðxÞ; x2R
where f : Rn ! R is coercive, i.e. limkxk!þ1 f ðxÞ ! þ1, then there exists a closed bounded box X 2 Rn containing all global minimizers of (P0) in its interior. Then (P0) is equivalent to the following problem: ðP Þ
min f ðxÞ. x2X
Throughout the paper, x* is a global minimizer of f(x), and xi ði P 1Þ is the isolated minimizer of f(x). Moreover, k Æ k is taken as k Æ k2. Some concepts of filled functions are introduced as follows: Definition 1.1. The basin of f(x) at an isolated minimizer x1 is a connected domain B1 which contains x1 , and in which the steepest descent trajectory of f(x) starting form any point converges to x1 , while the descent trajectory of f(x) starting from any initial point x 62 B1 does not converge to x1 . Conversely, if x1 is an isolated maximizer, the basin of f(x) at x1 is defined as the hill of f(x). A basin of f(x) at x2 , B2 , is higher (lower) than B1 , if f ðx2 Þ P f ðx1 Þ (f ðx2 Þ 6 f ðx1 Þ respectively). Definition 1.2. A connected domain S 1 B1 is defined as the simple basin at x1 if the equality ðx x1 ÞT 5 f ðxÞ > 0 holds for any point x 2 S 1 and x 6¼ x1 . A function P(x) is said to be a filled function of f(x) at the local minimizer x1 if it satisfies the following properties: (p1) x1 is a local maximizer of P(x) and the whole basin B1 of f(x) at x1 is a part of the hill of P(x).
X. Wang, G. Zhou / Appl. Math. Comput. 174 (2006) 419–429
421
(p2) P(x) has no minimizers or saddle points in any basin of f(x) higher than B1 . (p3) If f(x) has a basin B2 lower than B1 , then for any point x 2 B2 , there is a point x0 2 B2 that minimizes p(x) on the line through x and x1 . Ge proposed the filled function in [5]: ! x x 2 1 1 exp P ðx; r; qÞ ¼ ; r þ f ðxÞ q2
ð1:1Þ
where r, q are parameters. In (1.1), when the value of kx x1 k is quite large, P(x, r, q) and the exponenkxx1 k2 tial term exp q2 will tend to zero which means the change of P(x, r, q) and $P(x, r, q) will be undistinguished. According to the determining criterion of the algorithm, the inappropriate point will be got when pursuing the stationary point of P(x, r, q), then the global minimizer of f(x) will probably be lost. In addition, whether the function satisfies the ‘‘filling properties’’ mainly depends on the value of parameters. In case that the value is not appropriate, the efficiency of the algorithm will be influenced, and we even cannot find the global minimizer of the problem possibly, which can be shown in Table 1. Table 1 includes the results of numerical experiments solving the 6-hump back function with the algorithm in [1]. Where the initial point is (1.6, 0.9)T, and the initial given value of r þ f ðx1 Þ is 3, and Kf means the total number of evaluations for the objective function, the filled function and their gradients, and NF means that the global minimizer cannot be found. Consequently, putting forward filled functions whose parameters are easy to adjust is an improving method. So filled functions with one parameter were proposed. For example, Li and Sheng [9] proposed the filled function as follows which were extended to solve the Lipschitz programming: h 2 i Qðx; AÞ ¼ f ðxÞ f x1 exp Ax x1 ; ð1:2Þ we can see it still has the same shortcoming as (1.1). Kong and Zhuang [7] proposed the function with the form as follows: 2 ð1:3Þ Qðx; AÞ ¼ ln 1 þ f ðxÞ f x1 exp Ax x1 . Obviously, the above two filled functions still have the same shortcoming that is be affected by the exponential term as (1.1). Considering such a fact that if f(x) > 0, then f(x) and ln f(x) have the same extreme points [15]. Therefore, to Table 1 q2
1
101
102
103
104
105
106
Kf
NF
358
93
80
561
1029
NF
422
X. Wang, G. Zhou / Appl. Math. Comput. 174 (2006) 419–429
tackle the above handicap, we can take logarithm of (1.1), then the effect of the exponential term will be eliminated. Then a new filled function is got: 2 x x1 P 1 ðx; qÞ ¼ ln f ðxÞ f x1 . ð1:4Þ q2 Similarly, a filled function as follows have been proposed in [11]: 2 Qðx; AÞ ¼ A ln 1 þ f ðxÞ f x1 þ x x1 .
ð1:5Þ
Obviously, x 2 X in (1.3) and (1.5) must satisfy f ðxÞ > f ðx1 Þ 1, similarly, x 2 X in (1.4) must satisfy f ðxÞ > f ðx1 Þ, which restricts the application of this function. Liu proposed successively in [10,11] the following filled functions: 1 2 akx x1 k ; arctan½f ðxÞ f ðx1 Þ a 1=m . LðxÞ ¼ p ½f ðxÞ f ðx1 Þ kx x1 k
LðxÞ ¼
ð1:6Þ ð1:7Þ
We can see from the above two forms that they are not continuous functions, just as mentioned in [10,11] that we need to suppose f ðxÞ > f ðx1 Þ. On the basis of the above filled functions, we propose a new improved one that combines the advantages of these functions. If the new filled function has a local minimizer x1, it must belong to a lower basin than the current one B1 . Importantly, if the parameters are chosen appropriately, we can ensure that the minimizer of the filled function x1 2 S 1 ¼ fx 2 Xjf ðxÞ < f ðx1 Þg, thus the new minimum of f(x) will be lower than f ðx1 Þ evidently, which obviously satisfies the property (p3). This paper is organized as four sections. In Section 2, we propose a new filled function, and prove that the function satisfies the filling properties. In Section 3, we give a solution algorithm and make numerical implementation which testifies the effectiveness of the function. Finally, we draw some conclusions in Section 4. 2. A new filled function The new filled function proposed in this paper is as follows: n o 2 W ðx; a; kÞ ¼ a ln max 0; f ðxÞ f1 þ 1 kx x1 k2 n 2 o 1 k max 0; f1 f ðxÞ ;
ð2:1Þ
where f1 is the value of the function f(x) at the present local minimizer x1 , and a, k > 0 are parameters.
X. Wang, G. Zhou / Appl. Math. Comput. 174 (2006) 419–429
423
It can be evidently observed from the form of (2.1) that to ensure the continuance of W(x, a, k), the term of square is given as the above, and to realize that W(x, a, k) has minimizers in the region S1 ¼ fx 2 Xjf ðxÞ < f1 g when the parameters are chosen properly, the last term is added on the basis of (1.5). When f ðxÞ P f1 , h i 2 2 W ðx; a; kÞ ¼ a ln f ðxÞ f1 þ 1 x x1 ; ð2:2Þ and when f ðxÞ < f1 , 2 h 2 i W ðx; a; kÞ ¼ x x 1 k f f ðxÞ . 1
ð2:3Þ
1
In the following part, we will prove that W(x, a, k) really satisfies the properties of a filled function under some conditions of parameters a, k. Theorem 2.1. Assume that x1 is a local minimizer of f(x) and W(x, a, k) is defined as (2.1), then x1 is a local maximizer of W(x, a, k). Proof. For 8x 2 B1 , and x 6¼ x1 , we have f ðxÞ > f1 . Since parameter a > 0, then W ðx; a; kÞ ¼ a ln½ðf ðxÞ f1 Þ2 þ 1 kx x1 k2 2
2
< a ln½ðf1 f1 Þ þ 1 kx1 x1 k ¼ 0 ¼ W ðx1 ; a; kÞ: Thus x1 is a local maximizer of W(x, a, k).
h
Theorem 2.2. Assume that x1 is a local minimizer of f(x), and x is a point different from x1 and satisfies f ðxÞ P f1 . (i) If the given direction d satisfies d T ðx x1 Þ > 0 and d$f(x) P 0 or d T ðx x1 Þ P 0 and dT$f(x) > 0, then for whatever value of a > 0, d is a descent direction of W(x, a, k) at point x. (ii) If the given direction d is a descent direction at point x, that is dT$f(x) < 0, when i h 2 d T x x1 f ðxÞ f1 þ 1 0
ð2:5Þ
424
X. Wang, G. Zhou / Appl. Math. Comput. 174 (2006) 419–429
(i) Since d T ðx x1 Þ > 0 dT$f(x) > 0, we have d T rW ðx; a; kÞ ¼
d$f(x) P 0
and
2aðf ðxÞ f1 Þ ðf ðxÞ f1 Þ2 þ 1
or
d T ðx x1 Þ P 0
and
d T rf ðxÞ 2d T ðx x1 Þ < 0.
(ii) Since dT$f(x) < 0 and condition (2.4) holds, we also obviously have d T rW ðx; a; kÞ < 0; so d is a descent direction of W(x, a, k).
h
Theorem 2.1 and the first half of Theorem 2.2 show that W(x, a, k) satisfies the filling property (p1) of a filled function. Provided the direction d ¼ x x1 and f ðxÞ P f1 , Theorem 2.2 shows that the point x cannot be the stationary point of W(x, a, k) when a always satisfies condition (2.4). That is to say, if (2.4) holds, any point x of B1 or some basin higher than B1 cannot be the stationary point or saddle point, so the function W(x, a, k) satisfies the filling property (p2). The following theorem shows this property of W(x, a, k) in a further degree. Theorem 2.3. Assume that x1 is an isolated local minimizer of f(x), and point x satisfies f ðxÞ P f1 . When 2D ; ð2:6Þ 0
x2X
W(x, a, k) has not any minimizer or saddle point in the region S 2 ¼ fx 2 Xjf ðxÞ P f1 g. Proof. Suppose that ^x is a minimizer or saddle point of f(x) which is different from x1 , and ^x 2 S 2 ¼ fx 2 Xjf ðxÞ P f1 g. Then we have krW ð^x; a; kÞk ¼ 0. That is aðf ð^xÞ f1 Þ ðf ð^xÞ f1 Þ2 þ 1 then a¼
krf ð^xÞk ¼ kx x1 k;
i h ^x x f ð^xÞ f 2 þ 1 1 1 krf ð^xÞkðf ð^xÞ f1 Þ
.
ð2:8Þ
X. Wang, G. Zhou / Appl. Math. Comput. 174 (2006) 419–429
425
It is obvious that ^x 62 S 1 , then k^x x1 k P D > 0. In addition, we have krf ð^xÞk 6 L. Consider the function 2 f ðxÞ f1 þ 1 ; GðxÞ ¼ f ðxÞ f1 when f ðxÞ f1 P 0, it has minimum at the point x satisfying f ðxÞ f1 ¼ 1, that is 2 f ðxÞ f1 þ 1 GðxÞ ¼ P 2; ð2:9Þ f ðxÞ f1 so i h ^x x f ð^xÞ f 2 þ 1 1 1 2D ; ð2:10Þ P a¼ L krf ð^xÞkðf ð^xÞ f1 Þ this obviously contradicts with (2.6). So any minimizer or saddle point of W(x, a, k) cannot belong to the region S 2 ¼ fx 2 Xjf ðxÞ P f1 g. h Theorem 2.3 shows that there is not any stationary point of W(x, a, k) in the region S 2 ¼ fx 2 Xjf ðxÞ P f1 g when a is properly chosen, which further indicates that W(x, a, k) satisfies the property (p2). Theorem 2.4. Assume that x1 is a local minimizer of f(x), and f ðxÞ P f1 , and the given direction d satisfies dT$f(x) < 0 and d T ðx x1 Þ > 0. When i h 2 d T x x1 f ðxÞ f1 þ 1 a> ; ð2:11Þ ðd T rf ðxÞÞðf ðxÞ f1 Þ d is an ascent direction of W(X, a, k) at point X. 2aðf ðxÞf Þ
Proof. When (2.11) holds, d T rW ðx;a;kÞ¼ ðf ðxÞf Þ21þ1 d T rf ðxÞ2d T ðxx1 Þ>0, 1
then d is an ascent direction of W(x, a, k) at point x. h Theorem 2.5. Assume that x1 is a local minimizer of f(x), and x2 is another local minimizer satisfying f ðx2 Þ < f1 . There exists r > 0, so that for 8x 2 N ðx2 ; rÞ, when a, k respectively satisfy 2D 1 ; k > 2; 0
426
X. Wang, G. Zhou / Appl. Math. Comput. 174 (2006) 419–429 2
2
2
W ðx; a; kÞ ¼ kx x1 k ½1 kðf1 f ðxÞÞ P kx x1 k ð1 ke2 Þ > 0; and while W ðx1 ; a; kÞ ¼ 0. According to Theorems 2.2 and 2.3, when 0 < a < 2D L f ðxÞ P f1 , the trajectory starting from x1 is descendent, then there is a point x0 2 B2 certainly which minimizes W(x, a, k) on the line through x1 and x. h Theorems 2.4 and 2.5 indicate that W(x, a, k) satisfies the property (p3) of a filled function when the parameters a, k are chosen appropriately. So we have proved that W(x, a, k) is a filled function. In fact, the parameters a, k are easy to adjust, and if they are adjusted properly, as long as W(x, a, k) has local minimizers, they must be in the region S 1 ¼ fx 2 Xjf ðxÞ < f1 g. 3. Solution algorithm and numerical experiments 3.1. Solution algorithm In this section, we present an algorithm as follows to solve the problem (P). In the algorithm, d > 0, a > 0, 0 < e < 1, and the large enough positive integer Nm are all given constants. Step 0: Set a :¼ 1, k :¼ 105, i :¼ 1, N :¼ 1. Step 1: Starting from an initial point x0, minimize f(x) and obtain the first local minimizer x1 and the value f1 . Step 2: Construct the filled function: W ðx; a; kÞ ¼ a lnf1 þ ½maxð0; f ðxÞ f1 Þ2 g 2
2
kx x1 k f1 k½maxð0; f1 f ðxÞÞ g. Step 3: If i 6 2n, starting from the point x1 :¼ x1 þ dd i , where di is a randomly selected direction whose norm are less than 1, minimize W(x, a, k) and obtain the minimizer x2, turn to Step 4; otherwise, set N :¼ N + 1, turn to Step 6. Step 4: If x2 2 X turn to Step 5; otherwise, set i = i + 1, and turn to Step 3. Step 5: Minimize the objective function f(x) starting from x2, and find a local minimizer x2 . If f ðx2 Þ < f1 . Set x1 :¼ x2 and f1 :¼ f ðx2 Þ, then turn to Step 2; otherwise, turn to Step 6. Step 6: If N 6 Nm, then if a > a, set a :¼ e Æ a, i :¼ 1; otherwise, set a :¼ 0, and turn to Step 3; otherwise, stop the algorithm and take x1 as the global minimizer of the problem (P). According to the feature of W(x, a, k), when the parameter k is large enough and a is appropriately small, W(x, a, k) must be an effective filled function. So in the implement, we can firstly give the parameter k a large enough value, then
X. Wang, G. Zhou / Appl. Math. Comput. 174 (2006) 419–429
427
we only need to adjust the parameter a, just as the above algorithm shows. In addition, since the initial point x1 is randomly chosen around x1 when minimizing the filled function W(x, a, k), if we solve the problem (P) using this algorithm, we may get different global minimizers for the same initial point x0. Thus if we have numerical experiments for proper times, we can get all the global minimizers of (P) just as the examples shown as follows.
3.2. Numerical experiments and results The results of numerical experiments show that the above solution algorithm is effective for solving multi-modal problems like (P). Take the following problems as examples, after running the algorithm procedure for proper times, all the global minimizers of these problems can be found successfully. (1) 6-hump back camel function: f ðxÞ ¼ 4x21 2.1x41 þ x61 =3 x1 x2 4x22 þ 4x42 ;
3 6 x1 ; x2 6 3.
Initial point x0: (1.6, 0.9)T. Two global minimizers x*: (0.0898, 0.7127)T, (0.0898, 0.7127)T. Global minimum: f * = 1.0316. (2) Goldstein and Price function: f ðxÞ ¼ ½1 þ ðx1 þ x2 þ 1Þ2 ð19 14x1 þ 3x21 14x2 þ 6x1 x2 þ 3x22 Þ 2
½30 þ ð2x1 3x2 Þ ð18 32x1 þ 12x21 þ 48x2 36x1 x2 þ 27x22 Þ; 3 6 x1 ; x2 6 3. Initial point x0: (1.6, 0.9)T. One global minimizer x*: (0.0000, 1.0000)T. Global minimum: f * = 3.0000. (3) Two-dimensional Shubert function: ( )( ) 5 5 X X f ðxÞ ¼ i cos½ði þ 1Þx1 þ i i cos½ði þ 1Þx2 þ i ; i¼1
i¼1
10 6 x1 ; x2 6 10. Initial point x0: (1, 1)T. Eighteen global minimizer x*: (0.8003, 1.4251)T, (0.8003, 7.7083)T, (0.8003, 4.8581)T, (4.8581, 5.4829)T, (4.8581, 0.8003)T, (4.8581, 7.0835)T, (5.4829, 7.7083)T, (5.4829, 4.8581)T, (5.4829, 1.4251)T,
428
X. Wang, G. Zhou / Appl. Math. Comput. 174 (2006) 419–429
(7.0835, 7.7083)T, (7.0835, 1.4251)T, (7.0835, 4.8581)T, (7.7083, 5.4829)T, (7.7083, 0.8003)T, (7.7083, 7.0835)T, (1.4251, 0.8003)T, (1.4251, 7.0835)T, (1.4251, 5.4829)T. Global minimum: f * = 186.7309. (4) n-dimension function: ( ) n1 X n 2 2 2 2 10sin ðpx1 Þ þ f ðxÞ ¼ ½ðxi 1Þ ð1 þ 10sin ðpxiþ1 Þ þ ðxn 1Þ ; p i¼1 10 6 xi 6 10. n = 3: Initial point x0: (2, 3, 5.1)T. One global minimizer x*: (1.0000, 1.0000, 1.0000). Global minimum f * = 6.0873e010. n = 6: Initial point x0 (2, 3, 5.1, 6, 4, 3)T. One global minimizer x*: (1.0000, 0.9993, 1.0002, 1.0003, 1.0027, 1.0001)T. Global minimum: f * = 2.6426e009. n = 10: Initial point x0: (8.9, 5.4, 4.7, 7.3, 9.8, 0, 2.6, 6, 1, 0.5)T. One global minimizer x*: (1.0000, 0.9956, 0.9932, 1.0027, 0.9988, 1.0012, 0.9996, 1.0009, 1.0020, 0.9990)T. Global minimum: f * = 1.5992e008.
4. Conclusions The filled function method is to seek the global minimum of some kinds of problems such as (P). The basic thought is leaving from a local minimizer to a lower one by an constructed auxiliary function. The filled function proposed in this paper improves the defect in [5] that the global minimizers may be lost because of the effect of the exponential term, and the parameters of this one are easier to adjust. Especially, if the parameters are properly chosen, provided the filled function has any local minimizer x1, x1 must belong to the region S 1 ¼ fx 2 Xjf ðxÞ < f ðx1 Þg, so we can obtain the minimizer xi , and the value of f(x) at xi must be lower than the present one. So it unquestionably further strengthen the ‘‘filling performance’’ of the filled function. Numerical experiments also show that the filled function W(x, a, k) is effective. We should point out that the filled function proposed in this paper is also be applicable to solve the global optimization problems in nonsmooth programming.
X. Wang, G. Zhou / Appl. Math. Comput. 174 (2006) 419–429
429
References [1] N. Baba, Global optimization of functions by the random optimization method, International Journal of Control 30 (1979) 1061–1065. [2] P. Basso, Iterative methods for the localization of the global maximum, SIAM Journal on Numerical Analysis 19 (1982) 781–792. [3] F. Branin, Widely convergent methods for finding multiple solutions of simultaneous nonlinear equations, IBM Journal of Research Developments 16 (1972) 504–522. [4] C. Dorea, Limiting distribution for random optimization methods, SIAM Journal on Control and Optimization 24 (1986) 76–82. [5] R.P. Ge, A filled function method for finding a global minimizer of a function of several variables, Mathematical Programming 46 (1990) 191–204. [6] R.P. Ge, Y.F. Qin, A class of filled functions for finding global minimizers of a function of several variables, Journal of Optimization Theory and Applications 54 (1987) 241–252. [7] M. Kong, J.-n. Zhuang, A class of new filled functions for pursuing the global minima of a non-smooth function with multi variables, Collegial Transaction of Computational mathematics 18 (2) (1996) 165–174 (in Chinese). [8] A.V. Levy, A. Montalvo, The tunneling algorithm for the global minimization of functions, SIAM Journal on Scientific and Statistical Computing 6 (1985) 15–29. [9] B.-x. Li, Y. Sheng, Filled functions method for the Lipschitz programming, Systems Mathematics and Science 11 (4) (1991) 346–348 (in Chinese). [10] X. Liu, A class of generalized filled functions with improved computability, Journal of Computational and Applied Mathematics 137 (2001) 61–69. [11] X. Liu, Several filled functions with mitigators, Applied Mathematics and Computation 133 (2002) 375–387. [12] J. Snyman, L. Fatti, A multi-start global minimization algorithm with dynamic search trajectories, Journal of Optimization Theory and Applications 54 (1987) 121–141. [13] A. To¨rn, Cluster analysis using seed points and density determined hyperspheres as an aid to global optimization, IEEE Transactions on Systems, Man, and Cybernetics 7 (1977) 610–616. [14] D.J. Wales, H.A. Scheraga, Global optimization of clusters, crystals and biomolecules, Science 285 (1999) 1368–1372. [15] Z. Xu, H.-x. Huang, P.M. Pardalos, C.-x. Xu, Filled functions for unconstrained global optimization, Journal of Global Optimization 20 (2001) 49–65. [16] L.-y. Zhang, S.-y. Liu, Z.-h. Ge, A revised filled function for the global optimzation of the Lipschitz programming, Collegial Transaction of Computational Mathematics 25 (2) (2003) 153–159 (in Chinese). [17] L.-s. Zhang, N.G. Chi-kong, Duan Li, Wei-wen Tian, A new filled function method for global optimization, Journal of Global Optimization 28 (2004) 17–43. [18] Wen-xing Zhu, Qing-xiang Fu, A sequential convexification method (SCM) for continuous global optimization, Journal of Global Optimization 26 (2003) 167–182.