- Email: [email protected]
Cloud theory-based simulated annealing algorithm and application
ARTICLE IN PRESS Engineering Applications of Artificial Intelligence 22 (2009) 742–749
Contents lists available at ScienceDirect
Engineering Applications of Artificial Intelligence journal homepage: www.elsevier.com/locate/engappai
Cloud theory-based simulated annealing algorithm and application Pin Lv a,b,, Lin Yuan a,b, Jinfang Zhang a a
National Key Laboratory of Integrated Information System Technology, South No. 4 Street, Zhong Guancun, Institute of Software, Chinese Academy of Sciences, Beijing 100190, PR China Graduate University, Chinese Academy of Sciences, Beijing 100049, PR China
b
a r t i c l e in f o
a b s t r a c t
Article history: Received 31 March 2008 Received in revised form 6 June 2008 Accepted 4 March 2009 Available online 23 April 2009
Using the randomness and stable tendency of a Y condition normal cloud generator, a cloud theorybased simulated annealing algorithm (CSA) is originally proposed, whose characteristic is approximately continuous decrease in temperature and implied ‘‘Backfire & Re-Annealing’’. It fits the annealing process of solid matter in nature much better, overcomes the traditional simulated annealing algorithm (SA)’s disadvantages, which are slow searching speed and being trapped by local minimum easily, then enhances the veracity of final solution and reduces the time cost of the optimization process simultaneously. Theory analysis proves that CSA is convergent and typical function optimization experiments show that CSA is superior to SA in terms of convergence speed, searching ability and robustness. The result of the application using CSA for multiple observers sitting problem (MOST) in visibility-based terrain reasoning (VBTR) also declares the new algorithm’s usefulness and effectiveness adequately. & 2009 Elsevier Ltd. All rights reserved.
Keywords: Cloud theory Improved simulated annealing algorithm Backfire & re-annealing Function optimization
1. Introduction Simulated annealing algorithm (SA) is a random optimization method that is based on Monte Carlo iterative strategy. It comes from a strong analogy between combinatorial optimization and the physical process of crystallization (Laarhoven and Aarts, 1987). From a high initial temperature, SA searches the solution space randomly using metropolis criterion, and escapes from local minimum through the acceptance of ‘‘bad solution’’ with a certain probability. As the temperature descends gradually, it repeats the process above and finds the global optimization solution finally. As an effective non-linear combinatorial optimization method, SA has been proved to be strict in theory and useful in many application fields (Anderson and McGeehan, 1994; Jeroen and Gerard, 2002; Kim et al., 2004). SA simulates the physical annealing process from theory aspect; nevertheless, the temperature of each step is discrete and unchangeable in the annealing course from implementation aspect, which does not meet the requirement of continuous decrease in temperature in actual physical annealing processes. On the one hand, the algorithm is easy to accept deteriorate solution with high temperature and does not converge quickly; on Corresponding author at: National Key Laboratory of Integrated Information System Technology, South No. 4 Street, Zhong Guancun, Institute of Software, Chinese Academy of Sciences, Beijing 100190, PR China. Tel.: +86 10 62614140 8217; fax: +86 10 62558764. E-mail address: [email protected] (P. Lv).
0952-1976/$ - see front matter & 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.engappai.2009.03.003
the other hand, it is hard to escape from local minimum trap with low temperature and has low searching accuracy. Therefore, it is necessary to find a new implement method of annealing mechanism that measures up the physical rules better, and makes the improved method have high searching accuracy and great converging ability. Cloud theory is the innovation and development of membership function in fuzzy theory (Deyi and Yi, 2005), and is a model of the uncertainty transformation between quantitative representation and qualitative concept using language value (Deyi et al., 1995). It is successfully used in intelligence control (Deyi et al., 1998; Feizhou et al., 1999), knowledge representation (Deyi et al., 2000; Cheng et al., 2005), data mining (Kaichang et al., 1998; Di et al., 1999; Shuliang et al., 2003; Yingjun and Zhongying, 2004), spatial analysis (Cheng et al., 2006; Haijun and Yu, 2007), target recognition (Fang et al., 2007), intelligent algorithm improvement (Yunfang et al., 2005) and so on. The physical annealing process is like a fuzzy system in which the molecules move from large scale to small scale randomly as the temperature descends. It is easy to describe this process in natural language but difficult to simulate it using computer, which is the real reason for the problems of SA’s searching accuracy and speed above. Nevertheless, because the cloud theory can realize the transformation between a qualitative concept described in words and its numerical representation (Deyi et al., 1995), it can be used to guide the implementation of SA so as to avoid those problems. In this paper, after introducing the preliminaries of cloud theory in Section 2, we propose a cloud theory-based simulated
ARTICLE IN PRESS P. Lv et al. / Engineering Applications of Artificial Intelligence 22 (2009) 742–749
annealing algorithm (CSA) in Section 3. The innovations of CSA are new temperature annealing process and new state acceptation process according to the cloud drop’s randomness and stable tendency in cloud theory and using Y condition normal cloud generator to produce nearly continuous annealing temperature in each step. Section 4 proves that the CSA is convergent in theory. Furthermore, Sections 5 and 6 are the illustrations of using CSA for some function optimizing experiments and a typical application. A conclusion in Section 7 based on those results in the two sections above shows that the CSA can not only make suitable between quantitative implementation of annealing process and qualitative concept in physical annealing theory, but also have ‘‘Backfire & Re-Annealing’’ process impliedly and it can enhance the searching veracity and reduce the time cost of optimization process evidently at the same time.
2. Preliminaries Cloud model is a model that contains the transferring procedure of uncertainty between qualitative concept and quantitative data representation by using natural language (Deyi et al., 1995). And it mainly reflects the fuzziness and randomness of the concept within the affair and human knowledge in the objective world (Di et al., 1999). 2.1. Basic concepts of cloud theory Suppose that T is the language value of domain u, and mapping C T ðxÞ : u ! ½0; 1; 8x 2 u; x ! C T ðxÞ, then the distribution of CT(x) in u is called membership cloud of T, or cloud for short. If the distribution of CT(x) is normal, it is named normal cloud model. It is a random number set that obeys normal distributive rule and has stable tendency, and it is determined by expectation Ex, entropy En and super entropy He (Deyi et al., 1995). They reflect the quantitative characteristics of concept CT(x). As illustrated in Fig. 1, Ex determines the center of the cloud, En determines the range of the cloud and, according to ‘‘3En’’ rule (Deyi et al., 1995), about 99.74% of the total cloud drops distribute between [Ex3En,Ex+3En]; He determines the cloud drops’ dispersive degree, which means the larger the He is the more dispersively the cloud drops locate. 2.2. Y condition cloud generator If the three digital characteristics (Ex,En,He) and a certain u0 are given, then a drop of cloud drop(xi,u0) can be generated by a
743
generator, which is called Y condition cloud generator (Deyi and Yi, 2005): INPUT:{Ex,En,He},n,u0 OUTPUT:{(x1,u0),y,(xn,u0)} FOR i ¼ 1 to n En0 ¼ randn(En,He) pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi xi ¼ Ex En0 2 lnðu0 Þ drop(xi,u0) where randn(En,He) produces random number with normal distribution whose expectation is En and standard deviation is He.
3. Cloud theory-based simulated annealing algorithm The cloud-based simulated annealing algorithm follows the Metropolis rule, and uses the Y condition normal cloud generator to produce nearly continuous annealing temperature, which influences the state generating and accepting process, and then complies with the physical laws better. Because the normal cloud model has the characteristics of randomness and stable tendency (Deyi et al., 1995), the random change of annealing temperature after importing the cloud theory can preserve diversity of the searched individual in order to avoid being trapped by local minimum. Moreover, the stable tendency of annealing temperature can protect the excellent individual effectively and fix the global minimum’s position quickly as well. 3.1. Algorithm flow Firstly, unify SA’s initial temperature. Commonly, the initial temperature is proportional to the difference degree of state values in SA (Laarhoven and Aarts, 1987). The method of this algorithm is as follows. Generate a group of states, and get their standard deviation, which is recorded as STD, then the initial normalization temperature is t 0 ¼ 1:0 arc cotðSTD=pÞ. Let tq be the quit temperature and s0 be the initial state. The new algorithm is as follows: Algorithm: the Cloud theory-based Simulated Annealing Algorithm 1: t ¼ t0, s ¼ s0, k ¼ 0; 2: Repeat 3: repeat 4: /* Generate the annealing temperature tk0 of the corresponding state, which is based on 5: * Y condition cloud generator. */ 6: He ¼ tk, En ¼ tk, u0 ¼ 1.0tk; 7: En0 ¼ (En+He-rand(0,1)/3); 8: /* rand(a,b) generates random numbers that equally distribute between a and b */ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 9: t 0 ¼ En0 2 lnðu0 Þ; k
10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20:
Fig. 1. Three digital characteristics of a normal cloud.
sj ¼ si+tk0 rand(1,1)Max; /* Max is the maximal difference of every two states’ value */ if min 1; expððCðsi Þ Cðsj ÞÞ=t 0k Xrandð0; 1Þ then s ¼ sj; else s ¼ si; end if until the Metropolis sampling stable criteria is satisfied; k ¼ k+1, tk ¼ t0 lk; until tk is smaller than the quit temperature tq; output the final result.
Because the reference temperature tkX0, then let Ex ¼ 0 in the new algorithm, which can satisfy the Y condition cloud generator’s requirement for the range of discussion domain u (Deyi and Yi, 2005). At the high-temperature stage, if the changeable range of annealing temperature is much wider and more
ARTICLE IN PRESS 744
P. Lv et al. / Engineering Applications of Artificial Intelligence 22 (2009) 742–749
dispersive, it can ensure the randomness of the annealing process adequately. Meanwhile, if that range is much narrower and less dispersive at the low-temperature stage, it can guarantee the stable tendency of the annealing process. In order to satisfy the requirement above, the entropy En and the super entropy He should be positively correlated with the temperature stage, since En determines the range of the cloud and the super entropy He determines the cloud drops’ dispersive degree in cloud theory (Deyi et al., 1995). So in the new algorithm, they are both equal to reference temperature tk. 3.2. New temperature annealing process The temperature update function of the simulated annealing algorithm is exponential in general (Aarts and Korst, 1989). Suppose the initial temperature is t0 and the annealing index is l, then the temperature of the No. k annealing step is tk ¼ t0 l
k
k ¼ 1; 2; 3; . . . ;
0olo1
(1)
Although the annealing temperature with this type of temperature update function is gradually falling, the temperature is a fixed value in every annealing step and the changing process of temperature between two neighbor steps is not continuous. In other words, the temperature does not change although the states update frequently during each annealing step. This phenomenon appears to be the same as other types of temperature update functions such as arithmetical, geometrical or logarithmic one (Anderson and McGeehan, 1994). Apparently, it does not accord with the physical annealing process of the solid material. Therefore, this temperature generation process does not grasp SA’s essentials. Using a Y condition cloud generator and taking a certain value as a reference, it can randomly generate a group of new values that distribute around the given value like ‘‘cloud’’. If we introduce the Y condition cloud generator to the temperature generation process and let the fixed temperature point of each step become a changeable temperature zone in which the temperature of each state generation in every annealing step is chosen randomly, the course of temperature changing in the whole annealing process is nearly continuous and fits the physical annealing process better. Fig. 2 is the comparison of annealing temperature with each state between CSA and SA when states update 800 times. This figure clearly shows that CSA can preserve the characteristic of gradual
Fig. 3. Comparison of backfire & re-annealing process for two algorithms, l ¼ 0.9.
decrease in annealing temperature and convergence of SA. Meanwhile, the changeable bound of annealing temperature of CSA is far wider than that of SA, and then CSA overcomes the disadvantage of SA whose temperature is discrete and simple during each annealing step; hence, the annealing process of CSA is consistent with the physical annealing process. In order to improve the performance of SA, Ingber proposed a very fast simulated annealing algorithm (VFSA) in which the temperature can raise apparently during the annealing process under certain circumstance, which is called ‘‘Backfire & ReAnnealing’’, which means the annealing temperature can rise back to a high stage and then the annealing process can be repeated, and this strategy had been testified effectively (Ingber, 1989). With the same effect as VFSA has, the random change in temperature in each annealing step of CSA implies the ‘‘Backfire & Re-Annealing’’ process in each step, and the overlap of temperature range between two neighbor annealing steps also implies the ‘‘Backfire & Re-Annealing’’ process among multi-steps. Fig. 3 shows the ‘‘Backfire & Re-Annealing’’ process within one step and multi-step. In contrast, the SA does not have those peculiarities. Therefore, through the introduction of the Y condition cloud generator, the new annealing temperature generation process of CSA combines the temperature’s continuous decrease with ‘‘Backfire & Re-Annealing’’ skillfully, and improves the new algorithm’s ability of jumping from local minimum trap further.
3.3. New state acceptation process
Fig. 2. Comparison of temperature annealing process for two algorithms, l ¼ 0.9.
One of the simulated annealing algorithm’s keys is the new state acceptation process (Anderson and McGeehan, 1994). Because the temperature of each annealing step is fixed for SA, it is easy to accept the deteriorative state and does not converge quickly at the low-temperature stage; on the contrary, it is hard to escape from local minimum trap and then decrease the searching accuracy. To solve these problems, the annealing temperature’s stable tendency of CSA can make the new algorithm satisfy the basic requirement that it is increasingly harder to accept the bad solution as the temperature’s descending on one hand. On the other hand, the continuous change in temperature and ‘‘Backfire & Re-Annealing’’ assures that the new algorithm can escape from local minimum trap easily and has faster convergence speed. Fig. 4
ARTICLE IN PRESS P. Lv et al. / Engineering Applications of Artificial Intelligence 22 (2009) 742–749
745
When C(j)4C(i), Z ai;j ðTÞ ¼ Pr acceptði; j; xÞjt ¼ x d PrðtpxÞ Z ¼ f t ðx; TÞeðCðiÞCðiÞ=xÞ dx
(4)
The equation above uses all probability formula. Suppose the probability density function of random variable t is ft(t,T), and accept(i,j,t) is the Metropolis acceptation function of SA under temperature t (Metropolis et al., 1953): acceptði; j; tÞ ¼ eðCðjÞCðiÞÞ=t
(5)
Therefore, the acceptation function ai,j(T) can be summarized as Z (6) ai;j ðTÞ ¼ min 1; f t ðx; TÞeðCðiÞCðiÞ=xÞ dx
Fig. 4. Comparison of acceptance probability of deteriorative solution for two algorithms, d ¼ 0.9.
is the comparison of acceptance probability with deteriorative solution for two algorithms in each annealing step. The figure shows that the CSA is much easier to accept the bad state in the front half annealing phase (the searching range of the new state is wide, corresponding to the high temperature) so as to expand the distribution scope of the new solution and avoid being trapped by local optimal solution effectively. But in the back half annealing phase (the searching range of the new state is narrow, corresponding to the low temperature), it is much harder for CSA to accept deteriorative solution, thereby reducing the invalid searching times and then improving the convergence speed.
Compared with the classical SA, whose acceptation function is totally determined by DC and t, this new algorithm introduces an extra uncertain factor – a distribution function of acceptance under a given temperature. This new uncertainty, which allows the search to escape from local optimal states more easily especially under low temperature, leads to the algorithm’s distinctive characteristic of ‘‘backfire and reannealing’’. However, the main structure of this algorithm still conforms to that of the classical SA, and thus, as is analyzed below, the new feature will not inflict extra problems in the convergence. In this algorithm, t is generated by a random variable U, which has uniform distribution according to Section 3.1: En þ He randð0; 1Þ T þ T randð0; 1Þ ¼ 3 3 pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi t ¼ U 2 lnðu0 Þ ¼ U 2 lnð1 TÞ U¼
(7) (8)
Thus, the probability density function of this random variable t is
4. Analysis of CSA’s convergence
f t ðt; TÞ ¼
Theorem 1. The CSA is corresponding to a Markov chain. Proof. The Markov property of CSA is related to its state generator function: X nþ1 ¼ X n þ t randð0; 1Þ Max
(2)
According to the state generating method of CSA, the generation of next state Xn+1 is only correlative with current state Xn. Therefore, this state transformation has Markov property. Moreover, for each temperature T, the temperature decrease in CSA occurs only when the solutions does not change anymore, so this algorithm is time homogeneous (Athreya et al., 1996). The transferring probability for the next annealing step under the temperature T of this Markov chain is 8 j 2 Di and jai g i;j ðTÞai;j ðTÞ; > > P < pi;k ðTÞ; j ¼ i 1 (3) pi;j ðTÞ ¼ k2Di > > : 0; otherwise gi,j(T) is the probability of the state transferring from i to j under temperature T, and ai,j(T) is the acceptation function under this temperature, Di is the neighbor domain of state i. According to the analysis of CSA in Section 3.1, the acceptation function ai,j(T) can be presented as When C(j)pC(i), then ai,j(T) ¼ 1
8 3T < pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi; :
2 lnð1TÞ
0;
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
2 lnð1TÞ 2 ptp 3T
otherwise
2 lnð1TÞ 3T
(9)
&
Theorem 2. The state space of CSA’s Markov chain is limited. Proof. The scale of state space is determined by actual problem. However, this requirement is satisfied for most combinational optimization problem. Furthermore, for the problem of continuous function optimization whose state space is limitless, because of the precision restriction of real number in the computer, it also considers that the state space under this circumstance is limited only if the independent variable has limited definition domain. & Lemma 1. All states of the Markov chain whose state is limited and unreduced have natural returning (Karin and Taylor, 1975). Theorem 3. This state-limited Markov chain cannot be reduced. Proof. According to the neighbor selection method and state generation function, it is clear that if only the feasible domain has strong connectivity, 8i; j 2 O, ( s0 ; s1 ; . . . ; sn 2 O; s0 ¼ i; sn ¼ j and since every state is not the same as each other, it assures that gsk,sk+140,k ¼ 0,1, y, n1. Moreover, because 8i; j 2 O; ai;j ðTÞ40, ðTÞXps0 ;s1 ðTÞps1 ;s2 ðTÞ . . . psn1 ;sn ðTÞ pðnÞ i;j ¼ g s0 ;s1 as0 ;s1 g s1 ;s2 as1 ;s2 . . . g sn1 ;sn asn1 ;sn 40
(10)
ARTICLE IN PRESS 746
P. Lv et al. / Engineering Applications of Artificial Intelligence 22 (2009) 742–749
the arbitrary state of this Markov chain is connectivity to each other. Therefore, this Markov chain cannot be reduced. & Theorem 4. This state-limited Markov chain is non-periodic. Proof. Because this state-limited Markov chain cannot be reduced, all states of this chain are equivalent and have the same period. Consider the state acceptation function ai,j(T), When C(j)pC(i), then ai,jp (T)ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ¼1 ffi When C(j)4C(i), pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi let a ¼ 2 lnð1 TÞ=3T; b ¼ 2 2 lnð1 TÞ=3T
i¼1
t¼a
(11)
If 9i 2 O; j 2 Di , which are satisfied with 0oai,j(T)o1, consider the transferring probability of i X pi;k ðTÞ pi;i ðTÞ ¼ 1 k2O;kai
¼1
X
g i;k ai;k
k2O;kai
¼1
X
g i;k ai;k g i;j ai;j
k2O;kai;kaj
41
X
g i;k g i;j
k2O;kai;kaj
¼1
X
g i;k
k2O;kai
¼0
Using 7 typical functions to conduct the function optimization experiment, we compare the performance of the CSA to that of SA. The experiment is realized on a PC, which has P4 3.0 GHz CPU and 1 G memory using Matlab platform: F1: Sphere model function P P F1 ¼ 5i¼1 x2i ; xi p30F1 ¼ 5i¼1 x2i , F2: Hyper-ellipsoid function P 2 2 xi p1 F2 ¼ 10 i¼1 i xi ; F3: Generalized Rastrigin’s function P 2 xi p2:56 F3 ¼ 30 i¼1 xi 10 cosð2pxi Þ þ 10 ; F4: Branin function
2
2 5 F4 ¼ x2 45:1 þ 10 1 81p cos x1 þ 10; xi p5 p2 x1 þ p x1 6 F5: Griewangk’s function
30 30 Q P 1 F5 ¼ 4000 x2i cos pxiffii þ 1;
0oai;j ðTÞ Z ¼ f t ðx; TÞeðCðjÞCðiÞÞ=x dx Z p f t ðx; TÞ eðCðjÞCðiÞÞ=x dx Z o f t ðx; TÞ dx Z 3T dt pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ¼ 2 lnð1 TÞ t¼b 3Tt ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2 lnð1 TÞ ¼1
5. Typical function optimization
(12)
Thus the state i is non-periodic. And also because all states have the same period, this Markov chain is non-periodic. Contrarily, for 8i 2 O; j 2 Di, condition 0oai,j(T)o1 is not tenable; consequently, it has ai,j(T) ¼ 1. It shows that 8i; j 2 O; CðiÞ ¼ CðjÞ, all states have the same cost value; hence this optimization problem is ordinary. Lemma 2. The state-limited, unreduced and non-periodic Markov chain has unique smooth distribution (Karin and Taylor, 1975). Lemma 3. The sufficient and necessary condition of natural returning for a chain, which is non-periodic and unreduced, is that it has smooth distribution, and this distribution is its ultimate distribution (Chung, 1967). Theorem 5. The Markov chain of CSA is convergent. Proof. Theorems 1 and 2 prove that the CSA corresponds to a state-limited and unreduced Markov chain. According to Lemma 1, all states of this Markov chain are natural returning. Theorems 3 and 4 prove that this Markov chain is unreduced and nonperiodic. According to Lemma 2 and 3, this Markov chain is convergent, which means the ultimate distribution of this chain is equal to its smooth distribution. &
xi p256
i¼1
F6: DeJong’s f2 function F6 ¼ 100ðx21 x2 Þ2 þ ð1 x1 Þ2 ; xi p2:56 F7: Six hump camel back function F7 ¼ 4x21 2:1x41 þ x61 =3 þ x1 x2 4x22 þ 4x42 ; xi p5 Among those functions, F1, F2 and F3 are continuous singlepeak functions, and it means that they have only one best solution. F4 is continuous multi-peaks function, which means that it has several best solutions. All those four functions are used to test the searching ability of the two algorithms; F5 and F6 are non-linear multi-minimums function. p The former one’s local ffi minimums locate at xi ¼ k p i; i ¼ 1; 2; . . . ; 256; k ¼ 1; 2; . . . ; 256 and the latter one has many minimums near x1 ¼ x2 ¼ 1. In addition, their function values do not change linearly with change in xi, so they are used to test the ability of escaping from local minimum and quick converging of the two algorithms. F7 is a special non-linear multi-minimum function. Its function values are not only small but also close to each other. Moreover, it has infinite local minimums, which can easily trap the searching. It has two optimal solutions, which are F7(0.08983, 0.7126) ¼ F7(0.08983, 0.7126) ¼ 1.0316285. Because it is hard to find the optimal solutions, it is used to compare the integrated ability of the two algorithms. For different functions, repeat the experiment using two algorithms 100 times separately. In each experiment, the initial temperature t0, initial state s0, and convergence temperature tq are all same, and the annealing index l is 0.99 for both algorithms too. From Table 1, for F1–F4, irrespective of the adjacency degree to the reference solution, the time cost or the stableness of optimization process, the outcomes of CSA are all better than SA. It shows that the CSA has stronger searching ability. Fig. 5 is the comparison of state generation times (vertical axis) in each annealing step (horizontal axis) comparison for F5 (Fig. 5a) and F6 (Fig. 5b) using CSA and SA, respectively. In Fig. 5a, the state generation times increase close to 40 000 in the no. 450 annealing step by using SA, which means it is trapped by a local minimum. Nevertheless, this trouble does not appear when using CSA whose state generation times are all below 60 during all annealing steps. Similarly, the phenomenon occurs in Fig. 5b when the annealing process approaches the end. From this figure, it is clear to see that whether in the front half annealing phase (Fig. 5a) or back half annealing phase (Fig. 5b), CSA can escape from local minimum quickly by using less state generation times, and then increase the convergence speed. The conclusion is also testified by the experiment data in Table 1. Fig. 6 is the comparison of a certain optimization process for two algorithms with F7. In this figure, the vertical axis represents the solution’s x1 value and the horizontal axis represents the solution’s x2 value. The deeper the color is, the smaller the function value is. The circle in the figure shows the position of two
ARTICLE IN PRESS P. Lv et al. / Engineering Applications of Artificial Intelligence 22 (2009) 742–749
747
Table 1 Performance comparison for two algorithms (time cost unit: second). F
Ref. value
Best value
F1
0
7.9069E03 3.0499E04
2.6564E02 9.9594E04
1.5946E02 5.6471E04
4.5563E03 1.2198E04
1.375 0.203
2.531 0.219
F2
0
1.7026E03 2.0351E05
1.2903E02 3.6407E04
6.0582E03 1.3301E04
2.5875E03 7.9466E05
7.359 0.234
F3
0
9.3298E03 1.6092E04
4.1971E02 5.8142E04
1.7330E02 3.7062E04
5.0919E03 9.8637E05
F4
3.9780 E01
3.9852E01 3.9789E01
4.0654E01 3.9792E01
4.0034E01 3.9790E01
F5
0
2.0957E03 4.0275E04
1.4686E01 3.5222E02
F6
0
6.6253E06 5.9961E07
F7
1.0316 E+00
1.0310E+00 1.0316E+00
Worst value
Average value
Std of all value
Min time cost
Max time cost
Average time cost
Std of time cost
Algorithm
1.847 0.208
0.201 0.007
SA CSA
9.641 0.266
8.229 0.241
0.568 0.008
SA CSA
7.953 0.078
17.797 0.110
12.285 0.094
2.354 0.008
SA CSA
1.5495E03 1.8587E05
62.985 0.500
79.781 0.532
70.608 0.526
3.369 0.008
SA CSA
4.3497E02 1.0646E02
2.7675E02 9.3398E03
3.938 0.609
21.516 0.672
14.113 0.626
4.223 0.012
SA CSA
3.8965E03 1.0014E04
8.8630E04 1.7697E05
9.8820E04 2.0205E05
1.296 0.203
1.344 0.219
1.315 0.207
0.014 0.007
SA CSA
1.0255E+00 1.0315E+00
1.0290E+00 1.0316E+00
1.4716E03 1.8389E05
99.688 0.390
130.656 0.437
112.442 0.407
6.727 0.012
SA CSA
Fig. 6. Comparison of a certain optimization process for two algorithms.
Fig. 5. Comparison of escaping from local extremism for two algorithms.
reference solutions, ‘‘ ’’ illustrates the local optimal solution of each state generation process, and ‘‘+’’ represents the final optimal solution’s position. From a global aspect, it is obvious to see that
ARTICLE IN PRESS 748
P. Lv et al. / Engineering Applications of Artificial Intelligence 22 (2009) 742–749
the CSA can let the current optimal solution approach the reference optimization much quickly by using less state generation times than by using SA, which accelerates the convergence speed. And from a local aspect, the state generation times by using CSA are similar as using SA when the current optimal solution is close to reference optimization, which lets the CSA have local precision searching ability as well. Combining with the data in Table 1 and the fact in Fig. 5, it can be concluded that because of the integration of annealing temperature’s continuous and random changing property and the ‘‘Backfire & Re-Annealing’’ process, the CSA can enhance the searching veracity and reduce the time cost of the optimization process evidently at the same time.
6. Using CSA for multiple observer sitting problem The problem of multiple observers sitting on terrain (MOST) is an important part in visibility-based terrain reasoning (VBTR). Consider a given terrain, with an observer O at a certain height H. Define the viewshed as the specific terrain visible from O that lies within O’s Region Of Interest (ROI), of radius R. The MOST problem consists of finding the fewest possible observers to make the united viewshed of those observers cover a certain ratio area, given the kind of observer (person, radar, etc.) and the characteristic of the observer (height, the radius of viewshed, etc.) (Pin et al., 2006). The MOST problem has many applications, such as locating a telecommunication base station (De Floriani et al., 1994; Franklin and Vogt, 2004), protecting endangered species (Camp et al., 1997; Aspbury and Gibson, 2004), and locating wind turbines (Kidner et al., 1999). It is difficult because of the unacceptable computing time. Recent developments in this field focus on involving spatial optimization techniques, and Kim et al. (2004) had reported that the simulated annealing algorithm is a good choice considering both efficiency and accuracy. In this case, we compare the traditional simulated algorithm and the cloud theory-based simulated algorithm by three steps to solve the MOST problem for six representative terrains (described in Table 2) as follows. Step 1: According to the desired number of the observers s, partition the terrain into k average-sized smaller blocks and make each block contain s/k observers. The distribution of observers should be fairly uniform across the terrain. Step 2: Pick s/k observers randomly and independently in each block. Compute each observer’s viewshed (Camp et al., 1997) and the united-viewshed coverage ratio of the observer set after the individual viewsheds have been combined. Step 3: Let the result of Step 2 be the initial state, and apply a different algorithm to get the approximately best observer set. We use those steps above in two experiments. Each experiment is repeated 10 times for each terrain. The traditional SA algorithm is used in the first experiment and the CSA algorithm is used in the second experiment. The two algorithms stop when the temperature decreases to 10% of the original temperature stage. All the experiments are done by using a PC that has 2.4 GHz Pentium CPU and 1 Gbytes
Fig. 7. 4 blocks, 2 observer per block, R ¼ 256 sample points, l ¼ 0.9, H ¼ 1.6 m.
Fig. 8. 4 blocks, 4 observer per block, R ¼ 256 sample points, l ¼ 0.9, H ¼ 1.6 m.
Table 2 Statistical characteristic of sample terrain.
Sample Sample Sample Sample Sample
1 2 3 4 5
Min
Max
Diff
Mean
SD
693.1 939.3 250.0 2153.5 930.4
754.2 2531.5 461.3 2570.1 2481.7
61.1 1592.2 211.3 416.6 1551.3
712.2 1731.2 364.4 2372.7 2023.6
84.28 250.3 1272.6 2030.2 3677.1 Fig. 9. 16 blocks, 2 observer per block, R ¼ 128 sample points, l ¼ 0.9, H ¼ 1.6 m.
ARTICLE IN PRESS P. Lv et al. / Engineering Applications of Artificial Intelligence 22 (2009) 742–749
RAM to get the comparison of united-viewshed coverage and time cost between using SA and CSA. The experiment results are presented below. Figs. 7–9 show the comparison results of united-viewshed coverage ratio and the time cost with different observer number and a different radius of ROI. It is seen that the average time cost of the optimal solution based on the CSA decreases by 30–40% and the accuracy improves by 5% as compared with the one based on the SA. Take the experiment using sample 1 terrain for example. As the observer’s number increases, the time cost of using SA increases quickly from 700 s to nearly 6000 s. That is because the computational complexity rises exponentially as the observer number increases, it needs much more searching time to use SA. However, the time cost of using CSA increases not so much, just from 500 s to nearly 3000 s, and it is much less than that of using SA. It illustrates that the gain of using CSA is much more than using SA if the complexity of the problem increases.
7. Conclusions In this paper, according to the cloud drop’s randomness and stable tendency in cloud theory, a cloud theory-based simulated annealing algorithm is proposed. CSA not only realizes the suitableness between quantitative implementation of algorithm and qualitative theory of physical annealing, but also implies ‘‘Backfire & Re-Annealing’’ process. So it can enhance the searching veracity and reduce the time cost of optimization process evidently at the same time. It has been proved that the CSA is convergent theoretically. Typical function optimization experiment shows that CSA is superior to SA in terms of convergence speed, searching ability and robustness. The experiment result of the application using CSA for multiple observers sitting problem (MOST) in visibility-based terrain reasoning (VBTR) demonstrates that the new algorithm is useful and effective adequately.
Acknowledgements This research was supported by National 863 Hi-tech Developing Project Fund (2005AA121120) and Knowledge Innovation Project of Chinese Academy of Sciences Fund (5206020). References Aarts, E.H.L., Korst, J.H.M., 1989. Simulated Annealing and Boltzmann Machines. Wiley, New York, 284pp. Anderson, H.R., McGeehan, J.P., 1994. Optimizing microcell base station locations using simulated annealing techniques. In: Proceedings of IEEE VTC, vol. 2, pp. 858–862. Aspbury, A.S., Gibson, R.M., 2004. Long-range visibility of greater sage grouse leks: a GIS-based analysis. Animal Behaviour 67, 1127–1132. Athreya, K.B., Doss, H., Sethuraman, J., 1996. On the convergence of the Markov chain simulation method. Annual of Statistic 24, 69–100.
749
Camp, R.J., Sinton, D.T., Knight, R.L., 1997. Viewsheds: a complementary management approach to buffer zones. Wildlife Society Bulletin 25, 612–615. Cheng, T., Li, Z., Deng, M., Xu, Z., 2005. Representing indeterminate spatial objects by cloud theory. In: Proceedings of the 4th International Symposium on Spatial Data Quality, pp. 70–77. Cheng, T., Zhilin, L., Deren, L., Deyi, L., 2006. An integrated cloud model for measurement errors and fuzziness. In: Proceedings of the 12th International Symposium on Spatial Data Handling, pp. 699–718. Chung, K.L., 1967. Markov Chains with Stationary Transition Probabilities. Springer, Berlin, 302pp. De Floriani, L., Marzano, L., Puppo, P.E., 1994. Line-of-sight communication on terrain models. International Journal of Geographical Information Systems 8 (4), 329–342. Deyi, L., Haijun, M., Xuemei, S., 1995. Membership clouds and membership cloud generators. Journal of Computer Research and Development 32 (6), 15–20. Deyi, L., Cheung, D., Shi, X.M., Ng, V., 1998. Uncertainty reasoning based on cloud models in controllers. Computers and Mathematics with Applications 35 (3), 99–123. Deyi, L., Di, K., Deren, L., 2000. Knowledge representation and uncertainty reasoning in GIS based on cloud theory. In: Proceedings of the 9th International Symposium on Spatial Data Handling, vol. 3a, pp. 3–14. Deyi, L., Yi, D., 2005. Artificial intelligence with uncertainty. Chapman & Hall, 376pp. Di, K., Deyi, L., Deren, L., 1999. Cloud theory and its applications in spatial data mining knowledge discovery. Journal of Image and Graphics 4A (11), 930–935. Fang, W., Yanpeng, L., Xiang, L., 2007. A new performance evaluation method for automatic target recognition based on forward cloud. In: Proceedings of the Asia Simulation Conference 2007, pp. 337–345. Feizhou, Z., Yuezu, F., Chengzhi, S., Deyi, L., 1999. Intelligent control based membership cloud generators. Acta Aeronautica et Astronautica Sinica 20 (1), 89–92. Franklin, W.R., Vogt, C., 2004. Multiple observer siting on terrain with intervisibility or Lo-Res data. In: Proceedings of the XXth Congress of International Society for Photogrammetry and Remote Sensing, pp. 12–23. Haijun, W., Yu, D., 2007. Spatial clustering method based on cloud model. In: Proceedings of the Fourth International Conference on Fuzzy Systems and Knowledge Discovery, vol. 7, pp. 272–276. Ingber, L., 1989. Very fast simulated annealing. Mathematical and Computer Modeling 12, 967. Jeroen, C.J., Gerard, B.M., 2002. Using simulated annealing for resource allocation. International Journal of Geographical Information Science 16 (6), 571–587. Kaichang, D., Deyi, L., Deren, L., 1998. Knowledge representation and discovery in spatial databases based on cloud theory. International Archives of Photogrammetry and Remote Sensing 32 (3/1), 544–551. Karin, S., Taylor, H.M., 1975. A First Course in Stochastic Process. Academic Press, New York, 573pp. Kim, Y.H., Rana, S., Wise, S., 2004. Exploring multiple viewshed analysis using terrain features and optimisation techniques. Computers and Geosciences 30 (9), 1019–1032. Kidner, D., Sparkes, A., Dorey, M., 1999. GIS and Wind Farm Planning. Geographical Information and Planning. Springer, London, pp. 203–222. Laarhoven, V., Aarts, E., 1987. Simulated Annealing: Theory and Applications. Kluwer Academic, Norwell, 186pp. Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E., 1953. Equations of state calculations by fast computing machines. Journal of Chemical Physics 21, 1087–1091. Pin, L., JinFang, Z., Min, L., 2006. An optimal method for multiple observers sitting on terrain Based on improved simulated annealing techniques. In: Proceedings of the 19th International Conference on Industrial, Engineering & Other Applications of Applied Intelligent Systems, pp. 373–382. Shuliang, W., Deren, L., Wenzhong, S., Deyi, L., Xinzhou, W., 2003. Cloud model-based spatial data mining. Geographical Information Science 9 (2), 67–78. Yingjun, W., Zhongying, Z., 2004. Pattern mining for time series based on cloud theory pan-concept-tree. In: International Conference on Rough Sets and Current Trends in Computing, pp. 618–623. Yunfang, Z., Chaohua, D., Weirong, C., 2005. Adaptive probabilities of crossover and mutation in genetic algorithms based on cloud generators. Journal of Computational Information Systems 1 (4), 67l–678.
Contents lists available at ScienceDirect
Engineering Applications of Artificial Intelligence journal homepage: www.elsevier.com/locate/engappai
Cloud theory-based simulated annealing algorithm and application Pin Lv a,b,, Lin Yuan a,b, Jinfang Zhang a a
National Key Laboratory of Integrated Information System Technology, South No. 4 Street, Zhong Guancun, Institute of Software, Chinese Academy of Sciences, Beijing 100190, PR China Graduate University, Chinese Academy of Sciences, Beijing 100049, PR China
b
a r t i c l e in f o
a b s t r a c t
Article history: Received 31 March 2008 Received in revised form 6 June 2008 Accepted 4 March 2009 Available online 23 April 2009
Using the randomness and stable tendency of a Y condition normal cloud generator, a cloud theorybased simulated annealing algorithm (CSA) is originally proposed, whose characteristic is approximately continuous decrease in temperature and implied ‘‘Backfire & Re-Annealing’’. It fits the annealing process of solid matter in nature much better, overcomes the traditional simulated annealing algorithm (SA)’s disadvantages, which are slow searching speed and being trapped by local minimum easily, then enhances the veracity of final solution and reduces the time cost of the optimization process simultaneously. Theory analysis proves that CSA is convergent and typical function optimization experiments show that CSA is superior to SA in terms of convergence speed, searching ability and robustness. The result of the application using CSA for multiple observers sitting problem (MOST) in visibility-based terrain reasoning (VBTR) also declares the new algorithm’s usefulness and effectiveness adequately. & 2009 Elsevier Ltd. All rights reserved.
Keywords: Cloud theory Improved simulated annealing algorithm Backfire & re-annealing Function optimization
1. Introduction Simulated annealing algorithm (SA) is a random optimization method that is based on Monte Carlo iterative strategy. It comes from a strong analogy between combinatorial optimization and the physical process of crystallization (Laarhoven and Aarts, 1987). From a high initial temperature, SA searches the solution space randomly using metropolis criterion, and escapes from local minimum through the acceptance of ‘‘bad solution’’ with a certain probability. As the temperature descends gradually, it repeats the process above and finds the global optimization solution finally. As an effective non-linear combinatorial optimization method, SA has been proved to be strict in theory and useful in many application fields (Anderson and McGeehan, 1994; Jeroen and Gerard, 2002; Kim et al., 2004). SA simulates the physical annealing process from theory aspect; nevertheless, the temperature of each step is discrete and unchangeable in the annealing course from implementation aspect, which does not meet the requirement of continuous decrease in temperature in actual physical annealing processes. On the one hand, the algorithm is easy to accept deteriorate solution with high temperature and does not converge quickly; on Corresponding author at: National Key Laboratory of Integrated Information System Technology, South No. 4 Street, Zhong Guancun, Institute of Software, Chinese Academy of Sciences, Beijing 100190, PR China. Tel.: +86 10 62614140 8217; fax: +86 10 62558764. E-mail address: [email protected] (P. Lv).
0952-1976/$ - see front matter & 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.engappai.2009.03.003
the other hand, it is hard to escape from local minimum trap with low temperature and has low searching accuracy. Therefore, it is necessary to find a new implement method of annealing mechanism that measures up the physical rules better, and makes the improved method have high searching accuracy and great converging ability. Cloud theory is the innovation and development of membership function in fuzzy theory (Deyi and Yi, 2005), and is a model of the uncertainty transformation between quantitative representation and qualitative concept using language value (Deyi et al., 1995). It is successfully used in intelligence control (Deyi et al., 1998; Feizhou et al., 1999), knowledge representation (Deyi et al., 2000; Cheng et al., 2005), data mining (Kaichang et al., 1998; Di et al., 1999; Shuliang et al., 2003; Yingjun and Zhongying, 2004), spatial analysis (Cheng et al., 2006; Haijun and Yu, 2007), target recognition (Fang et al., 2007), intelligent algorithm improvement (Yunfang et al., 2005) and so on. The physical annealing process is like a fuzzy system in which the molecules move from large scale to small scale randomly as the temperature descends. It is easy to describe this process in natural language but difficult to simulate it using computer, which is the real reason for the problems of SA’s searching accuracy and speed above. Nevertheless, because the cloud theory can realize the transformation between a qualitative concept described in words and its numerical representation (Deyi et al., 1995), it can be used to guide the implementation of SA so as to avoid those problems. In this paper, after introducing the preliminaries of cloud theory in Section 2, we propose a cloud theory-based simulated
ARTICLE IN PRESS P. Lv et al. / Engineering Applications of Artificial Intelligence 22 (2009) 742–749
annealing algorithm (CSA) in Section 3. The innovations of CSA are new temperature annealing process and new state acceptation process according to the cloud drop’s randomness and stable tendency in cloud theory and using Y condition normal cloud generator to produce nearly continuous annealing temperature in each step. Section 4 proves that the CSA is convergent in theory. Furthermore, Sections 5 and 6 are the illustrations of using CSA for some function optimizing experiments and a typical application. A conclusion in Section 7 based on those results in the two sections above shows that the CSA can not only make suitable between quantitative implementation of annealing process and qualitative concept in physical annealing theory, but also have ‘‘Backfire & Re-Annealing’’ process impliedly and it can enhance the searching veracity and reduce the time cost of optimization process evidently at the same time.
2. Preliminaries Cloud model is a model that contains the transferring procedure of uncertainty between qualitative concept and quantitative data representation by using natural language (Deyi et al., 1995). And it mainly reflects the fuzziness and randomness of the concept within the affair and human knowledge in the objective world (Di et al., 1999). 2.1. Basic concepts of cloud theory Suppose that T is the language value of domain u, and mapping C T ðxÞ : u ! ½0; 1; 8x 2 u; x ! C T ðxÞ, then the distribution of CT(x) in u is called membership cloud of T, or cloud for short. If the distribution of CT(x) is normal, it is named normal cloud model. It is a random number set that obeys normal distributive rule and has stable tendency, and it is determined by expectation Ex, entropy En and super entropy He (Deyi et al., 1995). They reflect the quantitative characteristics of concept CT(x). As illustrated in Fig. 1, Ex determines the center of the cloud, En determines the range of the cloud and, according to ‘‘3En’’ rule (Deyi et al., 1995), about 99.74% of the total cloud drops distribute between [Ex3En,Ex+3En]; He determines the cloud drops’ dispersive degree, which means the larger the He is the more dispersively the cloud drops locate. 2.2. Y condition cloud generator If the three digital characteristics (Ex,En,He) and a certain u0 are given, then a drop of cloud drop(xi,u0) can be generated by a
743
generator, which is called Y condition cloud generator (Deyi and Yi, 2005): INPUT:{Ex,En,He},n,u0 OUTPUT:{(x1,u0),y,(xn,u0)} FOR i ¼ 1 to n En0 ¼ randn(En,He) pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi xi ¼ Ex En0 2 lnðu0 Þ drop(xi,u0) where randn(En,He) produces random number with normal distribution whose expectation is En and standard deviation is He.
3. Cloud theory-based simulated annealing algorithm The cloud-based simulated annealing algorithm follows the Metropolis rule, and uses the Y condition normal cloud generator to produce nearly continuous annealing temperature, which influences the state generating and accepting process, and then complies with the physical laws better. Because the normal cloud model has the characteristics of randomness and stable tendency (Deyi et al., 1995), the random change of annealing temperature after importing the cloud theory can preserve diversity of the searched individual in order to avoid being trapped by local minimum. Moreover, the stable tendency of annealing temperature can protect the excellent individual effectively and fix the global minimum’s position quickly as well. 3.1. Algorithm flow Firstly, unify SA’s initial temperature. Commonly, the initial temperature is proportional to the difference degree of state values in SA (Laarhoven and Aarts, 1987). The method of this algorithm is as follows. Generate a group of states, and get their standard deviation, which is recorded as STD, then the initial normalization temperature is t 0 ¼ 1:0 arc cotðSTD=pÞ. Let tq be the quit temperature and s0 be the initial state. The new algorithm is as follows: Algorithm: the Cloud theory-based Simulated Annealing Algorithm 1: t ¼ t0, s ¼ s0, k ¼ 0; 2: Repeat 3: repeat 4: /* Generate the annealing temperature tk0 of the corresponding state, which is based on 5: * Y condition cloud generator. */ 6: He ¼ tk, En ¼ tk, u0 ¼ 1.0tk; 7: En0 ¼ (En+He-rand(0,1)/3); 8: /* rand(a,b) generates random numbers that equally distribute between a and b */ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 9: t 0 ¼ En0 2 lnðu0 Þ; k
10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20:
Fig. 1. Three digital characteristics of a normal cloud.
sj ¼ si+tk0 rand(1,1)Max; /* Max is the maximal difference of every two states’ value */ if min 1; expððCðsi Þ Cðsj ÞÞ=t 0k Xrandð0; 1Þ then s ¼ sj; else s ¼ si; end if until the Metropolis sampling stable criteria is satisfied; k ¼ k+1, tk ¼ t0 lk; until tk is smaller than the quit temperature tq; output the final result.
Because the reference temperature tkX0, then let Ex ¼ 0 in the new algorithm, which can satisfy the Y condition cloud generator’s requirement for the range of discussion domain u (Deyi and Yi, 2005). At the high-temperature stage, if the changeable range of annealing temperature is much wider and more
ARTICLE IN PRESS 744
P. Lv et al. / Engineering Applications of Artificial Intelligence 22 (2009) 742–749
dispersive, it can ensure the randomness of the annealing process adequately. Meanwhile, if that range is much narrower and less dispersive at the low-temperature stage, it can guarantee the stable tendency of the annealing process. In order to satisfy the requirement above, the entropy En and the super entropy He should be positively correlated with the temperature stage, since En determines the range of the cloud and the super entropy He determines the cloud drops’ dispersive degree in cloud theory (Deyi et al., 1995). So in the new algorithm, they are both equal to reference temperature tk. 3.2. New temperature annealing process The temperature update function of the simulated annealing algorithm is exponential in general (Aarts and Korst, 1989). Suppose the initial temperature is t0 and the annealing index is l, then the temperature of the No. k annealing step is tk ¼ t0 l
k
k ¼ 1; 2; 3; . . . ;
0olo1
(1)
Although the annealing temperature with this type of temperature update function is gradually falling, the temperature is a fixed value in every annealing step and the changing process of temperature between two neighbor steps is not continuous. In other words, the temperature does not change although the states update frequently during each annealing step. This phenomenon appears to be the same as other types of temperature update functions such as arithmetical, geometrical or logarithmic one (Anderson and McGeehan, 1994). Apparently, it does not accord with the physical annealing process of the solid material. Therefore, this temperature generation process does not grasp SA’s essentials. Using a Y condition cloud generator and taking a certain value as a reference, it can randomly generate a group of new values that distribute around the given value like ‘‘cloud’’. If we introduce the Y condition cloud generator to the temperature generation process and let the fixed temperature point of each step become a changeable temperature zone in which the temperature of each state generation in every annealing step is chosen randomly, the course of temperature changing in the whole annealing process is nearly continuous and fits the physical annealing process better. Fig. 2 is the comparison of annealing temperature with each state between CSA and SA when states update 800 times. This figure clearly shows that CSA can preserve the characteristic of gradual
Fig. 3. Comparison of backfire & re-annealing process for two algorithms, l ¼ 0.9.
decrease in annealing temperature and convergence of SA. Meanwhile, the changeable bound of annealing temperature of CSA is far wider than that of SA, and then CSA overcomes the disadvantage of SA whose temperature is discrete and simple during each annealing step; hence, the annealing process of CSA is consistent with the physical annealing process. In order to improve the performance of SA, Ingber proposed a very fast simulated annealing algorithm (VFSA) in which the temperature can raise apparently during the annealing process under certain circumstance, which is called ‘‘Backfire & ReAnnealing’’, which means the annealing temperature can rise back to a high stage and then the annealing process can be repeated, and this strategy had been testified effectively (Ingber, 1989). With the same effect as VFSA has, the random change in temperature in each annealing step of CSA implies the ‘‘Backfire & Re-Annealing’’ process in each step, and the overlap of temperature range between two neighbor annealing steps also implies the ‘‘Backfire & Re-Annealing’’ process among multi-steps. Fig. 3 shows the ‘‘Backfire & Re-Annealing’’ process within one step and multi-step. In contrast, the SA does not have those peculiarities. Therefore, through the introduction of the Y condition cloud generator, the new annealing temperature generation process of CSA combines the temperature’s continuous decrease with ‘‘Backfire & Re-Annealing’’ skillfully, and improves the new algorithm’s ability of jumping from local minimum trap further.
3.3. New state acceptation process
Fig. 2. Comparison of temperature annealing process for two algorithms, l ¼ 0.9.
One of the simulated annealing algorithm’s keys is the new state acceptation process (Anderson and McGeehan, 1994). Because the temperature of each annealing step is fixed for SA, it is easy to accept the deteriorative state and does not converge quickly at the low-temperature stage; on the contrary, it is hard to escape from local minimum trap and then decrease the searching accuracy. To solve these problems, the annealing temperature’s stable tendency of CSA can make the new algorithm satisfy the basic requirement that it is increasingly harder to accept the bad solution as the temperature’s descending on one hand. On the other hand, the continuous change in temperature and ‘‘Backfire & Re-Annealing’’ assures that the new algorithm can escape from local minimum trap easily and has faster convergence speed. Fig. 4
ARTICLE IN PRESS P. Lv et al. / Engineering Applications of Artificial Intelligence 22 (2009) 742–749
745
When C(j)4C(i), Z ai;j ðTÞ ¼ Pr acceptði; j; xÞjt ¼ x d PrðtpxÞ Z ¼ f t ðx; TÞeðCðiÞCðiÞ=xÞ dx
(4)
The equation above uses all probability formula. Suppose the probability density function of random variable t is ft(t,T), and accept(i,j,t) is the Metropolis acceptation function of SA under temperature t (Metropolis et al., 1953): acceptði; j; tÞ ¼ eðCðjÞCðiÞÞ=t
(5)
Therefore, the acceptation function ai,j(T) can be summarized as Z (6) ai;j ðTÞ ¼ min 1; f t ðx; TÞeðCðiÞCðiÞ=xÞ dx
Fig. 4. Comparison of acceptance probability of deteriorative solution for two algorithms, d ¼ 0.9.
is the comparison of acceptance probability with deteriorative solution for two algorithms in each annealing step. The figure shows that the CSA is much easier to accept the bad state in the front half annealing phase (the searching range of the new state is wide, corresponding to the high temperature) so as to expand the distribution scope of the new solution and avoid being trapped by local optimal solution effectively. But in the back half annealing phase (the searching range of the new state is narrow, corresponding to the low temperature), it is much harder for CSA to accept deteriorative solution, thereby reducing the invalid searching times and then improving the convergence speed.
Compared with the classical SA, whose acceptation function is totally determined by DC and t, this new algorithm introduces an extra uncertain factor – a distribution function of acceptance under a given temperature. This new uncertainty, which allows the search to escape from local optimal states more easily especially under low temperature, leads to the algorithm’s distinctive characteristic of ‘‘backfire and reannealing’’. However, the main structure of this algorithm still conforms to that of the classical SA, and thus, as is analyzed below, the new feature will not inflict extra problems in the convergence. In this algorithm, t is generated by a random variable U, which has uniform distribution according to Section 3.1: En þ He randð0; 1Þ T þ T randð0; 1Þ ¼ 3 3 pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi t ¼ U 2 lnðu0 Þ ¼ U 2 lnð1 TÞ U¼
(7) (8)
Thus, the probability density function of this random variable t is
4. Analysis of CSA’s convergence
f t ðt; TÞ ¼
Theorem 1. The CSA is corresponding to a Markov chain. Proof. The Markov property of CSA is related to its state generator function: X nþ1 ¼ X n þ t randð0; 1Þ Max
(2)
According to the state generating method of CSA, the generation of next state Xn+1 is only correlative with current state Xn. Therefore, this state transformation has Markov property. Moreover, for each temperature T, the temperature decrease in CSA occurs only when the solutions does not change anymore, so this algorithm is time homogeneous (Athreya et al., 1996). The transferring probability for the next annealing step under the temperature T of this Markov chain is 8 j 2 Di and jai g i;j ðTÞai;j ðTÞ; > > P < pi;k ðTÞ; j ¼ i 1 (3) pi;j ðTÞ ¼ k2Di > > : 0; otherwise gi,j(T) is the probability of the state transferring from i to j under temperature T, and ai,j(T) is the acceptation function under this temperature, Di is the neighbor domain of state i. According to the analysis of CSA in Section 3.1, the acceptation function ai,j(T) can be presented as When C(j)pC(i), then ai,j(T) ¼ 1
8 3T < pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi; :
2 lnð1TÞ
0;
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
2 lnð1TÞ 2 ptp 3T
otherwise
2 lnð1TÞ 3T
(9)
&
Theorem 2. The state space of CSA’s Markov chain is limited. Proof. The scale of state space is determined by actual problem. However, this requirement is satisfied for most combinational optimization problem. Furthermore, for the problem of continuous function optimization whose state space is limitless, because of the precision restriction of real number in the computer, it also considers that the state space under this circumstance is limited only if the independent variable has limited definition domain. & Lemma 1. All states of the Markov chain whose state is limited and unreduced have natural returning (Karin and Taylor, 1975). Theorem 3. This state-limited Markov chain cannot be reduced. Proof. According to the neighbor selection method and state generation function, it is clear that if only the feasible domain has strong connectivity, 8i; j 2 O, ( s0 ; s1 ; . . . ; sn 2 O; s0 ¼ i; sn ¼ j and since every state is not the same as each other, it assures that gsk,sk+140,k ¼ 0,1, y, n1. Moreover, because 8i; j 2 O; ai;j ðTÞ40, ðTÞXps0 ;s1 ðTÞps1 ;s2 ðTÞ . . . psn1 ;sn ðTÞ pðnÞ i;j ¼ g s0 ;s1 as0 ;s1 g s1 ;s2 as1 ;s2 . . . g sn1 ;sn asn1 ;sn 40
(10)
ARTICLE IN PRESS 746
P. Lv et al. / Engineering Applications of Artificial Intelligence 22 (2009) 742–749
the arbitrary state of this Markov chain is connectivity to each other. Therefore, this Markov chain cannot be reduced. & Theorem 4. This state-limited Markov chain is non-periodic. Proof. Because this state-limited Markov chain cannot be reduced, all states of this chain are equivalent and have the same period. Consider the state acceptation function ai,j(T), When C(j)pC(i), then ai,jp (T)ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ¼1 ffi When C(j)4C(i), pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi let a ¼ 2 lnð1 TÞ=3T; b ¼ 2 2 lnð1 TÞ=3T
i¼1
t¼a
(11)
If 9i 2 O; j 2 Di , which are satisfied with 0oai,j(T)o1, consider the transferring probability of i X pi;k ðTÞ pi;i ðTÞ ¼ 1 k2O;kai
¼1
X
g i;k ai;k
k2O;kai
¼1
X
g i;k ai;k g i;j ai;j
k2O;kai;kaj
41
X
g i;k g i;j
k2O;kai;kaj
¼1
X
g i;k
k2O;kai
¼0
Using 7 typical functions to conduct the function optimization experiment, we compare the performance of the CSA to that of SA. The experiment is realized on a PC, which has P4 3.0 GHz CPU and 1 G memory using Matlab platform: F1: Sphere model function P P F1 ¼ 5i¼1 x2i ; xi p30F1 ¼ 5i¼1 x2i , F2: Hyper-ellipsoid function P 2 2 xi p1 F2 ¼ 10 i¼1 i xi ; F3: Generalized Rastrigin’s function P 2 xi p2:56 F3 ¼ 30 i¼1 xi 10 cosð2pxi Þ þ 10 ; F4: Branin function
2
2 5 F4 ¼ x2 45:1 þ 10 1 81p cos x1 þ 10; xi p5 p2 x1 þ p x1 6 F5: Griewangk’s function
30 30 Q P 1 F5 ¼ 4000 x2i cos pxiffii þ 1;
0oai;j ðTÞ Z ¼ f t ðx; TÞeðCðjÞCðiÞÞ=x dx Z p f t ðx; TÞ eðCðjÞCðiÞÞ=x dx Z o f t ðx; TÞ dx Z 3T dt pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ¼ 2 lnð1 TÞ t¼b 3Tt ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2 lnð1 TÞ ¼1
5. Typical function optimization
(12)
Thus the state i is non-periodic. And also because all states have the same period, this Markov chain is non-periodic. Contrarily, for 8i 2 O; j 2 Di, condition 0oai,j(T)o1 is not tenable; consequently, it has ai,j(T) ¼ 1. It shows that 8i; j 2 O; CðiÞ ¼ CðjÞ, all states have the same cost value; hence this optimization problem is ordinary. Lemma 2. The state-limited, unreduced and non-periodic Markov chain has unique smooth distribution (Karin and Taylor, 1975). Lemma 3. The sufficient and necessary condition of natural returning for a chain, which is non-periodic and unreduced, is that it has smooth distribution, and this distribution is its ultimate distribution (Chung, 1967). Theorem 5. The Markov chain of CSA is convergent. Proof. Theorems 1 and 2 prove that the CSA corresponds to a state-limited and unreduced Markov chain. According to Lemma 1, all states of this Markov chain are natural returning. Theorems 3 and 4 prove that this Markov chain is unreduced and nonperiodic. According to Lemma 2 and 3, this Markov chain is convergent, which means the ultimate distribution of this chain is equal to its smooth distribution. &
xi p256
i¼1
F6: DeJong’s f2 function F6 ¼ 100ðx21 x2 Þ2 þ ð1 x1 Þ2 ; xi p2:56 F7: Six hump camel back function F7 ¼ 4x21 2:1x41 þ x61 =3 þ x1 x2 4x22 þ 4x42 ; xi p5 Among those functions, F1, F2 and F3 are continuous singlepeak functions, and it means that they have only one best solution. F4 is continuous multi-peaks function, which means that it has several best solutions. All those four functions are used to test the searching ability of the two algorithms; F5 and F6 are non-linear multi-minimums function. p The former one’s local ffi minimums locate at xi ¼ k p i; i ¼ 1; 2; . . . ; 256; k ¼ 1; 2; . . . ; 256 and the latter one has many minimums near x1 ¼ x2 ¼ 1. In addition, their function values do not change linearly with change in xi, so they are used to test the ability of escaping from local minimum and quick converging of the two algorithms. F7 is a special non-linear multi-minimum function. Its function values are not only small but also close to each other. Moreover, it has infinite local minimums, which can easily trap the searching. It has two optimal solutions, which are F7(0.08983, 0.7126) ¼ F7(0.08983, 0.7126) ¼ 1.0316285. Because it is hard to find the optimal solutions, it is used to compare the integrated ability of the two algorithms. For different functions, repeat the experiment using two algorithms 100 times separately. In each experiment, the initial temperature t0, initial state s0, and convergence temperature tq are all same, and the annealing index l is 0.99 for both algorithms too. From Table 1, for F1–F4, irrespective of the adjacency degree to the reference solution, the time cost or the stableness of optimization process, the outcomes of CSA are all better than SA. It shows that the CSA has stronger searching ability. Fig. 5 is the comparison of state generation times (vertical axis) in each annealing step (horizontal axis) comparison for F5 (Fig. 5a) and F6 (Fig. 5b) using CSA and SA, respectively. In Fig. 5a, the state generation times increase close to 40 000 in the no. 450 annealing step by using SA, which means it is trapped by a local minimum. Nevertheless, this trouble does not appear when using CSA whose state generation times are all below 60 during all annealing steps. Similarly, the phenomenon occurs in Fig. 5b when the annealing process approaches the end. From this figure, it is clear to see that whether in the front half annealing phase (Fig. 5a) or back half annealing phase (Fig. 5b), CSA can escape from local minimum quickly by using less state generation times, and then increase the convergence speed. The conclusion is also testified by the experiment data in Table 1. Fig. 6 is the comparison of a certain optimization process for two algorithms with F7. In this figure, the vertical axis represents the solution’s x1 value and the horizontal axis represents the solution’s x2 value. The deeper the color is, the smaller the function value is. The circle in the figure shows the position of two
ARTICLE IN PRESS P. Lv et al. / Engineering Applications of Artificial Intelligence 22 (2009) 742–749
747
Table 1 Performance comparison for two algorithms (time cost unit: second). F
Ref. value
Best value
F1
0
7.9069E03 3.0499E04
2.6564E02 9.9594E04
1.5946E02 5.6471E04
4.5563E03 1.2198E04
1.375 0.203
2.531 0.219
F2
0
1.7026E03 2.0351E05
1.2903E02 3.6407E04
6.0582E03 1.3301E04
2.5875E03 7.9466E05
7.359 0.234
F3
0
9.3298E03 1.6092E04
4.1971E02 5.8142E04
1.7330E02 3.7062E04
5.0919E03 9.8637E05
F4
3.9780 E01
3.9852E01 3.9789E01
4.0654E01 3.9792E01
4.0034E01 3.9790E01
F5
0
2.0957E03 4.0275E04
1.4686E01 3.5222E02
F6
0
6.6253E06 5.9961E07
F7
1.0316 E+00
1.0310E+00 1.0316E+00
Worst value
Average value
Std of all value
Min time cost
Max time cost
Average time cost
Std of time cost
Algorithm
1.847 0.208
0.201 0.007
SA CSA
9.641 0.266
8.229 0.241
0.568 0.008
SA CSA
7.953 0.078
17.797 0.110
12.285 0.094
2.354 0.008
SA CSA
1.5495E03 1.8587E05
62.985 0.500
79.781 0.532
70.608 0.526
3.369 0.008
SA CSA
4.3497E02 1.0646E02
2.7675E02 9.3398E03
3.938 0.609
21.516 0.672
14.113 0.626
4.223 0.012
SA CSA
3.8965E03 1.0014E04
8.8630E04 1.7697E05
9.8820E04 2.0205E05
1.296 0.203
1.344 0.219
1.315 0.207
0.014 0.007
SA CSA
1.0255E+00 1.0315E+00
1.0290E+00 1.0316E+00
1.4716E03 1.8389E05
99.688 0.390
130.656 0.437
112.442 0.407
6.727 0.012
SA CSA
Fig. 6. Comparison of a certain optimization process for two algorithms.
Fig. 5. Comparison of escaping from local extremism for two algorithms.
reference solutions, ‘‘ ’’ illustrates the local optimal solution of each state generation process, and ‘‘+’’ represents the final optimal solution’s position. From a global aspect, it is obvious to see that
ARTICLE IN PRESS 748
P. Lv et al. / Engineering Applications of Artificial Intelligence 22 (2009) 742–749
the CSA can let the current optimal solution approach the reference optimization much quickly by using less state generation times than by using SA, which accelerates the convergence speed. And from a local aspect, the state generation times by using CSA are similar as using SA when the current optimal solution is close to reference optimization, which lets the CSA have local precision searching ability as well. Combining with the data in Table 1 and the fact in Fig. 5, it can be concluded that because of the integration of annealing temperature’s continuous and random changing property and the ‘‘Backfire & Re-Annealing’’ process, the CSA can enhance the searching veracity and reduce the time cost of the optimization process evidently at the same time.
6. Using CSA for multiple observer sitting problem The problem of multiple observers sitting on terrain (MOST) is an important part in visibility-based terrain reasoning (VBTR). Consider a given terrain, with an observer O at a certain height H. Define the viewshed as the specific terrain visible from O that lies within O’s Region Of Interest (ROI), of radius R. The MOST problem consists of finding the fewest possible observers to make the united viewshed of those observers cover a certain ratio area, given the kind of observer (person, radar, etc.) and the characteristic of the observer (height, the radius of viewshed, etc.) (Pin et al., 2006). The MOST problem has many applications, such as locating a telecommunication base station (De Floriani et al., 1994; Franklin and Vogt, 2004), protecting endangered species (Camp et al., 1997; Aspbury and Gibson, 2004), and locating wind turbines (Kidner et al., 1999). It is difficult because of the unacceptable computing time. Recent developments in this field focus on involving spatial optimization techniques, and Kim et al. (2004) had reported that the simulated annealing algorithm is a good choice considering both efficiency and accuracy. In this case, we compare the traditional simulated algorithm and the cloud theory-based simulated algorithm by three steps to solve the MOST problem for six representative terrains (described in Table 2) as follows. Step 1: According to the desired number of the observers s, partition the terrain into k average-sized smaller blocks and make each block contain s/k observers. The distribution of observers should be fairly uniform across the terrain. Step 2: Pick s/k observers randomly and independently in each block. Compute each observer’s viewshed (Camp et al., 1997) and the united-viewshed coverage ratio of the observer set after the individual viewsheds have been combined. Step 3: Let the result of Step 2 be the initial state, and apply a different algorithm to get the approximately best observer set. We use those steps above in two experiments. Each experiment is repeated 10 times for each terrain. The traditional SA algorithm is used in the first experiment and the CSA algorithm is used in the second experiment. The two algorithms stop when the temperature decreases to 10% of the original temperature stage. All the experiments are done by using a PC that has 2.4 GHz Pentium CPU and 1 Gbytes
Fig. 7. 4 blocks, 2 observer per block, R ¼ 256 sample points, l ¼ 0.9, H ¼ 1.6 m.
Fig. 8. 4 blocks, 4 observer per block, R ¼ 256 sample points, l ¼ 0.9, H ¼ 1.6 m.
Table 2 Statistical characteristic of sample terrain.
Sample Sample Sample Sample Sample
1 2 3 4 5
Min
Max
Diff
Mean
SD
693.1 939.3 250.0 2153.5 930.4
754.2 2531.5 461.3 2570.1 2481.7
61.1 1592.2 211.3 416.6 1551.3
712.2 1731.2 364.4 2372.7 2023.6
84.28 250.3 1272.6 2030.2 3677.1 Fig. 9. 16 blocks, 2 observer per block, R ¼ 128 sample points, l ¼ 0.9, H ¼ 1.6 m.
ARTICLE IN PRESS P. Lv et al. / Engineering Applications of Artificial Intelligence 22 (2009) 742–749
RAM to get the comparison of united-viewshed coverage and time cost between using SA and CSA. The experiment results are presented below. Figs. 7–9 show the comparison results of united-viewshed coverage ratio and the time cost with different observer number and a different radius of ROI. It is seen that the average time cost of the optimal solution based on the CSA decreases by 30–40% and the accuracy improves by 5% as compared with the one based on the SA. Take the experiment using sample 1 terrain for example. As the observer’s number increases, the time cost of using SA increases quickly from 700 s to nearly 6000 s. That is because the computational complexity rises exponentially as the observer number increases, it needs much more searching time to use SA. However, the time cost of using CSA increases not so much, just from 500 s to nearly 3000 s, and it is much less than that of using SA. It illustrates that the gain of using CSA is much more than using SA if the complexity of the problem increases.
7. Conclusions In this paper, according to the cloud drop’s randomness and stable tendency in cloud theory, a cloud theory-based simulated annealing algorithm is proposed. CSA not only realizes the suitableness between quantitative implementation of algorithm and qualitative theory of physical annealing, but also implies ‘‘Backfire & Re-Annealing’’ process. So it can enhance the searching veracity and reduce the time cost of optimization process evidently at the same time. It has been proved that the CSA is convergent theoretically. Typical function optimization experiment shows that CSA is superior to SA in terms of convergence speed, searching ability and robustness. The experiment result of the application using CSA for multiple observers sitting problem (MOST) in visibility-based terrain reasoning (VBTR) demonstrates that the new algorithm is useful and effective adequately.
Acknowledgements This research was supported by National 863 Hi-tech Developing Project Fund (2005AA121120) and Knowledge Innovation Project of Chinese Academy of Sciences Fund (5206020). References Aarts, E.H.L., Korst, J.H.M., 1989. Simulated Annealing and Boltzmann Machines. Wiley, New York, 284pp. Anderson, H.R., McGeehan, J.P., 1994. Optimizing microcell base station locations using simulated annealing techniques. In: Proceedings of IEEE VTC, vol. 2, pp. 858–862. Aspbury, A.S., Gibson, R.M., 2004. Long-range visibility of greater sage grouse leks: a GIS-based analysis. Animal Behaviour 67, 1127–1132. Athreya, K.B., Doss, H., Sethuraman, J., 1996. On the convergence of the Markov chain simulation method. Annual of Statistic 24, 69–100.
749
Camp, R.J., Sinton, D.T., Knight, R.L., 1997. Viewsheds: a complementary management approach to buffer zones. Wildlife Society Bulletin 25, 612–615. Cheng, T., Li, Z., Deng, M., Xu, Z., 2005. Representing indeterminate spatial objects by cloud theory. In: Proceedings of the 4th International Symposium on Spatial Data Quality, pp. 70–77. Cheng, T., Zhilin, L., Deren, L., Deyi, L., 2006. An integrated cloud model for measurement errors and fuzziness. In: Proceedings of the 12th International Symposium on Spatial Data Handling, pp. 699–718. Chung, K.L., 1967. Markov Chains with Stationary Transition Probabilities. Springer, Berlin, 302pp. De Floriani, L., Marzano, L., Puppo, P.E., 1994. Line-of-sight communication on terrain models. International Journal of Geographical Information Systems 8 (4), 329–342. Deyi, L., Haijun, M., Xuemei, S., 1995. Membership clouds and membership cloud generators. Journal of Computer Research and Development 32 (6), 15–20. Deyi, L., Cheung, D., Shi, X.M., Ng, V., 1998. Uncertainty reasoning based on cloud models in controllers. Computers and Mathematics with Applications 35 (3), 99–123. Deyi, L., Di, K., Deren, L., 2000. Knowledge representation and uncertainty reasoning in GIS based on cloud theory. In: Proceedings of the 9th International Symposium on Spatial Data Handling, vol. 3a, pp. 3–14. Deyi, L., Yi, D., 2005. Artificial intelligence with uncertainty. Chapman & Hall, 376pp. Di, K., Deyi, L., Deren, L., 1999. Cloud theory and its applications in spatial data mining knowledge discovery. Journal of Image and Graphics 4A (11), 930–935. Fang, W., Yanpeng, L., Xiang, L., 2007. A new performance evaluation method for automatic target recognition based on forward cloud. In: Proceedings of the Asia Simulation Conference 2007, pp. 337–345. Feizhou, Z., Yuezu, F., Chengzhi, S., Deyi, L., 1999. Intelligent control based membership cloud generators. Acta Aeronautica et Astronautica Sinica 20 (1), 89–92. Franklin, W.R., Vogt, C., 2004. Multiple observer siting on terrain with intervisibility or Lo-Res data. In: Proceedings of the XXth Congress of International Society for Photogrammetry and Remote Sensing, pp. 12–23. Haijun, W., Yu, D., 2007. Spatial clustering method based on cloud model. In: Proceedings of the Fourth International Conference on Fuzzy Systems and Knowledge Discovery, vol. 7, pp. 272–276. Ingber, L., 1989. Very fast simulated annealing. Mathematical and Computer Modeling 12, 967. Jeroen, C.J., Gerard, B.M., 2002. Using simulated annealing for resource allocation. International Journal of Geographical Information Science 16 (6), 571–587. Kaichang, D., Deyi, L., Deren, L., 1998. Knowledge representation and discovery in spatial databases based on cloud theory. International Archives of Photogrammetry and Remote Sensing 32 (3/1), 544–551. Karin, S., Taylor, H.M., 1975. A First Course in Stochastic Process. Academic Press, New York, 573pp. Kim, Y.H., Rana, S., Wise, S., 2004. Exploring multiple viewshed analysis using terrain features and optimisation techniques. Computers and Geosciences 30 (9), 1019–1032. Kidner, D., Sparkes, A., Dorey, M., 1999. GIS and Wind Farm Planning. Geographical Information and Planning. Springer, London, pp. 203–222. Laarhoven, V., Aarts, E., 1987. Simulated Annealing: Theory and Applications. Kluwer Academic, Norwell, 186pp. Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E., 1953. Equations of state calculations by fast computing machines. Journal of Chemical Physics 21, 1087–1091. Pin, L., JinFang, Z., Min, L., 2006. An optimal method for multiple observers sitting on terrain Based on improved simulated annealing techniques. In: Proceedings of the 19th International Conference on Industrial, Engineering & Other Applications of Applied Intelligent Systems, pp. 373–382. Shuliang, W., Deren, L., Wenzhong, S., Deyi, L., Xinzhou, W., 2003. Cloud model-based spatial data mining. Geographical Information Science 9 (2), 67–78. Yingjun, W., Zhongying, Z., 2004. Pattern mining for time series based on cloud theory pan-concept-tree. In: International Conference on Rough Sets and Current Trends in Computing, pp. 618–623. Yunfang, Z., Chaohua, D., Weirong, C., 2005. Adaptive probabilities of crossover and mutation in genetic algorithms based on cloud generators. Journal of Computational Information Systems 1 (4), 67l–678.