A hybrid evolutionary algorithm for recurrent neural network control of a three-dimensional tower crane

Automation in Construction 23 (2012) 55–63

Contents lists available at SciVerse ScienceDirect

Automation in Construction journal homepage: www.elsevier.com/locate/autcon

A hybrid evolutionary algorithm for recurrent neural network control of a three-dimensional tower crane Sam Chau Duong a, Eiho Uezato b, Hiroshi Kinjo b,⁎, Tetsuhiko Yamamoto c a b c

Graduate School of Engineering and Science, University of the Ryukyus, Japan Faculty of Engineering, University of the Ryukyus, Senbaru 1, Nishihara, Okinawa 903-0213, Japan Tokushima College of Technology, Japan

a r t i c l e

i n f o

Article history: Accepted 7 December 2011 Available online 9 February 2012 Keywords: Hybrid evolutionary algorithm Recurrent neural network Tower crane Underactuated system Nonlinear system control

a b s t r a c t This paper is concerned with the control of an underactuated three-dimensional tower crane system using a recurrent neural network (RNN) which is evolved by an evolutionary algorithm. In order to improve the performance in evolving the RNN, a hybrid evolutionary algorithm (HEA) which utilizes the operators of a constricted particle swarm optimization (PSO) and a binary-coded genetic algorithm (GA) is proposed. Simulation results show that the proposed HEA has superior performance in a comparison with the canonical algorithms and that the control system works effectively. © 2011 Elsevier B.V. All rights reserved.

1. Introduction Crane systems are widely used in transportation and construction [1]. They are used to move a load from an initial point to a desired point and, at the same time, the load swing needs be suppressed. This task is usually performed by a skillful operator. Nevertheless, control of cranes has attracted much research [1–5], motivated by the fact that automated cranes can reduce the cycle time for suppressing the load swing with accuracy. The potentials of automated cranes are especially well suited when they are mounted in fixed locations repeating almost the same trajectories, like tower cranes in construction. Crane systems usually exhibit underactuated behavior [2,3]. Underactuated systems, which are characterized by the fact that they process fewer actuators than degrees of freedom, have been the subject of much study during the last decades due to its pervasive applications. Many real-world mechanical systems are examples of such systems, where underactuated behavior can even be utilized intentionally [6] in order to reduce the cost, complexity, size and weight by reducing the number of required actuators and related designs. Such systems generate interesting but difficult control problems as they usually exhibit complex dynamics and nonholonomic behavior. Following the potential trend of applying NNs in control systems design [7–9], this study considers the control of a three-dimensional rotary tower crane, which is a five degree-of-freedom model with underactuated nature, by using a recurrent neural network (RNN). As to be

⁎ Corresponding author. Tel./fax: + 81 98 895 8632. E-mail address: [email protected] (H. Kinjo). 0926-5805/$ – see front matter © 2011 Elsevier B.V. All rights reserved. doi:10.1016/j.autcon.2011.12.005

shown, the use of RNN-based control system provide convenience to deal with the problem while considering constraints and damping coefficients of the system, which are not always considered adequately in the literature. Aside from several advantages of NNs, such as good nonlinear processing ability and robustness with inherently parallel architecture, RNNs have interesting properties with great potential and they have shown superiority to feedforward neural networks (FNNs) [10,11]. Although FNNs have been very popular, with only feedforward connections, they are static networks and lack dynamic memory since the response of such networks depends only on current inputs [10]. They are thus unable to represent dynamic mappings without external feedback [11]. In contrast, RNNs have several important capabilities, including attractor dynamics and internal memory. Since a recurrent neuron already has an internal feedback loop, it captures the dynamic response of a system without external feedback through tapped delays [11], RNNs are thus dynamic mappings and are more appropriate than FNNs when applied to dynamical systems. However, despite important capabilities, RNNs are much less popular than FNNs because it is hard to develop a convenient learning algorithm as well as classic gradientbased algorithms, such as backpropagation [12,13], are apparently insufficient/limited [14]. As NNs training can be considered as an optimization problem which apparently contains several local optimum, it should thus be more suitable to utilize global optimization techniques. In order to overcome the problems of RNNs training, as stochastic optimization techniques become available and effective, the last two decades have shown an increasing amount of interest in applying evolutionary algorithms (EAs) to construct RNNs (e.g., see [14,15]). In this paper, attention is paid to the use of an EA as an optimization algorithm to construct an RNN-based controller for the crane.

56

S.C. Duong et al. / Automation in Construction 23 (2012) 55–63

Notations φ s ‘ θ1 θ2 Mb F Mw Jb Jw mt m Dφ Ds Dα rw g

rotation angle of bridge trolley position rope length out-of-plane swing angle in-plane swing angle torque actuating tower force actuating trolley winch torque moment of inertia of jib moment of inertia of winch mass of trolley mass of load damping coefficient (tower) damping coefficient (girder) damping coefficient (winch) winch radius acceleration of gravity

EAs have been emerging as a growing field of study and applied pervasively in various areas, particularly in control engineering [16–18]. Among the EA-family, genetic algorithms (GAs) [18–21] and particle swarm optimization (PSO) [22–26] seem to be most well-known, and in fact they have shown good performance in NNs optimization [14,19,22,23]. Although it is considered to be a newly developed tool, PSO has appeared to be a simple but fast algorithm with only a few parameters [22–26]. However, it has difficulties in balancing between exploration and exploitation [25], since PSO seems to be easily “trapped” into a better solution previously found, which could possibly be a local optima. In contrast, and with a long history, GAs have emerged to be one of the most successful in the EA family. As they contain several operators and tunable parameters, it is intuitively easier to control the balance between exploration and exploitation in GAs. Although the competences of EAs have been shown to be superior over those of conventional methods, an experienced combination of operations (either fully or partially) from different approaches may provide an even more efficient performance. In fact, the hybridizations of EAs have been the focus of much research recently, and one of the popular and interesting frameworks are the hybrids of GAs and PSO which are capable of handling several real-world problems [27–31]. For example, an interesting work in [28] presents a hybrid of PSO and GA with use of three fuzzy systems: a main fuzzy system is for decision making and the other two fuzzy systems are used to tune the parameters of PSO and GA. In [29], while the lower-half is eliminated, the top-half of the best-performing individuals is considered as elites and they are first enhanced by PSO before applying GA operators. In [30], a hybrid between GA and PSO is introduced where the population is randomly divided into two parts to be evolved with the two methods and then combined in the updated population, which is then again divided randomly into two parts for the next run. To control the percentage of the population developed by GA, a hybridization coefficient with a self-adaptive strategy is employed. In [31] the upper-haft of the best individuals is instead processed by a real-coded GA to create the same amount of new individuals which are then used to adjust the remaining particles by PSO. Since it is desirable to have an efficient algorithm that can provide a good solution with a small population and within a small number of generation/iteration, in this article a hybrid of a constricted PSO and a binary-coded GA, called a hybrid evolutionary algorithm (HEA), is proposed. In order to develop the HEA, genetic operators of the GA (recombination and mutation) are embedded into the PSO to generate

offspring with a fitness-based selection of parents, and an elimination of the inferior individuals is applied to maintain a constant population. While several hybrid EAs have involved the problems of mathematical models and functions optimizations (e.g. [27–29,31]), our focus is to apply the HEA to construct an RNN control of a complex underactuated mechanical system. The rest of this article is organized as follows. Section 2 introduces a rotary tower crane system and its dynamics. A control system using RNN and EA is also presented. In Section 3, brief backgrounds on the PSO and GA used to construct the HEA are first provided. The proposed hybrid algorithm based on the PSO and GA is then introduced. Simulations are demonstrated in Section 4, where the performance of the proposed method is investigated and compared with those of the original PSO and GA. Control simulations are also shown. Section 5 is a discussion of the results and Section 6 is the conclusion of the study. 2. Three-dimensional tower crane system and control 2.1. Three-dimensional tower crane system This paper considers a rotary tower crane as shown in Fig. 1. This is a concrete model that has been studied by W. Blajer et al. [2,3],

Z

s Y

X’

ϕ

F

mt

Y’

Mw

X

θ2 Jb

Mb

θ1

l m Fig. 1. Tower crane model.

S.C. Duong et al. / Automation in Construction 23 (2012) 55–63

where the crane dynamic equations were transformed using dependent coordinates. This study deals with the control of the system using state–space representation of the system described directly by the systems states. The generalized coordinate of the system is q ¼ ½φ; s; ‘; θ1 ; θ2 T which defines the position of the load mass m. While the height of the payload is defined by the hoisting robe length ‘, the pendulations/oscillations of the payload are characterized by two swing angles θ1 and θ2. The out-of-plane angle θ1 is the angle between the robe and the Y′Z-plane. The in-plane angle θ2 is the angle which the robe makes with the Z-axis in the Y′Z-plane. The control input is u = [ Mb, F, Mw ] T, containing the torque Mb regulating the girderrotation angle φ, the force F actuating the trolley position s, and the winch/hoist torque Mw changing the rope length ‘. Using Lagrangian approach for the system, the equation of motion can be derived as: € þ Dðq; q_ Þ ¼ Fðq; q_ Þ þ Bu MðqÞq

Input layer

57

Hidden layer

Output layer

ref

u1

x1 - x 1

z

-1

z z

ref

uK

xI - x I I

K

H

z

ð1Þ

-1

z z

T The system state is defined as x ¼ ½ q q_  ¼ ½ xi T ði ¼ 1; 2; ⋯10Þ. The state–space representation of the system is

" x_ ¼ f ðx; uÞ ¼

−1

MðqÞ

# q_ f−Dðq; q_ Þ þ Fðq; q_ Þ þ Bug

ð2Þ

More details of the system dynamics can be found in the Appendix A to this article. As the system is a five-degree-of-freedom system (5-DOF), which is larger than the number of control inputs (3 inputs), the system is thus an underactuated system [6]. The control task is to regulate the load mass from an arbitrary position x init to a desired one, x ref. 2.2. Controller design for the crane In this study, a control method is introduced using an RNN which is evolved by an EA. Fig. 2 shows the proposed control system. Fig. 3 shows the RNN model used in this paper as a state-feedback controller. The network consists of three layers: the input layer with I neurons, the hidden layer with H neurons, and the output layer with K neurons (i.e., it is an I-H-K structured RNN). The input layer contains only feedforward connections (directly to both hidden and output layers) and there is no recurrent/feedback loop in its own neurons nor from/to the other layers. In the hidden and output layers, aside from feedforward connections, each neuron is connected to itself (self-connected neuron) and is also fully connected to all other output and hidden units. That is, the hidden and output layers are fully recurrent. An EA is employed to evolve the RNN. That is to say, the task of the EA herein is to search for an optimal set of the RNN's connection weights (because the numbers of input neurons I and output neurons K are respectively equal to the number of the system states and control inputs while the number of hidden neurons H can be tuned easily by trial and error for the most suitable value regarding to performance criteria or can be determined with respect to computational cost). In order to improve the efficiency in evolving the RNN, a HEA is proposed.

−

x

EA

ref

+

RNN

u

Fig. 2. Control system.

Plant

x

-1

-1

-1

-1

Fig. 3. Recurrent neural network.

A sum-squared error is used as the evaluation function (cost function) for the EA, which is: E¼

I X

  ref end 2 Q i xi −xi

ð3Þ

i¼1

where Qi are the weight coefficients, xiend represents the values at the final state (at the end of the control starting fromxinit ), and xiref represents the desired values (xref). As most engineering problems involve constraints [16], this study also considers controlling the crane with constraints by using a penalty factor. That is, the following constraints must be satisfied: 8 < smin ≤s≤smax ðmÞ ‘ ≤‘≤‘max ðmÞ : _min _ jφ j≤φ max ðrad=sÞ

ð4Þ

where, smin and ‘min should be positive. The constraints relating to s and ‘ are considered due to the mechanical/physical properties of the crane system (the height and length of the jib), while the constraint φ_ max is needed in order to avoid large swings of the load mass due to fast rotation caused by high rotating/angular velocity as well as it may relate to the specifications/limits of the actuator motor. If a controller violates one of the constraints, a penalty Epenlt is added to its evaluation value. In addition, this study fully considers damping coefficients in the system (i.e., Dφ, Dα, Ds), which are not always taken into account in the literature [1–5]. 3. A hybrid evolutionary algorithm of particle swarm optimization and genetic algorithm 3.1. Brief backgrounds on particle swarm optimization and genetic algorithms 3.1.1. Particle swarm optimization Inspired by the social behavior of a flock of birds, PSO is a population-based stochastic approach which provides an efficient performance with simple operators. In contrast with other EAs, PSO has a random velocity associated with each potential solution [23–26]. Fig. 4 shows the algorithm of a traditional PSO. Assuming that the search space is D-dimensional, the n-th particle (solution) of the swarm is represented by a D-dimensional vector W n ¼ ½wn1 ; wn2 ; ⋯; wnD T (in this study, Wn represents a set of the RNN's connection weights). The nth particle's velocity is also a D-dimensional vector, denoted as V n ¼ ½vn1 ; vn2 ; ⋅s; vnD T . The best position of the n-th particle is PBn ¼

58

S.C. Duong et al. / Automation in Construction 23 (2012) 55–63

Fig. 4. Pseudocode of a PSO. Fig. 5. Pseudocode of a GA. T

½pbn1 ; pbn2 ; ⋯; pbnD  (the local best, i.e., the smallest objective value that the n-th particle has obtained so far), and the best position of the T swarm is denoted as gb (gb ¼ ½gb1 ; gb2 ; ⋯; gbD  , the global best, i.e., the smallest objective value achieved by any particle in the population). After finding the two best values in the j-th iteration, the particle updates its velocity and position in the next iteration (j+1) with the following equations:     jþ1 j j j j j vnd ¼ λvnd þ c1 r1 pbnd −wnd þ c2 r 2 gbd −wnd jþ1

j

jþ1

wnd ¼ wnd þ vnd

ð5Þ ð6Þ

where d=1,2,⋯,D; n=1,2,⋯,N (N is the swarm population size); j=1,2, ⋯,J is the iteration (or generation) number; λ is the inertia weight; c1 and c2 are positive constants, called the cognitive and social components respectively; and r1 and r2 are random values in the range [0, 1]. In the constricted version proposed in [24], the original PSO following Eqs. (5) and (6) is modified to      jþ1 j j j j j vnd ¼ χ vnd þ c1 r 1 pbnd −wnd þ c2 r 2 gbd −wnd jþ1

j

jþ1

wnd ¼ wnd þ vnd

ð7Þ ð8Þ

where χ is the constriction coefficient defined as: 2 χ ¼  pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2−ε− ε 2 −4ε

shown in Fig. 6. First, individuals Wn in the population are evaluated and then parents are selected based on their fitness. After being produced by the GA via recombination and mutation, and then being evaluated, the new offspring WN + m (m = 1, 2, ⋯, M) sets its own value to be its local best PBN + m. These offspring enter the pool with zero initial velocity and join the search process. Then, the nextgeneration individuals of the Wn in the PSO are determined. A ranking procedure based on the fitness of each individual is performed for the pool of all Wn (associated with PBn) and WN + m (associated with PBN + m), from which the individuals with poor fitness are then eliminated to maintain a constant population. The individual with the highest fitness is set to be the global best gb, i.e., the solution to the problem being considered. To visualize the idea of the HEA, Fig. 7 is plotted, where W′n and PB′n are the updated position and local best by Eq. (8). The reproduction with fitness-based selection is the main difference in the HEA and GA compared with PSO. The GA and the proposed HEA would produce M = ξ × N offspring and also eliminate the same amount (the inferiors) to keep a constant population size N at each generation. We denote the GA and HEA accompanied by ξ value as GA-ξ and HEA-ξ. In general, a large value of ξ would result in better performance (since individuals have to overcome a severe selection) but also require higher computational cost. In the GA and HEA, the roulette wheel technique is used to select parents for reproduction in proportion to their relative fitness, which is defined as:

ð9Þ Fitness ¼

1 1þE

ð10Þ

where ε = c1 + c2. where E is the cost function value. 3.1.2. Genetic algorithms Inspired by Darwin's theory of evolution, GAs are adaptive heuristic search techniques based on the ideas of natural selection and genetics [20,21]. Generally, the main driving operators of GAs are selection, recombination via crossover, and mutation. These operators are used to generate the offspring based on the fitness of the parents. While recombination and mutation serve as a source of exploration, the fitness-based selection contributes to exploitation. Suppose that the probability of mutation is μ and that of the new offspring to be generated is ξ (i.e., there will be M = ξ× N offspring produced), Fig. 5 shows the flowchart of a canonical GA. In this study, the genetic operators of a binary-coded GA are embedded into the constricted PSO in order to develop a hybrid algorithm. 3.2. A hybrid evolutionary algorithm In this research, a hybrid of a PSO and GA is proposed, where the PSO is employed to update the population and the GA's genetic operators (crossover and mutation) are used to produce offspring. The flowchart of the proposed hybrid evolutionary algorithm (HEA) is

Fig. 6. Pseudocode of the HEA.

S.C. Duong et al. / Automation in Construction 23 (2012) 55–63

PSO & GA

Ranking

Survival PBN+8 WN+8

PB1 W1

PB1’

W1’

PBN+8 WN+8

PB2

PB2’

W2’

PB1’

. . .

PBN

. . .

PSO

PBN’ WN’

WN

Offspring WN+m production by GA

PBN+m=WN+m

Selection

W1’ . . .

PBN+5 WN+5

PBN+5 WN+5

W2’

PB2’

PBN+2 WN+2

PBN+m WN+m

. . .

. . .

PB6’

W1’

. . .

PBN+1 WN+1

PBN+M WN+M

PB1’

Eliminated W6’

j

Table 2 Parameters of the crane system. Parameter

Value

Jb Jw mt m Ds Dφ Dα rw [smin, smax] ½‘min ; ‘max  ϕ_ max Epenlt

480 (kgm2) 0.1 (kgm2) 10 (kg) 100 (kg) 75 (Ns/m) 15 (Nms) 15 (Nms) 0.1 (m) [0.5, 10.0] (m) [0.5, 10.0] (m) 1.0 (rad/s) 106

j+1 Fig. 7. Process of the HEA.

4. Simulation results 4.1. Evolutionary performance In order to validate the proposed method, a performance comparison will be made between the HEA and the original PSO and GA. In the comparison, the performances of the three EAs in evolving the recurrent neurocontroller (RNC) are evaluated by the mean rate of the successfully evolved controllers Rmean, i.e., the rate of successfully-evolved controllers over the total performed runs (or success rate), and the mean error value Emean of the best controllers obtained at each generation. A controller is considered to be successfully evolved if it can obtain an error smaller than a required accuracy after running the EA, that is, Eb Esuc = 10− 3. The above two factors are calculated over the replications of varying the initial system positions/configurations and the initial EA populations. The initial system configuration xinit is defined by the initial positions sinit and ‘init and the angle φinit, forming xinit ¼ ½φ; s; ‘; 0; 0; 0; 0; 0; 0; 0T (for simplicity, in the comparison we set zero-swing angles at the beginning of the control process, i.e., θ1init =θ2init =0). Tests are carried out for 20 times of varying initial system configurations which are generated by drawing random values uniformly distributed in the ranges sinit ∈ [0.5, 5.0], ‘init ∈½0:5; 5:0, and φinit ∈[0, π]. For each of the initial positions, we implement 20 times of randomly changing the initial population of the EAs, which has a strong effect to the competence of the algorithms. Without loss of generality, to make any possible movements of the crane from the above initial configuration ranges, the desired configuration is set to be xref ¼ ½π=2; 2:5; 2:5; 0; 0; 0; 0; 0; 0; 0T . Under a specific condition, it is generally possible to tune for the most suitable values of the hidden neuron number H in NNs. However, in order to reduce computational cost, only H = 2 hidden neurons are used herein for the RNN. Namely, the structure of the RNN is 10-2-3. In the network, we use a linear function fio. act(x) = x as the activation function of the input and output layers and a hyperbolic tangent

function fh. act(x) = tanh(x) for the hidden layer. The parameters of the GA are shown in Table 1, which are tuned based on trial and error such that the GA obtains the best performance. For the PSO, it is suggested in [24] that the values of the constants in the constricted PSO should be set as c1 = c2 = 2.05, hence ε = 4.1 and χ ≈ 0.7298, for assuring the convergence. The above parameters are then applied to the HEA in the comparison. It should be noted that these parameters are probably not the optimal values for the HEA, and so is the number of hidden neurons for the RNN. That is to say, we have favored the conventional GA and PSO in the comparison when selecting their parameters. In the comparison, we use the same population size and number of generations for the EAs, which are selected to be not too large with respect to computational cost as well as to show the advantage of the HEA since it is desirable that an EA can obtain the goal with as small population size and generation number as possible, especially in control systems design. On the other hand, as the probability of offspring production ξ has strong influence to the performance of the HEA and GA, we shall use only a small value of ξ in the HEA, that is, the HEA-0.1 is used, even though the optimal value in GA is ξ = 0.7 as shown in Table 1. Tests are executed using the physical parameters and constraints of the crane system shown in Table 2, and the fourth-order Runge–

(a) Mean success rate ( the values described in the line titles are the success rates [%] at the end of evolution processes)

R mean [%]

W2

59

90 80 70 60 50 40 30 20 10 0

HEA-0.1 (68.75%) PSO (65.50%) GA-0.7 (63.75%) HEA-0.7 (85.50%) 50

100

150

200

Generation

(b) Mean best error Table 1 Parameters of GA.

10 Value/scheme

Population size N Probability of offspring production ξ No. of generations Bit number Initial range Solution range of variables Crossover operator Mutation operator Mutation probability μ Selection scheme Qi

50 0.7 (M = 0.7 × N) 200 16 [− 1.5, 1.5] [− 250, 250] Two-point One-bit 0.3 Roulette wheel 1.0

10

Emean

Parameters

2

HEA-0.1 PSO GA-0.7 HEA-0.7

1

1 10 10 10

-1 -2 -3

50

100

150

200

Generation Fig. 8. Results of the comparative evolutionary performance test.

60

S.C. Duong et al. / Automation in Construction 23 (2012) 55–63

(a)

Table 3 Mean best error Emean and standard deviation S (Emean/S).

9.3658/3.2103 0.0316/.0581 0.0102/.0241 0.0051/.0122 0.0030/.0086

HEA-01

0.1574/.1548 0.0131/.0180 0.0051/.0098 0.0032/.0061 0.0022/.0048

0.0660/.0647 0.0092/.0126 0.0040/.0077 0.0027/.0051 0.0020/.0043

HEA-07 0.0489/.0456 0.0073/.0101 0.0031/.0059 0.0020/.0038 0.0014/.0030

Kutta method with time step of 0.01 s for the system simulation. The mean success rates and mean best errors of the EAs at each generation are computed, and the results are respectively shown in Figs. 8 (a) and (b). It turns out that, even with the unfavored parameter settings, the HEA-0.1 could obtain the highest mean success rate and lowest mean error compared to the conventional means. For reference, the result of the HEA-0.7 was also plotted in the figure to show that in case the HEA uses as high a value of ξ as the GA-0.7, it could obtain a more attractive result. In order to show further the advantage of the proposed method, we compare the algorithms using the standard deviation of the best evaluation function values calculated over the 400 runs of the test (20 times of changing initial system configurations and 20 times of changing initial populations) as shown in the following equation: vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u 400  2 u 1 X S¼t Ep −Emean 400−1 p¼1

ð11Þ

where Emean is the mean value of the best errors as depicted in Fig. 8 (b) and Ep are the best errors obtained in each run (i.e., at p-th trial, p = 1,2 ⋯, 400). The results are provided in Table 3, showing the small deviation values of the HEA. This outcome indicates that the HEA could provide good performance/robustness in most of the implemented trails. Clearly, the obtained results have shown that the superiority of the HEA has exhibited from the first generations of the evolution. In other words, the HEA could quickly find solutions at the early stage of evolution. 4.2. Control simulations As examples, control simulations are performed using the HEA-0.1 and RNN with the test conditions given in Table 4, where the second test seems to be more difficult with non-zero initial swing angles. As the results, the trajectories of the payload are shown in Figs. 9 and 10 for the two tests, respectively. More details of the control results are provided in Fig. 11, in the case of the Test no. 2 with xinit:2 , for example. It is clear that the load mass is regulated to the desired configuration within the constraints. 5. Discussion

Table 4 Test conditions for control simulations. Test no.

Condition (xinit and xref)

No. 1

xinit.1 = [π/2, 1.0, 0.5, 0, 0, 0, 0, 0, 0, 0]T xref.1 = [0, 5.0, 3.0, 0, 0, 0, 0, 0, 0, 0]T xinit.2 = [0, 10.0, 5.0, π/12, π/18, 0, 0, 0, 0, 0]T xref.2 = [π, 1.0, 1.0, 0, 00, 0, 0, 0, 0]T

No. 2

Start

0 End -0.5 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5

X [m]

(b) 3.5 3 2.5 2 1.5 1 0.5 0 0

1

2

3

4

5

6

7

8

9 10

Time [s] Fig. 9. Trajectories of the load mass on (a) X-Y plane and (b) Z axis in the case of test no. 1.

each other. The RNN is thus considered to be suitable for such a complex system like the crane. In order to improve the evolutionary performance, a hybrid of PSO and GA has been proposed. In our method, PSO is applied to the entire population and the individuals created by GA have to compete with the updated particles of PSO and the elimination of inferiors is performed only after the competition. This is one of the differences between our method and the algorithms introduced in [29–31], where PSO is applied to only a portion of the population. Besides, compared to the method proposed in [29], our method applies the GA operators to the unupdated swarm of PSO while in [29] the upper-half of the population (elites) is updated by PSO before being fed to GA. According to our tests, this sequence is not as good as the sequence used in the herein proposed method (selecting parents in GA before updating by PSO) for the problem being considered. In addition, while the methods in [29,31] employ truncation selection technique (i.e., use a fixed percentage of the best individuals) for later breeding and the algorithm in [30] randomly divides the population into two parts for being processed by PSO and GA, we use the fitness-proportionate selection (or roulettewheel technique [20]). This selection technique at first does not erase any individual deterministically. Instead, it probabilistically selects parents from the entire population in such a way that it can guarantee that the individuals with higher fitness will be more likely selected but

(a) 2 1.5 1 0.5 0 -0.5 End -1 -1.5 -2 -2 -1

Start

0

(b) Z [m]

This study has presented an intelligent control method for a constrained underactuated crane by using an RNN optimized by EA. Aside from the self-connected neurons, the RNN used in this paper includes the hidden and output layers that have other feedbacks interacting

1 0.5

Z [m]

10 50 100 150 200

GA-07

Y [m]

Methods PSO

Y [m]

Generation

2 1.5

1

5 4.5 4 3.5 3 2.5 2 1.5 1 0.5

2

3

4

5

6

X [m]

7

8

9

10

11

12

0 1 2 3 4 5 6 7 8 9 10

Time [s] Fig. 10. Trajectories of the load mass on (a) X-Y plane and (b) Z axis in the case of test no. 2.

S.C. Duong et al. / Automation in Construction 23 (2012) 55–63

vs. time 3.5 3 2.5 2 1.5 1 0.5 0 -0.5 1

2

3

4

5

6

7

8

9

10

Time [s] vs. time

φ dot [rad/s]

1 0.5

2

3

4

5

6

1.5 1 0.5 0 -0.5 -1 -1.5

7

8

9

10

7

8

9

10

7

8

9

10

7

8

9

10

7

8

9

10

θ1 dot θ2 dot

0

1

2

3

4

5

6

Time [s]

-0.5

(g) Mb vs. time 0

1

2

3

4

5

6

7

8

9

500 400 300 200 100 0 -100 -200 -300

10

Mb [Nm]

Time [s]

(c) s and a vs. time 10 9 8 7 6 5 4 3 2 1 0

s l

0

1

2

3

4

5

6

Time [s]

(h) Mw vs. time 1

2

3

4

5

6

7

8

9

10

Time [s]

(d) s and a vs. time

Mw [Nm]

100 0

0 -100 -200 -300

1 0.5 0 -0.5 -1 -1.5 -2 -2.5 -3

-400

0

1

2

3

4

5

6

Time [s]

(i) F vs. time s dot l dot 0

1

2

3

4

5

6

7

8

9

10

Time [s]

F [N]

s, l [m]

1

vs. time

0

-1

s dot, l dot [m/s]

θ1 θ2

Time [s]

(f) 0

(b)

vs. time 0.3 0.2 0.1 0 -0.1 -0.2 -0.3 -0.4 0

θ1 dot , θ2 dot [rad/s]

φ [rad]

(a)

θ1 , θ2 [rad]

(e)

61

100 0 -100 -200 -300 -400 -500 0

1

2

3

4

5

6

Time [s] Fig. 11. Details of the control result in the case of test no. 2.

still remains a chance for some week candidates to survive and possibly become good solutions after the reproduction process. Moreover, erasing a portion of the PSO swarm at the beginning would be a drawback because there might be a situation in which the new offspring created by GA are even worse than the inferior particles in PSO. Keeping the entire population in this case is useful to maintain better solutions compared to the low-grade new offspring of GA. The performance of the HEA was compared with those of the canonical approaches, and showed the good performance of the proposed method regardless of the small values of the generation number, population size, and reproduction probability. In the tests, although its parameters were tuned for the most suitable values, the GA exhibited a slightly poor performance at the end. This is

apparently due to the small values of generation and population size because, in general, GAs usually require larger values. Overall, its performance increased with increasing generation and it provided a good mean-error curve. The PSO appeared to converge rapidly regarding to the mean success rate Rmean, but only after the first 50 generations and it resulted in a high mean-error curve. At the beginning there might be a strong random population, the PSO therefore resulted in low success rates and several large-error individuals. Basically, PSO can maintain good diversity as it updates all of the individuals at every generation. However, without a selection operator, individuals in PSO tend to follow the best solution found so far, and only get into the neighborhood of the solution. This may be the cause of high mean error values as well as it may lead to a situation where the algorithm is sometimes unable to

62

S.C. Duong et al. / Automation in Construction 23 (2012) 55–63

explore other regions. In contrast, the proposed HEA could utilize the advantages of both PSO (with a fine search around the feasible solution) and the GA's genetic operators (with a widespread search with fitnessbased selection). In the HEA, the GA's crossover and mutation operators serve as an important role of introducing new individuals to the pool so as to enrich the diversity of the population, which is much useful for the complex problem being considered. It thus allows the algorithm to search over a wide space beside the PSO. In addition, with a fitnessbased selection process and elimination of inferior individuals, along with the PSO operator, the HEA tends to focus on feasible regions. These two features (wide search and focus) make the HEA able to achieve a good performance, even from the beginning of the evolution. The result of the HEA-0.7 showed that it is possible to improve the performance of the HEA using a higher value of ξ, because that would mean a more severe selection. 6. Conclusion Although underactuated systems seem to be a potential approach with several benefits bringing it to broad applications, it is a fact that control of such systems is a challenging task. In this study, we have presented a control method for an underactuated three-dimensional rotary tower crane system by an RNN, which was used due to its several advantages. The damping coefficients and constrained states have been taken into account. To overcome the difficulties when using gradient-based learning algorithms, EAs have been employed to optimize the RNN. To improve the optimizing performance, a hybrid (HEA) of PSO and GA has been proposed. It was shown that, although the parameters of the HEA were set unfairly in the comparison, the HEA has provided an attractive performance compared to the conventional GA and PSO. Also, by using the scheme, it is convenient to consider the constrains as well as the damping in the system. As shown in the control simulations, the controller was effective, allowing it able to regulate the system to the desired point, even in the later test with non-zero swing angles (test no. 2). As our first research stage, this work has only shown the numerical simulations. An experimental work should be carried out for future work, where more concrete considerations may be needed for the control problem. Also, a set of optimal parameters of the HEA for the problem should be examined. Appendix A. Dynamics of the tower crane system Equation of motion € þ Dðq; q_ Þ ¼ Fðq; q_ Þ þ Bu MðqÞq 2

M 11 6 M 12 6 6 MðqÞ ¼ 6 6 M 13 6 4M 14 M 15

M 12 mt þ m M 23 0 M 25

M13 M23

ðA:1Þ

M14 0

Jw þm 0 r 2w 0 M44 0 0

3 M 15 M 25 7 7 7 0 7 7 7 0 5 M 55

8 2 2 2 2 2 > M11 ¼ J b þ mt s þ mðs þ ‘sinθ2 Þ þ m‘ sin θ1 cos θ2 > > > > M 12 ¼ m‘sinθ1 cosθ2 > > > > M13 ¼ −mssinθ1 cosθ2 > > > > M14 ¼ −mðs þ ‘sinθ2 Þ‘cosθ1 cosθ2 < M15 ¼ mðssinθ2 þ ‘Þ‘sinθ1 > > M 23 ¼ msinθ2 > > > > M 25 ¼ m‘cosθ2 > > > 2 2 > > M 44 ¼ m‘ cos θ2 > > : 2 M55 ¼ m‘

ðA:2Þ

2

3 d1 6 d2 7 6 7 7 Dðq; q_ Þ ¼ 6 6 d3 7 4d 5 4 d5

ðA:3Þ

8 2 2 > __ > > d1 ¼ 2fðmt þ mÞs þ m‘sinθ2 gφ_ s_ þ 2mfðs þ ‘sinθ2 Þsinθ2 þ ‘sin θ1 cos θ2 gφ ‘ > > > 2 2 2 > > þ2m‘ sinθ1 cosθ1 cos θ2 φ_ θ_ 1 þ 2mðs þ ‘cos θ1 sinθ2 Þ‘cosθ2 φ_ θ_ 2 > > > > > > > −2mðs þ ‘sinθ2 Þcosθ1 cosθ2 ‘_ θ_ 1 þ 2mðssinθ2 þ ‘Þsinθ1 ‘_ θ_ 2 > > > > > 2 > > þm‘ðs þ ‘sinθ2 Þsinθ1 cosθ2 θ_ 1 þ 2m‘ðs þ ‘sinθ2 Þcosθ1 sinθ2 θ_ 1 θ_ 2 > > > > > 2 > > þms‘sinθ1 cosθ2 θ_ 2 > > > > > 2 > > d ¼ −fðmt þ mÞs þ m‘sinθ2 gφ_ þ 2msinθ1 cosθ2 φ_ ‘_ þ 2m‘cosθ1 cosθ2 φ_ θ_ 1 > > 2 < 2 −2m‘sinθ1 sinθ2 φ_ θ_ 2 þ 2mcosθ2 ‘_ θ_ 2 −m‘sinθ2 θ_ 2 > > > 2 2 2 2 > > d3 ¼ −mðssinθ2 þ ‘sin θ2 þ ‘sin θ1 cos θ2 Þφ_ −2msinθ1 cosθ2 φ_ s_ > > > > > 2 2 2 > > þ2m‘cosθ1 sinθ2 cosθ2 φ_ θ_ 1 −2m‘sinθ1 φ_ θ_ 2 −m‘cos θ2 θ_ 1 −m‘θ_ 2 > > > > > > > d4 ¼ −m‘2 sinθ1 cosθ1 cos2 θ2 φ_ 2 −2m‘cosθ1 cosθ2 φ_ s_ −2m‘cosθ1 sinθ2 cosθ2 φ_ ‘_ > > > > > 2 2 2 2 > > −2m‘ cosθ1 cos θ2 φ_ θ_ 2 þ 2m‘cos θ2 ‘_ θ_ 1 −2m‘ sinθ2 cosθ2 θ_ 1 θ_ 2 > > > > > 2 2 > d ¼ −mðs þ ‘cos θ sinθ Þ‘cosθ φ_ þ 2m‘sinθ sinθ φ_ s_ þ 2m‘sinθ φ_ ‘_ > > 5 1 2 2 1 2 1 > > > > 2 2 2 2 : þ2m‘ cosθ1 cos θ2 φ_ θ_ 1 þ 2m‘‘_ θ_ 2 þ m‘ sinθ2 cosθ2 θ_ 1

2

3 −Dφ φ_ 6 7 _ −Ds s 6 7 6 Dα 7 _ þ mgcosθ cosθ 7 − ‘ Fðq; q_ Þ ¼ 6 1 27 6 r2 6 7 w 4 5 −mg‘sinθ1 cosθ2 −mg‘cosθ1 sinθ2 2

1 60 6 6 Bu ¼ 6 0 6 40 0

3 2 3 0 Mb 0 72 M 3 6 F 7 7 1 7 74 b 5 6 7 0 ¼6 F 6 Mw =rw 7 rw 7 7 4 0 5 0 0 5 Mw 0 0 0

ðA:4Þ

0 1

ðA:5Þ

References [1] E.M. Abdel-Rahman, A.H. Nayfeh, Z.N. Masoud, Dynamics and control of cranes: a review, Journal of Vibration and Control 9 (2003) 863–908. [2] W. Blajer, K. Kolodziejczyk, Dynamics and control of rotary cranes executing a load prescribed motion, Journal of Theoretical and Applied Mechanics 44 (4) (2006) 929–948 Warsaw. [3] W. Blajer, K. Kolodziejczyk, Improved DAE formulation for inverse dynamics simulation of cranes, Multibody System Dynamics, Springer, Oct. 2010, doi: 10.1007/s11044-010-9227-6. [4] Y. Rosenfeld, A. Shapira, Automation of existing tower cranes: economic and technological feasibility, Automation in Construction 7 (1998) 285–298. [5] Omar H.M., Control of Gantry and Tower Cranes, PhD Thesis, Virginia Polytechnic Institute and State University, Virginia, USA, 2003. [6] M.W. Spong, Underactuated mechanical systems, Control Problems in Robotics and Automation, in: B. Siciliano, K.P. Valavanis (Eds.), Lecture Notes in Control and Information Sciences, vol. 230, Spinger-Verlag, London, 1998, pp. 135–150. [7] H. Ohno, T. Suzuki, K. Aoki, A. Takahasi, G. Sugimoto, Neural network control for automatic braking control system, Neural Networks 7 (8) (1994) 1303–1312. [8] M.L. Corradini, G. Ippoliti, S. Longhi, Neural networks based control of mobile robots: development and experimental validation, Journal of Robotic Systems 20 (10) (2003) 587–600. [9] S. Omatu, M.B. Khalid, R. Yusof, Neuro-Control and Its Applications (Advances in Industrial Control), Springer, 1996. [10] E.N. Sanchez, M.A. Bernal, Adaptive recurrent neural control for nonlinear system tracking, IEEE Transactions on Systems, Man, and Cybernetics Part B: Cybernetics 30 (6) (2000) 886–889. [11] S. Zerkaoui, F. Druaux, E. Leclercq, D. Lefebvre, Stable adaptive control with recurrent neural networks for square MIMO non-linear systems, Engineering Applications of Artificial Intelligence 22 (2009) 702–717. [12] D.E. Rumelhart, G.E. Hinton, R.J. Williams, Learning representations by backpropagating errors, Nature 323 (1986) 533–536. [13] R.J. Williams, D. Zipser, A learning algorithm for continually running recurrent neural networks, Neural Computation 1 (2) (1989) 270–280. [14] A. Blanco, M. Delgado, M.C. Pegalajar, A real-coded genetic algorithm for training recurrent neural networks, Neural Networks 14 (2001) 93–105.

S.C. Duong et al. / Automation in Construction 23 (2012) 55–63 [15] P.J. Angeline, G.M. Saunders, J.B. Pollack, An evolutionary algorithm that constructs recurrent neural networks, IEEE Transactions on Neural Networks 5 (Jan. 1994) 54–65. [16] P.J. Fleming, R.C. Purshouse, Evolutionary algorithms in control systems engineering: a survey, Control Engineering Practice 10 (2002) 1223–1241. [17] R. Martinez, A. Rodriguez, O. Castillo, L.T. Aguilar, Type-2 fuzzy logic controllers optimization using genetic algorithms and particle swarm optimization, Proc. IEEE International Conference on Granular Computing (GrC), San Jose, CA, 14–16 Aug. 2010, pp. 724–727. [18] K. Kristinsson, G.A. Dumont, System identification and control using genetic algorithms, IEEE Transactions on Systems, Man, and Cybernetics 22 (5) (1992) 1033–1046. [19] R.S. Sexton, R.E. Dorsey, J.D. Johnson, Optimization of neural networks: a comparative analysis of the genetic algorithm and simulated annealing, European Journal of Operational Research 114 (3) (1999) 589–601. [20] D.E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Addison-Wesley, 1989. [21] M. Srinivas, L.M. Patnaik, Genetic algorithms: a survey, IEEE Computer 27 (6) (1994) 17–26. [22] V.G. Gudise, G.K. Venayagamoorthy, Comparison of particle swarm optimization and backpropagation as training algorithms for neural networks, Proc. IEEE Swarm Intelligence Symposium, SIS'03, Indiana, USA, 2003, pp. 110–117. [23] J. Kennedy, R.C. Eberhart, Particle swarm optimization, Proc. of IEEE Intl. Conference on Neural Networks, Piscataway, NJ, 1995, pp. 1942–1948. [24] M. Clerc, J. Kennedy, The particle swarm-explosion, stability, and convergence in a multidimensional complex space, IEEE Transactions on Evolutionary Computation 6 (1) (2002) 58–73.

63

[25] P.J. Angeline, Evolutionary optimisation versus particle swarm optimisation: philosophy and performance differences, Lecture Notes in Computer Science, vol. 1447, Springer, 1998, pp. 601–610. [26] K. Yasuda, N. Iwasaki, G. Ueno, E. Aiyoshi, Particle swarm optimization: a numerical stability analysis and parameter adjustment based on swarm activity, IEEJ Transactions on Electrical and Electronic Engineering, vol. 3, Wiley InterScience, 2008, pp. 642–659. [27] F. Valdez, P. Melin, O. Castillo, Fuzzy logic for combining particle swarm optimization and genetic algorithms: preliminary results, MICAI 2009: Advances in Artificial Intelligence, Lecture Notes in Computer Science, vol. 5845, Springer, 2009, pp. 444–453. [28] F. Valdez, P. Melin, O. Castillo, An improved evolutionary method with fuzzy logic for combining particle swarm optimization and genetic algorithms, Applied Soft Computing 11 (2) (2011) 2625–2632. [29] C.-F. Juang, A hybrid of genetic algorithm and particle swarm optimization for recurrent network design, IEEE Transactions on Systems, Man, and Cybernetics Part B: Cybernetics 34 (2) (2004) 997–1006. [30] F. Grimaccia, M. Mussetta, R.E. Zich, Genetical swarm optimization: self-adaptive hybrid evolutionary algorithm for electromagnetics, IEEE Transactions on Antennas and Propagation 55 (3) (2007) 781–785 [Piscataway, NJ]. [31] Y.T. Kao, E. Zahara, A hybrid genetic algorithm and particle swarm optimization for multimodal functions, Applied Soft Computing, vol. 8, iss. 2, Elsevier B.V., 2008, pp. 849–857.