- Email: [email protected]
Neural network modelling and control for underwater vehicles
Artificial fnfelligence in Engineering 1 (1996) 203-212
0954-1810(95)00029-l
EISEVIER
Copyright 0 1996 Elsevier Science Limited Printed in Great Britain. All rights reserved 0954-1810/96/$15.00
Neural network modelling and control for underwater vehicles V. S. Kodogiannis, P. J. G. Lisboa* & J. Lucas Department
of Electrical Engineering and Electronics,
University of Liverpool, Brownlow Hill, PO Box 147, Liverpool,
UK, L69 3BX
(Received 24 August 1994; revised version received 24 August 1995; accepted 5 October 1995)
Neural networks are currently finding practical applications ranging from ‘soft’ regulatory control in consumer products to accurate control of non-linear plant in the process industries. This paper describes the application of neural networks to modelling and control of a prototype underwater vehicle, as an example of a system containing severe non-linearities. The most common implementation strategy for neural control is model predictive control, where a model of the process is developed first and is used off-line to design an appropriate compensator. The accuracy and robustness of this control strategy relies on the quality of the non-linear process model, in particular its ability to predict the plant response accurately multiple-steps ahead. In this paper, several neural network architectures are used to evaluate a long-range model predictive control
structure, both in simulation and for on-line control of vehicle depth, achieving accurate control with a smooth actuator signal. Key words: underwater vehicles, recurrent neural networks, model predictive control strategies.
1 INTRODUCTION
linearities in practical plant with all of the fluctuations inherent in industrial sensors and actuators.3 The control loop used typically in these applications follows the model predictive control architecture used in this paper. In the pH example, the process model is needed in order to select appropriate control actions to return the controlled value to the set point while avoiding any undershoot when the desired value of the pH is near the shoulder at the top of the titration curve. In this paper we report a controller design for depth control of a prototype underwater vehicle with a single degree of freedom, as a preliminary stage in the selection of a control strategy for an unmanned vehicle with six degrees of freedom for underwater inspection. The design of controllers for underwater vehicles (UVs) is challenging because of difficulties in accurately modelling the inherently non-linear dynamics of UVs in a hazardous environment with persistent unmodelled disturbances. In general, real-time control of non-linear systems with unknown structure and parameter uncertainty remains an open area of research. Several extensions of linear system design techniques have been studied. For nonlinear systems, the most direct approach is modelling of the plant dynamics using physical laws in conjunction with off-line experiments to determine the relationship between the inputs and outputs of the plant. Modelling techniques, such as fitting linear
Continuing developments in soft information processing, typified by fuzzy logic and neural networks, have led to a more pragmatic approach towards the control of systems with uncertainties or non-linearities. Examples of the former are a number of products in the consumer industry, such as auto-focus systems, the control of kerosene heaters and air-conditioning, all of which have already marketed products with fuzzy and neural components.’ In order to deal with the variability of sensor conditions which influence the controller settings, for instance image contrast has an important bearing on the speed of response of the auto-focus controller, neural and fuzzy systems are used in combination. The neural network may define the membership functions for a fuzzy controller, or it may act in parallel to add a correction term to the output from the fuzzy controller, or a fuzzy pre-processor may be cascaded with a neural supervisory controller.’ In addition, neural controllers are increasingly used in process control with a number of practical applications of this technology.“2 Our own experience with on-line control of pH in a sulphonation loop reactor shows the robustness of these methods in handling severe non*To whom correspondence
should be addressed. 203
204
V. S. Kodogiunnis, P. .I. G. Lishorr, J. Lucas
models to frequency response curves, finite element analysis and stochastic modelling have been successful at characterising classes of unknown plants. Although conventional control techniques (both adaptive and non-adaptive) using standard mathematical methods have performed remarkably well, their appeal tapers off when one is confronted with the control of complex systems characterised by poor models, high dimensionality of the decision space, distributed sensors. distributed decision makers, multiple time scales, multiple time constants, high noise levels, drifting parameter values, stringent performance requirements, non-linearities, and so on. As a system becomes more complex, so does the complexity associated with the computation of the control law and the task of implementing the control in a timely fashion. In view of the very difficult control problems presented by UVs operation and to handle uncertainties, intelligent techniques have been deployed. One such application involves a fuzzy six degree of freedom flight control system for the Ocean Voyager, a UV currently under development by the Advanced Marine Systems Group at Florida Atlantic University (FAU).4 The control system comprises a simple proportional controller of forward speed, and three separate fuzzy logic controllers for heading, pitch and depth. Each fuzzy controller has two components: a rule-based fuzzy controller and a fuzzily constrained integrator. The Ocean Voyager was simulated with a highly accurate six degree of freedom non-linear simulation model developed at FAU. The fuzzy control system seems to be robust to changes in vehicle speed. The emergence of neural networks (NNs) as effective learning systems for a wide variety of applications has resulted in the use of these networks as learning models for dynamical systems. For an overview of the learning control techniques using NN and discussions on some of the applications, the reader is referred to Ref. 5. More recently, NN approaches have been developed to control underwater vehicles. One of the important advantages of using NNs for control applications is that the dynamics of the controlled system (plant) need not be completely known as a prior condition for controller design. This is a very desirable feature in the design of the controller, because of the non-linearities arising from rigid-body coupling and also the unpredictable nature of disturbances from the sea environment. Also, the ability of these networks for adaptation and disturbance rejection and their highly parallel nature of computation make this approach suitable for real-time control applications. In one of the simplest approaches, a neural network was trained using the chemotaxis algorithm to invert a linearised UVA model.6 The inverse network replaced a PID controller as a feedforward controller. Although that NN controller provided a smoother output than the oscillatory PID, its mean-squared-error was considerably worse. However, there is no guarantee that an inverse model will exist, therefore the generality of the proposed method cannot
be guaranteed. A simulated neural network based control scheme for UV is described by Yuh in Ref. 7. Direct control is used with the error back propagation as the learning algorithm applied to a three-layer network, in simulation, and good trajectory tracking is achieved for the vehicle. The above control scheme does not require any information about the system dynamics except an estimate of the inertial terms. Although there is a growing interest in applying neural networks to non-linear systems, little work has been reported on using such networks for practical realtime control. In this paper, we describe the application of NNs to the modelling and positional control of a prototype SISO UV, which we call the Aquacube. The goal is to develop a non-linear controller to keep station. or ‘hover’, the vehicle at specified depths. To meet this goal, two objectives for control of the vehicle are proposed. First to develop accurate model architectures for the prototype UV. Such architectures include recurrent networks and memory neuron networks, and second to develop a technique for controlling the UV using the previous neural network models. These models are then incorporated into a long-range predictive control strategy which is evaluated both in simulation and on-line. Results are shown for both the modelling and real-time control of the Aquacube.
2 NON-LINEAR
SYSTEM
MODELLING
The investigation of NNs for system modelling and control has been carried out on an underwater prototype, the Aquacube, which comprises a single thruster aligned vertically and mounted in a steel endoskeleton. All information and power is transmitted via an umbilical which is non-tensioned and made approximately neutrally buoyant, in order to minimise the static forces. Buoyancy was also added to the vehicle to make it slightly positively buoyant. Thrust is provided by a brushless d.c. motor driven by a PWM inverter, and motor speed is commanded by applying an analogue voltage (It 1OV) to the motor controller. In this arrangement, striction can lead to pronounced dead bands for small thrust values which are crucial in fine position control. Although the vehicle can be considered as a SISO plant, its dynamics are highly non-linear. The hydrodynamic coefficients of the vehicle are unknown, and the drift due to water and the buoyancy generate additional forces and moments. Similarly, the acceleration and deceleration of the vehicle affects its overall dynamics. 2.1 Underwater vehicle characteristics Knowledge of the system indicates that the vehicle and the surrounding environment are likely to exhibit predominantly fourth order dynamics.8 To calculate
Neural network modelling and control for underwater vehicles
the system’s overall response a series of frequency response tests were carried out. The test used was to give a sinusoidal control input to the PWM unit, whilst measuring the vehicle’s depth throughout. From repetitions of this test at different frequencies in the range 0.35-5.236rad/s, the frequency response of the vehicle can be calculated. The frequency response of the Aquacube prototype for small oscillations of the control signal can be calculated using Fourier methods. MATLAB has a rich collection of functions useful for system identification. It is easy to compute the FFT for the sinusoidal inputs and the corresponding vehicle positions, and therefore the magnitude of the Aquacube’s transfer function can be computed as the ratio of the output magnitude to the input magnitude. Similarly, the phase of the actuator signal is subtracted from that of the Aquacube’s response. 2.2 Modelling and approximation using neural networks A critical element in any control task is to build an adequate model of the plant. This is where it is claimed that neural networks have an advantage over classical techniques because they can be trained with data observed from the real plant, to reproduce the response of the plant treated as a black box model. The neural networks are in fact non-linear, multi-variable function approximation tools. The training involves the optimisation of the weights in the network until the knowledge distributed among them is sufficient to accurately approximate the response of the plant. A training set of input output pairs is needed to provide sufficient information to model the non-linear system. An input signal must be chosen which will excite all the dynamic modes of the system and cover the whole amplitude range of interest, otherwise the validity of the model will become highly input sensitive. As a consequence of the slightly positive buoyancy of the vehicle, the thruster requires a minimum value, which ranges in region of 2-3 V, in order to maintain or increase the current vehicle depth position. It is critical for the NN to learn the response characteristics due to the buoyancy and actuator deadband. In order to achieve this, a series of step control signals were taken for training, and a further set collected for validation tests. The neural network representing the forward dynamics of the Aquacube has the following NARX (nonlinear autoregressive model with exogenous input) configuration: y”(t) =f(u(t
205
based on the network achieving an acceptable mean square error (MSE) on the training data without excessive computation time. A network with one hidden layer provided a poor performance, while a NN having two hidden layered structures was much more accurate. For modelling the Aquacube, 18 and 12 nodes were used in the two hidden layers respectively, and standard sigmoidal functions as the non-linear neural activation function.” We also tested the effect of having more neurons at each layer without any significant improvement. Therefore, the network with a 5/18/12/l topology was trained as a one-step ahead predictor using the back-propagation (BP) algorithm.’ ’ Despite the fact that the forward neural model is very accurate, as a one-step-ahead predictor, it cannot be used independently as a plant simulator, or to provide long range predictions, because it is unstable when used recursively.
3 RECURRENT
NETWORKS
FOR MODELLING
The application of NNs to problems involving time varying or dynamic patterns is an active research area. In contrast to the series-parallel model, the network can be arranged completely parallel to the system, which is therefore referred to as a parallel ident@ation method. While the series-parallel method requires only a feedforward network, topologically, the parallel identification method results in a neural network with delayed recurrent connections from its output neurons back to its own input neurons as shown in Fig. 1. The use of recurrent neural networks as system identification networks offers a number of potential advantages over the use of static layered networks. Recurrent neural networks provide a means for encoding and representing internal or hidden states, albeit in a potentially distributed fashion, which leads to capabilities that are
- l),_YP(t - l),.YP(t - 21,
yP(t - 3),yP(t - 4))
(1)
where y m(t) is the model predicted output of the network. In eqn (1) the network is supplied with the input voltage to the thruster, i.e. u(t - l), and the vehicle’s delayed depth positions, i.e. yP(t - n). The internal topology was
Fig. 1. Parallel identification
method using neural networks.
206
V. S. Kodogiannis,
P. J. G. Lisboa, J. Lucas
similar to those of an observer in modern control theory. Recurrent network architectures provide increased flexibility for filtering of noisy inputs.12 An accurate recurrent plant model is highly desirable since it can be used independently from the plant for separate dynamic simulation, whereas in the one-step-ahead predictive structure the network is dependent on data immediately available from the plant. The most commonly used training algorithms for recurrent networks, to date, BPTT (backpropagation through time)13 and RTRL (real-time recurrent learning),i4 have large memory requirements, therefore alternative methods must be considered for use in a real-time control strategy. It has been found that a major factor affecting the neural model prediction accuracy when it is acting as a recurrent model is the method by which data are coded in the network. The reason for this is because any errors in the predicted output will tend to accumulate. The conventional method of conditioning the data is to rescale and represent using a single node at the input or output layers of the network. An alternative representation, called spread encoding (SE), has been shown to enable a network to maintain a high degree of accuracy when operated in a recurrent mode by exploiting the network’s fault tolerance.i5 Figure 2 illustrates the internal architecture of this technique. In the SE technique, each data value is represented as the mean value of a sliding Gaussian pattern of excitation over several nodes at the network input and output. A similar and reverse procedure is applied at the network output to decode the output back into the original variable range. For our problem, we used 36 input nodes and 6 outputs. Although the training time was considerably less than the time for BPTT, the size of the network makes it unsuitable for real-time control. Figure 3 illustrates the validation of the one-stepahead and the recurrent SE network respectively. The performance of the network operating recurrently is indicated by a MSE of depth 0.008. It is clearly seen to track the Aquacube’s response over the complete duration of the test.
The representation of temporal knowledge and time correlation in neural networks presents a continuous challenge to researchers in the field of artificial neural networks. In the previous section, the so called windowed input network has been applied for an iterative response modelling of the Aquacube. Another approach has been that of explicitly including the time dependency into the network structure. The commonly used backpropagation algorithm contains no memory, hindering the learning of temporal patterns. Here, the alternative is to use a dynamic network that is given some kind of memory to encode past history. In this section, our primary objective is to design recurrent models of our underwater prototype to carry out five-steps-ahead accurate predictions with the additional requirements of short training time and small network size. Each time step corresponds to the sampling interval which was set to 0.3 seconds. This section describes an extension to error backpropagation that allows the nodes in a neural network to encode state information. The two recurrent architectures, are the autoregressive network and a modified version of the well known Elman network, both of which are examples of internal state networks.
Fig. 2. Spread encoding neural architecture.
3.1.1 The autoregressive backpropagation network In this section, a novel network architecture, called the autoregressive recurrent network (ARNN), is proposed. It was designed to contain internal memory of the previous states, while training rapidly using a generalised BP algorithm. The idea is that we still use a recurrent neural network model but the recurrent neurons are decoupled so that each neuron only feeds back to itself. With this modification, the ARNN model is considered to converge easier and to need less training cycles than a fully recurrent network. In a standard multilayer perceptron, the inputs to a neuron are multiplied by feedforward weights and summed, along with a node bias term. The sum is then passed through a smooth sigmoidal transfer function, producing the neuron’s output value. This neural model
NO.
or pattmu (mocc)
Fig. 3. System modelling using MLPs. 3.1 Internal state recurrent networks
Neural network modelling and control for underwater vehicles
207
model for our underwater prototype. The five memories in each node at the first hidden layer allow the network to encode state information. A structure of 6/20/14/l nodes has been used, with inputs consisting of one past control signal and five past depth indications. 3.1.2 Elman network
Input UnIta
Fig. 4. ARNN architecture.
has no memory, because the output value is not explicitly dependent upon previous outputs. The ARNN is a hybrid feedforward/feedback neural network, with the feedback represented by recurrent connections appropriate for approximating the dynamic system. The structure of the ARNN is shown in Fig. 4. There are two hidden layers, with sigmoidal transfer functions, and a single linear output node. The ARNN topology allows recurrence only in the first hidden layer. For this layer, the memoryless backpropagation model has been extended to include an autoregressive memory, a form of self-feedback where the output depends also on a weighted sum of previous outputs. A modified backpropagation algorithm is developed to train the ARNN which includes dynamic recursive equations in time. The mathematical definition of the ARNN is shown below:
_v(t)= o(t) = Sl = C
zj(t) =f
cI w?Qr(t),
QI =f (S/)
(4
WjyZj(t)
As an alternative to the ARNN, the network developed by Elman16 has a simpler architecture and it can be trained using the standard backpropagation learning algorithm. The output nodes are linear. The context nodes interact only with the hidden nodes and have fixed recurrent connections. The context units of the Elman network memorise past states of the hidden units, so the output of the network depends on an aggregate of the previous states and the current input. This network has been proved to be effective for modelling linear systems not higher than the first order.” For this reason, an idea based on the work of Hertz et al.‘* was employed to configure a modified Elman network which is shown in Fig. 5. Here, self-connections are introduced in the context units of the network in order to give these units a certain amount of inertia. The introduction of self-feedback in the context units increases the possibility of the Elman network to model high-order systems. We use this network to generate five-steps-ahead predictions for the Aquacube. For this purpose, the output of the jth context unit in this modified Elman network structure is given by X,j(t
+
1) =
Xj(t)
+
OZXj(t
-
+
0J3Xj(t
-
3) +
+
a5Xj(t
-
5)
1) + a4Xj(t
0Z2Xj(t
-
-
4)
CHjCt))
k=l
WjyZj(t
- k) + C
Wi;Zi i
where Zi(t) is the ith input to the ARNN, Hi(t) is the sum of inputs to the ith recurrent neuron in the first hidden layer, Zj(t) is the output of the jth recurrent neuron, S/(t) is the sum of inputs to the Ith neuron in the second hidden layer, Q,(t) is the output of the Ith neuron in the second hidden layer and O(t) is the output of the ARNN. Here, f (-) is the sigmoid function and W’, WD, WH and W” are input, recurrent, hidden and output weights, respectively. Since the objective is to demonstrate a five-step-ahead recurrent model of the Aquacube, therefore the number of internal memories in the hidden nodes was set to five. The ARNN was trained as a five-step-ahead prediction
(4)
Usually, o is between 0 and 1. A value of (Ynearer to 1 enables the network to trace further back into the past. In order to enhance network’s performance we have added an extra hidden layer and replaced the linear output function with a standard sigmoidal one. Therefore
k=5
Hj(t) = C
2)
Fig. 5. Modified Elman architecture.
208
V. S. Kodogiannis, P. J. G. Lisboa, J. Lucas
No.
-‘Nmud
Fig. 6. System modelling
using internal
state networks.
net -Aq-be
of phmnm (mma) ‘-input
Fig. 7. Elman’s autonomous
a 3/12/14/l Elman network was applied with self-feedback in the 12 context units, with Q equal to 0.8. The input to the network consisted of the current control signal and the current and previous system positions, to enable the network to emulate externally the integrator action of the Aquacube from the input-output data. Figure 6 illustrates the performances of the modified Elman network and the ARNN as five-step-ahead predictors. An MSE of 0.011 has been achieved for the former case while an MSE of 0.02 for the latter one. In summary, the approach of producing a dynamic memory is clearly simpler than the SE network with the result that the computational burden is substantially reduced. Although the ARNN architecture is dependent on the number of memories in the recurrent nodes, it proved to be the fastest in training time. We proceeded to examine the performance of the Elman network in an autonomous iterative response. The SE network has already proved its ability to work practically independent from a real system. For a singleinput single-output system, such as the underwater prototype, we are interested in a network with a single input and output line. The integrator in the response of the Aquacube was removed by modelling
0.61 1
61
101
li@d
response
161
test 1.
eoi
No. d pattamm (mw)
1-.Nowmlnet -Aq-be.
-.input rnw
Fig. 8. Elman’s autonomous
response
1 test 2.
Y(s) = V(S)
(5) Equivalent, in the z-plane domain, the above relation is transformed to y(t) = l’(t - 1) + TY(t)
(6)
where T is the sampling interval, equal to 0.3 s. A network structure of 1/14/l was used to predict iteratively the Y(t) term, and eqn (6) was employed to reconstruct the actual position. Figures 7 and 8 illustrate the validation of this approach in a number of autonomous iterative response tests. The proposed modification has improved the versatility of the network. As shown from these results, the network appears to have a drift similar to the real underwater vehicle. This fact, in conjunction with the simpler network complexity, proves that the current approach is better suited to this kind of sequence processing than the SE network.
-Elmm Il. +Elman Ph. *@mube Fig. 9. Open-loop
Y *41mmaba
Bode diagram for the Aquacube network.
Ph.
and Elman
The open-loop Bode plot of the Aquacube is shown in Fig. 9, superimposed upon the frequency response derived using the same Fourier methods applied to a recursive neural network model. An important consideration in the design and implementation of neural network plant models is the extent to which they capture the process dynamics. We have investigated this in our neural network models using standard spectral methods by applying small PRBS driving signals to the recursive neural models. The results were analysed using the
Neural network modelling and control for underwater vehicles 100
AMPLITUDE
PLOT, input
#
1 output
209
# 1
10-l
10-Z
1 O-3 10-2
1 o-’
100
8 o -150 .C m
-200
PHASE
102
10’
102
(md/ssc)
frequency -100
10’
PLOT, input
#
1 output
# 1
-
I
1
1o-2
10-l
lgo frequency
Fig. 10. Elman’s ‘continuous’
(md/aec)
Bode plot obtained by using PRBS signals.
MATLAB Signal Processing Toolbox” and are shown in Fig. 10. The results are consistent with those we obtained from the already mentioned Bode plot, while the noise makes its appearance for frequencies beyond about 4 rad/s. Notice that the best fully recurrent neural network model matches the magnitude of the frequency response of the Aquacube very well, but it shows a sharp transition between the phase at low and high frequencies.
4 NEURAL PREDICTIVE CONTROL STRATEGIES
The conventional method of controlling dynamic systems is to build a detailed mathematical model based upon the physics of the system under study, to approximate the input-output relationship observed in the real system. For the sake of practicality, this usually involves the engineer making simplifying assumptions during the modelling phase, which results in poorer control performance, and requires a great deal of experience. Non-linear systems present particular problems since there is no simple and unifying body of theory available for them. Model predictive control (MPC) is now widely recognised as a powerful methodology to address industrially important control problems.20 The general strategy of MPC algorithms is to utilise a model to predict several steps ahead and minimise the difference between the predicted output and the desired output21 We now propose a five-steps-ahead prediction controller architecture with the goal of providing a nonlinear learning controller with stability assurances. Central to the controller design process is the feasibility of actual use. To address this issue, the proposed control
architecture was tested with real-time experiments using the prototype underwater robot. Our investigation relies on using a forward neural model directly, as part of a closed loop model predictive control scheme. Model based predictive control is characterised by the following strategy: A reference trajectory, a prediction horizon, is calculated at each sampling time. Calculation of the control input is based on the prediction of plant output, and its comparison with the reference trajectory, repetitively over the whole process from now until the prediction horizon. Calculation of the best control action is based on minimising the difference between predicted plant output and the reference trajectory. With the requirement of fewer weights and a shorter training time for the neural network model, the modified Elman and the ‘autoregressive’ networks are proposed. The proposed MPC structure is shown in Fig. 11.22 The recurrent MPC structure consists of a recurrent neural model which is used to predict the next five output responses of the non-linear system. According to this
Fig. 11. Model predictive control scheme.
V. S. Kodogiannis, P. J. G. Lisboa, J. Lucas
210 configuration,
the following cost function is used:
i=l
j=O
+~bn(f+ 1)-ml2
(7)
where X is a weight in the output error and 0 is a damping factor to suppress oscillations. The prediction horizon specifies the range of future outputs which are being considered in the minimisation, while the control horizon defines the number of control moves required to reach the desired objective. Alpha (a) is a parameter which introduces a difference timescale for changes in the modified set point and therefore improves the robustness of the control scheme by reducing the effects of plant/model mismatch. The role of the recurrent network is to provide the output predictions up to the specified prediction horizon. Depending on the control horizon specified, a range of future control moves may have been calculated at each sample instant. However, control is implemented in a ‘receding horizon’ manner; i.e. only the first of the calculated control sequences is applied, so as to avoid uncertainties associated with extrapolation.23 The optimal control, u(t), that will minimise the cost function, can be determined by numerical methods such as the simplex method or the Rosen gradient projection method. If the control horizon is shorter than the prediction horizon, the future controls after N, is held constant to u(t + NU).24 In our case the prediction horizon was set at five and the control horizon at zero. In this configuration, the recurrent networks identify the one-step-ahead model and use it recursively up to the pre-specified prediction horizon. Figures 12 and 13 show a simulation test of the long range prediction controller in which the prediction is made by the modified Elman and the ‘autoregressive’ networks respectively. Here, the real underwater system was replaced by a SE network. The control algorithm was then tested with real-time experiments using the Aquacube in a swimming pool and performances can be seen in Figs 14 and 15, while the complete results are illustrated in Table 1. Although the ‘autoregressive’ version of the long term
u 1
:
i
21
41
‘,; _---j-----i
__...._;..____.
a-
,
:
:
:
:
:
Fig. 14. ARNN-MPC
0.1
.____. j..
u I
.. j . .. . ..
scheme.
.. . . ... .. . .. .. . . .. . .. . . .. .
i
:
:
:
:
:
21
41
a1
e1
101
121
1
21
. . . . . . . . . ____
. 41
;
. . + __.....;
.
;__......
: 01
El
101
181
141 tiraa b-d
Fig. 12. Elman-MPC
scheme in simulation.
jF... . ..;. .. ... . . . :
:
:
141 161
181
tlmc
Table
(ccc)
scheme.
1. Performance comparison of two predictive network
architectures
ARNN (Y= 0.999, x = 1.8, e = 4 Elman (Y= 0.5, X = 1.8, 6’ = 4 _......;
:
the (mm)
Method
. . . . . .._.
121
Fig. 13. ARNN-MPC scheme in simulation
Fig. 15. Elman-MPC
: ; 8.6 .. .. . .. .i.. . .. ... .i.. ... . .. . .
0
101
81
tlm. (rc)
s
a.0
81
MSE setpoint-plant
MSE plant-NN
0.0523
0.000 516
0.0555
0.000 748
prediction control scheme proved to be superior to the Elman network, both experiments demonstrated the effectiveness of the recurrent MPC in keeping the vehicle at a specified constant depth with the minimum required control effort.
Neural network modelling and control for underwater vehicles
: _1o 1
,I 1
:
:
:
:
:
:
tl
41
SC
al
lol
l&l
:
:
:
:
j : :
61
41
81
81
:
141 18l
:
. . :
. :
, :
:
o
101 161 141 161 161
- -8.tpodat-0oatml
181
-0
211
Input
Fig. 18. Control input for one-step-ahead MPC.
Fig. 16. One-step-ahead MPC scheme.
On the other hand, the performance of the one-stepahead predictive control scheme was rather inferior to the present schemes as shown in Fig. 16. The one-stepahead predictive control scheme seems to be sensitive to the NARX structure. The improvements obtained with recurrent networks are due to the fact that a minimum control effort was used to achieve the specific performance. The control values for the ARNN are much smoother than those of the one-step-ahead control, and this can be seen in Figs 17 and 18.
5 CONCLUSIONS This paper demonstrates practical techniques for nonlinear control using learning methods. Although the particular application considered here concerns a prototype underwater vehicle, the model predictive control strategy and the learning methods apply generally to SISO control of process plant. The backpropagation algorithm was applied successfully to several network architectures, including multilayer perceptrons, recurrent networks and memory neuron networks. The dynamics captured by the recurrent network architectures can be evaluated locally using spectral
methods and this was found to agree with experimental results from the Aquacube. Explicit tests of the dynamic characteristics of recurrent neural network models are an important indication of their faithfulness to the plant, which is often not reported in the literature. The preferred neural network is the ARN network, on account both of its modelling accuracy and fast recall speed. The implementation of a neural control strategy for an underwater system demonstrated a more accurate performance than alternative control strategies without the need for ad hoc gain scheduling to combat the actuator’s dead-band and the positive static buoyancy.25’26 The main limitation of the controller design procedure described in the paper concerns the sampling period required to optimise the controller output by searching several steps ahead using the plant model. In this study, the required minimum sampling period of 300ms was achieved using a standard PC workstation programmed in C . An additional factor which was not considered in this paper is the control of processes with multiple coupled variables. This will be the subject of the next stage of this study when the six thrusters are combined in the final vehicle, the Aquasphere.
ACKNOWLEDGEMENTS
V. S. Kodogiannis would like to thank NATO and EU (MAST Initiative Project No: 0030-C) for their financial support.
REFERENCES
1. Takagi, H., Cooperative systems of neural networks and fuzzy logic and their application to consumer products. In Industrial Applications of Fuzzy Control and Intelligent Systems, ed. J. Yen, R. Langari & L. Zadeh. IEEE Press,
1--wtpolnt
-control
8UmlI
Fig. 17. Control input used in ARNN-MPC.
New York, 1995 (in press). 2. Werbos, P. J., Optimal neurocontrol: practical benefits, new results and biological evidence. Proc. World Congress on Neural Networks, Vol. 2, Washington, DC, IEEE Press, 1995, pp. 318-25. 3. Lisboa, P. J. G. & To, Q. S., Control of pH in a
212
4.
5. 6.
7. 8.
9. 10.
11.
12.
13.
14.
15.
V. S. Kodogiannis,
P. J. G. Lisboa, J. Lucas
sulphonation loop reactor. Proc. World Congress on Neural Networks, Vol. 2, Washington, DC. IEEE Press. 1995, pp. 348851. Smith, S. M., Rae, G. J. S. & Anderson, D. T.. Applications of fuzzy logic to the control of an autonomous underwater vehicle. Proc. 1st IEEE Int. Fuzz>, Svstems Conf., 1992, pp. 1099-106. Miller III, W. T., Sutton, R. S. & Werbos, P. J. (eds), Neural Networks,for Control. MIT Press. Cambridge, MA, 1990. Johnson, C., Sutton, R. & Roberts, G. N., Bottom profiling control of an UUV using neural networks. IEE Colloquium on Control and Guidance of Underwater Vehicles, December, 1993, pp. 3/l-3/15. Yuh, J., A neural net controller for underwater robotic vehicles. IEEE J. Ocean Engng, 1990, 15(3), 161-6. Kodogiannis, V. S., Neural network techniques for modelling and learning control of an underwater robotic vehicle. PhD thesis, Liverpool University, May, 1994. MATLAB, System Identification Toolhos. The MathWorks Inc., 1991. Kodogiannis, V. S., Lisboa, P. J. G. & Lucas, J., A neural predictive controller for underwater robotic applications. Proc. 2nd IEEEIBCS Workshop on Neural Network Applications and Tools, Liverpool, September 1993. IEEE Computer Society Press, pp. 126633. Rumelhart, D. & McClelland, T. L. (eds), Parallel Distributed Processing: E.xp1oration.s in the Microstructure of Cognition, Vol. I: Foundations. MIT Press. Cambridge, MA, 1986. Polycarpou, M. M. & Ioannou, P. A., Stable nonlinear system identification using neural networks models. In Neural Networks in Robotics, ed. G. A. Bekey & K. Y. Goldberg. Kluwer Academic Publishers, Dordrecht. 1993. pp. 165577. Werbos, P. J., Backpropagation through time: What it is and how to do it. IEEE Proceedings, 1990, 78(10), 155060. Williams, R. J. & Zipser, D., A learning algorithm for continually running fully recurrent neural networks. Neural Computation, 1989, 1, 270-80. Evans, J. T., Gomm, J. B., Williams, D. & Lisboa. P. J. G.,
16.
17.
18.
19. 20.
21. 22.
23.
24.
25.
26.
Identification and on-line control of a non-linear process via an artificial neural network approach. Proc. 2nd IEEE/ BCS Workshop on Neural Network Applications and Tools, Liverpool, September, 1993, IEEE Computer Society Press, pp. 1099 18. Pearlmutter, B. A., Dynamic recurrent neural networks. Carnegie Mellton University, Report No. CMU-CS-90196. December, 1990. Pham, D. T. & Liu, X., Identification of linear systems using recurrent neural networks. In Applications of Neural Networks to ModeNing and Control, ed. G. F. Page, J. B. Gomm & D. Williams. Chapman and Hall, London, 1993, pp. 24-34. Hertz, J., Krogh, A. & Palmer, R. G., Introduction to Theory of Neural Computation. Addison-Wesley, Reading, MA. 1991. MATLAB, Signal Processing Toolbox User’s Guide. The MathWorks Inc., 1992. Saint-Donat, J., Bhatt, N. & McAvoy, T. J., Neural net based predictive control. Znt. J. Control, 1991,54(6). 145368. Clark, D., Mohtadi, C. & Tuffs, S., General predictive control. Automatica, 1987, 23(1, II), 137-60. Kodogiannis, V. S., Lisboa, P. J. G. & Lucas, J.. Long range predictive controller for underwater robotic vehicles using recurrent neural networks. 2nd IEEE Mediterranean Symposium on New Directions in Control & Automation, Chania, Crete, Greece, June 1994, pp. 217-24. Willis, M. J., Di Massimo, C., Montague, G. A., Tham, M. T. & Morris, A. J., On neural networks in chemical processes. Proc. ZEE-D, 1991, 138(3), 256-66. Koivisto, H., Kimpimaki, P. & Koivo, H., Neural net based control of the heating process. In Artificial Neural Networks, ed. T. Kohonen & K. Makisara. NorthHolland. Amsterdam, 1991, pp. 1277-80. Swainston, J. R., Positional control of underwater vehicles. MSc thesis, Liverpool University, December, 1992. Stephens, M., Intelligent adaptive control of remotely operated vehicles (ROVs). PhD thesis, Liverpool University, March, 1994.
0954-1810(95)00029-l
EISEVIER
Copyright 0 1996 Elsevier Science Limited Printed in Great Britain. All rights reserved 0954-1810/96/$15.00
Neural network modelling and control for underwater vehicles V. S. Kodogiannis, P. J. G. Lisboa* & J. Lucas Department
of Electrical Engineering and Electronics,
University of Liverpool, Brownlow Hill, PO Box 147, Liverpool,
UK, L69 3BX
(Received 24 August 1994; revised version received 24 August 1995; accepted 5 October 1995)
Neural networks are currently finding practical applications ranging from ‘soft’ regulatory control in consumer products to accurate control of non-linear plant in the process industries. This paper describes the application of neural networks to modelling and control of a prototype underwater vehicle, as an example of a system containing severe non-linearities. The most common implementation strategy for neural control is model predictive control, where a model of the process is developed first and is used off-line to design an appropriate compensator. The accuracy and robustness of this control strategy relies on the quality of the non-linear process model, in particular its ability to predict the plant response accurately multiple-steps ahead. In this paper, several neural network architectures are used to evaluate a long-range model predictive control
structure, both in simulation and for on-line control of vehicle depth, achieving accurate control with a smooth actuator signal. Key words: underwater vehicles, recurrent neural networks, model predictive control strategies.
1 INTRODUCTION
linearities in practical plant with all of the fluctuations inherent in industrial sensors and actuators.3 The control loop used typically in these applications follows the model predictive control architecture used in this paper. In the pH example, the process model is needed in order to select appropriate control actions to return the controlled value to the set point while avoiding any undershoot when the desired value of the pH is near the shoulder at the top of the titration curve. In this paper we report a controller design for depth control of a prototype underwater vehicle with a single degree of freedom, as a preliminary stage in the selection of a control strategy for an unmanned vehicle with six degrees of freedom for underwater inspection. The design of controllers for underwater vehicles (UVs) is challenging because of difficulties in accurately modelling the inherently non-linear dynamics of UVs in a hazardous environment with persistent unmodelled disturbances. In general, real-time control of non-linear systems with unknown structure and parameter uncertainty remains an open area of research. Several extensions of linear system design techniques have been studied. For nonlinear systems, the most direct approach is modelling of the plant dynamics using physical laws in conjunction with off-line experiments to determine the relationship between the inputs and outputs of the plant. Modelling techniques, such as fitting linear
Continuing developments in soft information processing, typified by fuzzy logic and neural networks, have led to a more pragmatic approach towards the control of systems with uncertainties or non-linearities. Examples of the former are a number of products in the consumer industry, such as auto-focus systems, the control of kerosene heaters and air-conditioning, all of which have already marketed products with fuzzy and neural components.’ In order to deal with the variability of sensor conditions which influence the controller settings, for instance image contrast has an important bearing on the speed of response of the auto-focus controller, neural and fuzzy systems are used in combination. The neural network may define the membership functions for a fuzzy controller, or it may act in parallel to add a correction term to the output from the fuzzy controller, or a fuzzy pre-processor may be cascaded with a neural supervisory controller.’ In addition, neural controllers are increasingly used in process control with a number of practical applications of this technology.“2 Our own experience with on-line control of pH in a sulphonation loop reactor shows the robustness of these methods in handling severe non*To whom correspondence
should be addressed. 203
204
V. S. Kodogiunnis, P. .I. G. Lishorr, J. Lucas
models to frequency response curves, finite element analysis and stochastic modelling have been successful at characterising classes of unknown plants. Although conventional control techniques (both adaptive and non-adaptive) using standard mathematical methods have performed remarkably well, their appeal tapers off when one is confronted with the control of complex systems characterised by poor models, high dimensionality of the decision space, distributed sensors. distributed decision makers, multiple time scales, multiple time constants, high noise levels, drifting parameter values, stringent performance requirements, non-linearities, and so on. As a system becomes more complex, so does the complexity associated with the computation of the control law and the task of implementing the control in a timely fashion. In view of the very difficult control problems presented by UVs operation and to handle uncertainties, intelligent techniques have been deployed. One such application involves a fuzzy six degree of freedom flight control system for the Ocean Voyager, a UV currently under development by the Advanced Marine Systems Group at Florida Atlantic University (FAU).4 The control system comprises a simple proportional controller of forward speed, and three separate fuzzy logic controllers for heading, pitch and depth. Each fuzzy controller has two components: a rule-based fuzzy controller and a fuzzily constrained integrator. The Ocean Voyager was simulated with a highly accurate six degree of freedom non-linear simulation model developed at FAU. The fuzzy control system seems to be robust to changes in vehicle speed. The emergence of neural networks (NNs) as effective learning systems for a wide variety of applications has resulted in the use of these networks as learning models for dynamical systems. For an overview of the learning control techniques using NN and discussions on some of the applications, the reader is referred to Ref. 5. More recently, NN approaches have been developed to control underwater vehicles. One of the important advantages of using NNs for control applications is that the dynamics of the controlled system (plant) need not be completely known as a prior condition for controller design. This is a very desirable feature in the design of the controller, because of the non-linearities arising from rigid-body coupling and also the unpredictable nature of disturbances from the sea environment. Also, the ability of these networks for adaptation and disturbance rejection and their highly parallel nature of computation make this approach suitable for real-time control applications. In one of the simplest approaches, a neural network was trained using the chemotaxis algorithm to invert a linearised UVA model.6 The inverse network replaced a PID controller as a feedforward controller. Although that NN controller provided a smoother output than the oscillatory PID, its mean-squared-error was considerably worse. However, there is no guarantee that an inverse model will exist, therefore the generality of the proposed method cannot
be guaranteed. A simulated neural network based control scheme for UV is described by Yuh in Ref. 7. Direct control is used with the error back propagation as the learning algorithm applied to a three-layer network, in simulation, and good trajectory tracking is achieved for the vehicle. The above control scheme does not require any information about the system dynamics except an estimate of the inertial terms. Although there is a growing interest in applying neural networks to non-linear systems, little work has been reported on using such networks for practical realtime control. In this paper, we describe the application of NNs to the modelling and positional control of a prototype SISO UV, which we call the Aquacube. The goal is to develop a non-linear controller to keep station. or ‘hover’, the vehicle at specified depths. To meet this goal, two objectives for control of the vehicle are proposed. First to develop accurate model architectures for the prototype UV. Such architectures include recurrent networks and memory neuron networks, and second to develop a technique for controlling the UV using the previous neural network models. These models are then incorporated into a long-range predictive control strategy which is evaluated both in simulation and on-line. Results are shown for both the modelling and real-time control of the Aquacube.
2 NON-LINEAR
SYSTEM
MODELLING
The investigation of NNs for system modelling and control has been carried out on an underwater prototype, the Aquacube, which comprises a single thruster aligned vertically and mounted in a steel endoskeleton. All information and power is transmitted via an umbilical which is non-tensioned and made approximately neutrally buoyant, in order to minimise the static forces. Buoyancy was also added to the vehicle to make it slightly positively buoyant. Thrust is provided by a brushless d.c. motor driven by a PWM inverter, and motor speed is commanded by applying an analogue voltage (It 1OV) to the motor controller. In this arrangement, striction can lead to pronounced dead bands for small thrust values which are crucial in fine position control. Although the vehicle can be considered as a SISO plant, its dynamics are highly non-linear. The hydrodynamic coefficients of the vehicle are unknown, and the drift due to water and the buoyancy generate additional forces and moments. Similarly, the acceleration and deceleration of the vehicle affects its overall dynamics. 2.1 Underwater vehicle characteristics Knowledge of the system indicates that the vehicle and the surrounding environment are likely to exhibit predominantly fourth order dynamics.8 To calculate
Neural network modelling and control for underwater vehicles
the system’s overall response a series of frequency response tests were carried out. The test used was to give a sinusoidal control input to the PWM unit, whilst measuring the vehicle’s depth throughout. From repetitions of this test at different frequencies in the range 0.35-5.236rad/s, the frequency response of the vehicle can be calculated. The frequency response of the Aquacube prototype for small oscillations of the control signal can be calculated using Fourier methods. MATLAB has a rich collection of functions useful for system identification. It is easy to compute the FFT for the sinusoidal inputs and the corresponding vehicle positions, and therefore the magnitude of the Aquacube’s transfer function can be computed as the ratio of the output magnitude to the input magnitude. Similarly, the phase of the actuator signal is subtracted from that of the Aquacube’s response. 2.2 Modelling and approximation using neural networks A critical element in any control task is to build an adequate model of the plant. This is where it is claimed that neural networks have an advantage over classical techniques because they can be trained with data observed from the real plant, to reproduce the response of the plant treated as a black box model. The neural networks are in fact non-linear, multi-variable function approximation tools. The training involves the optimisation of the weights in the network until the knowledge distributed among them is sufficient to accurately approximate the response of the plant. A training set of input output pairs is needed to provide sufficient information to model the non-linear system. An input signal must be chosen which will excite all the dynamic modes of the system and cover the whole amplitude range of interest, otherwise the validity of the model will become highly input sensitive. As a consequence of the slightly positive buoyancy of the vehicle, the thruster requires a minimum value, which ranges in region of 2-3 V, in order to maintain or increase the current vehicle depth position. It is critical for the NN to learn the response characteristics due to the buoyancy and actuator deadband. In order to achieve this, a series of step control signals were taken for training, and a further set collected for validation tests. The neural network representing the forward dynamics of the Aquacube has the following NARX (nonlinear autoregressive model with exogenous input) configuration: y”(t) =f(u(t
205
based on the network achieving an acceptable mean square error (MSE) on the training data without excessive computation time. A network with one hidden layer provided a poor performance, while a NN having two hidden layered structures was much more accurate. For modelling the Aquacube, 18 and 12 nodes were used in the two hidden layers respectively, and standard sigmoidal functions as the non-linear neural activation function.” We also tested the effect of having more neurons at each layer without any significant improvement. Therefore, the network with a 5/18/12/l topology was trained as a one-step ahead predictor using the back-propagation (BP) algorithm.’ ’ Despite the fact that the forward neural model is very accurate, as a one-step-ahead predictor, it cannot be used independently as a plant simulator, or to provide long range predictions, because it is unstable when used recursively.
3 RECURRENT
NETWORKS
FOR MODELLING
The application of NNs to problems involving time varying or dynamic patterns is an active research area. In contrast to the series-parallel model, the network can be arranged completely parallel to the system, which is therefore referred to as a parallel ident@ation method. While the series-parallel method requires only a feedforward network, topologically, the parallel identification method results in a neural network with delayed recurrent connections from its output neurons back to its own input neurons as shown in Fig. 1. The use of recurrent neural networks as system identification networks offers a number of potential advantages over the use of static layered networks. Recurrent neural networks provide a means for encoding and representing internal or hidden states, albeit in a potentially distributed fashion, which leads to capabilities that are
- l),_YP(t - l),.YP(t - 21,
yP(t - 3),yP(t - 4))
(1)
where y m(t) is the model predicted output of the network. In eqn (1) the network is supplied with the input voltage to the thruster, i.e. u(t - l), and the vehicle’s delayed depth positions, i.e. yP(t - n). The internal topology was
Fig. 1. Parallel identification
method using neural networks.
206
V. S. Kodogiannis,
P. J. G. Lisboa, J. Lucas
similar to those of an observer in modern control theory. Recurrent network architectures provide increased flexibility for filtering of noisy inputs.12 An accurate recurrent plant model is highly desirable since it can be used independently from the plant for separate dynamic simulation, whereas in the one-step-ahead predictive structure the network is dependent on data immediately available from the plant. The most commonly used training algorithms for recurrent networks, to date, BPTT (backpropagation through time)13 and RTRL (real-time recurrent learning),i4 have large memory requirements, therefore alternative methods must be considered for use in a real-time control strategy. It has been found that a major factor affecting the neural model prediction accuracy when it is acting as a recurrent model is the method by which data are coded in the network. The reason for this is because any errors in the predicted output will tend to accumulate. The conventional method of conditioning the data is to rescale and represent using a single node at the input or output layers of the network. An alternative representation, called spread encoding (SE), has been shown to enable a network to maintain a high degree of accuracy when operated in a recurrent mode by exploiting the network’s fault tolerance.i5 Figure 2 illustrates the internal architecture of this technique. In the SE technique, each data value is represented as the mean value of a sliding Gaussian pattern of excitation over several nodes at the network input and output. A similar and reverse procedure is applied at the network output to decode the output back into the original variable range. For our problem, we used 36 input nodes and 6 outputs. Although the training time was considerably less than the time for BPTT, the size of the network makes it unsuitable for real-time control. Figure 3 illustrates the validation of the one-stepahead and the recurrent SE network respectively. The performance of the network operating recurrently is indicated by a MSE of depth 0.008. It is clearly seen to track the Aquacube’s response over the complete duration of the test.
The representation of temporal knowledge and time correlation in neural networks presents a continuous challenge to researchers in the field of artificial neural networks. In the previous section, the so called windowed input network has been applied for an iterative response modelling of the Aquacube. Another approach has been that of explicitly including the time dependency into the network structure. The commonly used backpropagation algorithm contains no memory, hindering the learning of temporal patterns. Here, the alternative is to use a dynamic network that is given some kind of memory to encode past history. In this section, our primary objective is to design recurrent models of our underwater prototype to carry out five-steps-ahead accurate predictions with the additional requirements of short training time and small network size. Each time step corresponds to the sampling interval which was set to 0.3 seconds. This section describes an extension to error backpropagation that allows the nodes in a neural network to encode state information. The two recurrent architectures, are the autoregressive network and a modified version of the well known Elman network, both of which are examples of internal state networks.
Fig. 2. Spread encoding neural architecture.
3.1.1 The autoregressive backpropagation network In this section, a novel network architecture, called the autoregressive recurrent network (ARNN), is proposed. It was designed to contain internal memory of the previous states, while training rapidly using a generalised BP algorithm. The idea is that we still use a recurrent neural network model but the recurrent neurons are decoupled so that each neuron only feeds back to itself. With this modification, the ARNN model is considered to converge easier and to need less training cycles than a fully recurrent network. In a standard multilayer perceptron, the inputs to a neuron are multiplied by feedforward weights and summed, along with a node bias term. The sum is then passed through a smooth sigmoidal transfer function, producing the neuron’s output value. This neural model
NO.
or pattmu (mocc)
Fig. 3. System modelling using MLPs. 3.1 Internal state recurrent networks
Neural network modelling and control for underwater vehicles
207
model for our underwater prototype. The five memories in each node at the first hidden layer allow the network to encode state information. A structure of 6/20/14/l nodes has been used, with inputs consisting of one past control signal and five past depth indications. 3.1.2 Elman network
Input UnIta
Fig. 4. ARNN architecture.
has no memory, because the output value is not explicitly dependent upon previous outputs. The ARNN is a hybrid feedforward/feedback neural network, with the feedback represented by recurrent connections appropriate for approximating the dynamic system. The structure of the ARNN is shown in Fig. 4. There are two hidden layers, with sigmoidal transfer functions, and a single linear output node. The ARNN topology allows recurrence only in the first hidden layer. For this layer, the memoryless backpropagation model has been extended to include an autoregressive memory, a form of self-feedback where the output depends also on a weighted sum of previous outputs. A modified backpropagation algorithm is developed to train the ARNN which includes dynamic recursive equations in time. The mathematical definition of the ARNN is shown below:
_v(t)= o(t) = Sl = C
zj(t) =f
cI w?Qr(t),
QI =f (S/)
(4
WjyZj(t)
As an alternative to the ARNN, the network developed by Elman16 has a simpler architecture and it can be trained using the standard backpropagation learning algorithm. The output nodes are linear. The context nodes interact only with the hidden nodes and have fixed recurrent connections. The context units of the Elman network memorise past states of the hidden units, so the output of the network depends on an aggregate of the previous states and the current input. This network has been proved to be effective for modelling linear systems not higher than the first order.” For this reason, an idea based on the work of Hertz et al.‘* was employed to configure a modified Elman network which is shown in Fig. 5. Here, self-connections are introduced in the context units of the network in order to give these units a certain amount of inertia. The introduction of self-feedback in the context units increases the possibility of the Elman network to model high-order systems. We use this network to generate five-steps-ahead predictions for the Aquacube. For this purpose, the output of the jth context unit in this modified Elman network structure is given by X,j(t
+
1) =
Xj(t)
+
OZXj(t
-
+
0J3Xj(t
-
3) +
+
a5Xj(t
-
5)
1) + a4Xj(t
0Z2Xj(t
-
-
4)
CHjCt))
k=l
WjyZj(t
- k) + C
Wi;Zi i
where Zi(t) is the ith input to the ARNN, Hi(t) is the sum of inputs to the ith recurrent neuron in the first hidden layer, Zj(t) is the output of the jth recurrent neuron, S/(t) is the sum of inputs to the Ith neuron in the second hidden layer, Q,(t) is the output of the Ith neuron in the second hidden layer and O(t) is the output of the ARNN. Here, f (-) is the sigmoid function and W’, WD, WH and W” are input, recurrent, hidden and output weights, respectively. Since the objective is to demonstrate a five-step-ahead recurrent model of the Aquacube, therefore the number of internal memories in the hidden nodes was set to five. The ARNN was trained as a five-step-ahead prediction
(4)
Usually, o is between 0 and 1. A value of (Ynearer to 1 enables the network to trace further back into the past. In order to enhance network’s performance we have added an extra hidden layer and replaced the linear output function with a standard sigmoidal one. Therefore
k=5
Hj(t) = C
2)
Fig. 5. Modified Elman architecture.
208
V. S. Kodogiannis, P. J. G. Lisboa, J. Lucas
No.
-‘Nmud
Fig. 6. System modelling
using internal
state networks.
net -Aq-be
of phmnm (mma) ‘-input
Fig. 7. Elman’s autonomous
a 3/12/14/l Elman network was applied with self-feedback in the 12 context units, with Q equal to 0.8. The input to the network consisted of the current control signal and the current and previous system positions, to enable the network to emulate externally the integrator action of the Aquacube from the input-output data. Figure 6 illustrates the performances of the modified Elman network and the ARNN as five-step-ahead predictors. An MSE of 0.011 has been achieved for the former case while an MSE of 0.02 for the latter one. In summary, the approach of producing a dynamic memory is clearly simpler than the SE network with the result that the computational burden is substantially reduced. Although the ARNN architecture is dependent on the number of memories in the recurrent nodes, it proved to be the fastest in training time. We proceeded to examine the performance of the Elman network in an autonomous iterative response. The SE network has already proved its ability to work practically independent from a real system. For a singleinput single-output system, such as the underwater prototype, we are interested in a network with a single input and output line. The integrator in the response of the Aquacube was removed by modelling
0.61 1
61
101
li@d
response
161
test 1.
eoi
No. d pattamm (mw)
1-.Nowmlnet -Aq-be.
-.input rnw
Fig. 8. Elman’s autonomous
response
1 test 2.
Y(s) = V(S)
(5) Equivalent, in the z-plane domain, the above relation is transformed to y(t) = l’(t - 1) + TY(t)
(6)
where T is the sampling interval, equal to 0.3 s. A network structure of 1/14/l was used to predict iteratively the Y(t) term, and eqn (6) was employed to reconstruct the actual position. Figures 7 and 8 illustrate the validation of this approach in a number of autonomous iterative response tests. The proposed modification has improved the versatility of the network. As shown from these results, the network appears to have a drift similar to the real underwater vehicle. This fact, in conjunction with the simpler network complexity, proves that the current approach is better suited to this kind of sequence processing than the SE network.
-Elmm Il. +Elman Ph. *@mube Fig. 9. Open-loop
Y *41mmaba
Bode diagram for the Aquacube network.
Ph.
and Elman
The open-loop Bode plot of the Aquacube is shown in Fig. 9, superimposed upon the frequency response derived using the same Fourier methods applied to a recursive neural network model. An important consideration in the design and implementation of neural network plant models is the extent to which they capture the process dynamics. We have investigated this in our neural network models using standard spectral methods by applying small PRBS driving signals to the recursive neural models. The results were analysed using the
Neural network modelling and control for underwater vehicles 100
AMPLITUDE
PLOT, input
#
1 output
209
# 1
10-l
10-Z
1 O-3 10-2
1 o-’
100
8 o -150 .C m
-200
PHASE
102
10’
102
(md/ssc)
frequency -100
10’
PLOT, input
#
1 output
# 1
-
I
1
1o-2
10-l
lgo frequency
Fig. 10. Elman’s ‘continuous’
(md/aec)
Bode plot obtained by using PRBS signals.
MATLAB Signal Processing Toolbox” and are shown in Fig. 10. The results are consistent with those we obtained from the already mentioned Bode plot, while the noise makes its appearance for frequencies beyond about 4 rad/s. Notice that the best fully recurrent neural network model matches the magnitude of the frequency response of the Aquacube very well, but it shows a sharp transition between the phase at low and high frequencies.
4 NEURAL PREDICTIVE CONTROL STRATEGIES
The conventional method of controlling dynamic systems is to build a detailed mathematical model based upon the physics of the system under study, to approximate the input-output relationship observed in the real system. For the sake of practicality, this usually involves the engineer making simplifying assumptions during the modelling phase, which results in poorer control performance, and requires a great deal of experience. Non-linear systems present particular problems since there is no simple and unifying body of theory available for them. Model predictive control (MPC) is now widely recognised as a powerful methodology to address industrially important control problems.20 The general strategy of MPC algorithms is to utilise a model to predict several steps ahead and minimise the difference between the predicted output and the desired output21 We now propose a five-steps-ahead prediction controller architecture with the goal of providing a nonlinear learning controller with stability assurances. Central to the controller design process is the feasibility of actual use. To address this issue, the proposed control
architecture was tested with real-time experiments using the prototype underwater robot. Our investigation relies on using a forward neural model directly, as part of a closed loop model predictive control scheme. Model based predictive control is characterised by the following strategy: A reference trajectory, a prediction horizon, is calculated at each sampling time. Calculation of the control input is based on the prediction of plant output, and its comparison with the reference trajectory, repetitively over the whole process from now until the prediction horizon. Calculation of the best control action is based on minimising the difference between predicted plant output and the reference trajectory. With the requirement of fewer weights and a shorter training time for the neural network model, the modified Elman and the ‘autoregressive’ networks are proposed. The proposed MPC structure is shown in Fig. 11.22 The recurrent MPC structure consists of a recurrent neural model which is used to predict the next five output responses of the non-linear system. According to this
Fig. 11. Model predictive control scheme.
V. S. Kodogiannis, P. J. G. Lisboa, J. Lucas
210 configuration,
the following cost function is used:
i=l
j=O
+~bn(f+ 1)-ml2
(7)
where X is a weight in the output error and 0 is a damping factor to suppress oscillations. The prediction horizon specifies the range of future outputs which are being considered in the minimisation, while the control horizon defines the number of control moves required to reach the desired objective. Alpha (a) is a parameter which introduces a difference timescale for changes in the modified set point and therefore improves the robustness of the control scheme by reducing the effects of plant/model mismatch. The role of the recurrent network is to provide the output predictions up to the specified prediction horizon. Depending on the control horizon specified, a range of future control moves may have been calculated at each sample instant. However, control is implemented in a ‘receding horizon’ manner; i.e. only the first of the calculated control sequences is applied, so as to avoid uncertainties associated with extrapolation.23 The optimal control, u(t), that will minimise the cost function, can be determined by numerical methods such as the simplex method or the Rosen gradient projection method. If the control horizon is shorter than the prediction horizon, the future controls after N, is held constant to u(t + NU).24 In our case the prediction horizon was set at five and the control horizon at zero. In this configuration, the recurrent networks identify the one-step-ahead model and use it recursively up to the pre-specified prediction horizon. Figures 12 and 13 show a simulation test of the long range prediction controller in which the prediction is made by the modified Elman and the ‘autoregressive’ networks respectively. Here, the real underwater system was replaced by a SE network. The control algorithm was then tested with real-time experiments using the Aquacube in a swimming pool and performances can be seen in Figs 14 and 15, while the complete results are illustrated in Table 1. Although the ‘autoregressive’ version of the long term
u 1
:
i
21
41
‘,; _---j-----i
__...._;..____.
a-
,
:
:
:
:
:
Fig. 14. ARNN-MPC
0.1
.____. j..
u I
.. j . .. . ..
scheme.
.. . . ... .. . .. .. . . .. . .. . . .. .
i
:
:
:
:
:
21
41
a1
e1
101
121
1
21
. . . . . . . . . ____
. 41
;
. . + __.....;
.
;__......
: 01
El
101
181
141 tiraa b-d
Fig. 12. Elman-MPC
scheme in simulation.
jF... . ..;. .. ... . . . :
:
:
141 161
181
tlmc
Table
(ccc)
scheme.
1. Performance comparison of two predictive network
architectures
ARNN (Y= 0.999, x = 1.8, e = 4 Elman (Y= 0.5, X = 1.8, 6’ = 4 _......;
:
the (mm)
Method
. . . . . .._.
121
Fig. 13. ARNN-MPC scheme in simulation
Fig. 15. Elman-MPC
: ; 8.6 .. .. . .. .i.. . .. ... .i.. ... . .. . .
0
101
81
tlm. (rc)
s
a.0
81
MSE setpoint-plant
MSE plant-NN
0.0523
0.000 516
0.0555
0.000 748
prediction control scheme proved to be superior to the Elman network, both experiments demonstrated the effectiveness of the recurrent MPC in keeping the vehicle at a specified constant depth with the minimum required control effort.
Neural network modelling and control for underwater vehicles
: _1o 1
,I 1
:
:
:
:
:
:
tl
41
SC
al
lol
l&l
:
:
:
:
j : :
61
41
81
81
:
141 18l
:
. . :
. :
, :
:
o
101 161 141 161 161
- -8.tpodat-0oatml
181
-0
211
Input
Fig. 18. Control input for one-step-ahead MPC.
Fig. 16. One-step-ahead MPC scheme.
On the other hand, the performance of the one-stepahead predictive control scheme was rather inferior to the present schemes as shown in Fig. 16. The one-stepahead predictive control scheme seems to be sensitive to the NARX structure. The improvements obtained with recurrent networks are due to the fact that a minimum control effort was used to achieve the specific performance. The control values for the ARNN are much smoother than those of the one-step-ahead control, and this can be seen in Figs 17 and 18.
5 CONCLUSIONS This paper demonstrates practical techniques for nonlinear control using learning methods. Although the particular application considered here concerns a prototype underwater vehicle, the model predictive control strategy and the learning methods apply generally to SISO control of process plant. The backpropagation algorithm was applied successfully to several network architectures, including multilayer perceptrons, recurrent networks and memory neuron networks. The dynamics captured by the recurrent network architectures can be evaluated locally using spectral
methods and this was found to agree with experimental results from the Aquacube. Explicit tests of the dynamic characteristics of recurrent neural network models are an important indication of their faithfulness to the plant, which is often not reported in the literature. The preferred neural network is the ARN network, on account both of its modelling accuracy and fast recall speed. The implementation of a neural control strategy for an underwater system demonstrated a more accurate performance than alternative control strategies without the need for ad hoc gain scheduling to combat the actuator’s dead-band and the positive static buoyancy.25’26 The main limitation of the controller design procedure described in the paper concerns the sampling period required to optimise the controller output by searching several steps ahead using the plant model. In this study, the required minimum sampling period of 300ms was achieved using a standard PC workstation programmed in C . An additional factor which was not considered in this paper is the control of processes with multiple coupled variables. This will be the subject of the next stage of this study when the six thrusters are combined in the final vehicle, the Aquasphere.
ACKNOWLEDGEMENTS
V. S. Kodogiannis would like to thank NATO and EU (MAST Initiative Project No: 0030-C) for their financial support.
REFERENCES
1. Takagi, H., Cooperative systems of neural networks and fuzzy logic and their application to consumer products. In Industrial Applications of Fuzzy Control and Intelligent Systems, ed. J. Yen, R. Langari & L. Zadeh. IEEE Press,
1--wtpolnt
-control
8UmlI
Fig. 17. Control input used in ARNN-MPC.
New York, 1995 (in press). 2. Werbos, P. J., Optimal neurocontrol: practical benefits, new results and biological evidence. Proc. World Congress on Neural Networks, Vol. 2, Washington, DC, IEEE Press, 1995, pp. 318-25. 3. Lisboa, P. J. G. & To, Q. S., Control of pH in a
212
4.
5. 6.
7. 8.
9. 10.
11.
12.
13.
14.
15.
V. S. Kodogiannis,
P. J. G. Lisboa, J. Lucas
sulphonation loop reactor. Proc. World Congress on Neural Networks, Vol. 2, Washington, DC. IEEE Press. 1995, pp. 348851. Smith, S. M., Rae, G. J. S. & Anderson, D. T.. Applications of fuzzy logic to the control of an autonomous underwater vehicle. Proc. 1st IEEE Int. Fuzz>, Svstems Conf., 1992, pp. 1099-106. Miller III, W. T., Sutton, R. S. & Werbos, P. J. (eds), Neural Networks,for Control. MIT Press. Cambridge, MA, 1990. Johnson, C., Sutton, R. & Roberts, G. N., Bottom profiling control of an UUV using neural networks. IEE Colloquium on Control and Guidance of Underwater Vehicles, December, 1993, pp. 3/l-3/15. Yuh, J., A neural net controller for underwater robotic vehicles. IEEE J. Ocean Engng, 1990, 15(3), 161-6. Kodogiannis, V. S., Neural network techniques for modelling and learning control of an underwater robotic vehicle. PhD thesis, Liverpool University, May, 1994. MATLAB, System Identification Toolhos. The MathWorks Inc., 1991. Kodogiannis, V. S., Lisboa, P. J. G. & Lucas, J., A neural predictive controller for underwater robotic applications. Proc. 2nd IEEEIBCS Workshop on Neural Network Applications and Tools, Liverpool, September 1993. IEEE Computer Society Press, pp. 126633. Rumelhart, D. & McClelland, T. L. (eds), Parallel Distributed Processing: E.xp1oration.s in the Microstructure of Cognition, Vol. I: Foundations. MIT Press. Cambridge, MA, 1986. Polycarpou, M. M. & Ioannou, P. A., Stable nonlinear system identification using neural networks models. In Neural Networks in Robotics, ed. G. A. Bekey & K. Y. Goldberg. Kluwer Academic Publishers, Dordrecht. 1993. pp. 165577. Werbos, P. J., Backpropagation through time: What it is and how to do it. IEEE Proceedings, 1990, 78(10), 155060. Williams, R. J. & Zipser, D., A learning algorithm for continually running fully recurrent neural networks. Neural Computation, 1989, 1, 270-80. Evans, J. T., Gomm, J. B., Williams, D. & Lisboa. P. J. G.,
16.
17.
18.
19. 20.
21. 22.
23.
24.
25.
26.
Identification and on-line control of a non-linear process via an artificial neural network approach. Proc. 2nd IEEE/ BCS Workshop on Neural Network Applications and Tools, Liverpool, September, 1993, IEEE Computer Society Press, pp. 1099 18. Pearlmutter, B. A., Dynamic recurrent neural networks. Carnegie Mellton University, Report No. CMU-CS-90196. December, 1990. Pham, D. T. & Liu, X., Identification of linear systems using recurrent neural networks. In Applications of Neural Networks to ModeNing and Control, ed. G. F. Page, J. B. Gomm & D. Williams. Chapman and Hall, London, 1993, pp. 24-34. Hertz, J., Krogh, A. & Palmer, R. G., Introduction to Theory of Neural Computation. Addison-Wesley, Reading, MA. 1991. MATLAB, Signal Processing Toolbox User’s Guide. The MathWorks Inc., 1992. Saint-Donat, J., Bhatt, N. & McAvoy, T. J., Neural net based predictive control. Znt. J. Control, 1991,54(6). 145368. Clark, D., Mohtadi, C. & Tuffs, S., General predictive control. Automatica, 1987, 23(1, II), 137-60. Kodogiannis, V. S., Lisboa, P. J. G. & Lucas, J.. Long range predictive controller for underwater robotic vehicles using recurrent neural networks. 2nd IEEE Mediterranean Symposium on New Directions in Control & Automation, Chania, Crete, Greece, June 1994, pp. 217-24. Willis, M. J., Di Massimo, C., Montague, G. A., Tham, M. T. & Morris, A. J., On neural networks in chemical processes. Proc. ZEE-D, 1991, 138(3), 256-66. Koivisto, H., Kimpimaki, P. & Koivo, H., Neural net based control of the heating process. In Artificial Neural Networks, ed. T. Kohonen & K. Makisara. NorthHolland. Amsterdam, 1991, pp. 1277-80. Swainston, J. R., Positional control of underwater vehicles. MSc thesis, Liverpool University, December, 1992. Stephens, M., Intelligent adaptive control of remotely operated vehicles (ROVs). PhD thesis, Liverpool University, March, 1994.