Automatic Initialization of Robust Adaptive Controllers

Copyright © IFAC Adaptive Systems in Control and Signal Processing, Grenoble, France, 1992

AUTOMATIC INITIALIZATION OF ROBUST ADAPTIVE CONTROLLERS M. Lundh and K.J. Astrom Department of Automatic Control, Lund Institute of Technology, Box 118, 5·22100 Lund, Sweden

Abstract. All adaptive controllers contain design parameters that must be chosen. This choice often requires considerable skilL This paper describes a procedure for automatically choosing the design parameters. The procedure gives controllers that can be started simply by pushing a button. Keywords. Automatic tuning, Adaptive Control, Self-Tuning Control, Robust Control, Initialization, Pretuning, Dead-time compensation.

1.

2.

Introduction

The Adaptive Controller

The structure of adaptive controllers is quite well known. See [Astrom and Wittenmark, 1989]. In this paper we consider indirect discrete time controllers based on estimation of a discrete time model of the type

Adaptive techniques have been used industrially since the late seventies. Much has been learned about operational issues over that period. Although adaptive controllers have the potential to give excellent performance, it has been observed that adaptive systems may also be difficult to commission. For this reason the most extensive use of adaptive techniques has been automatic tuning of simple controllers of the PID type [Astrom and Hagglund, 1988a]. These controllers have been developed to the stage where tuning is performed simply by pushing a tuning button, so called onebutton-tuning. The purpose of this paper is to develop initialization procedures that make adaptive controllers as easy to use as the simple autotuners. The key idea is to use an experiment with relay feedback to obtain the information required to start an adaptive controller properly. The paper is organized as follows . A prototype adaptive controller is described in Section 2. This controller is based on pole placement design which is modified to improve its robustness. Relay feedback is discussed in Section 3. In Section 4 it is shown how information from an experiment with relay feedback can be used to determine the design parameters required to start an adaptive controller. Particular attention is given to the choice of control performance. An initialization procedure is described in Section 5 and an example of its use is given in Section 6.

H(z)

= B(z)

(1)

A(z)

where

Controller Design

The controller is specified by requiring that the closed loop pulse transfer function relating output to set point is

(2) where Am has the same degree as A. The process zeros are thus retained in the closed loop system. The controller is linear with a two-degree-offreedom structure. It is described by

R(q)u(k)

= T(q)Y,p(k) -

S(q)y(k)

(3)

where Rand S are a solution to

A(z)R(z) + B(z)S(z) 279

= Ao(z)Am(z)

(4)

Initialization: The recursive least squares estimator has the estimates, their covariances, and the regression vector as states. They must be initialized properly before starting adaptation.

and T( ) z

= Am(l) A B(I)

( ) oZ

There exists a solution to (4) if A and Bare relatively prime and if Ao and Am are of the same degree as A. The characteristic polynomial of the closed loop system is AoAm where Ao can be interpreted as the observer polynomial. The polynomials Ao and Am are the design parameters. They are chosen to have the same structure as A, i.e. with degree n+d where d zeros are at the origin. The remaining zeros are chosen to be discrete time polynomials corresponding to S+W o ,

n=1

+ V2wos + w~, (s + wo)(s2 + V2wos + w~),

n=2

s2

Regreuion Filterll: It is important to filter the input-output data before sending them to the estimator. This filter which is called the regression filter should have bandpass character. It is chosen as the sampled version of

The pass band of the filter determines the frequency range where the Nyquist curves of the identified model and the real process are close. To obtain robust estimates it is important that the frequency response of the estimated model is accurate in the frequency range where the Nyquist curve for the loop transfer function is close to -l. To achieve this the frequency W J should be related to WIS0 which is the frequency where the open loop process has a phase shift of -180 degrees. The values wJ = 0.75wIS0, and a = .JIO are reasonable choices.

n=3

where Wo is a design parameter. There are cases where it is useful to have the polynomials A o and Am different . An extra parameter could be introduced for this purpose. The parameter n is assumed to be less than or equal to three. Robust Control

Forgetting Factor: Discounting of old data is essential in adaptive algorithms. It is known, however, that discounting may cause some problems [Astrom and Wittenmark, 1989]. There are many ways to do the discounting. The particular choice depends on the nature of the parameter variations. Ultimately it would be desirable to let the algorithm explore this itself. Good ways to do this have not been explored. In the simulations we have used a simple method with exponential forgetting. The forgetting factor has been chosen with respect to the sampling period.

There are many controllers that give a closed loop characteristic equation with the desired properties. If one controller R o, So, satisfying (4) is found, another controller is given by the Youla factorization R=Ro+QB S

= So - QA Q = N / zdegN.

where The polynomial N can be used to improve the robustness of the feedback law. In [Lundh, 1991] it is shown how to choose N to achieve various robustness criteria. The calculations to do this are in general to complicated to do on line in an adaptive algorithm. There are, however, some simple choices that will lead to controllers with significantly improved performance. First N is chosen so that z - 1 is a factor of R. This means that controller gain at high frequencies is infinite, i.e. integral action. To have a controller that is robust with respect to high frequency modeling errors we also require that z + 1 is a factor of S. This means that the controller gain is zero at the Nyquist frequency.

Design Parameters With the choices discussed above the design parameters of the controller have been reduced to •

Wo -

•

h-

pole distance from the origin sampling period

•

rn, nand d -

•

8(0) -

model structure

•

P(O) - initial covariance

initial estimates

Parameter Estimation

3.

In the adaptive controller the parameters of the model (1) are estimated recursively and the parameters of the controller (3) are computed replacing A and B in (4) by their estimates. The control signal is then computed from (3). Recursive least squares [Ljung and Soderstrom, 1983] are used for the estimation. There are several parameters in the parameter estimator that also have to be chosen.

Relay Feedback

To initialize the adaptive controller it is necessary to give the design parameters. It will be shown that these parameters can be determined from an experiment with relay feedback. Consider a system where the relay amplitude is d and the hysteresis is €. For a large class of processes there will be a symmetric limit cycle oscillation. There are 280

approximate and exact methods to determine the amplitude and the period of the oscillation. See [Tsypkin, 1984]. The amplitude and the period of the oscillation gives approximately the frequency 0 WISO where the process has a phase lag of 180 • In [Holmberg, 1991] it is shown that convergence to the limit cycle is very fast for low order systems. It is also shown that very complicated behavior can be obtained with systems having poorly damped poles. The method is therefore restricted to certain classes of systems.

Frequency Domain Information

When there exists a stable limit cycle under relay feedback the input to the process is a square wave with frequency components at W OlC , 3w olc , 5w olc etc. The values of the transfer function at those frequencies can be determined by frequency response analysis. Introduce the normalized frequencies

which corresponds to choosing the sampling period as the time unit. Let the signals u( le) and y( le) be defined at le = I, 2, ... , N. The discrete Fourier transforms

Using The Waveform It is possible to extract more information from the results of the relay experiment by analyzing the waveform. Dominant oscillatory poles manifest themselves in an oscillatory waveform. In [Astrom and Hagglund, 1988b] and [Hagglund and Astrom, 1991] it is shown how the transfer function

G(s)=le

e-· L + sT l

(5) are calculated for m = 1,3, .... The frequency response estimate is then

can be computed from the samples of the waveform at three points. Parameter L can be estimated independently from the time where the peak occurs. The model given by (5) is often used as an approximation of the transfer function for a system with a monotone step response. The estimates of le /T and L are often good even for high order systems. The estimate of T may however be unreliable. A better estimate is obtained by first determining L and then computing T from

Assuming that the oscillation is dominated by the first harmonic, the variance of the estimate satisfies iOm

wo.cL

+ arctan wo.cT = -

Var [H(e )] i01)12 IH(e

(6)

arg G( iw o• c )

A slightly more general model is

4 (

=N

a )2 -y max

2 m

(9)

where a 2 is the variance of the measurement noise and Ymax is the oscillation amplitude.

(7) EstiInation of Slope

The slope of the amplitude curve at frequencies larger than 1/ ../T1 T 2 can be used to discriminate this model from (5). Another possibility is the following procedure. Let r be the difference between the times where the limit cycle y and its derivative dy/dt have their extrema. It follows that r = 0 for the model (5). By straightforward but tedious calculations [Lundh, 1991], it can be shown that

The slope of the transfer function at the crossover frequency is of interest to discriminate between different models. The slope at the frequency 0 = v'OmlOm2 is given by

n=

log IH(e iOm1 ) I-log IH(e iOm2 ) I log(md -log(m2)

(10)

The estimate is unbiased with variance for the model (7). Notice that r -+ Tl if T2 For a system with the transfer function

-+

e-· L G(s) = le -0------,-(1 + sT)n it can be shown that r increases with n. For n we have approximately r = 1.5T.

Var [n] ~ (n~J2 Var [lH(eiO m1 )1]

T1 •

+ (n~.)2Var [lH(eiOm2)1]

(11)

where n~ is the first derivative of n with respect to IH(e iOm )1. An example illustrates the accuracy of the estimate that can be expected.

(8)

=3 281

Standard deviation of .Iope in Exam-

Table 1. ple 1.

n

ml

m2

Un

1

1 1 3 1 1 3 1 1 3

3 5 5 3 5 5 3 5 5

0.12 0.22 0.74 0.35 1.10 3.54 1.04 5.49 17.45

2

3

EXAMPLE

i

V ./

;•

./' i r ···········

1V~

V

~

L

V

lL ..... .........

. .. -

j

o o

0.5

1..5

25

3.5

1

Consider a relay experiment over N = 200 samples. Assume that the signal to noise ratio is Ymoz / U = 10. Table 1 shows the standard deviation of the estimates of the slope n for different local n . The standard deviation increases fast with increasing harmonic and increasing order of the process. Both these effects are due to decreasing frequency response magnitude of the open loop [] process.

Figure 1. Relation (12) between Wo and WOle . Empirical results for n = 1 (0). n = 2 (+) . and

=

n

3

(*) .

processes (8) with different values of n, L, and T. Parameter Wo was increased until the sensitivity reached the value 2. In this way the largest values of Wo that can be achieved subject to a sensitivity constraint is obtained. The results obtained are illustrated in Figure 1. The function

The example shows that the slope can be determined reasonably well at frequency wol eVl for first, second and third order systems. The estimated slope n is hereafter considered to represent the local slope at this frequency

4.

............... ,. . . j

c··········-

-

Wo

W o•e

1 wol eT

= 0.5 + - -

1 2 + 0.12(--) wo. eT

(12)

is a reasonable approximation to the points in Figure 1. For processes with small relative delay, T /T, the quantity wo. eT is large. Consider, e.g. a first order process with T 1 and T 0.1. For this process Equation (12) gives Wo ;:::: 0.6w o• e • This means that the controller is similar to a PID-controller, tuned by Ziegler-Nichols method [Ziegler and Nichols, 1942]' which has Wo ;:::: 0.7W1SO for this process. For processes with larger relative delay, T /T, it follows from (12) that it is possible to have a significant larger ratio wo/w o • e • This is equivalent to classical controllers with dead time compensation. Such controllers can give significant faster response than PID-cont rollers, tuned by ZieglerNichols method, which in this case have Wo ;:::: 0. 3W 1SO·

Finding Design Parameters

=

In Section 2 it was shown that the adaptive controller can be described by a few design parameters. It will now be shown how these parameters can be determined.

Model Order: The model order can be determined by analyzing the higher harmonics of the limit cycle. The slope of the amplitude curve is estimated by Equation (10) . Only values n = 1, 2 or 3 are expected. Let be the slope obtained from (10) . The order n is then chosen as n = 1 if n < 1.3, as n = 2 if 1.3 ~ n < 2.1, and as n = 3 if 2.1 ~ n. The interval boundaries have been adjusted empirically.

n

=

Sampling Inter'fJal: The sampling interval h is chosen to match the bandwidth of the closed loop system. We use woh = 0.3, 0.5, and 0.7 for first, second, and third order processes respectivelY. For processes with considerable delay h is adjusted such that L < 4h.

Desired Closed Loop Poles: The pattern of the closed loop poles is determined by parameters d and n. The numerical values are given by parameter Wo which is the distance of the poles from the origin. This parameter is critical, its choice depends on the properties of the plant. It is well known that too large a value gives a system that is extremely sensitive to parameter variation. See [Lilja, 1989] . Similarly too Iowa value may give controllers with positive feedback. It is expected that parameter Wo is related to W o • e obtained from the relay experiment. To investigate this, the design procedure was applied to many

Delay: The delay is determined by least-squares estimation of the model A(j where deg > n + L/h and deg A(j = n. The delay d in (1) is chosen as the index of the leading coefficient in B~ whose magnitude is considerably smaller than the largest coefficient.

Ba/

282

Ba

5.

An Initialization Procedure

A procedure for automatic initialization of the adaptive controller will now be given. From an operational point of view the algorithm is similar to the procedure given in [Astrom and Hagglund, 1988a] for tuning a PID-controller using relay feedback. The procedure has the following steps.

i

:f-, -1

{/PT5=t=fl'

..........

o

1

~

1

J

I.. _............. j

w

s

~

i

j

l.o .................. time •

u

w

~

Figure 2. Process input and output during relay experiment

Step 1: Bring the process output close to the desired operating value by manual control and wait for stationarity. Estimate the noise level and determine the hysteresis level and relay amplitude. Step!: Introduce relay feedback. Measure process input and output. Stop the experiment after 2 to 3 periods of a stable limit cycle. Step 3: Determine the period of oscillation Estimate the slope of the amplitude curve from (10) and determine the order of the process.

Wale'

Step -4: Determine the times when the limit cycle and its derivative have their extrema. Estimate the parameters T and L. Step 5:

Determine

Wo

from Equation (12) . Figure in (13).

Step 6: Determine sampling interval hand delay d as was discussed in Section 4. Step 7: Initialize the recursive estimator with parameters computed from n, L, T, and h. Run the estimator with the data from step 2.

Adaptive control of the process G(.)

where white noise with standard deviation Un 0.05 is added to the output. Figure 2 shows the process input (dotted line) and the process output (solid line) during the relay experiment. The oscillation frequency is Wale = 0.71 radians/so The slope of the frequency response was estimated to 11. = 1.32 for W = v'3w o • e which gives n = 2. The time constant of (8) is then estimated to T = T = 1.08 S. Equation (12) gives Wo = 1.42 radians/so The sampling interval is chosen to satisfy L ~ 4h. This gives woh = 0.71. The delay of the discrete time process model is then estimated to d = 5 and m=2.

Step 8: Select a reference input signal that excites the system so that the controller will tune.

6.

s.

Example

Part of the heuristics for determining the design parameters, like Figure 1 were based on simple models. To determine the validity of the approach the initialization procedure was tested on a large test batch. See [Lundh, 1991] for details. Since the procedure is based on relay feedback it will only work for systems where stahle limit cycles are obtained under relay feedback. Extensive experience indicate that this occurs for a wide variety of processes that are encountered in industry, see [Hagglund and Astrom, 1991] . There are however systems where a stable limit cycle is not obtained. The double integrator is a simple example. There are also systems that may give a very complicated behavior under relay feedback. Oscillatory system with low damping and time delays are such cases. See [Holmberg, 1991]. The procedure is illustrated by an example.

Figure 3 shows a simulation where the process is controlled by an adaptive controller. Table 2 shows the parameters after the initialization procedure (t = 0) and after 67 seconds of adaptive control. For comparison, the true parameters of the process are also given. The estimated parameters at t = 0 and at t = 67 s do not differ much. The estimated parameters deviate somewhat from the true parameters 9tr1£ •• The main reason for this deviation is insufficient excitation during the Table 2.

al

2 Consider a process with the transfer function EXAMPLE

az bl bl bz

(13)

283

True and estimated parameters.

9(0)

9(67)

9tr1£e

-1.0850 0.2736 0.1233 0.0317 0.0308

-1.1073 0.2875 0.1057 0.0463 0.0282

-1.2198 0.3720 0.0819 0.0702 0.0001

8. 1 . 5~ ·-·······+···· .. ·······;·-··· ····;···· .. ·;,

Astrom, K. J. and T. Hagglund (1988a): Automatic Tunings of PID Controllers. Instrument Society of America, Research Triangle Park, North Carolina.

0.5

Astrom, K. J. and T. Hagglund (1988b): "A new auto-tuning design." In Preprints IFAC Int. Symposium on Adaptive Control of Chemical Processes, ADCHEM '88, Lyngby, Denmark.

-I - 1.5~ ·"····+·-"···"··+·-····f

Astrom, K. J. and B. Wittenmark (1989): AdaJr tive Control. Addison-Wesley, Reading, Massachusetts.

Rc

Hagglund, T. and K. J. Astrom (1991): "Industrial adaptive controllers based on frequency response techniques." Automatica, 21, pp. 599609.

Figure 4. Nyquist plot of the loop transfer functions based on the estimated process model 8(67) (solid line) and on the true process 8,...< (dashed line). The frequencies Wo.< and 3wo.< are marked.

Holmberg, U. (1991): Relay Feedback of Simple Systems, PhD thesis TFRT-1034. Dept. of Automatic Control, Lund Inst. of Technology, Lund, Sweden.

relay feedback and the adaptive control. In spite of this, the loop transfer function based on the estimated model 8(67) and the loop transfer function based on the true process 8true do not differ much. The Nyquist curves of these loop transfer functions are shown in Figure 4. The solid curve corresponds to the estimated model and the dashed curve corresponds to the true process. The same eighth order controller, calculated from the model 8(67) is used for both curves. The curves are adjacent at W ole , which means that the estimate is good in the critical region. The robustness margin is satisfactory since the loop transfer function is sufficiently far away from -1. Notice also that the sensitivity of the closed loop system is slightly larger than 2 although the specifications for this system were chosen to give a sensitivity of 2. This is due to the fact that the excitation is not able to make 118(67) - 8true ll = O.

7.

References

Lilja, M. (1989): Controller Design by Frequency Domain Approximation, PhD thesis TFRT1031. Dept. of Automatic Control, Lund Inst. of Technology, Lund, Sweden. Ljung, L. and T. Soderstrom (1983): Theory and Practice of Recursive Identification. MIT Press, Cambridge, Massachusetts. Lundh, M. (1991) : Robust Adaptive Control, PhD thesis TFRT-1035. Dept. of Automatic Control , Lund Inst. of Technology, Lund, Sweden. Tsypkin, Y . Z. (1984) : Relay Control Systems. Cambridge University Press, Cambridge, UK. Ziegler, J. G. and N. B. Nichols (1942): "Optimum settings for automatic controllers." Trans. ASME, 64, pp. 759-768.

Conclusions

The paper has described a procedure for automatic initialization of an adaptive controller. The procedure is based on information obtained from an experiment with relay feedback . It is shown how the data required can be obtained automatically. The result is an adaptive controller that can be started simply by pushing a button.

284