Sensitivity of nonlinear dynamic modeling

SENSITIVITY OF NONLINEAR DYNAMIC MODELING...

14th World Congress ofIFAC

H-3a-Ol-4

Copyright © 1999 IFAC 14th Triennial World Congress, Beijing, P.R. China

SENSITIVITY OF NONLINEAR DYNAMIC MODELING

Istvan Vajk

Department ofAutomation, Technical Llniversit)/ of Budapest H-1521 Budapest, Hungary e-mail: vajkCijJaut.bme.hu~fax: 361-463-2871

Abst.ract: The paper discusses the sensitivity of Inodeling with respect to the discretization and structural realization of sampled continuous nonlinear dynamic processes. As a tool for the sensitivity analysis equivalent input-output equivalent structures are introduced. The aim of the paper is to explain tile relationships behind the modeling error and to offer a systematic approach to discuss input/output equivalent structures and to sho,"' how to reduce sensitivity by choosing the proper structure. The results are verified by simulation examples, as well. Copyright ~91999 IFAC

Keywords: nonlinear approximation~ modeling, sensitivity analysis, error reduction

I. INTRODUCTION

Proper solutions for a aumber of practical control and identification problellls require accurate and high fidelity system description. System modeling is commonly used for control synthesis~ designing fault det.ection systelDs or modeJing complex large-scale plants. Recend)' several approximation 11lethods have been reported to describe the nonIinear behavior of the different processes. In general~ for nonlinear system description \ve can apply lookup tables as piece.. \vise approximation tools or "'le can approximate a nonlinear relationship by using neural nets or fuzzy base models. Several theoretical results can be found in the literature which prove the attractive properties of the developed approximation algoritluns (e.g. Jang et al. 1997, Isennann et al. 1997). The above listed methods most conlffionly used for approximation arc quite good for modeling static nonlinearities. Ho",·ever, the same methods are often used \\rhen modeling nonlinear dynamic systerns. In particular:- most of the models used for nonlinear dynamic systems can usually be separated into nvo

parts: one part describes the static nonIinearity and another contains stoTage-like elements to describe the system dynamics. The paper shows the sensluvlty of the commonly used system descriptions with respect to the modeling error. It demonstrates that although the approxilnation of the static nonlinearity itself is good enough~ in most cases the modeling error may prevent the practical application of the complete model. The Inain reason of exhibiting large modeling errors is that the feedback applied around the static nonlinearit)' may drastically increase the modeling error and make the model unusable in practice.

The paper is organized as follows. Section 2 contains the problem formulation by introducing the notion of the approxiolation error and the modeling error. Section 3 is devoted to classify the nOlllinear dynamic system models as state and input-output representations. Conditions to find equivalent realizations are also discussed here. Section 4 is about the key questions~ the role of the equivalent transfonnations is defined in the sense whether an appropriate model selection can or can not reduce

3790

Copyright 1999 IFAC

ISBN: 0 08 043248 4

SENSITIVITY OF NONLINEAR DYNAMIC MODELING...

14th World Congress ofIFAC

the IIlOdclillg error in case of modeling. Section 5 concludes the paper.

is to be used (Narendra, 1990). The applied seriesparallel and parallel structures are shown in Fig. 2. y

2_ APPROXIMATION ERROR vs MODELING ERROR

u

~OI\. single-input sing1e-output nonlinear sampled data dynamic system is often described by the following nonlincar input-output relationship: Y t = .{(u t _ 1 ,Ut-2~··.um,.Yt-l ,Yt--2,···Yt- n) -:' where .Yt is the process output, Ut is the process input, f(.··) is a nonlincar function and t=OJ1,2 ... denotes the discrete time instants. This description is a USCflll unified representation for a wide class of nonlinear. It is often called NARlVIA (nonlinear aUlorcgressive tlloving average) model in the literature. The structure of the nlode! is shown in Fig. 1, where .S denotes a static nonlincar mapping. As far as the dynamics is concerned, for the sake of simplicity n=m wiJJ be asslllned further on.

s

y

YSP

Fig. 2: Series-parallel and parallel models

Even if the approximation of the static nonlinearity S is good it can be shown (Vajk et al., 1998) that modeling of the complete nonlincar dynanllc system could be still very poor. The feedback applied around the st.atic nonlincarity Inay drastically increase the approximation error and result in large modeling errors. In other words, the modeling of, nonlinear dynamic systetns is very sensitive "vith respect to the discretization and to the structure of the model applied.

To demonstrate the approximation error propagation in a dynamic model a simple simulation example is presented_ The continuous tilne nonlillear system

Fig. 1. Static nonlinearity with external dynamics

For the approximation of the nonlinear part of the model we can apply different kinds of spline methods~ 100 kup table approach or various approximations using fuzzy base model or neural nets. For example the capability of neural net\vorks to ensure excellent approximation for nonlinear mapping is we]] known. However~ according to the nature of the approximation the real process output and the model output will not exactly be the salne, an approximation error will always be present. This error is due to the lUlIllodeled part of the nonlinearity and to the finite length number representation used when storing infonnation. In general. the identification of nonlinear dynalnic systems is perfonned in series-parallel structures. This Ineans that the process model to be identified takes the output samples of the Teal process into account ,,,hen updating the model parameters. Nanlrally~ depending on tIle infoTlnalion represented by the input-output samples the identification exhibits restricted accuracy. The samples of the real process outputs~ howcvcc arc not al,"'ays available for mooeling. This is the case lvhen perfonning silnulation or doing prediction. If the samples of the real process are not available fOT rnodeling the identification scheme works on the output samples generated by the nlodel itself, Le. a parallel scheme

considered is described by the following equation: y = (u _ y)3 As far as the approximation is concerned the lookup table approach is used~ i.e. values of the nonlinearity are stored in a lookup table. The [-4 .. 4 J interval of the input variables has been divided for 80 equal intervals. It is assumed that the output of t.he nonlinear system is llleasured by an 8 bit AJD converter. The sanlpling tline is 0.04. The step response of the tuodeled system is shown in Fig. 3. The example verifies that the approxilnation error of the nonlinearity in steady state is magnified by a

0_8r--1--"--"---"-~~~~~~~~---'--J O-7

r

a,B

r

.,v

O-St

0.4

0.3-

0.21---

O.1~/

U' °O~--1:oJob=o----"'-'2Do"--------::;;;3"""OO=-----"';400~-

500

----ebo

steps Fig.3: The step response of the rnodeled first order system

3791

Copyright 1999 IFAC

ISBN: 0 08 043248 4

SENSITIVITY OF NONLINEAR DYNAMIC MODELING...

14th World Congress of IFAC

factor of about 100. So the steady state error of the model is significant. Simulation studies show that the Inodeling error can be drastically larger if the order of the system is higher and the sampling time is small. This phenoluenon may ccrlainly jeopardize many practical applications. Accordingly ~ there is a need to elaborate alternative modeling luethods with considerably lo\ver sensitivit),. To prepare the systcluatic analysis of the sensitivity issues outlined so far.. nonlinear system model classes ,viII be introduced first~ then equivalence alllong nonlinear systelns will be considered in the next Section. The properties of this classification will be utilized in further Sections of the paper.

3 . EQUIVALENCE AMONG NONLINEAR SYSTEMS

In this Section t",,·o model classes "",ill be discussed. As far as tile fundamental structure of these classes are concerned~ the ~State Model is using a dynamical subsysteln as a feedback around a static nonlincarily ~ \vhile the Input-Output A-fodel is using a cascaded structure of the static nonlinearity and the dynamic subsystetll. At this point it should be nlentioned that all the results to be derived in this Section for discrete time systems can be generalized for continuous tilne systelns~ as \vell.

State representation When modeling nonlinear dynamic systems a typical decomposition is applied in most cases, nmnely the complete S)'stenl is built. up by someho\v combining a static non1inear and a dynamic linear subsystem. One way to set up a model is to use a static nonlinearity assumed to be unknown and driven by the process input and internal states involved by the state vector x (Fig. 4). The outputs of the static nonJinearity are the process output y and the input vector z exciting the dynamic subsystenl. The d:Yl1amic subsystelTI is assulued to involve knO\\ln memory-like elements. The dynamic subsystem can be as simple as a diagonal structure with luenlorylike elements. The complete stale lnode' is sho\vn in Fig. 4. Note that assuming that the state vectors have identical size n tlle static nonlinearity involves n / 1 functions each with n-+-l variables. The static nonlinear subsystem will be assumed to realize a single valued nonlinear mapping. Thus this part can be represented by universal (spline, neural net or fuzzy) approximators. T~ypicall)'~ t.he dynalnic subsysterll is assumed to be linear~ however~ it can be

u

y

x

z

Fig. 4: State model of nonlinear dynamic systems generalized to have nonlinear dyn(l1nics~ as ,",'ell. Also~ the dynamic subsystem may have its o,"vn internal state variables p Pt+l

== f(pp Zt)

Xl = h(Zl) It is well known that any systelll may have several inputJoutput equivalent representations. For example~ equivalent system descriptions can be looked for by introducing different x and z state veclors. It is clear that to ensure input/output equivalence various state variables ",ill result in various static and dynamic subsy'stelns.

To be explicit~ the following problctn wiH be addressed first. A nonlinear dynamic system is assumed to be modeled by a pair of a static nonlinearity SI and a dynalnic subsystelll DJ. Assume further on that we define another D 2 subsystem as part of another pair lS2~ D 2 ]. Then we ask the following question: are there properties of D 1 and D 2 which guarantee the existence of 52 such that [Sh Dd and lS2~ D 2 ] realize equivalent systems? For the sake of siluplicity restrict ourselves to discuss the case of linear dynamics and assume identical size for the z, x and p vectors. Then in the equations describing the dynaIuical subsysterll Pt+l == APt +B(Zt +d) Xl

=CPt +e

the rnatrices .04. Band C-: are quadratic matrices and the offsets are denoted by e and ,f. Then two systems are equivalent if the B and (~ nlatrices are invertible. This can be easily verified by adding B- 1B to the SI ~ D l path and adding C:- J (~ to the D 1 -75 1 path. Additionally, system realization aSSUlnes stability., as well. This condition requires an appropriate A ruatrix.

Input-output representation

Unlike state models the input-output models express the process output ~v as a function of the previous input and output samples: Yt ~ .r(ut - b U t -2 ,··.ut - n , Yt-l ?cYt-2, ..._Vt-n) To set up a model a dynamical subsystelll is introduced with the responsibility to generate the

3792

Copyright 1999 IFAC

ISBN: 0 08 043248 4

SENSITIVITY OF NONLINEAR DYNAMIC MODELING...

14th World Congress of IFAC

input-output samples as function variables for the static nOlllinearity. The input-output model then is of cascaded structure, \vhere the process input u drives the dynamic subsystem rather than the static nonlinearity as "ve have seen for the state models (Fig. 5). y

where It/[

=

[u 1 ~ U t-

u ~

(-2.. ..

f-n.

)-"(.,

.V t -

1

"~)/1-2'" .Yt_n]T .

This is the system we are going to transfonn into an input-output equivalent realization. The linear transfonnation no\\r is to be perfonncd for l-ft as follows: which leads to g~(1t}t ')

Fig. 5: Input-output model of the nonlinear dynamic systeulS The derivation of the input-output model from the state representation can be based on the implicit function theorem taking into consideration that the system is obsen;able. Some recent results in this field can be found in (Narendra and Mukhopadhyay

=0

.

In tenus of block diagrams g'(wt'J=O can be visualized as Fig.6 shows. Here L denotes a static linear subS)Jslenl. The notion of the equivalent structures will be utilized in the next Section.

1997). Comparing the number of the input variables

taken into account by the static nonlinearity it is seen that for input-output models this number has been doubled 10 2n as "ve had it ]ike n ~] for the state rnodeL However, a clear advantage of the inputoutput model is that the underlying dynamic subsystelTI is driven by the samples of the process input and output. This was not the case for the state model as there was no direct access to tIle input (z) and output (x) states handled by the dynanlicaJ sub system_ Next \ve are going to derive conditions lO find equivalent [S.. DJ pairs for input-output realization. The standard notation bv Yt

:=

~f(Ut_),Ut 2 ,~.Jlt -n ,Ytl ~Yj-2, "'YJ- n)

Fig. 6: A useful schenle for equivalent Input-Output models

4. SENSITIVITY ANALYSIS OF NOl\l"LlNEAR MODELING In this Section the lllodeling of unknown nOlllinear dynalnic systclns will be considered. As a key point the relation between the equivalent transfonnations discussed earlier and the sensitivity reduction will be shown.

can be changed to "'here

r.

Zr == [Ut -) ,u t -2, .,.u t - n ,Yt-l ~Yt-2 ~ ....Yt-n The admissible transformations for linear dynamic subsysteulS are restricted to have the fOfrll of Zlt

== Az( +b

with a nonsingular A matrix. Then substituting tIle ne,"' states Yt can be expressed as Yt ::::: ,('(Z'l ) Considering the set of the available equivalent transformations it is seen that comparing to the state models this set is rather restricted for the inputoutput models. Tile linear transforulation by z ~=A z --t b \\fill not change the structure of the dynamic

subsystem. To derive a more useful structure re\vrite the output

equation in the folloV\.ring form:

In general, the identification of nonlinear dynamic systenls is perfonned in series-parallel structure. First ~'e are going to discuss the impact of the approximation error on the model output. The discussion relates to tlle scheme sho\vIJ in Fig. R in the sense that the static nonlinearity and the linear subsystem work in parallel ,vay. Following the idea of the parallel decomposition assume the nlodel is using a state vector z and has a nominal model ~vNt=fN(Zt). The difference between the real process output and the nominal Dlodel is described by a deviation rnodel y D t == ~fD (z t ) producing the real process output by }-'t = .y~h.J' t + },D t

When identifying the syrstem we are going to estimate the deviation model. Assulue that tIle output of the deviation model is "'D ~v r

=

"() ID Zl

3793

Copyright 1999 IFAC

ISBN: 0 08 043248 4

SENSITIVITY OF NONLINEAR DYNAMIC MODELING...

14th WorId Congress of IFAC

which results in an estimated output sequence by "

Yt :: Y

N

I'D

+y

t

r.

The above equation allows to rewrite the first order error estilnation equation above as QVt +al~t-l + ... +an~t-n ~O)\ '

The error in the output estimation is

~t == S't

- Yt = iD (Zt)- iD (Zt)

Let us have a closer look at the error tertH in the above equation. Keeping in nlind that the real process outputs are not available for the modeling the state vector z can not involve such samples~ the estilnated output samples ace to be used. Accordingly, the output of the parallel model becomes

al

D

,"There at

~---.

Ozt-i

ASSUlue now that an equivalent state model is available. As the admissible transfonnations among the equivalent tDodels are restricted to linear mappings via nonsingular luatrices, any equivalent state can be expressed as Zlt == AZ t +b . The error of the estimation expressed by the ne\,,' state variables becolnes

where the differential model is

() ~ bV

yD t ==]D(2,).

.Yt"""

of

t

+

qf'v (z', ) &' '" ,

t

.

(,"'fZ t

Also, the rnodeling error can be wTi tlen as

CfYt ==Yt-Y t =JD(Zt)~.rD(Zt). Next the relation between the adaptive lllodeling and the equivalent transfonnations will be discussed. The tenn of the adaptive modeling covers the following n~{o phase procedure. As first phase ident.ificat.ion is assunled to start by taking samples of series-parallel model~ i.e. real process outputs arc used for the underlying estimation schenles. As a next phase the modeling is continued in parallel structure~ i.e. estimations ",,.ill be based on the model output rather than the real process output. Then concerning the equivalent structures the follo"ving questions need to be answered: 1. Will a suitable selection among the equivalent models modify the propagation of the error when sVfitching over from series-parallel model to parallel model? 2. Will a suitable selection among the equivalent models reduce the error in the series-parallel model?

To ans\\'er the first question consider the relation between errors in the series-parallel and t.he parallel lllodels. Assume that tlle state variables are the input-output samples of the process: Zt =

To estimate the error a first order Taylor series approximation gives

..-.

•

.-.J

s:-!'

OYt ~

+

lfD(Zt) ~

az,

v..:::: t

OF

lfD(Zt)A- 14>':-" _5..-- +qfD(Zt)8z ~ V~( -~Yt aZ[ t Ozt

The above equation sho\-vs that the error propagation is independent from the equivalent transfonnation applied. The Illcssage of this relationship is that the error propagation can not be affected by equivalent transfornlations. The result itself is not surprising as the equivalent transfonnation does not Inodify the ~,1 oap gain~~. No,"' we turn to ans"l'er the second question. It is to be investigated \"'hether or not the modcling error obLained for series-parallel Illodels depends on the equivalent transfonnations. The modcling error is due to the estimation ofJD and can be expressed as

kl\

:=

~~'l -

t

op

[Jp

= L oiu

p Op,

apt

i

1

,

Pi

where oP is a diagonal matrix "\vith Opj! Pl eleluents along its main diagonal. For an equivalent model assmne p'=. Cp + d (with a nonsingular (-:' Inatrix ) to describe the relation bet\veen the estiluated parameters. Then the luodeling error expression becomes -..J",.

QVl

where

JD (Zt)- .iD(Zl) .

~ "" oin ~ = aiv bP p

~r"

~

J-'r =:

Assuming that the error is due to sOllte estimated parameters p, a first order Taylor series approxirnation for the estimated output error can be obtained by ~

'1

UVt ......

.~

--

Q}'t ;::,::uv r +

(u t - 1 ~ .. Ut~Yl~Yt-l ~· ••Yt-n]

In a parallel lnodel we have the rnodel outputs as entries in the state vector: Zt :::::. (u t-l , .. U t- n Yt- r ~ .••yt - rz ]

~

Taking into account the relationship betw-een the state variables z and z ~ the first order error estimation can be expressed as:

a ;=::;:_J_D_s.-.'=~opt

aP

arn

~f'1t

I

l/}/

a' P

1=

P

o!-"J'

~_-:J_D_pri _L_i L Gp'. pI . r·

I

I

3794

Copyright 1999 IFAC

ISBN: 0 08 043248 4

14th World Congress of IFAC

SENSITIVITY OF NONLINEAR DYNAMIC MODELING...

Using the relationship between the allo\vs to re\\'Tite the above equation as

paralueters

Si, ~ afllo 8P' pI =: afD C- 1 iJpt (Cp + d) _ ~. t

opt

ap

It is seen that there is no specific choice for C'! to influence the modeling error ~ because in practice §P1 has alnlost identical entries. At the same time "rith a proper choice of d the modeling error can be reduced significantly. Specifically, by reducing some measure of p + Cl d the modcling error will be reduced as well via the equivalence transfonnation.

Up to this point the relative paralneler sensitivity of the models have been analyzed and we have found tbat an equivalent transfornlation (Le. special selection of the matrix A) "rill not affect the modeling error. However~ in tenus of numerical processing I.he numerical conditions of the estimations can be improved by proper orthogonalization of all the inputs exciting the static nonlineari •.1l • This consideration luay support the selection of the . 4 matrix. More involved discussion of the orthogonalization can be found in (Bokor, 1997). In case of slnall sanlplillg time the system can be well approximated by integrators. Thus it is worth to supply the L subsystem with operations capable for the integration. SpecificaJl)!, if the dynamics contains only cascaded shift operators, then the L subsystem perfonns

for a first order Jag and

yf'l == 2Yt-l - Yt-2 for second order systetus. The parameters of the L subsystem can be determined in a silniJar way for higher order systems. The output of the subsystelll 50,1 produces then the nth derivative of the system output. This selection corresponds to the delta transformation applied to linear systems (Middleton and Good"vin~ 1990) as the output of the dynamic subsystem is U t-n ~ U t-n-l -

U [-n~·· ·,J/t-n ~ Yt-n-+-I -

Yt-n~···

The sensitivity with respect to the approximations \vhen Inodclinglidentifying nonlinear dynarnic systems can be improved significantly by using the above modeling procedure.

5. CONCLUSIONS

Models and the Input-Output Models introduced in the paper arc not only applicable for lnodeling purposes, but they also offer tools for sensitivity analysis. When modeling mtknown nonlinear dynamic systetllS experience shows that even sInal1 errors in the approximation of the static nonlinearity lead to large errors in the complete modeling. The sensitivity analysis explains the fact that the modeling error of series-parallel structures is 111agnified in a large extent \vhen s"vitchillg over to parallel structures. Due to the luain result present.ed in the paper equivalent transfonnations can not effect the rate as the error of the series-parallel structure is gained for parallel structures. Ho\Yever~ equivalent syst.em realizations may help to significantly reduce the error in the series-parallel structure i tse If.

Ackno\vledgements

Part of the research "vas supported by the Hungarian National Research Fund (grant number T015776) and the fund of Hungarian Academy of Sciences for con tro) research.

REFERENCES Bokor,J. (1997). Approximate identification for robust control. 3rd joint COSY \vorkshop. ESF scientific program on control of conlplex systems, Hungary, pp.181-192. Havldn, S. (1994). Neural networks. Maclllillan - College Publishing Company~ Nel\! York. lsermann~R., Emst,S. and Nelles,O. (1997). Identification v.rith dYllaulic neural netv{orks architectuTes~ comparisons~ applications. SYSID~97. Vo1.3. pp.997-1022. Jang~J.S.R_~ C.T.Sun and E.Mizutani (1997). Neurofuzzv and soft computing. Prenticc-Hall.lnc. Middlet';n,R.H.~ and Goodvlin:-G.C. (1990). Digital control and estimation. A unified approach. Prentice-Hall IntemationaC luc. Narendra~K.S. and Parthasarathy~K. (1990). Identification and control of dynalnical systems using neural nel\vorks. IEEE Transactions OIl NeuraJ Networks~ VoJ. 1, No. J. pp. 4-27, Narendra,K.S. and S.Mukhopadhyay (1997). Neural net\vork for system identification~ SYSID ~ 97 Vo1.2. pp.763-770. VajkJ.~ He1.thessy~J_ and Charaf~H.(1998). Reduction of Inodeling error in nonlinear dynamic systems. NOLCOS~98 IFAC Conference. Enschede~ The Netherlands. pp.354-359.

Modeling nonlinear dynamic systems require mode-l structures witll building eletncnts to realize nonlinear mapping and dynamical actions. The State

3795

Copyright 1999 IFAC

ISBN: 0 08 043248 4