- Email: [email protected]
Box-Jenkins Continuous-Time Modeling
Copyright © IFAC System Identification Santa Barbara, California, USA, 2000
BOX-JENKINS CONTINUOUS-TIME MODELING R. PINTELON, J. SCHOUKENS and Y. ROLAIN
Vrije Universiteit Brussel, Departmelll ELEC: Pleinlaan 2, 10.50. Brussel, Belgium Abstract. This paper treats the identification of continuous-time models using arbitrary bandlimited excitation signals. A modeling approach is presented that has the two follOWing advantages: 1) asymptotically (the amount of data tends to infinity) there is no approximation error over the complete frequency band from DC to Nyquist, 2) it allows to identify general parametric noise models. The key idea is to combine a continuous-time plant model with a discrete-time noise model (= hybrid Box-jenkins model structure) . Copyright @20001FAC Key Words. system identification, continuous-time systems and models, frequency domain
1. INTRODUCTION
model parameters, and the identification can be done using standard time domain methods (see Liung, 1999 and the references therein). A parametric noise model is identified Simultaneously with the plant model (LJung, 1999).
This paper treats the identification of linear, time invariant, continuous-time systems (see Fig. 1) "no
a y(n)(t) = "ni,
£...n = 0 n
£...m
=0
b u(m)(t)
(1)
m
If the illput u(t) is non-periodiC and ha7ld-limited, then differential equation (I) is transformed to a discretetime equation by explicit or impliCit approximation of the time-derivatives using digital filters. According to the digital filters used these methods are known as the Poisson moment functional approach, the integrated sampling approach, the instantaneous sampling approach , the Laguerre approach . . , (see Chou et al., 1999; Sinha and Rao, 1991 , and the references therein). These modeling approaches work reasonably well (small systematic errors) for frequencies below f / 4, but in the neighbourhood of the Nyquist frequency (f > f / 4) they either introduce large errors or require (very) high order digital filters (Van hamme et al., 1991). Weighted Cnon-)linear least squares (Sinha and Rao, 1991), maximum likelihood (Van hamme et aI. , 1991), instrumental variable (Sinha and Rao, 1991) and subspace identiflGltion (Johansson et al., 1999) methods have been developed. All these methods, except the ARMAX-modeling in Johansson (1994), either do not identify a noise model or assume that it is known. Also the systematic approximation errors are mostly not taken into account in the consistency proofs.
from N samples u(rT s> and y(rT s) ' r = 0, I, ... , N - I , of the input u(t) and output yet) signals (T s stands for the sampling period). The basic problem of parametric continuous-time modeling is the correct reconstruction of the time-derivatives in differential equation (1) from the observed samples u(rT) and y(rTs)' r = 0, I, ... , N - I . In practise the output observations are always disturbed by noise and, besides a plant model, one should also identify a noise model for vet) (see Fig. 2), which complicates the problem. The solutions proposed in the literature can be classified according to the input signal used. If the input u(t) is periodic, then diffe rential equation (I) can be transformed without systematic errors to a
set of linear algebraic equations, and the identification can be done in the frequency domain (see Schoukens and Pintelon, 1991; Pintelon et al., 1994, and the references therein). A non-parametric noise model is obtained independently of the plant model in a preprocessing step CSchoukens et al .. 1997). If the illput u(t) is non-periodiC and piecewise constant, then differential equation (1) can be transformed without systematic errors to a difference equation which is parametrized in the original continuous-time
We conclude from this brief overview that the existing continuous·time modeling approaches for non-periodic band-limited signals firstly do not a llow accurate modeling near to the Nyquist frequency and hence always suffer from a (small) bias, and secondly are unable to identify a general parametric noise model. The contribution of this paper is to fill these two gaps:
This work is supponed by the Belgium National Fund for Scientific Research; the Belgian Government as a pan of the Belgian Programme on lnteruniversity Poles of Attraction (IUAP41 2) initiated by the Belgian State, Prime Minister'S Offlce, Science Policy Programming; and the Flemmish Community (GOA-IMMlJ.
193
u(t)
l >1
without any restriction on the bandwidth of the excitation signal U(/). In this section the asymptotic behaviour (N ~ (0) of the alias term &,sk) is studied for band-limited excitations Signals with finite power.
y(t)
G(s) [ "-
"
Ts
Ts
I
I
u(rTs)
y(rT)
Definition 1: A deterministic (random) Signal X(/) is band-limited if its amplitude (power) spectrum IX(jw)1 (S xx(jw» satisfies: IX(jw)1 = 0 (S xx(jw) = 0) for any Iwl > w B with w R < ws/2.
Figure 1: Identifi c3tion of a continuous-time system: prnh'el11
statement.
Definitirm 2: A band-limited signal x(r) has finite nonzero power if,
• It is shown that the approximation (alias) errors of the direct approach (Pintelon and Schoukens, 1997) in the complete frequency band from DC (f = 0) to Nyquist (f = f/2) converge to zero at the rate O(N-1/2) as N ~ 00.
I f+NT sl2 . E{x 2(r)}dl=O(No»0 NT s - NT ,I2
for any N,
00
(6)
included (E{) is the expected value) .
• A consistenl estimator for a hybrid Box-Jenkins model structure is developed: it combines a parametric continuous-time plant model with a parametric discrete-time noise model.
Defillition 3: A normalised periodic signal has the form
Note that the same hybrid modeling idea has been used in Johansson (994) for continuous-time ARMAX models.
where A k > 0 and where F increases with N. The peak value of X(I) is uniformly bounded (maxlx(t)1 ~ C < 00 for any N, 00 included) . (
Because of lack of space the proofs of the theorems have been omitted. The reader is referred to Pintelon and Schoukens (2000) and Pintelon et at. (2000) for more details .
For stationary random signals (6) implies that x(l) should have a finite non-zero variance. 2.2. Results
The convergence rate to zero of the plant transient term T(s k' e) and the alias term &.s k) is established in the following two lemmas.
2. CONTINUOUS-TIME MODEL FOR ARBITRARY SIGNALS 2 . ]. Introduction
Lemma 1 - crJr//.'erRence rate T(s k' e) : For hounded excitations u(t) with finite left (n b - I) th order derivative applied to stable plants G(s, e) or unstable plants captured in a stabilizing feedback loop, the plant transient term T(sk' e) tends to zero as O(N-1I2). Proof See Pintelon et al. (2000).
Consider the set up shown in Fig. 1. The relationship between the scaled discrete Fourier transforms (OFT) U(k), Y(k) of the input and output samples u(rT s)' y(rT s)
X(k) =
WIl2L~ ~ ~ x(rTs)Zk'
= exp(j21tk/ N),
X plant transfer function
(zk
G(s, e)
= U,
8(s, e) A(s, e)
Y and x
~nu Ln =
= u, y),
(2)
conllerp,cnce rate &,sk): Consider bandlimited Signals with finite non-zero power. The residual alias error &.s k) tends to zero as O(N- I12 ) in probability for random excitations with differentiable power spectrum Su/jw), dSuu(jw)/ dw < 00 for Iwl ~ w B ' and as O(N-1/2) for (normalised) periodic excitations. Proof See Pintelon and Schoukens (2000) and Pintelon et at. (2000). Lemma 2 -
and the
(3) a sn
°n
is given by (Pintelon and Schoukens, 1997) Y(k) = G(sk' ()U(k)
+ T(Sk' e) + &,sk)
where &,sk) is the alias error, sk T(s, e) the plant transient term
= j21tk/(NTs)'
(4)
Lemmas 1 and 2 allow to calculate how fast the extended transfer function model (4) converges to
and
Y(k)
T(s, e)
/(s, e)
A(s, e)
(S)
~na Ln =
= G(s k' e) U(k)
(H)
as N ~ 00. This is summarized in the follOWing theorem.
a sI!
°n
with n i = max(n a, n b ) - I. The parameter vector () contains the plant model parameters and the equivalent initial conditions of the plant model: eT = [a Tb TiT] with aT = [aOa, .. . a n ] , b T = [bOb] ... b n ] and iT = [ioi, ... i n ]. Transter function model (4/is made identifiable by'constraining e, for example, an = I or Ilalli + Ilblli + Ililli = 1. Formula (4) is exactly true
Theorem 1 - COllllerp,ence rate extended transfer functioll model: Under the assumptions of Lemmas 1 and 2 the rebtive convergence rate of (4) to (8) is O(N-1/2) in probability for random excitations with power spectrum, O(N-1/2) for differentiable normalised periodic signals (see Definition .:D, and O(N-') at the excited DFT frequencies for periodic 194
theorem.
R
c
Theorem 2 - c(}/wergence rate hyhrid Box~/ellkills model: Under the assumptions of Lemmas 1 to 3 the relative convergence rate of (12) to (3) is O(N - 1I2) in probability for random excitations with differentiable power spectrum, O(N-I/2) for normalised periodic signals, and O(N - I ) at the excited OFf frequencies for periodic signals with a fixed number of frequencies. Proof Follows immediately from Theorem 1 and Lemma 3. 3.2. The estimator
Figure 4: Block diagram of rhe electrical circuit with R = 16 n , C = 9.4 IlF, L = I Hand ( l - 1000 y - 2 . The inductance is consrructed using a capacitive loaded gyrator (Te mes and LaPatra , )977). The circuit basically acts as a seco nd order planr with nn top some disturbances due to the non-linear feedback . For broad band random excitarions and breadboard periodic excitations with random phases u(r) these disturbances act as noise (Schoukens et ai. , ) 9'-)H). The rms and peak value of the input signal used equal 16 mY and 60 mY respectively .
e
The frequency domain estimate SJ of the hybrid 13oxJenkins model parameters minimizes V(IJ, z) = .!..Ni'\Y(k) - G(St' 8)U(k) - T(St ,
8) - T H(Z,I, 8)\2
Cl 4)
consistency of the noise model parameters is maintained if the summation in (14) is replaced by the union of disjoint sets of indices k such that the corresponding zk values of each set cover uniformly the unit circle (see Pintelon and Schoukens, 1999). This allows filtering of the input and output spectra (for example , removal of every second Off line) without affecting the consistency of the noise model parameters.
H( Zk l , 8)
N,=o
w.r.t. 0, where z stands for the vector of the observed input and output OFf spectra. Some additional assumptions are required to prove the consistency of
eBJ ·
Assumption 1 - disturhillg noise: The noise sequence e(rT s) is independently distributed with zero mean , variance 0 2 and bounded moments of order 4. The noise e(t) is independent of the excitation u(t).
With some additional effort the presented method allows also hybrid ARMAX modeling. The denominators of the plant and the noise model are then parametrized in their poles, and the poles of H(Z - I, 8) are related to those of G(s, 8) by the impulse invariant transformation z = exp(sT s)' Starting values are obtained via the hybrid 13ox-Jenkins model.
Assumption 2 - existence oj a true model: The data is generated according to y(l) = G(s, ( 0)u(l) + H c(s)e(t) with u(t) band-limited, e(t) piecewise constant , G(s, ( ) stable, and H(z-I, (0) = (l-z - I)Z{s - IHc(s)} 0 a stable and inversely stable monic filter. G(s, (0) and H(z-I, (0) have no cancelling poles and zeros .
The approach is not consistent in feedback. The basic reason for this is that the observed input signal , which is a mixture of a band-limited Signal (originating from the reference signal) and a filtered zero-order-hold signal (originating from the process noise), is multiplied in the hybrid Box-Jenkins model structure by a continuous-time plant model.
Assumption 3 - identifiahili~v conditioll: There exists an NO such that for any N ~ NO' included, the expected value of the cost function (14), E ( V(O, z)} , has a unique global minimum 8 •. 00
Assumption 4 - regulari~y cOIldilion cost junctio1/.· There exists an NO such that for any N ~ No , 00 included, the cost function (14) , V(8, z), is a continuous function of fJ in a closed and hounded neighbourhood of 8 • .
In practise the continuous-time noise e(t) will mostly not be piecewise constant (Lowen and Teich, 1990; Pyati, 1992) and the discrete-time noise model will be an approximation. This only influences the uncertainty, but not the consistency, of the estimated plant model parameters.
Theorem 3 - consistency plant and noise model parameters: Under the conditions of Lemmas 1 to 3 and Assumptions 1 to 4 the estimates of the plant and noise model parameters are consistent:
plim [G
SJ ]
N --+"" hSJ
= [GO] bo
and plim [C N --+""
SJ
dBJ
]
Although the equivalent initial conditions of the plant and noise model are not conSistently estimated, they are added in the cost function (14) to improve the finite sample behaviour of the estimates of the plant and noise model parameters.
(15)
Proof See Pintelon et ai, (2000) .
Although the influence of the alias term 8,sk) in the cost function (14) is hidden in the observed output (see Cl 2) with = ( 0 ), it is present and causes a (small) bias in the estimated model parameters. This bias term tends to zero as N tends to infinity (see Theorem 3). To suppress its influence for finite values of N, the order of the numerator polynomial /(s, 8) in the plant transient term T(s, 8) is increased nj ~ max(n ll , n b ) - 1 .
3.3. Discussion
°
Although the input signal u(t) is band-limited, the summation in the cost function (14) is taken over all the OFf frequencies. Otherwise the estimate of the noise model parameters would not be consistent. The estimate of the plant model parameters is consistent even if the summation is only taken over a subset of the OFf frequenCies, for example, the bandwidth of the excitation (see the proof of Theorem 3). The
196
30 I
i 0 ;
Ol...-_ _-r
iD Ql
j
~ -30
~ -30 ~
c.
c. E
,
co-60
co-60 ..,
:£. Q)
E
I
-90
-90 ~·~ '~~'Io 50 100
1 ~~~-'-~~~'1~' --,--,--,- ~ ~
o
50
100
150
200
frequency (Hz)
Figure 5: Validation of identified plant model with random
200
C ross-v~l id~ti()n of the identifie d pl3nr model with the periodic excitation measurements . Bo ld line : fre4uency response function (FRf ) Gp(s,) oiJtained frolll the p e riodic e xcitation expe,iment. ",lid line: variance measured FRF var(Gp(s,)), and dashes: difference hetwee n the plant mode! G(S,. OOE) identified from rhe periodic excitation measure-tuenls and rhe plant mouei G(sk" &al) identified from the randoll1 excitation rnea SUfemen[s
Figure 7:
reSr{)no:.e fUn (1i()n ( FI{F) G,(s,) obtained from the random excitation experiment. solid line : difference berween FHF and plant model G,(s,) - G(s,. OSI) , dashes : variance of the plant model var(G(.".OSI» exc it;Jtio n
150
frequency (Hz)
Illeasurelllents . I)of.s : frequcn('Y
-30
iD
iD -70
:£.
:£.
Ql
Q)
'0
~ -60
c.
C.
.~
E co
E -110 co
-150
I
o
100
'
. ,.-- -y-----r-
~--y-
200
T
-90 o!~~T-'
1
50
300
100
150
200
frequency (Hz)
frequency (Hz)
Figure 8: cross-v;tlidalion of the ide ntified noise model with the lx..'riotiic excil :Hioll 1llt'
Figure 6: Validation o f the identified noise model with the r:u1oolll t.: xciration measurements. l)ols : noise resi du:ds \/0:) , solid line : Jloise llllHJd al-f(z.i. I , ORJ) , and Jashes : va rianct' o f (ht.~ noise modt'l v"r(aU(z;: l. eSI»
4. EX1'ERIMENTAL RESULTS
,
I '
Y(k) - T(sk' 8SJ ) - T H( Z-;' ,8BJ ) U(k)
The hybrid modeling approach is illustrate d on real measurements of an eleclrical circuil which acts :tS a second order plant (see Fig. 4). Two experimenls have heen performed, lhe first with a hand-limited random signal and the second with a hand-limited periodic signal with random phases. Both signals have the same power and the same bandwidth (200 Hz). For both experiments the sampling frequency I s is ahout 600 Hz (20 MHz / 2 IS ). The hybrid 13ox-Jmkins model is estimated using the random excitation experimenl and is cross-validaled using lhe periodic excitation experiment.
(16)
while figure 6 comp:tres the estimaled noise model H(z-;, J, 8sJ ) to the noise residuais V(k) ~ Y(k) - C(.". ORI)U(k) - rIsk'
OSI) - rH( li l , Osl)
(17)
Note thal the plant model is only shown within the bandwidth of the excitation signal while the noise model is shown from DC (I = 0) to Nyquist (j = 1/2) The estimated Box-Jenkins model is cross-validated with th e periodiC measure ments in Figures 7 and B. Figure 7 compares th e estim:.Ited planl model G(sk , 8 SJ ) to the frequency response fun ction Gp(sk) obtained from lhe periodic excitalion experiment, while Figure B compares lhe estimated noise mode l H(z-;, I, 8SJ ) 10 the non-parametric noise model var( Gp(s k»1 U(k)1 2 obtained from lhe periodic measuremenls. It follows lhat the estimated 13oxJenkins model explains very well lhe periodic excitation experiment. Note that lhe cross-validation of lhe noise model is only possible within the bandwidth of the excitation signal.
N = 10400 inpul/ oulpUl samples of the random excilation experimenls are used to identify the 13oxJenkins model (12). It lurns out that a model SlrUClure with second order plant model (nil = 2 , n" = 0 , n i = 2) and an eighth order noise mode l (n = 8 , c nd = 8, n j = 7) explains very well the data (see Figures 5 and 6). Figure 5 compares the estimated plant model G(sk,8 SJ ) 10 lhe frequency response funclion Gr(sk) ohtained from the random excitation experiment
197
5. CONCLUSION
Estimated from a Small Numher of Data Sets, Automatica, Vo!. 33, no 6, pp. 107.)-1086. Schoukens j., T Dobrowiecki and R. i'inte!on (998). Parametric Identification of Linear Systems in the Presence of Nonlinear Distortions. A Frequency Domain Approach, IEEE Trans. Autom. Contr., Vo!. AC-43, no 2, pp. 176-190. Sinha , N. K. and G. 1'. Rao, eds. (1991). Identification of COlltilluuus-time Systems. K1uwer Academic Publishers, Dordrecht (The Netherlands). Siiderstriim, T (1974). Convergence properties of the generalized least squares identification method, Automatica, Vo!. 10, pp. 617-626. Temes G. C. and j. W. LaPatra (977). Circuit Synthesis and Design. McGraw-Hill, New York (USA). Van hamme, H., R. Pintelon and J. Schoukens (991). Discrete-time mode!ing and identification of continuous-time systems: a general framework. Chapter 2 q[ 'Identification of Continuuus-time Systems', edited hy Sinha and Rao, Kluwer Academic Publishers, Dordrecht (The Netherlands), pp. 17-77.
The simplicity of the presented continuous-time modeling is very appealing: the method merely boils down to a classical processing of the input and output OFT spectra and does not require the design of any digital filter. The proposed hybrid Box-Jenkins model contains asymptotically no plant structure approximation errors and allows general parametric noise models. A disadvantage of the method is that it is not consistent in feedback. 6. REFERENCES llrillinger, D. R., Time Series: Data Analysis and 77Jeory. New York: McGraw-Hill, 198J. Chou, C T , M. Verhaegen and R. Johansson (999). Continuous-Time Identification of SISO Systems Using Laguerre Functions. IEEE Trans. Signal Processing, Vo!. 47, no 2, pp. 349-362. Johansson, R. (994). Identification of Continuous-Time Models. IEEE Tram . Signal Processing, Vo!. 42, no 4, pp. 887-897. Johansson, R., M. Verhaegen and C T Chou (1999). Stochastic Theory of Continuous-Time State-Space Identification. IEEE Trans. Signal Prucessing, Vo!. 47, no 1, pp. 41-51. Kailath, T (980). Linear Systems. I'rentice-Hall , Engl ewood Cliffs, NJ (USA). Ljung, L (1999). System Identification, Theory for the User. Prentice-Hall , Upper Saddle River, NJ (USA). Lowen, S. 13. and M. C Teich (990). Power-Law Shot Noise, IEEE Trans. Inform. Theory, Vo!. IT-36, n ' 6, pp. 1302-1318. Lukacs, E. (975) . Stochastic Convergence. Academic Press, New York (USA) . Middleton, R. H. and G. C Goodwin (1990). Digital Control and Estimation. Prentice-Hall, London. Pintelon , R.. P. Guillaume, Y. Rolain, J. Schoukens and H. Van hamme (994). Parametric Identification of Transfer Functions in the Frequency Domain - A Survey. Il:Eh' Tram. Automat. Contr. , Vo!. .39, no 11 , pp. 2245-2260. Pinte!on , R. and J. Schoukens (1997). Identification of Continuous-Time Systems Using Arbitrary Signals. Automatica, Vo!. 33, no 5, pp. 991-994. Pinte!on, R. and J. Schoukens (999). Time Series Analysis in the Frequency Domain, IEEE TrailS. Sign . Proc ., Vo!. 47, no 1, pp. 206-210. Pintelon, R. and J. Schoukens (2000). Box-Jenkins Continuous-time modeling, Internal Note 2000/ 01. Pinte!on, R., j. Schoukens and Y. Rolain (2000). BoxJenkins Continuous-time modeling, Automatica, Vo!. 36, no 7. Pintelon, R., J. Schoukens and G. Vandersteen (1997). Frequency Domain System Identification Using Arbitrary Signals, IEEE Tram. Autom. Contr., Vo!. AC-42, no 12, pp. 1717-1720. Pyati V. P. (992). An Exact Expression for the Noise Voltage Across a Resistor Shunted hy a Capacitor, IEEt: Trans. Circuits and SystemS-I, Vo!. CAS-39, n° 12, pp. 1027-1029. Schoukens, j. and R. Pintelon 099]). Identification of Linear Systems: A Practical Guideline to Accurate Modeling. Pergamon, London (GB). Schoukens, j., G. Vandersteen, R. Pintelon and P. Guillaume (997). Frequency Domain System Identification Using Non-parametric Noise Models
198
BOX-JENKINS CONTINUOUS-TIME MODELING R. PINTELON, J. SCHOUKENS and Y. ROLAIN
Vrije Universiteit Brussel, Departmelll ELEC: Pleinlaan 2, 10.50. Brussel, Belgium Abstract. This paper treats the identification of continuous-time models using arbitrary bandlimited excitation signals. A modeling approach is presented that has the two follOWing advantages: 1) asymptotically (the amount of data tends to infinity) there is no approximation error over the complete frequency band from DC to Nyquist, 2) it allows to identify general parametric noise models. The key idea is to combine a continuous-time plant model with a discrete-time noise model (= hybrid Box-jenkins model structure) . Copyright @20001FAC Key Words. system identification, continuous-time systems and models, frequency domain
1. INTRODUCTION
model parameters, and the identification can be done using standard time domain methods (see Liung, 1999 and the references therein). A parametric noise model is identified Simultaneously with the plant model (LJung, 1999).
This paper treats the identification of linear, time invariant, continuous-time systems (see Fig. 1) "no
a y(n)(t) = "ni,
£...n = 0 n
£...m
=0
b u(m)(t)
(1)
m
If the illput u(t) is non-periodiC and ha7ld-limited, then differential equation (I) is transformed to a discretetime equation by explicit or impliCit approximation of the time-derivatives using digital filters. According to the digital filters used these methods are known as the Poisson moment functional approach, the integrated sampling approach, the instantaneous sampling approach , the Laguerre approach . . , (see Chou et al., 1999; Sinha and Rao, 1991 , and the references therein). These modeling approaches work reasonably well (small systematic errors) for frequencies below f / 4, but in the neighbourhood of the Nyquist frequency (f > f / 4) they either introduce large errors or require (very) high order digital filters (Van hamme et al., 1991). Weighted Cnon-)linear least squares (Sinha and Rao, 1991), maximum likelihood (Van hamme et aI. , 1991), instrumental variable (Sinha and Rao, 1991) and subspace identiflGltion (Johansson et al., 1999) methods have been developed. All these methods, except the ARMAX-modeling in Johansson (1994), either do not identify a noise model or assume that it is known. Also the systematic approximation errors are mostly not taken into account in the consistency proofs.
from N samples u(rT s> and y(rT s) ' r = 0, I, ... , N - I , of the input u(t) and output yet) signals (T s stands for the sampling period). The basic problem of parametric continuous-time modeling is the correct reconstruction of the time-derivatives in differential equation (1) from the observed samples u(rT) and y(rTs)' r = 0, I, ... , N - I . In practise the output observations are always disturbed by noise and, besides a plant model, one should also identify a noise model for vet) (see Fig. 2), which complicates the problem. The solutions proposed in the literature can be classified according to the input signal used. If the input u(t) is periodic, then diffe rential equation (I) can be transformed without systematic errors to a
set of linear algebraic equations, and the identification can be done in the frequency domain (see Schoukens and Pintelon, 1991; Pintelon et al., 1994, and the references therein). A non-parametric noise model is obtained independently of the plant model in a preprocessing step CSchoukens et al .. 1997). If the illput u(t) is non-periodiC and piecewise constant, then differential equation (1) can be transformed without systematic errors to a difference equation which is parametrized in the original continuous-time
We conclude from this brief overview that the existing continuous·time modeling approaches for non-periodic band-limited signals firstly do not a llow accurate modeling near to the Nyquist frequency and hence always suffer from a (small) bias, and secondly are unable to identify a general parametric noise model. The contribution of this paper is to fill these two gaps:
This work is supponed by the Belgium National Fund for Scientific Research; the Belgian Government as a pan of the Belgian Programme on lnteruniversity Poles of Attraction (IUAP41 2) initiated by the Belgian State, Prime Minister'S Offlce, Science Policy Programming; and the Flemmish Community (GOA-IMMlJ.
193
u(t)
l >1
without any restriction on the bandwidth of the excitation signal U(/). In this section the asymptotic behaviour (N ~ (0) of the alias term &,sk) is studied for band-limited excitations Signals with finite power.
y(t)
G(s) [ "-
"
Ts
Ts
I
I
u(rTs)
y(rT)
Definition 1: A deterministic (random) Signal X(/) is band-limited if its amplitude (power) spectrum IX(jw)1 (S xx(jw» satisfies: IX(jw)1 = 0 (S xx(jw) = 0) for any Iwl > w B with w R < ws/2.
Figure 1: Identifi c3tion of a continuous-time system: prnh'el11
statement.
Definitirm 2: A band-limited signal x(r) has finite nonzero power if,
• It is shown that the approximation (alias) errors of the direct approach (Pintelon and Schoukens, 1997) in the complete frequency band from DC (f = 0) to Nyquist (f = f/2) converge to zero at the rate O(N-1/2) as N ~ 00.
I f+NT sl2 . E{x 2(r)}dl=O(No»0 NT s - NT ,I2
for any N,
00
(6)
included (E{) is the expected value) .
• A consistenl estimator for a hybrid Box-Jenkins model structure is developed: it combines a parametric continuous-time plant model with a parametric discrete-time noise model.
Defillition 3: A normalised periodic signal has the form
Note that the same hybrid modeling idea has been used in Johansson (994) for continuous-time ARMAX models.
where A k > 0 and where F increases with N. The peak value of X(I) is uniformly bounded (maxlx(t)1 ~ C < 00 for any N, 00 included) . (
Because of lack of space the proofs of the theorems have been omitted. The reader is referred to Pintelon and Schoukens (2000) and Pintelon et at. (2000) for more details .
For stationary random signals (6) implies that x(l) should have a finite non-zero variance. 2.2. Results
The convergence rate to zero of the plant transient term T(s k' e) and the alias term &.s k) is established in the following two lemmas.
2. CONTINUOUS-TIME MODEL FOR ARBITRARY SIGNALS 2 . ]. Introduction
Lemma 1 - crJr//.'erRence rate T(s k' e) : For hounded excitations u(t) with finite left (n b - I) th order derivative applied to stable plants G(s, e) or unstable plants captured in a stabilizing feedback loop, the plant transient term T(sk' e) tends to zero as O(N-1I2). Proof See Pintelon et al. (2000).
Consider the set up shown in Fig. 1. The relationship between the scaled discrete Fourier transforms (OFT) U(k), Y(k) of the input and output samples u(rT s)' y(rT s)
X(k) =
WIl2L~ ~ ~ x(rTs)Zk'
= exp(j21tk/ N),
X plant transfer function
(zk
G(s, e)
= U,
8(s, e) A(s, e)
Y and x
~nu Ln =
= u, y),
(2)
conllerp,cnce rate &,sk): Consider bandlimited Signals with finite non-zero power. The residual alias error &.s k) tends to zero as O(N- I12 ) in probability for random excitations with differentiable power spectrum Su/jw), dSuu(jw)/ dw < 00 for Iwl ~ w B ' and as O(N-1/2) for (normalised) periodic excitations. Proof See Pintelon and Schoukens (2000) and Pintelon et at. (2000). Lemma 2 -
and the
(3) a sn
°n
is given by (Pintelon and Schoukens, 1997) Y(k) = G(sk' ()U(k)
+ T(Sk' e) + &,sk)
where &,sk) is the alias error, sk T(s, e) the plant transient term
= j21tk/(NTs)'
(4)
Lemmas 1 and 2 allow to calculate how fast the extended transfer function model (4) converges to
and
Y(k)
T(s, e)
/(s, e)
A(s, e)
(S)
~na Ln =
= G(s k' e) U(k)
(H)
as N ~ 00. This is summarized in the follOWing theorem.
a sI!
°n
with n i = max(n a, n b ) - I. The parameter vector () contains the plant model parameters and the equivalent initial conditions of the plant model: eT = [a Tb TiT] with aT = [aOa, .. . a n ] , b T = [bOb] ... b n ] and iT = [ioi, ... i n ]. Transter function model (4/is made identifiable by'constraining e, for example, an = I or Ilalli + Ilblli + Ililli = 1. Formula (4) is exactly true
Theorem 1 - COllllerp,ence rate extended transfer functioll model: Under the assumptions of Lemmas 1 and 2 the rebtive convergence rate of (4) to (8) is O(N-1/2) in probability for random excitations with power spectrum, O(N-1/2) for differentiable normalised periodic signals (see Definition .:D, and O(N-') at the excited DFT frequencies for periodic 194
theorem.
R
c
Theorem 2 - c(}/wergence rate hyhrid Box~/ellkills model: Under the assumptions of Lemmas 1 to 3 the relative convergence rate of (12) to (3) is O(N - 1I2) in probability for random excitations with differentiable power spectrum, O(N-I/2) for normalised periodic signals, and O(N - I ) at the excited OFf frequencies for periodic signals with a fixed number of frequencies. Proof Follows immediately from Theorem 1 and Lemma 3. 3.2. The estimator
Figure 4: Block diagram of rhe electrical circuit with R = 16 n , C = 9.4 IlF, L = I Hand ( l - 1000 y - 2 . The inductance is consrructed using a capacitive loaded gyrator (Te mes and LaPatra , )977). The circuit basically acts as a seco nd order planr with nn top some disturbances due to the non-linear feedback . For broad band random excitarions and breadboard periodic excitations with random phases u(r) these disturbances act as noise (Schoukens et ai. , ) 9'-)H). The rms and peak value of the input signal used equal 16 mY and 60 mY respectively .
e
The frequency domain estimate SJ of the hybrid 13oxJenkins model parameters minimizes V(IJ, z) = .!..Ni'\Y(k) - G(St' 8)U(k) - T(St ,
8) - T H(Z,I, 8)\2
Cl 4)
consistency of the noise model parameters is maintained if the summation in (14) is replaced by the union of disjoint sets of indices k such that the corresponding zk values of each set cover uniformly the unit circle (see Pintelon and Schoukens, 1999). This allows filtering of the input and output spectra (for example , removal of every second Off line) without affecting the consistency of the noise model parameters.
H( Zk l , 8)
N,=o
w.r.t. 0, where z stands for the vector of the observed input and output OFf spectra. Some additional assumptions are required to prove the consistency of
eBJ ·
Assumption 1 - disturhillg noise: The noise sequence e(rT s) is independently distributed with zero mean , variance 0 2 and bounded moments of order 4. The noise e(t) is independent of the excitation u(t).
With some additional effort the presented method allows also hybrid ARMAX modeling. The denominators of the plant and the noise model are then parametrized in their poles, and the poles of H(Z - I, 8) are related to those of G(s, 8) by the impulse invariant transformation z = exp(sT s)' Starting values are obtained via the hybrid 13ox-Jenkins model.
Assumption 2 - existence oj a true model: The data is generated according to y(l) = G(s, ( 0)u(l) + H c(s)e(t) with u(t) band-limited, e(t) piecewise constant , G(s, ( ) stable, and H(z-I, (0) = (l-z - I)Z{s - IHc(s)} 0 a stable and inversely stable monic filter. G(s, (0) and H(z-I, (0) have no cancelling poles and zeros .
The approach is not consistent in feedback. The basic reason for this is that the observed input signal , which is a mixture of a band-limited Signal (originating from the reference signal) and a filtered zero-order-hold signal (originating from the process noise), is multiplied in the hybrid Box-Jenkins model structure by a continuous-time plant model.
Assumption 3 - identifiahili~v conditioll: There exists an NO such that for any N ~ NO' included, the expected value of the cost function (14), E ( V(O, z)} , has a unique global minimum 8 •. 00
Assumption 4 - regulari~y cOIldilion cost junctio1/.· There exists an NO such that for any N ~ No , 00 included, the cost function (14) , V(8, z), is a continuous function of fJ in a closed and hounded neighbourhood of 8 • .
In practise the continuous-time noise e(t) will mostly not be piecewise constant (Lowen and Teich, 1990; Pyati, 1992) and the discrete-time noise model will be an approximation. This only influences the uncertainty, but not the consistency, of the estimated plant model parameters.
Theorem 3 - consistency plant and noise model parameters: Under the conditions of Lemmas 1 to 3 and Assumptions 1 to 4 the estimates of the plant and noise model parameters are consistent:
plim [G
SJ ]
N --+"" hSJ
= [GO] bo
and plim [C N --+""
SJ
dBJ
]
Although the equivalent initial conditions of the plant and noise model are not conSistently estimated, they are added in the cost function (14) to improve the finite sample behaviour of the estimates of the plant and noise model parameters.
(15)
Proof See Pintelon et ai, (2000) .
Although the influence of the alias term 8,sk) in the cost function (14) is hidden in the observed output (see Cl 2) with = ( 0 ), it is present and causes a (small) bias in the estimated model parameters. This bias term tends to zero as N tends to infinity (see Theorem 3). To suppress its influence for finite values of N, the order of the numerator polynomial /(s, 8) in the plant transient term T(s, 8) is increased nj ~ max(n ll , n b ) - 1 .
3.3. Discussion
°
Although the input signal u(t) is band-limited, the summation in the cost function (14) is taken over all the OFf frequencies. Otherwise the estimate of the noise model parameters would not be consistent. The estimate of the plant model parameters is consistent even if the summation is only taken over a subset of the OFf frequenCies, for example, the bandwidth of the excitation (see the proof of Theorem 3). The
196
30 I
i 0 ;
Ol...-_ _-r
iD Ql
j
~ -30
~ -30 ~
c.
c. E
,
co-60
co-60 ..,
:£. Q)
E
I
-90
-90 ~·~ '~~'Io 50 100
1 ~~~-'-~~~'1~' --,--,--,- ~ ~
o
50
100
150
200
frequency (Hz)
Figure 5: Validation of identified plant model with random
200
C ross-v~l id~ti()n of the identifie d pl3nr model with the periodic excitation measurements . Bo ld line : fre4uency response function (FRf ) Gp(s,) oiJtained frolll the p e riodic e xcitation expe,iment. ",lid line: variance measured FRF var(Gp(s,)), and dashes: difference hetwee n the plant mode! G(S,. OOE) identified from rhe periodic excitation measure-tuenls and rhe plant mouei G(sk" &al) identified from the randoll1 excitation rnea SUfemen[s
Figure 7:
reSr{)no:.e fUn (1i()n ( FI{F) G,(s,) obtained from the random excitation experiment. solid line : difference berween FHF and plant model G,(s,) - G(s,. OSI) , dashes : variance of the plant model var(G(.".OSI» exc it;Jtio n
150
frequency (Hz)
Illeasurelllents . I)of.s : frequcn('Y
-30
iD
iD -70
:£.
:£.
Ql
Q)
'0
~ -60
c.
C.
.~
E co
E -110 co
-150
I
o
100
'
. ,.-- -y-----r-
~--y-
200
T
-90 o!~~T-'
1
50
300
100
150
200
frequency (Hz)
frequency (Hz)
Figure 8: cross-v;tlidalion of the ide ntified noise model with the lx..'riotiic excil :Hioll 1llt'
Figure 6: Validation o f the identified noise model with the r:u1oolll t.: xciration measurements. l)ols : noise resi du:ds \/0:) , solid line : Jloise llllHJd al-f(z.i. I , ORJ) , and Jashes : va rianct' o f (ht.~ noise modt'l v"r(aU(z;: l. eSI»
4. EX1'ERIMENTAL RESULTS
,
I '
Y(k) - T(sk' 8SJ ) - T H( Z-;' ,8BJ ) U(k)
The hybrid modeling approach is illustrate d on real measurements of an eleclrical circuil which acts :tS a second order plant (see Fig. 4). Two experimenls have heen performed, lhe first with a hand-limited random signal and the second with a hand-limited periodic signal with random phases. Both signals have the same power and the same bandwidth (200 Hz). For both experiments the sampling frequency I s is ahout 600 Hz (20 MHz / 2 IS ). The hybrid 13ox-Jmkins model is estimated using the random excitation experimenl and is cross-validaled using lhe periodic excitation experiment.
(16)
while figure 6 comp:tres the estimaled noise model H(z-;, J, 8sJ ) to the noise residuais V(k) ~ Y(k) - C(.". ORI)U(k) - rIsk'
OSI) - rH( li l , Osl)
(17)
Note thal the plant model is only shown within the bandwidth of the excitation signal while the noise model is shown from DC (I = 0) to Nyquist (j = 1/2) The estimated Box-Jenkins model is cross-validated with th e periodiC measure ments in Figures 7 and B. Figure 7 compares th e estim:.Ited planl model G(sk , 8 SJ ) to the frequency response fun ction Gp(sk) obtained from lhe periodic excitalion experiment, while Figure B compares lhe estimated noise mode l H(z-;, I, 8SJ ) 10 the non-parametric noise model var( Gp(s k»1 U(k)1 2 obtained from lhe periodic measuremenls. It follows lhat the estimated 13oxJenkins model explains very well lhe periodic excitation experiment. Note that lhe cross-validation of lhe noise model is only possible within the bandwidth of the excitation signal.
N = 10400 inpul/ oulpUl samples of the random excilation experimenls are used to identify the 13oxJenkins model (12). It lurns out that a model SlrUClure with second order plant model (nil = 2 , n" = 0 , n i = 2) and an eighth order noise mode l (n = 8 , c nd = 8, n j = 7) explains very well the data (see Figures 5 and 6). Figure 5 compares the estimated plant model G(sk,8 SJ ) 10 lhe frequency response funclion Gr(sk) ohtained from the random excitation experiment
197
5. CONCLUSION
Estimated from a Small Numher of Data Sets, Automatica, Vo!. 33, no 6, pp. 107.)-1086. Schoukens j., T Dobrowiecki and R. i'inte!on (998). Parametric Identification of Linear Systems in the Presence of Nonlinear Distortions. A Frequency Domain Approach, IEEE Trans. Autom. Contr., Vo!. AC-43, no 2, pp. 176-190. Sinha , N. K. and G. 1'. Rao, eds. (1991). Identification of COlltilluuus-time Systems. K1uwer Academic Publishers, Dordrecht (The Netherlands). Siiderstriim, T (1974). Convergence properties of the generalized least squares identification method, Automatica, Vo!. 10, pp. 617-626. Temes G. C. and j. W. LaPatra (977). Circuit Synthesis and Design. McGraw-Hill, New York (USA). Van hamme, H., R. Pintelon and J. Schoukens (991). Discrete-time mode!ing and identification of continuous-time systems: a general framework. Chapter 2 q[ 'Identification of Continuuus-time Systems', edited hy Sinha and Rao, Kluwer Academic Publishers, Dordrecht (The Netherlands), pp. 17-77.
The simplicity of the presented continuous-time modeling is very appealing: the method merely boils down to a classical processing of the input and output OFT spectra and does not require the design of any digital filter. The proposed hybrid Box-Jenkins model contains asymptotically no plant structure approximation errors and allows general parametric noise models. A disadvantage of the method is that it is not consistent in feedback. 6. REFERENCES llrillinger, D. R., Time Series: Data Analysis and 77Jeory. New York: McGraw-Hill, 198J. Chou, C T , M. Verhaegen and R. Johansson (999). Continuous-Time Identification of SISO Systems Using Laguerre Functions. IEEE Trans. Signal Processing, Vo!. 47, no 2, pp. 349-362. Johansson, R. (994). Identification of Continuous-Time Models. IEEE Tram . Signal Processing, Vo!. 42, no 4, pp. 887-897. Johansson, R., M. Verhaegen and C T Chou (1999). Stochastic Theory of Continuous-Time State-Space Identification. IEEE Trans. Signal Prucessing, Vo!. 47, no 1, pp. 41-51. Kailath, T (980). Linear Systems. I'rentice-Hall , Engl ewood Cliffs, NJ (USA). Ljung, L (1999). System Identification, Theory for the User. Prentice-Hall , Upper Saddle River, NJ (USA). Lowen, S. 13. and M. C Teich (990). Power-Law Shot Noise, IEEE Trans. Inform. Theory, Vo!. IT-36, n ' 6, pp. 1302-1318. Lukacs, E. (975) . Stochastic Convergence. Academic Press, New York (USA) . Middleton, R. H. and G. C Goodwin (1990). Digital Control and Estimation. Prentice-Hall, London. Pintelon , R.. P. Guillaume, Y. Rolain, J. Schoukens and H. Van hamme (994). Parametric Identification of Transfer Functions in the Frequency Domain - A Survey. Il:Eh' Tram. Automat. Contr. , Vo!. .39, no 11 , pp. 2245-2260. Pinte!on , R. and J. Schoukens (1997). Identification of Continuous-Time Systems Using Arbitrary Signals. Automatica, Vo!. 33, no 5, pp. 991-994. Pinte!on, R. and J. Schoukens (999). Time Series Analysis in the Frequency Domain, IEEE TrailS. Sign . Proc ., Vo!. 47, no 1, pp. 206-210. Pintelon, R. and J. Schoukens (2000). Box-Jenkins Continuous-time modeling, Internal Note 2000/ 01. Pinte!on, R., j. Schoukens and Y. Rolain (2000). BoxJenkins Continuous-time modeling, Automatica, Vo!. 36, no 7. Pintelon, R., J. Schoukens and G. Vandersteen (1997). Frequency Domain System Identification Using Arbitrary Signals, IEEE Tram. Autom. Contr., Vo!. AC-42, no 12, pp. 1717-1720. Pyati V. P. (992). An Exact Expression for the Noise Voltage Across a Resistor Shunted hy a Capacitor, IEEt: Trans. Circuits and SystemS-I, Vo!. CAS-39, n° 12, pp. 1027-1029. Schoukens, j. and R. Pintelon 099]). Identification of Linear Systems: A Practical Guideline to Accurate Modeling. Pergamon, London (GB). Schoukens, j., G. Vandersteen, R. Pintelon and P. Guillaume (997). Frequency Domain System Identification Using Non-parametric Noise Models
198