Parameter Bounding in Armax Models from Records with Bounded Errors in Variables

Copyright © IFAC Identification and System Parameter Estimation, Budapest, Hungary 1991

PARAMETER BOUNDING IN ARMAX MODELS FROM RECORDS WITH BOUNDED ERRORS IN VARIABLES V. Cerone Dipartimento di AUlomatica e Informatica, Politecnico di Torino, corso Duca degli Abruzzi 24,10129 Torino,ltaly

Abstract. The errors in variables problem is that of parameter estimation where all observed variables are corrupted by noise, So far, the computation of parameters bounds for dynamical models have been performed using the equation-error approach, assuming that the explanatory variables are exactly known, In this paper, parameter bounds for autoregressive-moving-average-exogenous (ARMAX) models are derived on the assumption that both input and output are affected by bounded noise , Parameter bounding by both the bounded equation error and the bounded errors-in-variables approach is outlined, together with the approximation of the feasible parameter region by an orthotope outer-bounding algorithm and tested on simulated data, ' Keywords, Identification; parameter bounding; equation error; errors-in-variables; bounded errors; ARMAX models; linear programming, consistent with the measurements , the error bounds and the assumed model structure, are considered as acceptable solutions of the identification problem , For linear models, when instantaneous constraints on the measurement errors are available , the jpr is a convex and simply connected polytope. Since the feasible parameter region may be quite complex, two methods can be found in the literature which compute outer approximations of the jpr, Milanese and Belforte (1982) suggested to bound the feasible parameter region by an orthotope aligned with the parameter coordinate axes; the orthotope is computed by linear programming, A recursive algorithm given by Fogel and Huang (1982) provides an ellipsoidal set outer-bounding the jpr. A combined use of the two algorithms , outer-bounding by linear programming and ellipsoidal outer-bounding, has been tested by Mo and Norton (1988) and Belforte , Bona and Cerone (1990). The interested reader can refer to Norton (1987b), Waiter and Lahanier (1988) as wide survey papers .

INTRODUCTION Most identification methods are based on the assumption that the system input is exactly known and all noise is considered as additive equation error (see, e,g" Soderstrom and Stoica, 1989) , However, the assumption of noise-free input may not be a realistic and correct one because of measurement errors , The equation error is then correlated with the measured input, leading to bias both in statistical parameter estimates and in parameter bounds (Norton, 1987a), Problems where both output and input are noisy are referred to as errors-in-variables problems, A wide collection of the main contributions when the variables are all considered to be affected by stochastic errors, can be found in Soderstrom (1981), together with sufficient conditions for identifiability, numerical investigations and accuracy analysis of some possible identification methods,

The purpose of this paper is to address the parameter bounding problem for ARMAX models when all the observed variables are affected by bounded noise.

When the number of samples is short, statistical parameter-estimation methods may fail to supply reliable estimates, In fact , available measurements records may not be long enough to check a posteriori the statistical hypothesis , Moreover, there are situations where the errors are intrisically deterministic: systematic and class errors in measurement equipment, rounding and truncation errors are some,

PROBLEM FORMULATION Consider the single-input single-output linear discrete-time system (ARMAX model) depicted in Fig. 1. The true input signal, XI, and the noise-free output, Wt, are related through the linear difference equation

An alternative to the stochastic description of measurement errors is the bounded-errors characterization , where the uncertain variables are assumed to belong to a given set. In the bounding context, all parameter vectors belonging to the feasible parameter region (fpr) , i,e, parameters

(1) 965

where A(-) and B(-) are polynomials in the backward shifting operator z - J, [z - JWt = Wt- d ,

Belforte , Bona, Cerone (1988) gave a conjecture how to construct t.he jPT for static linear models with bounded uncertainties in regressors; that conjecture is proved to be true in Cerone (1991) . Merkuryev (1989) presents a solution, based on interval analysis methods , which deals with linear static models when both input and output signals have interval nature. T empo , Barmish and Trujillo (1988) presented an approach to robust estimation and prediction of ARMA models when both input and output are contamined by bounded errors. Clement and Gentil (1988a , 1988b) addressed the problem of parameter bounding for ARMAX models when the output is a noisy vector , while the input is consid ered as exactly known. This is not a canonical errors-in-variables problem, since the input is assumed to be unaffected by noise ; hence the regressor variables are composed of noisy (autoregressive) and exact (moving-average) samples. For that problem, Clement and Gentil (1988a) gave a necessary condition for the description of D.

Let Yt and Ut be the noise-corrupted measurements of Wt and Xt respectively

Yt

= Wt + 17t,

Ut= Xt+~t,

t = l , ... ,N ;

(3)

where N is the number of output samples. Uncertainties are known to vary within given bounds , i.e ., 1

1]t 1~ t:.1]t, I ~ t:.~t·

(4) (5)

1~t

The unknown parameter vector i2. E RP is defined as: i2.T = raj ... ana bo bJ •• • bnb ], (6) where na + n b + 1 = p. The feasible parameter region is defined as

D = {i2.ERP:A(z - l)[Yt l1]t I~ t:.1]t ;

1~t

-1]tl =

B(Z-l)[Ut - ~tl ; (7)

I~ t:.~t ; t

PARAMETER BOUNDING IN ARMAX MODELS

= l , ... ,N}.

Bounded equation error approach

Since if N is large the set D may become fairly complex, orthotope-outer bounding algorithms are considered in this paper, which compute a tight orthotopic set B containing D and providing exact parameters uncertainties intervals (pui's). In formulas pU1.j

B

= [e jmm, emaxl j ,

J.

= 1 , ... , p ;

{i2.ERP : ej = 8j +bej ,

1bej

where

(8)

(9)

1]t +

e]c =

aj 17t- j -

L

bj ~t _ j)

(14)

j=O

denotes the equation error. The model (13) be equivalently written as:

e]min + e]max

' max min t:.e ] = I e ] _ eJ

nb

na

L

j=l

I~ t:.e j / 2, j = 1, ... , p},

where A

Reducing equations (1) and (3) yields the following noisy linear regression model :

r ai~

(10)

2

1

'

(11 )

where

and


(12)

[-Yt- I ... - Yt -na Ut Ut- l .. . Ut-nb] . (16)

If the equation error bounds are available , i.e. t:.T/ such that 1Tt I~ t:.Tt, the feasible parameter region D for linear regressions is known to be described by

The Chebyshev center of D , QC (also called the central estimate) , enjoys nice optimality properties (Kacewicz and co-workers , 1986). The problem of finding the feasible parameter set when bounds on the output errors are given has been approached by Milanese and Belforte (1982) , Fogel and Huang (1982) and a number of au thors (see the surveys by N orton (1987b) and WaIter and Lahanier (1988)).

1

Yt -
t = l, . .. ,N,

(17)

(Fogel and Huang , 1982; Milanese and Belforte , 1982). Unfortunately, the main difficulty arises in specifying noise bounds t:.1"t on the equation error 1"t according to those available on the input and output measurement errors. In fact , the equation error Tt depends on the measurement errors 1]t and ~t as well as on the unknown parameter vector i2. according to equation (14); hence, at least in principle, bounds on Tt cannot be computed. In practice, in the presence of noisy autoregressive and exact moving-average terms, Clement

Surprisingly, parameter bounding in the case of bounded errors-in-variables has received very little attention so far. Norton (1987a) gave an insight on the identification of ARMAX models in bounding context, paving the way for the bounded errors in all variables approach.

966

and Gentil (1988a) proposed the computat.ion of t:.rt by means of

na t:.rt = t:.TJt + L

1

aj 1 t:. 17t _j,

approaches outlined in Cerone (1991) (this Symposium), which makes it possible direct use of bounds on measurement errors. Let equations (1) and (3) be reduced to the following form:

(18)

j~ 1

na

nb

Yt=- L(Yt-j -TJt-j)aj+ L(Ut -j -~t _j )bj+TJt. (21) j=1 j=O

where i'ij is the least squares estimate of the true parameter a j' This particular choice is somewhat arbitrary and , what is more , the least square estimate may not belong to the feasible parameter region (Belforte, Bona and Cerone, 1987), thus leading to an inconsistent evaluation of t:.rt. A rough iterative and more proper evaluation of t:.rt, which implies the evaluation of the bounding set B , might consist of the following steps:

It can be seen that model (21) and conditions (4) and (5) fit in the framework of the bounded-errors-in-variables model outlined in Cerone (1991) , whose necessary and sufficient condition, when applied to this case gives: (<£t - ~~f(J.. ~ Yt

+ t:.TJt,

(22)

+ ~~ f(J.. ~ Yt

- t:.TJt,

(23)

(<£t

(a) compute

t:.r~ = t:.TJt

na

nb

j= 1

j=O

where

+ L 1ii j 1t:.I]t- j + L 1bj 1t:.~t -j,

where ii j and

b]

~;T

t:.~tsgn( bo),

are such that

t:.~t - I sgn( bl

minimum Chebyshev-norm esit is well known it belongs to parameter region (Tempo and 1988);

(b) compute B O (i.e. t:.rt = .6.r?) ;

) , ... ,

t:.~t - nbsgn( bnb )1 ,

and <£t is given by (16). Inequalities (22) and (23) describe the set V t , i.e. the feasible parameter region at the i-th step.

(19) and & is the timate of (J..; the feasible Wasilkowski,

= [t:.TJt _Isgn(aJ) , .. . , t:.TJt-nasgn(a na ), (24)

Remark -- When the input y, is assumed to be precisely known and the measurement vector Y is corrupted by noise , equation (25) reduces to: -

the bounding set when

t:.'P;T

(c) update .6.1'; according to

=

[t:.TJt - Isgn(aJ) , ... , t:.TJt-na sgn(ana), (25) 0,0 , ... , 0],

which, together with conditions (22) and (23) , is the result given by Clement and Gentil (1988a, 1988b).

t:.r; = t:.TJt +

I: t:. 7t - j [a,mE;z~.,{1 aTin 1,1 ajax I}] + 1

]= 1

f

]=0

t:.~t-j [bmDaZ~ l {1 bT in 1,1 bjax I}] ; ,E

i=l , ... ,N; (d)

--k

~N

ORTHOTOPE OUTER-BO UN DING ALGORITHM

(20)

When regressors are exactly known , V is defined by a set of 2N linear inequalities. Each pu.i's can be obtained by solving a linear programming (LP) problem; hence , the computation of the bounding orthotope requires the solution of 2p LP problems with 2N constraints and p variables (Milanese and Belforte, 1982) .

kj

compute.6.r = L:t=1 t:.rt N;

(e) update Bk (i.e. the bounding set when t:.rt = t:.r tk ) ., -

k

(f) repeat (c)-(e) until either t:.r or the volume of Bk (vol (Bk)) is converged within a given

When regressors are contaminated by noise , the region V is the union of at most 2P convex sets , each one being confined in an orthant of the parameter space (Cerone, 1991). Consequently, in order to find the pui's, at most 2P orthotope must be computed, i.e. 2p2 P LP problems with 2N + p constraints and p variables must be solved (as conjectured in Belforte , Bona and Cerone, 1988) . Therefore , in principle, the complexity of the 01'thotope bounding algorithm becomes fairly high as p increases. In practice , however , that complexity can be considerably reduced . In fact, if there are rn ~ p parameters whose sign is a priori known , then the number of LP problems to be solved becomes 2p2 P- m. The assumption on the sign of parameters is reasonably feasible

threshold. As conjectured by N orton (1987b), resulting bounds on rt may be very pessimistic and lead to an unacceptbale loose feasible parameter region; numerical simulations are also designed to investigate this possibility. Bounded errors-in-variables approach The main drawback of the equation error approach lies in the derivation of bounds on rt from those available on the measured variables errors. A different solution will be introduced here , based on the bounded errors-in-variables

967

when physical information about the identification problem under consideration are available. Another substantial reduction of complexity can be achieved when the jpr is simply connected. In that case , the following computational scheme is suggested, which performs LP only in those orthant containing a subset of 1):

DISCUSSION For a proper comparison of the approaches outlined in this paper , both the central es timate and the bounding-orthotope volmne were taken into account. The central estimate gives an idea to what extent the feasible parameter region is centered around the true parameters. The volume of the outer-bounding box is a good measure of the size of the jpr; in other words, it contains information about parameter uncertainty intervals, in fact vol(B) = n~= l puij. Furthermore, the computing time was observed in order to evaluate the efficiency of both the approaches.

i) choose the starting orthant in the parameter space as the one which contains a non empty subset of 1); this can be done using the point estimator &given by (19) which always provides a vector belonging to to the feasible parameter region. Thus, the starting orthant may be the one containing &;

For low noise level (6E ::; 10%), the central estimates from both the bee and the beiv approach give satisfactory estimation of the true parameters (Fig. 2 and Fig. 3); vol(B) obatined with the bee is only slightly larger (indipendently from the samples size) than that computed with the bei11 (Fig. 6). As the number of observations increases, vol(B) decreases unsurprisingly.

ii) compute the pui's in the start.ing orthant; iii) consider those pui's which have the zero value as an extreme, and perform LP in the orthants next to them, obtaining a new extreme (e't n or er;aX) of the considered pui; iv) repeat iii) until no zero value as extreme point in the pui's is found.

Increasing the noise level (61' 2: 20%), both central estimates become less and less accurate (Fig. 4 and Fig. 5); vol(B) from the bee is significantly larger than the one obtained via beiu (Fig. 6); this result confirm that parameter bounds in the case of bounded equation-error may be rather loose.

NUMERICAL EXAMPLE A numerical example is introduced in this section, which illustrate the behaviour of both the equation-error and the errors-in-variables approaches. The system considered here is an ARMAX model, characterized by (1), (3) and (1 - 1.1 z- 1 + 0.28z- 2 ), (Z-I + 0.5z- 2 ).

One of the aspects to be evaluated about algorithms is t.he computing time since it provides a measure of their efficiency. Simulations introduced in this paper were developed on a FAX STATION 3100 computer system. The CPU time required by the bee approach is from 4 (for low noise level) to 16 (in the case of high noise level) times larger than the beiv's. This is mainly because bee needs to iterate the basic orthotope bounding algorithm in order to assess bounds 6rt on the equation error rt. The relative convergence threshold for vol(B) was fixed to 10- 4 •

(26)

Bounded relative errors have been used, i.e., = ErYt, 1Er I::; 6E¥, ~t = EtUt, 1Et I::; 6Et; {xt}, {En and {En are random sequences belonging to the uniform distributions U[-p"',+p"'], U[-6EY, +6EY] and U[-6 EU, +6E U ] respectively. The parameter vector is = [- 1.1 0.28 1 0.5]. The system has two real poles for ZJ = 0.4 and Z2 = 0.7 and a zero at Z3 = -0.5. "7t

e

CONCLUSIONS Parameter bounding of ARMAX models has been outlined for syst.ems with bounded errors in both input and output data. Suitable equation-error approach has been proposed, pointing out a proper evaluation of bounds on the equation error. A bounded errors-in-variables method has been outlined, based on previous results of the author. Orthotope outer-bounding approximations of t.he /pr and related complexity problems have been considered and discussed; a computational scheme has been proposed to reduce such a complexit.y. Both the bounded equation-error (bee) and the bounded errors-in-variables (beiv) approaches have been tested on simulated data from a dynamic system, pointing out the superiority of beiv to bee as far as parameter uncertainty intervals and computing time are concerned. In fact , the jPT computed with beiv is significantly smaller than the one computed via bee, implying smaller uncertainties on paramet.ers.

In simulation, bounds on the errors at the input and the output were set as equal, i.e., 6EY = 6€U = 61'; this is a realistic assumption since one may think to use the same measurement equipment to collect samples of Ut and Yt. Four different values of uncertainties bounds were considered: 61' = 5% , 10%, 20% , 30%; for a given 61' ten indipendent sets of samples with different size have been generated (N = 100, 200,300, ... ,1000); p'" = 10 was kept fixed through the whole simulation. Parameter bounding from these records has been carried out for both the bounded equation-error (bee) and the bounded errors-in-variables (beiv) approaches, computing orthotopic approximations of the feasible parameter set according to algorithms outlined in this paper.

968

ACKNOWLEDGMENTS

Norton , J. P. (1987b). Identification and application of bounded-parameter models. Automatica. 23, 497-.597. Soderstroll1 , T. (1981). Identification of stochastic linear systems in presence of input. noise. Automatica, 17, No. 5, 713-72.5 . Soderstrom , T. , P. Stoica (1989). Syst em Id en tification. Prentice Hall International (UK). Tempo, R., B. R . Barmish , J . Trujillo (1988). 8th IFAC/ IFORS Symposium on Id entification and System Param eter Estimation, Beijing, 1136-1140 . Tempo, R ., G. W. Wasilkowski (1988). System fj Control Letters, 10, 265-270. WaIter, E., H. Piet-Lahanier (1988). Estimation of parameter bounds from bounded-error data: a survey. Proc. 12th IMACS Congress on Scien tific Computation, Paris , 467-472.

This research was partly developed while Dr. Cerone was honorary research fellow at. the School of Electronic and Electrical Engineering , University of Birmingham, UK, supported by a CNR scholarship, and concluded at the Politecnico di Torino , supported by the italian Ministry of Universities and Research in Science and Technology (MURST) , under the plan "Fondi per la Ricerca Scientifica .- 40% e 60 %". The author is thankful to Dr. J. P. Norton and Dr. S. M. Veres for discussions and their comments OIl several aspects of this paper.

REFERENCES Belforte G. , B. Bona, V. Cerone (1987). Bounded measurement error estimates: their properties and their use for small sets of dat.a. Measure ment, vol .5 No 4, 167-175. Belforte, G. , B. Bona, V. Cerone (1988). A bounded error approach to the tuning of a digital voltmeter. Proc. 12th IMACS Congress on Scientific Comp'utalion, Paris , 498-.501. Belforte , G. , B . Bona, V . Cerone (1990) . Parameter estimation algorithms for a setmembership description of uncertainty. A utomai1:ca.26 . No .5 , 887-898. Cerone, V., (1991). Parameter bounds for models with bounded errors in all variables. 9th IFAC/IFORS Symposium on Id en tification and System Parameter Estimation, Budapest, Hungary, 8-12 July 1991. Clement, T. , S . Gentil (1 988a) . Reformulation of parameter identification with unknown-butbounded errors. Math ematics and Computers in simulation, 30, 2.57-270. Clement, T. , S. Gentil (1988b). Recursive membership set estimation for ARMAX models: an output-error approach . Proc . 12th IMACS Congress on Scientific Computation, Paris, 484-486. Fogel, E. , Y.F. Huang (1982). On the value of information in system identification bounded noise case . Automatica, 18, 229-238. Kacewicz , B .Z., M. Milanese , R. Tempo , A. Vicino (1986). Optimality of cent.ral and projection algorithms for bounded uncertainty. System fj Control Letters, 8 , 161-171. Merkuryev , Y. A . (1989). Identification of objects with unknown bounded disturbances . In t. 1. Control, 50, No 6, 2333-2340. Milanese , M. , G. Belforte (1982). Estimation theory and uncertainty intervals evaluat.ion in presence of unknown but bounded errors. Linear families of models and estimators . IEEE Trans. on A ·u tom. Control, AC-27 , 408-414. Mo, S . H ., NortoIl, J. P. (1988). Parameter-bounding identification algorithms for boundednoise records. Proc. lEE, 13.5 , D , 2,127-132. Norton, J. P . (1987a). Identification of parameter-bounds for ARMAX models from records with bounded noise. Int. J. Control, 45 , 37.5390.

B( Z- l) A( z- l)

Fig. 1: Dynamic System with input and output corrupting noise .

-_ ..._._-.__ .... _--_.._-----_ ...._- -_ .._._-----.._---_ .._-----..,

0.5

o

. ...... bee .. beiv true parameters

-0.5

-I

-1.51 ----L~ L

__

..l_--1-.---'----'-~"___-'--_

100 200 300 400 500 600 700 800 900 1000 N - Number of observations Fig. 2: Central Estimates from noisy records with errors bounds ll EY = llE" = llE = .5%, and for different size of the samples set .

969

1.5 :----,--r---r--,-----,-----,--

-----,--

1r': ---: :-.-:-.-.-:-'-.:.:::.: ---.-: -.-' '"--'.-.' --, -.-" -.-.. -----, ---,~-,.- --' '.' ---" .-- '-.---, -.-" -.-' --,.,-.-"-~

I

O~

......:.

....-.0_5 -

0.5 " ...." . .-,.,-.- .---"-.-..- -.. ---.,.--.,.--_., ....... - ...-.,.-.. ------.------.-----.. -----" ~ : -::,,_-,.'--'., '-- •.---.. .... _ ,, __.• -.-.- ' _.- ,,-M.' " - -- ,_. '_"

I

'.- "_.,,,._ •• __ • __ . - -'"

.. . ... . .

o

- . bee

-- bee

------- beiv

-- -- --- -- beiv

i true parameters

-0.5 ·

-0_5

true parameters

I

l·......

.L:l ,• ... . . . ·....

-I ~

-1.5

1

·. m !

I

--'---, --~----

lOO 200 300 400 500 600 700 800 900 1000

lOO 200 300 400 500 600 700 800 900 1000

N - Number of observations Fig. 3: Central Estimates from noisy records with errors bounds tl.- EY = tl.- EU = tl.- E = 10%, and for different size of the samples set.

N - Number of observations Fig . 5: Central Estimates from noisy records with errors bounds tl.-E Y = tl.-Eu = tl.-E = 30%, and for different size of the samples set.

vol (B) - Outer-Bounding Orthotope Volume

1.5 ~---.---,---,--.------.---,--'

102 ~

---,--

bee

10 1 f

1 ~ - -- - - ------- ___ _____:._:::---:::.": . --.' ..-::---::-.-:._---:-.-::-:::-::·.-::-:·:-.-::-::-------7

---------- beiv

I·" O.5 ~:.:-,. :-:::---: -: :.:-:::': :-: ':-._' -.-.-,-.-' :-.-:---:: -.-::'::--____-:-:: -__:--:: co:' -- ' " :-: __ -__-

30 %

10- 1-""" 0-

. ::- .. -

------ bee 20 %

---------- beiv true parameters

-0_5-

-I - --_

"""',' ,

- ..

'

~

-' .-'..

10- 4

-..

.,-

.<:.:::..... ".

100 200 300 400 500 600 700 800 900 1000

10- 5'-------'--_ _ _-'------'-_-'-_-'----'-_----'---' 100 200 300 400 500 600 700 800 900 1000

N - Number of observations Fig. 4: Central Estimates from noisy records with errors bounds tl.- EY = tl.-Eu = tl.-E = 20%, and for different size of the samples set.

N - Number of observations Fig. 6: Outer-bounding orthotop e volumes for different values of the noise level and different size of the samples set.

-1.5 _--'-_---'----'-_--'-_-'-------L_ _-----'_ _

970