Robustness to Outliers of Bounded-Error Estimators, Consequences on Experiment Design

Copyright © IFAC Identification and System Parameter Estimation. Budapest. Hungary 1991

ROBUSTNESS TO OUTLIERS OF BOUNDED-ERROR ESTIMATORS, CONSEQUENCES ON EXPERIMENT DESIGN L. Pronzato and E. Waiter wboralOire des Signaux et Systemes, CNRSIEcole Superieure d'Electricite, Plateau de MOlllon, 91192 Gif-sur-Yvette Cedex, France

Abstrac!. If proper precautions are not taken, bounded-error estimators are not robust to outliers, i.e. to data points where the actual error is larger than assumed when specifying the error bounds. The Outlier Minimal Number Estimator (OM NE) has been designed to overcome this difficulty and has proved on various examples to be particularly insensitive to outliers. This paper is devoted to a theoretical study of its robustness. The notion of breakdown point, introduced to quantify the robustness of point estimators, is extended to setestimators. When the model output is linear in the parameters, OMNE is shown to possess the highest achievable breakdown poin!. A bound on the bias due to oUlliers is established and used to define a new policy for optimal experiment design aimed at providing a higher protection against outliers than conventional Doptimal design. Keywords. Robust estimation; parameter estimation; outliers; experiment design; bounded errors; breakdown point; set-membership estimation.

dimensional parameter vector ft, bounded-error estimation aims

1. INTRODUCTION

at characterizing the set of all ft's such that all differences Yi The purpose of robust estimation (see e.g. Launer and Wilkinson, 1979, and the references therein), is to provide estimates that are not dramatically affected if the hypotheses made on the measurement errors are not entirely satisfied, either because of a misspecification of the distribution or because of the presence of oUlliers. Least squares estimators are not robust to outliers, to the point where a single erroneous datum can ruin the estimate obtained from a large set of otherwise regular data. The notion of breakdown point, introduced in the context of point estimation (Rousseeuw and Leroy. 1987), is useful to quantify robustness and to compare the performances of estimators. Loosely speaking, the breakdown point of an estimator is the minimum percentage of oUlliers that must be introduced in a data set for the estimator to produce a meaningless resul!.

TJ(.(!, Ki) lie between some known bounds -Ei m and EjM. This posterior feasible parameter set (Norton, 1986) (or membership set (Bertsekas and Rhodes, 1971», denoted in what follows by 5, is then given by 5(Z) = (ft

E

RP I-Ei m ~ Yi - TJ(ft, Ki) ~ Ei M, i = I, ... , nl.(2.1)

As in classical point estimation, the observations Yi can be assumed to correspond to the model response TJm',Ki) obtained at some unknown true value ft' of the parameters, corrupted by some unknown errors bi, Yj =TJ(.(!',Ki) + bi, i = 1, .. , n.

In this paper, this notion is extended to set estimators such as those encountered in the context of bounded -error estimation (see e.g. the surveys by Norton (1987), Deller (1989), Milanese (1989), Waiter and Piet-Lahanier (1990)), which is recalled in Section 2. The aim of bounded-error estimation is to characterize the set of all parameter vectors such that the residuals lie between some prior bounds. In this context, outliers are any data points for which these bounds are too optimistic. Many bounded-error estimators are not robust, in the sense that a single outlier may make the set of possible values for the parameters empty. One of them, however, known as the Outlier Minimal Number Estimator (OM NE) has proved to be particularly insensitive to outliers (Waiter and Piet-Lahanier, 1986; Lahanier, Waiter and Gomeni, 1987; Waiter and PietLahanier, 1989a). When the model output is linear in the parameters, OMNE is shown in Section 3 to reach the highest possible breakdown point. A bound is given to the bias due to outliers, which suggests a new policy for optimal experiment design aimed at providing a high protection against outliers. This policy is described in Section 4, and compared on an illustrative example to conventional D-optimal design.

(2.2)

If the errors bi are only known to satisfy -Ei m ~ bi ~EiM, i = I, ... , n,

(2.3)

any ft in 5(Z) is a possible candidate to being the true value ft'. Note that if the bi'S are assumed to be random variables with a probability density function equal to zero when (and only when) (2.3) is not satisfied, then 5(Z) corresponds to the set of all parameter vectors with a non-zero likelihood. For that reason, 5(Z) has also been called posterior likelihood set. It must be emphasized that the definition of 5 (2.1) does not require the existence of a true parameter vector ft'. The structure of the model used in the definition of 5 can be quite different from that of the process having generated the data, which allows simple model structures to be used to describe the behaviour of complex processes. In such a situation, the errors bi may be essentially deterministic, so that the underlying assumptions of classical approaches for point-estimation such as maximum likelihood may no longer be valid. Note that 5(Z) can also be written as

2. BOUNDED-ERROR ESTIMATION '" = (e 01 ('Y) ,('.J _

Given an-sample Z of data points ~i, Yi), i = I, ... , n, where Yi denotes the measurement obtained under the ith experimental conditions

K.j,

E

1ll> ~

P I _1 < _ Zl. _ TJ (ft, li.i) < _ 1,1. -- I "." n l , Ei

and a model structure TJ (ft,~) with a p-

1133

(2.4)

~ + Ejm_EjM , so that we shall Ei 2Ei assume with no loss of generality that the bounds Ejrn and EiM are symmetrical and identical for all data points, i.e.

points by arbitrary outliers. One wishes the optimal set S*(Z: to satisfy

where Ej = EjM;Ejm, and Zj =

Ei m = Ei M = E, i = I, ... , n.

(3.4)

where the set Jr, corresponds to the regular data kept in Z', i.e.

(2.5)

In what follows, the model output is assumed to be a linear function ofl:!' so that it can be written as

Jr, = {ij E N I (Xij' Yij) E

= I ,... , n-m) .

(3.5)

This would correspond to the rejection of the m outliers and of no regular data. Note that in this case, if a true value !l.* can be defined for the model parameters, then ~* belongs to S*(Z). However, less favourable situations can be encountered where 5*( 7,, ) ;ot SJr,(Z), which corresponds to the existence of non-

(2.6)

or equivalently with a vector notation (2.7)

rejected outliers. There is then no reason for S*(Z) to contain

where the ith row of X is equal to KiT The posterior feasible parameter set S(Z) associated with the n measurements is then given by S(Z) = (!l. E RP I-E $ Yj - XiT!l. $ E, i = I, .. , n).

z" z', j

It. Practical experience indicates, however, that

S*(Z)

generally remains close to!l.* (Waiter and Piet-Lahanier, 1988). Denote by d(S , ,~) a suitable measure of the distance between th e two sets S I and S2 (e.g. the minimum distance between elements of S I and S2, or the Hausdorff metric). Intuitively , a maximal va lue m* should exist for m, such that d(S*(Z),

(2.8)

When rank(X) = p, S(Z) is a convex polyhedron which can be given an exact recursive parametric description (see e.g. Waiter and Piet-Lahanier, 1989b). An experiment design policy aimed at minimizing the volume of S(Z) has been described in (Pronzato and WaIter, 1988, 1989).

S *(Z)

remains bounded when m < m* . The ratio m * n corresponds to the notion of breakdown point of an estimator (see e.g. Rousseeuw and Leroy, 1987), extended here to set estimators such as those encountered in parameter bounding.

When the inequalities 1Yi - X; T!l.1 $ E, i = I, ... , n, cannot be satisfied simultaneously, S(Z) is empty. This can be due to either of two reasons. (i) The model structure is incorrect. (ii) The data are corrupted by outliers, which should be rejected. We shall assume in what follows that we are in the second situation. The rejection policy, motivated by robustness regarding outliers, is described in the next section.

1\

Definitio/l 1. The breakdown point of a set estimator S associated with a regular n-sample Zis given by

m

1\

1

1\

1\

m'(S(Z)) = min (- d(S(Z), S(Z) = OQ), Z' n

(3.6)

with the convention 1\

3. ROBUST PARAMETER BOUNDING

d(S(Z), 0) = OQ,

(3 .7)

3.1 Outlier Minimal Number Estimator whe re Z' is a corrupted n-sample obtained from Z by replacing m original points by arbitrary outliers. 0

Let J be a finite set of distinct indices, defined as follows

Remark I. The breakdown point of the posterior feasible set

J = (ij E NI ij

$

n, ij;ot ik if j ;ot k, j = 1, ... , h, h $ n).

(3.1)

.. . I . . . 5 (Z) as defllled III (2.8) IS equal to slllce a slllgle outher can

n

0

Let S f..Z) be the posterior feasible set associated with those

make S empty. S(Z) is therefore not robust to oucliers.

data points (X;, yj) from an-sample Z that are such that i E J. Define the set S#h(Z) as

To investigate the robustness of OMNE co olltliers, we shall need the following definition.

S# h(Z)

=

u

Sf.. Z ) ,

1\

(3.2)

Definitiol! 2. A set estimator S is regression equivariant if it

#(J)=h

satisfies

where #(J) denotes the cardinal of J. The Outlier Minimal Number Estimator (OMNE) then corresponds to the set S#h* (Z), with h*(Z) = Arg max{h 1S# h(Z);ot 0).

1\

1\

S(Z2) = 'Ty(S(ZI»

(3.8)

for any p-dimensional vector y, where Z I and Z2 are two data sets respectively defined by

(3.3)

Z , = (xl, y,), ... , (Xn, Yn»),

S#h*(Z) is denoted by S*(Z) in what follows. When Z consists of regular data points, which means that S as defined in (2 .8) is not empty, h*( Z) = n, and S*(Z) = s# n( Z) = S(Z). OMNE has proved on various examples to be particularly insensitive to numerous and severe outliers (Waiter and PietLahanier, 1986, 1989a). Its theoretical robustness properties will now be studied into more details in terms of its breakdown point.

(3.9)

and where '!y(.) is the translation associated with y.

0

Lemma I. The posterior feasible set S is regression equivariant. Proof. S(Z2) = (!l. E JRP I-E $ Yi - XiT(!l. - y) $ E, i = I, ... , n)

3.2 Breakdown point

= (!l.E RPI!l.-YE S(ZI») ='Ty(S(Z,».

Consider an-sample Z of regular data points (S(Z) ;ot 0), and

Corollary. OMNE is regression equiv ari ant.

a corrupted sample Z' obtained from Z by replacing m original

1134

0

Proof Notation is as in Lemma 1. OMNE for ZI2 can be written as

and A

A

dC S (Z ). SCZ'C -y)) <

S*("'~) V S.J?'~) ~L. #(J)=h*(Z2) J\~L.

p".

(3.14)

Taking (3 . 12) into account. one can easily check that Z'( -y) can also be deduced from Z '(y) by replacing each datum (Ki. A

Yi) by (h Yi-Ki T ay). The regression equivariance of S then A

implies that S(Z '( -y) = 'T.u'y'(S( Z '(Y»)). which contradicts

0

where h* is defined as in (3.3).

A

(3.13 -3.14) for values of a large enough.

Remark 2. The notions of scale equivariance and affine equivariance (Rousseeuw and Leroy. 1987) can also be extended to set estimators. and OMNE can be shown to be scale equivariant (provided that the bounds are modified according to 0 the same scale as the data) and affine equivariant.

(ii) -pan From the corollary of Lemma I . the breakdown point

of S * satisfies (3.10). Let us prove that the bound is reached. Suppose that m= points of Z are replaced to give a

The following theorem then extends to parameter bounding the results obtained by Rousseeuw and Leroy (1987) in the context of robust point estimation .

modified sample Z'. S*(7,) = S#h*(Z ). with h* defined as in (3.3). One obviously has h*(Z ) <' nom. Consider then one of

the sets Sf.-Z). with #(J)=h*( Z ). and denote it by S(z). Let .~ be th e set of regular data points that contribute to defining

Theorem 1. Ci) The breakdown point of any regression-equivariant set A

estimator S associated with an-sample

A

(

both S*(Z ) and S(Z). One has

z satisfies

#( q ) <' h*(Z) - m <' n - 2m = n -2 <' p.

~ 2 )+ I

m*(S(Z» ~ - n - '

(3.10)

(3 . 15)

Let Je; be the set of indices associated to data points in

q.

where (.) stands for the largest integer lower than or equal to the value inside the brackets.

3 J ,(Z) = S Jr:(Z) = SJG' The set S( Z ) is included in SJG' Y ':f 'cl j and any pxp submatrix of X has full rank. so that 3 Je; is

(ii) If the experimental conditions are chosen in such a way that any pxp submatrix of X has full rank. the breakdown point of OMNE satisfies

sets. so that the distance d(S*( Z ). S*( Z ) is bounded.

bounded. From (3.2). S*( Z ) is included in the union of such

Remark 3. Note that the bound in (3.10) only depends on Z through the number of data points. 0

(~>+I m*(S*(Z» = _2__.

(3.11)

n

Remark 4. When the number of measurements tends to infinity (3.11) indicates that S* can accommodate 50% outliers. This is obviously the largest possible percentage if the outliers are allowed to be organized in such a way that they can be de scribed by the model. Note that in practice the outliers are seldom organized so viciously. so that OMNE can perform satisfactorily even on cases where there is a large majority of outliers. 0

Proof ~ A ( 2 )+ I (i)-pari. Suppose that m*CS(Z)) > --n-' Any sample Z

deduced from Z by replacing + I points is then such that A

0

A

Remark 5. Other regression equivariant parameter bounding policies could be defined. with a high breakdown point. A possible choice corresponds to sets S #h(Z) with fixed h. Suppose that m points of Z are replaced to give a corrupted

d(S(Z). S(Z) < ~. with ~ bounded. Such a sample Z'

contains q = n - - I data points of Z. If n-p is odd. then 2q-(p-l) = n. otherwise 2q-(p-l) = n-1. Anyway. 2q -(p-l) ~ n. Consider two n-samples zty) and zt -y) whose 2q -(p-l) first points are respectively defined by

sample Z '. and that the experimental conditions are such that any pxp submatrix of X has full rank. We want d(S#h(Z). § #h(Z) to be bounded whatever the outliers may be. so that h

(:&1. yl).·· · . C:&p-l. yp-l). C:&p. yp) •...• (&!. yq).

should satisfy n - m <' h (one must have S#h(Z) ~ 0). and h - m <' p (any set Sf.-Z). with #(.1)=h. must contain at least p regular data points to be bounded). The maximal value for m

and

which allows these inequalities to be satisfied is m = <¥>. The value of h given by

~

= + then permits to reach the

bound (3.10) given in Theorem I (i). wi th a

E

R. y

~

Q and

0

Remark 6. The least median of squares (LMS) estimator A

(Rousseeuw and Leroy. 1987) and the set estimator S#h defined in the Remark 5 both neglect up to 50% of the data when n tends to infinity. This systematic rejection of a large part of the data leads to a loss of infom1ation when there are less than 50% outliers. S* does not reject any data a priori and therefore does not possess such a drawback. 0

(3.12) If n-p is even. the nth datum of Z '( -y) is chosen as (Kn. Yn Kn T ay) when the nth datum of Z '(y) is CKn. Yn). Z 'Cy) and Z'(-y) both contain q points of Z. so that A

A

d(S(Z), S(Z'(y)) <

W.

(3.13)

1135

Let X J be the pxp matrix the ith row of which is equal to ~T This matrix has full rank , and from (3.24-3 .25) S satisfies

3.3 Bias due to outliers The distance d(S'(Z), S'(Z), where Z' is a corrupted nsample obtained from a regular n-sample Z by replacing m original points by arbitrary outliers, can be seen as a bias due to these outliers. Provided that m S

and

(3.26) From (3.22), t.(SfZ» can therefore be written as

that any pxp

submatrix of X has full rank, this bias is known from Theorem I to be bounded. We now derive an expression for such a bound, which will be used in Section 4 to define an optimality criterion for experiment design.

t.(SfZ))

(3.27)

2E 11 ~ 11·

(3 .28)

or equivalently

MSfZ))

to the definition of S(z). Let J be a set of p indices associated with any subset (with cardinal p) of these regular data points. One obviously has ;d

2E 11 XiI!! 11,

.\lE IC p

Define S(Z) as in the proof of Theorem I, part (ii). Equation (3.15) implies that at least p regular data points of Z contribute

SJ(Z)

= max

=

max

XJ~ElCp

Replacing ~ by Yip, with 11 ylI = 1, one can write MSfZ» as (3.29)

(3.16)

S'(Z) ,

where

and

(3.30)

(3.17) The bound on d(S'(Z) , S'(Z) is finally obtained by

so that

d(S'(Z), 5(Z) S

max

d(ll., ll.').

(3.18)

d( S' (Z), S'(Z) S

fr, fr'E Sj(Z) Finally, from (3.2) and the definition of S( z ) ,

d(S'( Z ),S'(Z) S

max JI#(J)=p

max dell., ft'). ft ,ft'E S jZ)

2E max max JI#(j)=p YESJ'p P

(3.31)

or equivalently (3.19)

2E d(S'(Z), S'(Z) S - p'(X)

As dell., ft'), we shall use the Euclidean distance 11 ft - ft' 11. We first evaluate

(3.32)

with p'(X)=min (pl:3YE R P,lIylI= 1, :3 p rows ",iT of X,

t.(SfZ» =

max 11 ft - ft' 11. ft, ft'E S !Z)

(3.20)

I",h 1= p) .

Remark 7. If a true value ll.' can be defined for the model parameters, it belongs to S'(Z), and from (3.32) any ll.' in

We assume, with no loss af generality, that SfZ) is defined by the first p data. S fZ) is a convex polyhedron with p pairs of parallel faces (parallelotope). The ith pair of faces is defined by ftT ",i = Yi + E, ftT ",i = Yi - E, i E J.

(3.33)

S' (z) then satisfies 11 ft' - ft' 11 S ~

(3.21)

(3.34)

p'(X)

Take one of the vertices of SfZ) as the origin, and let lii, i = I, ... , P be the vectors of coordinates of the adjacent vertices. The maximum value of 11 ft -ll.' 11 is obtained when ll. and ll.' are vertices of SfZ), so that MS fZ» =

max 11 S .\l 11, .\lElC p

o 2E . . The bound - - can be used as a quantItatIve measure of the p'(X) robustness of the estimator. It depends on the experimental conditions through the value of p', hence the idea of designing

(3.22)

the experiment so as to make p' as large as possible. where

IC p = (J,l E RP I Ui = ± 1, i = I , ... , p),

(3.23)

4. EXPERIMENT ROBUSTNESS

and where the ith row of S is given by liiT The ordering of the vertices can be chosen such that all ~, k'# i, belong to the ith pair of faces of SfZ), which can be written as

DESIGN

TOWARDS

A first qualitative condition to ensure robustness of OMNE with respect to outliers is given in Theorem 1 (ii) : any pxp submatrix of X must have full rank. A quantitative criterion for designing experiments intended to yield a high protection against outliers can further be obtained from the expression of the bound (3 .32) on the bias due to outliers.

(3.24) The vertex ii does not belong to the ith face of S J
Definition 3. A nxp design matrix X is p-optimal if it maximizes the criterion p' given by (3.33).

(3.25)

1136

0

Experiment design for robust estimation seems to have received little attention in the literature. The only study we are aware of (Box and Draper, 1975) concerns the minimization of the di screpancy of the predicted outputs X(XTxy] XT~ obtained by standard least squares (SLS) when outliers are present, where ~ is the vector of measurement outputs. However, the SLS estimator has a breakdown point equal to I/n, and this policy should therefore be rejected when severe outliers are to be feared. Note that the bound on the bias due to outliers obtained in (Rousseeuw and Leroy, 1987) for the least median of squares (and other related) estimator(s), especially designed against severe outliers, is also related to

~*

10 .8

10.6

1" 10 .•

LO .2

e·, 10

' and maximizing p*

can be of interest for these estimators too. Further studies are required to investigate the theoretical properties of this new design policy, and to develop algorithmic procedures. The importance of a proper choice of the design matrix X for the robustness of OMNE is here simply stressed by an example.

I

'-

9 .6

..'---'--'-------~J 9 .6

Example. Assume that p = 2 and consider the following feasible region for the regressors

4 .4

4.6

4 .8

5.2

5.4

5.6

5.8

e1

X = (2). = (X] , X2)T E R P 1 0 ~ X] , 0 ~ X2, Xl 2 + X2 2 ~ I }.(4.1 )

When n = 4 measurements are to be performed, the maximal value of p* (3.33) is equal to sin ;2

Fig. 1. Estimate feasible set SeX, ftp) (frp = (5, JO)T) - and posterior feasible set S*(Z) ------- for regular data points, with the design matrix given by (4.2).

= 0.2588 and obtained for

the design matrix

Jt

X=

.

0 cos I6" sIn Jt

.

Jt

6"

2 2 ,-----~------~-----~------,

J

,, ~

------1

,, , -- -_. ----..... --- -'

, ~ .~ - ---- -- --

,

20

(4.2 )

Jt

( cos'3 sln '3 o 1

16

Note that no replications are involved, contrary to classical Doptimal design. The p-optimal experiment defined by (4.2) is

16

also ~-optimal (Pronzato and WaIter, 1988, 1989). It minimizes the volume of the estimate feasible set defined by SeX, ftp) = 1ft E R P 1-£ ~ 2>.i T (ft - ftp) ~ E, i

=

" 12

1, ... , n), (4.3)

where ftp is any prior value of fr. The estimate feasible set corresponding to the design matrix X given by (4.2) is presented in Figure I (solid lines). The volume of sex, ftp) is always greater than or equal to the volume of S(Z) . Assume that there are no outliers and that the four measurements are given by ~ = (5.1,9.5, I!., JO.3)T,

10

6 L-----~------~----~------~

•

5

5. 5

e,

(4.4) Fig. 2. OMNE for the p-optimal design matrix (4.2)-and for the D-optimal design matrix (4.5) -------in the presence of one outlier.

with bounds E = 0.5. Figure I presents S*(Z) (dashed lines), which coincides here with S(Z). The breakdown point of S* given by (3.11) is here equal to 50%, which means since n = 4 that up to one arbitrary outlier can be handled. Suppose that a problem occulTed in the registration of the last data point, so that it is replaced by the outlier 2>.4 = (0, I)T, Y4 = 20.3. The corresponding S* ( Z) is presented in Fig. 2, together with the outlier minimal number estimate associated with the measurements (5.1, 5.4, 10.3, 20.3)T and the D-optimal design matrix

5. CONCLUSIONS When outliers are to be feared and bounds are available on regular errors, OMNE is a powerful alternative to classical robust point estimators. Its breakdown point has been evaluated, and it reaches the highest achievable value. The bias due to the presence of outliers depends on the choice of the experimental conditions, which permits to define a new optimal design criterion. This criterion may also be of interest for robust point estimators such as the least median of squares. Further studies are required to investigate the properties of the corresponding design policy and to develop specific optimization procedures for experimental design. Contrary to this new design policy, classical D-optimal design which usually leads to replication of measurements may have disastrous consequences on robustness to outliers, as has been illustrated by a simple example.

(4.5)

The presence of replications implies that P(XD) = O. The conditions for OMNE to have a high breakdown point are therefore no longer fulfilled . Should Y4 tend to infinity, the maximum distance between S*(Z) and S*(Z) would tend to infinity. As a consequence, classical D-optimal design should be avoided if robustness to outliers is an issue.

1137

REFERENCES Bertseka s, D. P. and I. B. Rhodes (1971). Recursive state estimati on for a se t-me mber ship des cription of uncertain ty. IEEE Trans. Aur. Control, 16, 117- 128. Box, G. E. P. and N. R. Draper (l975). Robu st designs. Biometri ka, 62 , 347-352. Deller, J. R. (l989). Set members hip identifica tion in digital signal processin g. IEEE ASSP Magazine, 6(4}, 4-20. Lahanier, H ., E. Waiter and R. Gomeni (1987). OMNE: a new robust members hip set estimato r for the paramete rs of nonlinea r models. 1. Pharma cokin. Biopharm. , 15 , 203219. Launer, R. L. and G. N. Wilkin son (Eds) (1979). Robustn ess in Statistics . Academi c Press, New York. Milanese , M. (1989). Estimatio n and prediction in the presence of unknown but bounded un certai nty : a survey. In M. Milanese , R. Tempo and A. Vicino (Eds), Robustness in Identification alld Control, pp. 3-24. Plenum Press, New York. Norton, J. P. (1986). Problems in identifyin g the dynamic s of biologica l systems from very short record s. Proc. 25th IEEE Con/. on Decision and Control , Athens, pp. 286290. Norton, J. P. (1987). Identifica tion and applicati on of boundedparamete r model s. Auromatica , 23, 497-507. Pronzato , L. and E. Waiter (InS). Experim ent desi gn for member ship-set estimati on: lin ea r models with homogen eous and heteroge neous measurem ent errors. In Y. Dodge, V. V. Fedorov and H. P. Wynn (Eds), Optimal Design and Analysis of Experiments, pp. 195205. North-Ho lland, Amsterda m. Pronzato , L. and E. WaIter (1989). Experim ent design in a bounded -error context: comparis on with D-optim ality. Automat ica, 25 , 383-391. Rousseeu w, P. J. and A. M. Leroy (l987). Robust Regression and Outlier Detection . Wiley, New York. Waiter, E. and H. Piet-Lah anier (1986). Robust nonlinea r paramete r estimatio n in the bounded noise case. Pro c. 25th IEEE Con/. on Decision and Control, Athens , pp. 1037-104 2. WaIter, E. and H. Piet-Lah anier (1988). Estimati on of the paramete r uncertain ty resulting from bounded -error data. Math . Biosci. , 92,55-74. Waiter, E. and H. Piet-Lah anier (l989a). Robust linear and nonlinea r paramete r estimatio n in the bounded -error context. In M . Milanese , R. Tempo and A. Vicino (Eds), Robustn ess in Identific ation and Control, pp. 67-76. Plenum Press, New York. WaIter, E. and H . Piet-Lah anier (I 989b}. Exact recursive polyhedr al de sc ription of the feasible paramete r set for bounded -error models. IEEE Trans . Aur. Control, 34, 911-915. W aIter, E. and H . Piet-Lah anier (1990). Estimati on of paramete r bounds from bounded -error data: a survey. Mati!. Compur. Simul., to appear.

1138