Note on fuzzy regression

Fuzzy Sets and Systems 37 (1990) 65-75 North-Holland

65

NOTE ON FUZZY REGRESSION Andr~is B,ARDOSSY Institute for Hydrology and WaterResources, University of Karlsruhe, Kaiserstr. 12, D-7500 Karlsruhe, Germany Received September 1988 Revised February 1989

Abstract: Regression analysis has been widely used in several areas for many years. Fuzzy linear regression was introduced in 1980 by H. Tanaka, S. Uejima and K. Asai. In this paper the general form of regression equations is developed for fuzzy numbers. It is showed that the problem of fuzzy regression can be formulated as a mathematical programming problem. The special case of linear regression yields a linear programming problem. Different measures of vagueness are introduced to define the best estimate. Two simple numerical examples illustrate the methodology and allow a partial comparison among the different measures of vagueness.

Keywords: Fuzzy regression; fuzzy linear regression; measure of vagueness.

1. Introduction Regression analysis is used to model relationship between variables. The aim is to describe how the dependent variable is related to the independent variables. This relationship is based on existing data. The regression rarely describes the true relation between the variables. The deviations of the data set from the regression line can occur because of measurements errors and/or modelling errors. The concept of measurement error is clear, the modelling error is that which caused by imperfect knowledge, would occur even in the case of precise (error free) data. The modelling error occurs in almost each case because of omitting independent variables and/or the choice of the type of the regression line (linear, quadratic, etc.). Fuzzy regression was introduced by Tanaka et al. [7] for the linear case. An interesting application to forecasting can be found in Heshamaty and Kandel [3]. In fuzzy regression the data can differ from the calculated to a certain degree of belief. This difference is due to the modelling error. The purpose of this paper is to give a general definition of fuzzy regression, and review its properties. The L - R representation of fuzzy numbers is used throughout the paper. Like in the probabilistic case the main interest is the case of linear regression and those which can be transformed to the linear case. The paper is divided into 5 sections; Section 2 is a review of fuzzy numbers and fuzzy operations based on the extension principle. In Section 3 the general formulation of fuzzy regression is given. Different approaches to the choice of the regression parameters will be presented. In Section 4 the linear regression 0165-0114/90/$3.50 © 1990, Elsevier Science Publishers B.V. (North-Holland)

66

A. Bdrdossy

methods are presented. Section 5 contains two numerical examples to illustrate the methodology.

2. Fuzzy numbers, the extension principle This section recall the basic definitions of fuzzy number and fuzzy operations. Some of the definitions are not given in their most general forms; for convenience they are restricted to the form used in this paper.

Definition. A fuzzy set M on the set of real numbers is called an L - R fuzzy number if the membership of x can be calculated as:

orx
>0

formO.

Here L and R are continuous strictly decreasing functions on [0, 1] and

L(x)=R(x)=l

L(x)=R(x)=O

ifx<~0,

ifx~>l.

For M we write symbolically:

M = (m, or, fl)Z.R. A more general definition and further details can be found in Dubois and Prade [2] or Kandel [4] Zimmermann [10]. Basic operations and functions on these fuzzy numbers can be defined by the help of the extension principle (Zadeh [8]). Definition. If X and Y are two sets, and f is a mapping f : X ~ Y such that for all

x • X, f ( x ) = y • Y, then f can be extended to operate on fuzzy subsets of X in the following way: Let A be a fuzy subset of X with membership function #a, then the image of A in Y is the fuzzy subset B with the membership function/aR: ~sup{#;4(x); x: y = f ( x ) , x • X}, #B(Y) = l 0 if there is no x • X such that f ( x ) = y .

(1)

Applying this principle to functions like x + y or x • y we can define the basic operations on fuzzy numbers. For example the sum of n fuzzy numbers (mi, cri, fli)mR can be calculated as

(mi, o¢i, fli)gR = i=l

mi, --

O~i, i=1

J~i i=1

LR"

(2)

In this case we considered that all fuzzy numbers had the same L - R functions. In the case of different L - R functions the computation is slightly more difficult. Supposing that the i-th fuzzy number has the L - R functions Li, Ri, then the sum

Fuzzy regression

67

can be computed as above with the exception that the final L - R representation has the functions L=(/~

cri

•=

R =

L~_I)-1,

E T : 1 i~]

(3)

_ Si=,flj

(4)

Other operations like multiplication or division can be defined similarly. Details on fuzzy arithmetics can be found in Kaufmann and Gupta [6].

3. Fuzzy regression The general problem in regression is that given the set of data consisting of (x, Yt) (t = 1. . . . . T) where x, = (x,~ . . . . . xt,) is a vector in the n-dimensional Euclidian space and y, is a real number corresponding to the vector x , and given the function f ( x , a) we have to find the parameter vector a such that y, = f ( x , , a).

(5)

In general there is no solution of the above equation. Thus it has to be relaxed: The parameter vector a is supposed to be a fuzzy vector (a vector of fuzzy numbers). In the case of fuzzy regression the data y, can be crisp or fuzzy numbers. Instead of the equality in (5) it is supposed that for a selected level h for each element of the h level set of Yt there exists a parameter vector a of at least h membership satisfying (5). This means that Eq. (5) is 'fuzzified' in the form: For each y eyth = {y; /z(y) > h} there is a vector a* such that y = f ( x , a*),

a* = (a~ . . . . .

a*)

(6)

and /~.(al, *

..

. , a*) = min(/t,,(a~') . . . . .

a*r ) )

h.

(7)

Suppose the parameters ai are L - R fuzzy numbers with the representation ai = (mi,

(~i' Tli)LiRi"

There are infinitely many fuzzy parameter vectors satisfying the above conditions. The one with the 'minimal vagueness' will be the regression parameter vector. The vagueness of the regression can be defined in several different ways. 1. The vagueness is equal to the maximal vagueness of the individual parameters, expressed by the width of the corresponding fuzzy number. So V = max(max(61, ~/1). . . . .

max(6r, T/r)).

(8)

2. The average vagueness of the individual parameters, expressed by the width of the corresponding fuzzy number. This means V

= L ~ (I~i "t- ]e]i). 2r i=1

(9)

68

A. Bdrdossy

3. The vagueness of the resulting fuzzy function on the selected domain where the independent variables can take their values from. This means that for each i, xi- < xi < xT. In this case,

V .... fx,:+, n-

(

~

~ xx~ f ~ - +~ ~

sup{min(ua,(a*)l;y=f(x,a*)}dydx,...dx,,.

(10)

4. The vagueness of the resulting fuzzy function on the the at the xt vectors V = T,--~I

sup{min(#,~(a*)); y =f(x,, a*)} dy.

(11)

Vagueness measures (8) and (9) are related to the coefficient vector a. Measures (10) and (11) are performance related, the goodness of the fit depends on the domain for the vector x, or on the measurement vectors xt. These measures are more related to the traditional least squares regression, where the performance is measured also at the measurement vectors xt. These different measures of vagueness have to be minimized with respect to the constraints (6) and (7). Constraints (6) and (7) are formulated for each y e Yo~. For y~ fuzzy, (6) and (7) give infinitely many constraints. To be able to transform these constraints to a finite number of constraints additional assumptions have to be made. The first assumption is quite natural - the function f in (5) is assumed to be continuous in all its variables xi and parameters aj.

Proposition 1. The conditions 3a* such that y = f ( x , a*) and I~,(a*)>~h (i = 1. . . . .

r)

and 3a', a" such that y >~f(x, a'), with #a,(a~) >1h, l~(al.~ >i h for i = 1. . . . .

Proof. ( ~ ) This is trivial taking

y <-f(x, a"), r, are equivalent.

a' = a" = a*.

. ak+l . . . a. ' ) . for ( ~ ) Let bk = ( a .l , ... . . ,. ak, . .k =. O., f(X, b,) >i y there is an m such that

r. As f ( x , bo)<~y and

f(x, b~) <-y <~f(x, bm+O. Using the continuity of the function f there is a z' such that a m' + I ~-< Z

'~< ~ a m +"

1

and

y

= f ( x , al" . . . .

.

, am,

z',

t

am+2,

. . • ,

a'r).

By the convexity of the fuzzy n u m b e r / z . . . . (z')/> h. So taking a* = a" if i ~ m + 2 will do.

and

Corollary 1. The conditions

Vy ~ Yh 3a such that y = f ( x , a)

and

min(Ita,(al)) >t h

and

supyh <-f(x, a"),

and 3a', a" such that are equivalent.

I~a,(a') >>-h, infyh >~f(x, a')

Fuzzy regression

69

Using Corollary 1 we arrive at a finite number of inequalities describing the conditions for fuzzy regression: infyth>>-f(x, a'(t)), supyth <<-f(x, a"(t)),

t= 1,...,

T,

t = 1. . . . .

T,

and IZa,(a~(t)) >1h,

/za,(a~'(t))/> h

(t = 1 . . . . .

T, i = 1 . . . . .

Using a i = ( A i , 8i, r/i)L,R, the conditions I~a,(a~(t))~h, written as

r),

I~,(a"t(t))~h

can be

rI,RFl(h),

i = 1. . . . .

r,

(12)

A~ - tSiL;~(h) <~a"(t) <~A, + rliRT~(h),

i = 1. . . . .

r.

(13)

A i - 8 i L ; ~ ( h ) <~a~(t) <~Ai +

So finally 2T + 4rT inequalities describe the conditions for the fuzzy regression. Unknown quantities in these inequalities are a~(t), aT(t), Ai, 8i, Th. T h e number of unknowns is 2Tr + 3r. Functions (8), (9), (10) or (11) have to be minimized with respect to the above constraints. This nonlinear problem can be simplified by the help of additional assumptions. The function f in (5) is assumed to be non-increasing or non-decreasing in all its parameters. This restricts the class of possible f functions, but by the help of simple transformations functions not satisfying this condition (for example periodic) can be handled.

Proposition 2.

Suppose that for any x and i the function

g ( u ) = f ( x , al . . . . .

ai-1, u, ai+ 1. . . . .

ar)

is non-decreasing. Then the conditions o f f u z z y regression can be written as

infyth ~ f ( x , a - ) , supyth <-f(x, a+),

t = 1. . . . . t = 1. . . . .

T, T,

where a - = (A1 - 8 1 L [ l ( h ) . . . . .

A , - 6rLTX(h )),

a + = (A~ + rhRT~(h) . . . . .

Ar + r/rR~-X(h)).

Proof. If the conditions of the proposition are fulfilled then taking a'(t) = a - and T the conditions of fuzzy regression are fulfilled. Conversely, if

a " ( t ) = a ÷ for t = 1 . . . . .

infyth >-f(x, a'(t)), then ai-<-a'(t) for i = 1 . . . . .

t= 1,...,

T,

r. So

infyth >~f(x, a ' ( t ) ) >-f(x, a - ) ,

t = 1. . . . .

T.

Using the same for sup leads to supy, h <-f(x, a"(t)) <~f(x, a+),

t = 1,...,

T.

If the function f is non-decreasing in some of its parameters and non-increasing in the others a similar statement to Proposition 2 can be proved.

A. Bdrdossy

70

Using Proposition 2 the regression problem is transformed to a nonlinear optimization problem with 2T constraints and 3r unknowns. In the following section these constraints and objective functions will be examined in the case of linear regression.

4. Fuzzy linear regression Linear regression are most widely applied in the classical probabilistic framework. The conditions for fuzzy linear regression can be formulated in a relatively simple way, so their application has not to be restricted because of numerical problems. We suppose that

f(x, a)= ao + ~ ai(xi--Xii)" i=1

Here .~ -- ~ . . . . . x--~) is a selected reference point. (In the case of least squares regression this reference point is usually the average of the x~ values.)

Proposition 3. The conditions for fuzzy linear regression are infyth >~A o - 6oLol(h) + ~

(Ai - t~iL;l(h))(xi --Xii)

Xi>-Xi

+ E

(Ai + r l i R i - l ( h ) ) ( x i - x i )

X i "(X i

and sup yth <~Ao + ~/0Rot(h) + ~

(A~ + TliRi-l(h))(xi --Xii)

x i >~-x i

+ ~ (A,- 6,L;l(hl)(xi-~) Xi
Proof. The proof is the same as the proof of Proposition 2, but in this case one has to apply the same for each individual xt. Depending on the sign of x; - ~ the R, right or the L, left functions have to be considered. The conditioning inequalities for linear regression are linear with respect to the unknown parameters. Taking (8) or (9) as the measure of vagueness the problem to find the fuzzy regression parameters is a linear programming problem, and can be solved using traditional methods. If the measure of vagueness is defined as in (10) or (11) the problem can again be transformed to a linear one. For f linear we have:

Proposition 4. Supposing x7 = xii - di and x~- = xii + el, • ".

sup{min(#a,(a*));y = f ( x , a*)} dy d x l . . , dxn x-~ ~i -- ei

= FI (di + el) (6oL~ + tloR~) + ~ i=l

i=1

2(di

+

ei)

(61L~ + oiR~)

Fuzzy regression where L* =

£

Li(t) dr,

R* =

71

£

Ri(t) dt

Proof. For the proof the following lemma is needed: Lemma. Let h(x) be a continuous strictly decreasing .function on [0, 11 such that h(O) = 1 and h(1) = O. /n this case,

£

h(t) dt =

f2

h-'(t) dt.

The proof of this lemma is straightforward using the definition of the integral. Let y* = Ao + ~'=~ Ai(xi -xii). NOW

f+f sup{min(#,,(aT)); y =f(x, a*)} dy

=

f~" s u p { m i n . . . } + fy~=sup{min.--} dy.

Let D = ~/o + Ex,<~ 6i(Xi -- Xi) "~- ~xi~xii ~i(Xi -- Xi)" Using the extension principle and (1) for the case y > y * one has: sup{rain...} = [ ~ , , 0

~ x,<~

fy+~[O0O_, (~i(Xi--Xi) [~-"',o E *

x~<~

=O

.to

D

[...]-tdu=O

+ ~ tb(x~xj~>~

D

Oj(Xj--Xj' R}"1]-l(~_y_*) dy

L;"' + E ~i~zi

D

[..-l-ldu=O

[...]du

+x<~f/~ ~i(XoXi)

=o[f/~Roa(u,

R? 1

LZl(u) an

= rloR~ + Z 6i($,- x,)L* + ~, ~b(xj- x-j)R?.

xi<~i

xi>>-xj

Similarly one has f~"sup{min(#~,(a*)); y =f(x, a*)} dy

= ~o/~ + E o i ~ i - x,)R* + ~]

xi<~

xj~jj

,~j(x~-~j)z,;'.

Computing the integral on the given range leads to the desired result.

,

A. Bdrdossy

72

Similarly to Proposition 4 the following can also be proved: Proposition 5. T,=, J_® sup{min(~,~,(a*)); y = f ( x . a*)} dy

I~i - x,,I 6iL*+rl,g?).

= 6oLg + rioR8 + ~ ~

Remarks. 1. The vagueness V of the regression depends on the selection of the parameters x and h and the functions L and R. 2. The solution of the fuzzy linear regression in the above for its not always unique. For example it can be proved easily that if L -- R for i -- 1 . . . . . n then minimizing (9) or (10) the solution is never unique. If (Ai, dii, T/i) is a parameter of the solution then (B, qgi, ~p~) will result the same vagueness if Ai -

6iLTl(h) ~
<~A~ +

rhR.l(h)

and

Bi q)i =

Ai

A i - Bi

L71(h) + 6i,

~i - ~

+ rli.

In this case additional assumptions on the symmetry of the parameters have to be made (for example 6i = c ~ i ) . 5. Numerical examples The problem of the first example is also discussed in Abraham and Ledolter [1]. There it is handled with the classical least squares approach. Table 1 shows Table 1. Physical and performance characteristics of female runners; XI: Height X2: Weight, X3: Skinfold sum, X4: Relative body fat, Xs: VO2, Y: Running time, 6: The width of the fuzzy running time No.

X1

X2

g 3

X 4

g 5

Y

~5

1 2 3 4 5 6 7 8 9 10 11 12 13 14

163 167 166 157 150 151 162 168 152 161 161 165 157 154

53.6 56.4 58.1 43.1 44.8 39.5 52.1 58.8 44.3 47.4 47.8 49.1 50.4 46.4

76.4 62.1 65.0 44.9 59.7 59.3 98.7 73.1 59.2 51.5 61.4 62.5 60.3 76.7

17.9 15.2 17.0 12.6 13.9 19.2 19.6 19.6 17.4 14.4 7.9 10.5 12.6 19.6

61.32 55.29 52.83 57.94 53.31 51.32 52.18 52.37 57.91 53.93 47.88 47.41 47.17 51.05

39.37 39.80 40.03 41.32 42.03 42.37 43.93 44.90 44.90 45.12 45.60 46.03 47.83 48.55

0 0 0 0 0 0 0 0 0 0 0 0 0 0

Fuzzy regression

73

Table 2. Regression coefficients for female runner performance Ao

A1

A2

A3

A4

A5

43.492

-0.141

0.000

0.000

-0.059 0.148

0.150 0.437

0.195 0.000

-0.493 0.000

Coefficients Width t5 = r/

44.133 0.000

0.108 0.490

-0.234 0.000

0.038 0.034

-0.079 0.570

-0.498 0.000

Maximal Vagueness Coefficients Width t5 = 7/

44.631 0.173

0.013 0.173

-0.182 0.173

0.109 0.173

-0.088 0.173

-0.460 0.173

Average Vagueness Coefficients Width t5 = r/

Integral Vagueness

the running performance of 14 female runners in a 10km road race. As independent variables height, weight, skinfold sum, relative body fat, and maximal aerobic power were chosen. The measured running time is assumed to be crisp. The performance of the runners certainly depends on these parameters, but other non-quantitative factors like the 'mental state' of the runner also influence the running time. As these parameters are not all measurables it seems to be adequate to apply fuzzy linear regression. Linear regression using different measures of vagueness were performed. As L - R functions L(x)=R(x)= 1 - x were chosen. The level for the fit was h =0.75. The coefficients ai were supposed to be symmetrical; 6 ; = r/i. The reference points :~ = ½(max xt, + min xt,) was selected. Results for the different measures are shown in Table 2. Table 3 shows the running times calculated with the help of the fuzzy regression. Note that there is Table 3. Comparison between data and regression results: performance of female runners Average E

1 2 3 4 5 6 7 8 9 10

11 12 13 14

40.69 40.26 42.24 38.06 43.70 45.83 49.10 43.92 41.79 40.56 43.74 44.00 45.25 47.82

6

2.65 5.29 4.42 12.67 5.95 6.91 12.17 1.97 6.25 9.15

4.77 4.09 5.19 2.57

Integral

Maximal

E

6

E

6

Reality

39.88 42.33 42.79 42.60 44.21 46.11 45.39 43.40 41.95 44.13 47.94 48.13 46.84 45.59

4.48 5.08 5.51 2.55 4.91 7.45 5.72 7.79 5.94 2.04

40.78 41.78 42.56 41.20 44.43 45.81 47.53 43.51 42.07 42.88 47.24 47.16 46.51 46.58

4.17

39.37 39.80 40.03 41.32 42.03 42.37 43.93 44.90 44.90 45.12 45.60 46.03 47.83 48.55

4.67 5.11 2.03 5.95

4.72 4.89 6.91 4.62 6.69 7.03 4.76 5.52 4.35

4.52 4.43 3.95 3.78

A. Bdrdossy

74

Table 4. House prices; XI: Rank of material, X2: First floor space (m2), X3: Second floor space (m2), X4: Number of rooms, Xs:

Number of Japanese-style rooms, Y: House price (10 3 yen), 6: The width of the fuzzy house price No.

X1

Xz

X3

X4

X5

Y

6

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

1 1 1 1 1 2 2 2 2 2 3 3 3 3 3

38.09 62.10 63.76 74:52 75.38 52.99 62.93 72.04 76.12 90.26 85.70 95.27 105.98 79.25 120.50

36.43 26.50 44.71 38.09 41.40 26.49 26.49 33.12 43.06 42.64 31.33 27.64 27.64 66.81 32.25

5 6 7 8 7 4 5 6 7 7 6 6 6 6 6

1 1 1 1 2 2 2 3 2 2 3 3 3 3 3

6060 7100 8080 8260 8650 8520 9170 10310 10920 12030 13940 14200 16010 16320 16990

550 50 400 150 750 450 700 200 600 100 350 250 300 500 650

no measure of vagueness which could be called best. The width of the estimates varies most in the case of the average vagueness, and it is the most stable in the case of the maximal vagueness. The second example is taken from Tanaka et al. [7] Table 4 shows the fuzzy prices of 15 houses. The house prices were assumed to be fuzzy with linear L - R functions. The coefficient of the regression were supposed to be fuzzy with linear L - R functions too: L ( x ) = R ( x ) = 1 - x. Table 5 shows the results corresponding to the different measures of vagueness. Table 6 shows the calculated fuzzy house prices. In this case the maximal vagueness measure resulted the highest variances in the width of the calculated fuzzy number. The integral taken as measure seemed to give lower width in most of the cases than the other two measures. Table 5. Regression coefficients for house prices Ao

A1

A2

A3

A4

A5

11273.00 0.00

2049.51 0.00

78.07 7.84

62.67 163.61

- 270.88 0.00

- 71.88 0.00

11664.58 0.00

2062.82 0.00

89.28 0.00

69.62 120.06

-658.04 170.19

-213.95 0.00

10888.12 81.08

2078.26 75.41

73.94 81.08

83.81 81.08

Average Vagueness Coefficients Width 6 = r/

Integral Vagueness Coefficients Width 6 = r/

Maximal Vagueness Coefficients Width 6 = r/

197.44 81.08

453.47 40.54

Fuzzy regression

75

Table 6. Comparison between data and regression reults: House prices Average

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Integral

Maximal

E

6

E

6

E

6

Reality

5319 7081 8080 8235 8708 8888 9393 10177 10920 11997 13181 13697 14533 14901 15955

2034 3392 400 1402 867 3465 3387 2231 602 781 2596 3275 3359 3337 2718

5637 7324 8082 7924 8675 9675 9904 10307 10920 12153 13465 14063 15019 15359 16636

1397 2419 403 1368 801 2761 2591 1624 601 652 1839 2282 2282 2420 1729

3886 5766 7612 8050 8648 7228 8161 10041 10920 11930 12979 13378 14169 15476 15629

4772 2738 1209 990 670 3524 2637 1320 520 1701 2283 3358 4226 2152 5030

6060 7100 8080 8260 8650 8520 9170 10310 10920 12030 13940 14200 16010 14320 16990

Discussion and conclusions • Fuzzy regression leads to a mathematical programming problem. • Results of the regression depend on the choice of the L-R functions, the symmetry assumptions, the choice of the reference point and the choice of the measure of vagueness. These parameters and functions should be selected to conform the application. • Fuzzy regression can also be done using fiat fuzzy numbers. However in this case the measures of vagueness corresponding to the coefficients (8) and (9) have to be redefined. References [1] T.P. Abraham and L. Ledolter, Statistical Methods for Forecasting (Wiley, New York, 1984). [2] D. Dubois and H. Prade, Fuzzy Sets and Systems. Theory and Applications (Academic Press, New York, 1980). [3] B. Heshmaty and A. Kandel, Fuzzy linear regression and its applications to forecasting in uncertain environment, Fuzzy Sets and Systems 15 (1985) 159-191. [4] A. Kandel, Fuzzy Mathematical Techniques with Applications (Addison-Wesley, Reading, MA, 1986). [5] A. Kandel and W.J. Byatt, Fuzzy processes, Fuzzy Sets and Systems 4 (1980) 117-152. [6] A. Kaufmann and M.M. Gupta, Introduction to Fuzzy Arithmetic: Theory and Applications (Van Nostrand Reinhold, New York, 1985). [7] H. Tanaka, S. Uejima and K. Asai, Linear regression analysis with fuzzy model, IEEE Trans. Systems Man Cybernet. 12 (1982) 903-907. [8] L.A. Zadeh, Fuzzy sets, Inform. and Control 8 (1965) 338-353. [9] L.A. Zadeh, The concept of a linguistic variable and its application to approximate reasoning. Part 3, Inform. Sci. 9 (1975) 43-80. [10] H.-J. Zimmermann, Fuzzy Set Theory-And its Applications (Kluwer Nijhoff Publishing, Boston-Dordrecht, 1984).