- Email: [email protected]
A simple method for computation of fuzzy linear regression
European Journal of Operational Research 166 (2005) 172–184 www.elsevier.com/locate/dsw
Computing, Artificial Intelligence and Computer Technology
A simple method for computation of fuzzy linear regression Mehran Hojati a
a,*
, C.R. Bector b, Kamal Smimou
b
Department of Finance and Management Science, College of Commerce, University of Saskatchewan, 25 Campus Drive, Saskatoon, Saskatchewan, Canada S7N 5A7 b Asper School of Business, University of Manitoba, Winnipeg, Manitoba, Canada R3T 2N2 Received 6 November 2001; accepted 8 January 2004 Available online 15 April 2004
Abstract We propose a new method for computation of fuzzy regression that is simple and gives good solutions. We consider two cases: First, when only the dependent variable is fuzzy, our approach is given and is compared with those suggested in the literature. Secondly, when both dependent and independent variables are fuzzy, our approach is extended and compared with those given in the literature. In each case, a simple example is used to compare the competing approaches. Ó 2004 Elsevier B.V. All rights reserved. Keywords: Fuzzy sets; Fuzzy regression; Linear programming
1. Introduction Fuzzy linear regression was proposed by Tanaka et al. [13] to determine a fuzzy linear relationship: Yb ¼ A0 x0 þ A1 x1 þ A2 x2 þ þ Ak xk ;
ð1Þ
where each regression coefficient Aj , j ¼ 0; . . . ; k, was assumed to be a symmetric triangular fuzzy number with center aj (having membership ¼ 1) and half-width cj , cj P 0 (see Fig. 1). For example, a fuzzy linear relationship (with x0 ¼ a vector of 1 s and only another x variable x1 ) can be represented by a band (the bold lines having membership ¼ 0) with a centre line (the dashed line having a membership ¼ 1) as in Fig. 2. Two cases are considered in this article: Case 1: independent variables (x) are numbers ( ¼ crisp), and response variable (y) is fuzzy. Case 2: independent variables (x) are fuzzy, and response variable (y) is also fuzzy.
*
Corresponding author. Tel.: +1-306-966-8428; fax: +1-306-966-2515. E-mail address: [email protected] (M. Hojati).
0377-2217/$ - see front matter Ó 2004 Elsevier B.V. All rights reserved. doi:10.1016/j.ejor.2004.01.039
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
173
Fig. 1. A triangular fuzzy regression coefficient.
Fig. 2. A fuzzy linear relationship.
The input data are n sets of ‘‘values’’ (yi ; xi0 ; xi1 ; . . . ; xik ), i ¼ 1; . . . ; n, n P k þ 1, where xi0 ¼ 1. The ‘‘value’’ yi is assumed to be a symmetric triangular fuzzy number with center yi and half-width ei , ei P 0. In Case 2, each independent variable value xij , i ¼ 1; . . . ; n, j ¼ 1; . . . ; k, is also assumed to be a symmetric triangular fuzzy number having a center xij and half-width fij . Given a symmetric triangular fuzzy number for yi , if we are only interested in that part of yi which has a membership value of at least H , 0 6 H 6 1, we should use the interval [yi ð1 H Þei ; yi þ ð1 H Þei ]. This interval is the bold line segment in Fig. 3. Here, H represents the minimum degree of certainty acceptable, and we will refer to this interval as H-certain observed interval. Similarly, assuming that the independent variables xj , j ¼ 1; . . . ; k, have exact (crisp) values and regression coefficient Aj , j ¼ 0; . . . ; k, are symmetric triangular fuzzy numbers, the predicted interval to a specific set of x values (xi0 ; xi1 ; . . i. ; xik ) having membership value of at least H is hcorresponding Pk Pk j¼0 ðaj ð1 H Þ cj Þ xij ; j¼0 ðaj þ ð1 H Þ cj Þ xij . We will refer to this interval as H-certain predicted interval.
2. Earlier methods For Case 1, Tanaka et al. [13] proposed the following linear programming (LP) formulation to estimate Aj , j ¼ 0; . . . ; k:
174
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
Fig. 3. An H -certain observed interval.
ðTanÞ Minimize subject to:
c0 þ c1 þ c2 þ þ ck k X ðaj þ ð1 H Þ cj Þ xij P yi þ ð1 H Þei ;
ð2Þ i ¼ 1; . . . ; n;
ð3Þ
i ¼ 1; . . . ; n;
ð4Þ
j¼0 k X
ðaj ð1 H Þ cj Þ xij 6 yi ð1 H Þei ;
j¼0
aj ¼ free;
cj P 0;
j ¼ 0; . . . ; k:
ð5Þ
Note that the above LP forces the H-certain predicted intervals (dotted vertical lines in Fig. 4) to include the H-certain observed intervals (bold vertical lines). Also, note that in the above formulation cj s are assumed to be non-negative, because the imprecision in predicted intervals usually increases for larger values of xj . There have been a few criticisms of Tanaka et al.’s approach. One shortcoming is that the solution to (Tan) is xj -scale dependent and many cj s turn out to be zero (see e.g., Jozsef [4]). To rectify this problem,
Fig. 4. Illustration of (Tan).
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
175
instead of sum of half-width of regression coefficients (2), sum of half-width of the predicted intervals can be used as the objective function (see e.g., Tanaka et al. [11] or Redden and Woodall [8]): Minimize
n X k X i¼1
0
ð2 Þ
cj xij :
j¼0
Another problem with (Tan) is that the solution is xj point-of-reference dependent, i.e., the predicted function will be very different if we first subtract the mean of the independent variables, using ðxj xj Þ instead of xj ; see e.g., Bardossy et al. [2] or Bardossy [1]. Some articles have proposed major changes to Tanaka et al.’s approach. Savic and Pedrycz [9] suggest first to find the centers, aj s, using ordinary least squares method, and then solve (Tan) with these aj s. Tanaka and Ishibuchi [12] also suggest first to determine the centers aj s using ordinary least squares method, but then to solve a quadratic programming version of (Tan) given these aj s. Celmins [3] modified the least squares method to the case when both dependent and independent variables have triangular fuzzy number values in such a way that their joint membership function is a cone. Another shortcoming of Tanaka et al.’s approach is that each H-certain predicted interval is required to contain the corresponding H-certain observed interval. This results in large coefficient half-widths cj if any response ‘value’ has large half-width ei or if there is any ‘outlier’ response. Several articles have pointed this problem and have offered remedies. Tanaka et al. [11]’s Conjunctive model relaxes this requirement, only requiring that each H-certain predicted interval to intersect the associated H-certain observed interval. Sakawa and Yano [10] considered Case 2. First, depending on the expected range of values of fuzzy regression coefficients Aj , Sakawa and Yano would classify the independent variables into three groups: J1 ¼ those variables j; j ¼ 0; . . . ; k; which will have aj ð1 H Þcj P 0; J2 ¼ those j; j ¼ 0; . . . ; k; which will have aj ð1 H Þcj < 0 and aj þ ð1 H Þcj P 0; J3 ¼ those j; j ¼ 0; . . . ; k; which will have aj þ ð1 H Þcj < 0: Then, the following conjunctive problem, re-written in our notation, will be formulated: ðSak1Þ Minimize
n X ^yiU ^yiL i¼1
subject to:
X
ðaj þ ð1 H Þcj Þðxij þ ð1 H Þfij Þ þ
j2J1 [J2
X
ðaj þ ð1 H Þcj Þ xij ð1 H Þfij ¼ ^yiU ;
j2J3
i ¼ 1; . . . ; n; ^yiU P yi ð1 H Þei ; i ¼ 1; . . . ; n; X X ðaj ð1 H Þcj Þðxij ð1 H Þfij Þ þ ðaj ð1 H Þcj Þðxij þ ð1 H Þfij Þ ¼ ^yiL ; j2J2 [J3
j2J1
i ¼ 1; . . . ; n; ^yiL 6 yi þ ð1 H Þei ; aj ¼ free;
i ¼ 1; . . . ; n;
cj P 0; j ¼ 0; . . . ; k:
This LP is illustrated in Fig. 5, where the rectangles are the H-certain observed areas. In this case, the x values are fuzzy numbers with H-certain intervals depicted by horizontal lines of the rectangles. Fig. 5 illustrates the case of positive fuzzy relationship and x1 2 J1 . In this case, only the lower right and upper left corner of each rectangle is required by (SAK1) to be within the H-certain predicted band.
176
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
Fig. 5. Illustration of (Sak1).
Sakawa and Yano also formulated the following problem, re-written in our notation, which is similar to (Tan): ðSak2Þ Minimize
n X ^yiU ^yiL i¼1
subject to:
X j2J1 [J2
þ
ðaj þ ð1 H Þcj Þ xij þ ð1 H Þfij X
ðaj þ ð1 H Þcj Þ xij ð1 H Þfij ¼ ^yiU ;
i ¼ 1; . . . ; n;
j2J3
^yiU P yi þ Hei ; i ¼ 1; . . . ; n; X ðaj ð1 H Þcj Þ xij ð1 H Þfij j2J1
þ
X
ðaj ð1 H Þcj Þ xij þ ð1 H Þfij ¼ ^yiL ;
ð6Þ
i ¼ 1; . . . ; n;
j2J2 [J3
^yiL 6 yi Hei ; aj ¼ free;
i ¼ 1; . . . ; n;
ð7Þ
cj P 0; j ¼ 0; . . . ; k:
Note that in the right-hand-side of (6) and (7), H is used instead of (1 H ) in (3) and (4), based on the concept of ‘‘necessity’’. Sakawa and Yano also proposed an interactive approach to determine the appropriate value of H by balancing the increase in H ’s value vs. the increase in the value of objective function. The problems with this approach are that (a) ^yiU and ^yiL are only approximations of predicted values’ upper and lower surface values, and (b) classifying the independent variables into three groups ahead of performing the regression is difficult (although Sakawa and Yano provide an iterative procedure for this). Peters [7] considered Case 1 and used a Conjunctive model. His LP formulation, (Pet) below, is rather complicated to explain. Let yiU , yi and yiL be the upper, center, and lower points of ith observed interval, and ^yiU and ^yiL be the upper and lower point of the ith predicted interval. The LP (Pet) allows ^yiL to be larger than yiL but smaller than yiU , and ^yiU to be smaller than yiU but larger than yiL . In fact, the average (across all observations) of deviation of ^yiU from yi , if ^yiU < yi , and ^yiL from yi , if ^yiL > yi , is minimized. This objective is balanced against the total half-width of predicted intervals Eq. (20 ) by converting (20 ) into a goal (with an
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
177
appropriately chosen maximum P0 and a minimum of 0) and representing it as a constraint. The complete formulation of Peters’ approach, simplified slightly, is: ðPetÞ Maximize subject to:
k k X
ðaj þ cj Þxij P yi ð1 ki Þei ;
i ¼ 1; . . . ; n;
ðaj cj Þxij 6 yi þ ð1 ki Þei ;
i ¼ 1; . . . ; n;
j¼0 k X j¼0
k ¼ ðk1 þ k2 þ þ kn Þ=n; n X k X cj xij 6 P0 ð1 kÞ; i¼1
j¼0
0 6 ki 6 1;
i ¼ 1; . . . ; n; k P 0;
aj ¼ free;
cj P 0; j ¼ 0; . . . ; k:
Fig. 6 shows a typical relationship of observed intervals (in bold vertical lines) to the fuzzy band in (Pet). While Peters’ formulation allows the predicted intervals to include points other than those in the observed interval, they still have to intersect the observed intervals. Thus, an outlier response ‘value’ could still force wide half-widths cj s. In addition, it is hard to guess a good value for P0 , and the solution is sensitive to this value. Lee and Tanaka [5] have proposed a complex two stage procedure, where first a group of observations ‘‘close’’ to the center of trend in data are used to determine an approximation of the predicted fuzzy regression function, and then the other observations are used to expand this solution. Ozelkan and Duckstein [6] proposed a similar formulation to (Pet), but have not required the prediction intervals to intersect the observed intervals. Ozelkan and Duckstein’s formulation #7, as illustrated in their Example 1, can be written as: ðOzelÞ Minimize
n X
ðdiU þ diL Þ
i¼1
subject to:
k X
ðaj þ ð1 H Þ cj Þxij P yi þ ð1 H Þei diU ;
i ¼ 1; . . . ; n;
j¼0 k X
ðaj ð1 H Þ cj Þxij 6 yi ð1 H Þei þ diL ;
i ¼ 1; . . . ; n;
j¼0 n X k X i¼1
cj xij 6 v;
j¼0
diL ; diU P 0; aj ¼ free;
i ¼ 1; . . . ; n; cj P 0;
j ¼ 0; . . . ; k;
where v is a parameter which should be varied over all possible values of total half-width of predicted intervals, and diU and diL , i ¼ 1; . . . ; n, are upper and lower deviation variables. The problem with this formulation is that it has to be solved several times for different values of v, and that it is not easy to
178
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
Fig. 6. Illustration of (Pet).
determine which value of v is the best. Note that because the deviation variables are required to be nonnegative, diU only measures the difference between upper values of predicted and observed intervals when the upper value of the predicted interval is smaller than upper value of the observed interval, and diL only measures the difference between lower values of predicted and observed intervals when the lower value of the predicted interval is larger than lower value of the observed interval. Thus, the width of the predicted intervals could become arbitrarily large unless they are restricted explicitly using the constraint involving v. In our paper, we propose (a) for Case 1 a formulation similar to Peters and Ozelkan & Duckstein’s but easier to use, and (b) extend this formulation to Case 2 and compare it with the approach of Sakawa and Yano.
3. Our approach 3.1. Case 1: Crisp independent variables and fuzzy dependent variable In this case, we propose the following simple goal programming-like approach. Choose the fuzzy regression coefficients so that the total deviation of upper points of H-certain predicted and associated observed intervals and deviation of lower points of H-certain predicted and associated observed intervals are minimized. This can be achieved by using the following linear program: ðHBS1Þ Minimize
n X þ diU þ diU þ diLþ þ diL i¼1
subject to:
k X
þ ðaj þ ð1 H Þ cj Þxij þ diU diU ¼ yi þ ð1 H Þei ;
i ¼ 1; . . . ; n;
ð8Þ
ðaj ð1 H Þ cj Þxij þ diLþ diL ¼ yi ð1 H Þei ;
i ¼ 1; . . . ; n;
ð9Þ
j¼0 k X j¼0 þ ; diU P 0; i ¼ 1; . . . ; n; diLþ ; diL ; diU aj ¼ free; cj P 0; j ¼ 0; . . . ; k: þ Note that for each i, i ¼ 1; . . . ; n, at most one of diU and diU will be positive and at most one of diLþ and diL þ will be positive. In fact diU diU is the distance between
upper point of H-certain predicted interval and the upper point of the H-certain observed interval, and diLþ diL is the distance between lower point of Hcertain predicted interval and the lower point of the H-certain observed interval. The objective function is equivalent to minimizing the sum of these two distances.
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
179
If the values of the dependent variable are crisp, then it can be proved (by contradiction) that the solution to (HBS1) will also be crisp and it will be close to the solution of the Least Squares method. In this case, if a fuzzy solution is desired, one can use the approach of Savic & Petrycz or Tanaka & Ishibuchi to expand this solution to have fuzzy values. 3.2. Measures of existence of fuzzy linear relationship and goodness of fit Similar to the Least Squares method for the crisp data, it is reasonable to conclude that if the fuzzy relationship band contains a horizontal line, then there may be no fuzzy linear relationship between the dependent and independent variables. If there is a fuzzy linear relationship, then it is desirable to provide a measure of goodness of fit similar to the coefficient of determination R2 . In the fuzzy case, the goodness of fit depends both on the observations and the fuzzy band. The band should both be narrow enough to be of use and wide enough to contain as many observations as possible. To measure the width of the band, we can use the sum of half-width of predicted intervals (20 ) or the sum of half-width of regression coefficients (2). To measure the overlap of observed and predicted intervals, we could use (a) the average percentage of observed intervals contained in the predicted intervals, and (b) the average percentage of predicted intervals contained in the observed intervals. In addition, we would like to define a Similarity measure as follows: Similarity of fuzzy numbers yi & ^yi ¼ ðarea of intersection of yi & ^yi Þ=ðtotal area of yi & ^yi area of intersectionÞ. In Fig. 7, the similarity of yi and ^yi ¼ hashed area=ðA þ B þ hashed area). The similarity measure ranges from 0 ( ¼ no intersection) to 1 ( ¼ perfect overlap). To measure the goodness of fit of the fuzzy band and observations, we would average the similarity measure of all pairs of yi and ^yi , i ¼ 1; . . . ; n. Example 1. Consider the data given in the Example 1 of Tanaka et al. [11], reproduced below: i
xi1
yi ¼ ðyi ; ei Þ
Observed interval
1 2 3 4 5
1 2 3 4 5
(8.0, 1.8) (6.4, 2.2) (9.5, 2.6) (13.5, 2.6) (13.0, 2.4)
(6.2, 9.8) (4.2, 8.6) (6.9, 12.1) (10.9, 16.1) (10.6, 15.4)
Fig. 7. Illustration of similarity measure of yi and ^yi .
180
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
The results of using various approaches to estimate Yb ¼ A0 þ A1 x1 , for H ¼ 0, are given below. We chose the best values we could find for Peters’ and Ozelkan & Duckstein’s methods. Tan
Pet (P0 ¼ 40)
Ozel (v ¼ 25)
Ozel (v ¼ 15)
HBS1
A0 ¼ ða0 ; c0 Þ A1 ¼ ða1 ; c1 Þ cP 0 þ cP 1 n k i¼1 j¼0 cj xij
(3.84, 3.85) (2.10, 0) 3.85 19.25
(5.18, 1.04) (1.77, 0) 1.04 5.25
(5.9, 2.5) (1.4, 0) 2.5 12.5
(5.9, 1.2) (1.5, 0.1) 1.3 7.5
(6.75, 1.65) (1.25, 0.15) 1.8 10.5
Prediction intervals: 1 2 3 4 5
(2.1, 9.8) (4.2, 11.9) (6.3, 14.0) (8.4, 16.1) (10.5, 18.2)
(5.9, 8.0) (7.7, 9.8) (9.4, 11.5) (11.2, 13.3) (13.0, 15.1)
(4.8, 9.8) (6.2, 11.2) (7.6, 12.6) (9.0, 14.0) (10.4, 15.4)
(6.2, 8.8) (7.6, 10.4) (9.1, 12.1) (10.5, 13.7) (12.0, 15.4)
(6.2, 9.8) (7.3, 11.2) (8.4, 12.6) (9.5, 14.0) (10.6, 15.4)
60%
86%
74%
85%
78%
100%
39%
80%
56%
72%
42.8%
24.4%
48.0%
35.2%
55.0%
Average percentage of observed in predicted Average percentage of predicted in observed Average similarity measure
Fig. 8 displays the observation intervals (vertical lines) and the HBS1 fuzzy band. Looking at the results, HBS1 and Ozel (v ¼ 25) provide a more balanced result, which has a relatively large value (>70%) for average percentage of observed in predicted and average percentage of predicted in observed, and relatively low values for sum of half-width of regression coefficients and sum of half-width of predicted intervals. This conclusion is confirmed by the Average Similarity Measure which is the largest for the HBS1 method. Note that observation #2 seems to be lower than the general trend in observations and observation #4 seems to be higher. Also note that the Average Similarity Measure of 55% indicates a fairly good fuzzy linear relationship, which can also be observed graphically.
Fig. 8. Illustration of performance of (HBS1).
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
181
3.3. Case 2: Fuzzy independent variables and fuzzy dependent variable We choose the fuzzy regression coefficients such that the total deviation of upper points of predicted and associated observed intervals and deviation of lower points of predicted and associated observed intervals are minimized at both lower points (‘‘left’’) and upper points (‘‘right’’) of each of the independent variable values (except x0 ). For simplicity, the following LP is formulated for the case when there is only one independent variable (in addition to x0 ): ðHBS2Þ Minimize
n X
þ þ þ þ dilU þ dilL þ dilL þ dirU þ dirU þ dirL þ dirL þ dilU
i¼1
subject to:
1 X
þ ðaj þ ð1 H Þ cj Þ xij ð1 H Þ fij þ dilU dilU ¼ yi þ ð1 H Þei ;
j¼0
i ¼ 1; . . . ; n; 1 X
ð10Þ
þ ðaj þ ð1 H Þ cj Þ xij þ ð1 H Þ fij þ dirU dirU ¼ yi þ ð1 H Þei ;
i ¼ 1; . . . ; n;
j¼0 1 X
þ ðaj ð1 H Þ cj Þ xij ð1 H Þ fij þ dilL dilL ¼ yi ð1 H Þei ;
i ¼ 1; . . . ; n;
j¼0 1 X
þ ðaj ð1 H Þ cj Þ xij þ ð1 H Þ fij þ dirL dirL ¼ yi ð1 H Þei ;
j¼0
i ¼ 1; . . . ; n; þ þ þ þ ; dilU ; dirU ; dirU ; dilL ; dilL ; dirL ; dirL dilU
aj ¼ free;
ð11Þ P 0;
i ¼ 1; . . . ; n;
cj P 0; j ¼ 0; 1;
where in the indices of the deviation variables ‘‘l’’ refers to the left (lower) point and ‘‘r’’ refers to the right (upper) point of the independent variable intervals, and ‘‘U ’’ refers to the upper points and ‘‘L’’ refers to the lower points of the observed and predicted intervals. 3.4. Measures of goodness of fit in Case 2 The Similarity measure between yi and ^yi when xi is fuzzy is similar to the case when x is crisp, except that we would like to measure the overlap of observed and predicted areas: Similarity of fuzzy numbers yi & ^yi over range of xi ¼ (volume of intersection of yi & ^yi )/(total volume of yi & ^yi volume of intersection). Because the fuzzy band changes linearly as xi value changes, we could estimate the similarity measure over the range of xi by calculating the similarity measure at the two end points of xi , xil and xir , and then averaging these two numbers. In Fig. 9, the rectangle shows the observed area, and the parallelogram (in bold lines) shows the predicted area (the membership of points are not shown). Specifically, for H ¼ 0, the x dimension of observed area for observation i ranges from xi1 fi1 to xi1 þ fi1 , and the y dimension of observed area for observation i ranges from yi ei to yi þ ei . That is, the observed area is a rectangle. However, the predicted area is a parallelogram with the same x dimension as the observed area, and y dimension at xi1 fi1 ranging from a0 c0 þ ða1 c1 Þðxi1 fi1 Þ to a0 þ c0 þ ða1 þ c1 Þðxi1 fi1 Þ, called ‘‘left’’ in the table below and y dimension
182
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
at xi1 þ fi1 ranging from a0 c0 þ ða1 c1 Þðxi1 þ fi1 Þ to a0 þ c0 þ ða1 þ c1 Þðxi1 þ fi1 Þ, called ‘‘right’’ in table below. Example 2. Consider the data given in Sakawa and Yano [10], reproduced below: i
xi1 ¼ ðxi1 ; fi1 Þ
xi1 interval
yi ¼ ðyi ; ei Þ
yi interval
1 2 3 4 5 6 7 8
(2.0, 0.5) (3.5, 0.5) (5.5, 1.0) (7.0, 0.5) (8.5, 0.5) (10.5, 1.0) (11.0, 0.5) (12.5, 0.5)
(1.5, 2.5) (3.0, 4.0) (4.5, 6.5) (6.5, 7.5) (8.0, 9.0) (9.5, 11.5) (10.5, 11.5) (12.0, 13.0)
(4.0, 0.5) (5.5, 0.5) (7.5, 1.0) (6.5, 0.5) (8.5, 0.5) (8.0, 1.0) (10.5, 0.5) (9.5, 0.5)
(3.5, 4.5) (5.0, 6.0) (6.5, 8.5) (6.0, 7.0) (8.0, 9.0) (7.0, 9.0) (10.0, 11.0) (9.0, 10.0)
The results of solving (HBS2) with H ¼ 0, and the solutions obtained by Sakawa and Yano for (Sak1) and (Sak2), for their best H value, 0.6, are given below. Sak1
Sak2
HBS2
A0 ¼ ða0 ; c0 Þ A1 ¼ ða1 ; c1 Þ cP 0 þ cP 1 n k xij i¼1 j¼0 cj
(3.37, 0.43) (0.56, 0.11) 0.54 10.10
(4.00, 2.75) (0.48, 0.08) 2.83 26.78
(3.41, 0.41) (0.52, 0.02) 0.43 4.65
Predicted intervals: 1 Left Right 2 Left Right 3 Left Right 4 Left Right 5 Left Right 6 Left Right 7 Left Right 8 Left Right
(3.61, (4.06, (4.29, (4.74, (4.96, (5.86, (5.86, (6.31, (6.53, (6.98, (7.20, (8.10, (7.65, (8.10, (8.33, (8.77,
(1.85, (2.25, (2.44, (2.84, (3.04, (3.84, (3.84, (4.24, (4.43, (4.83, (5.03, (5.83, (5.43, (5.83, (6.03, (6.42,
(3.75, (4.25, (4.50, (5.00, (5.25, (6.25, (6.25, (6.75, (7.00, (7.50, (7.75, (8.75, (8.25, (8.75, (9.00, (9.50,
Average percentage of observed area in predicted area Average percentage of predicted areain observed area Avgerage similarity measure
4.80) 5.47) 5.80) 6.47) 6.81) 8.15) 8.15) 8.82) 9.15) 9.83) 10.16) 11.50) 10.83) 11.50) 11.84) 12.51)
7.58) 8.13) 8.41) 8.97) 9.25) 10.36) 10.36) 10.92) 11.20) 11.76) 12.03) 13.15) 12.59) 13.15) 13.43) 13.99)
41%
19%
45%
83%
100%
45%
23.9%
16.8%
22.2%
4.64) 5.18) 5.45) 6.00) 6.27) 7.36) 7.36) 7.91) 8.18) 8.73) 9.00) 10.10) 9.55) 10.10) 10.36) 10.90)
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
183
Fig. 9. Illustration of similarity measure when x is fuzzy.
Fig. 10. Illustration of peformance of (HBS2).
Looking at the results, our approach (HBS2) and (SAK1) both provide fairly balanced results, with reasonable values for average percentage of observed area in the predicted area and average percentage of predicted area in the observed area, and fairly low values for sum of half-widths and half sum of predicted intervals. Note that HBS2 is easier to implement then SAK1 because it does not need to classify the x variables into three categories ahead of the computation. However, both approaches result in fairly low Average Similarity Measure (22–24%). In Fig. 10, the observations are represented by rectangles and the HBS2 fuzzy band is shown by two lines. Note that the observations (rectangles) do not generally fit inside the fuzzy band (hence the low Average Similarity Measure) but it is evident that there is a fuzzy linear relationship.
4. Conclusions In this article we have reviewed the relevant articles on fuzzy regression and provided a simple approach to determine the coefficients of fuzzy linear relationship. We have compared the approaches using two small examples. Besides simplicity, our approach gave balanced results, matching the observed and predicted values reasonably well while providing reasonably narrow fuzzy bands. We have tried our approach on several data sets and have always obtained good solutions. Three cautions in using fuzzy regression and linear programming for obtaining the results follow:
184
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
1. Because (a) fuzzy regression has double the number of parameters as the ordinary regression (i.e., aj and cj , j ¼ 0; 1; . . . ; k) and (b) the linear programming formulation uses sum of deviations as opposed to sum of squared deviations (used in the Least Squares method), it follows that there are alternate optimal solutions for almost all data sets and for all methods used. In order to obtain the ‘‘best’’ solution out of the alternate solutions, after problem (HBS1) or (HBS2) is solved, let the objective value be d0 . Add the following constraint to the problem: objective Pnfunction Pk ¼ d0 and replace the problem’s objective function with Minimize c0 þ þ ck or Minimize i¼1 j¼0 cj xij and re-solve. 2. Because of interaction of some of aj ’s and cj ’s, it may be necessary to add finite bounds on values of these variables in order to prevent an unbounded solution. For example, 99999 6 aj 6 99999 and 0 6 cj 6 99999. 3. Problem (HBS2) requires 2kþ1 constraints for each observation i (k ¼ number of independent variables other than intercept). This is a shortcoming of HBS2. However, all these constraints may be required. As an example, we solved a reduced version of (HBS2) with Example 2 data. We omitted constraints (10) and (11), the left upper and the right lower constraints, as suggested by Sakawa and Yano (because x1 2 J1 for Example 2). The solution was A0 ¼ ða0 ; c0 Þ ¼ ð3:94; 0:28Þ and A1 ¼ ða1 ; c1 Þ ¼ ð0:44; 0Þ, with average percentage of observed area in predicted area ¼ 48% and only average percentage of predicted area in observed area ¼ 21% (because the prediction areas are too narrow). Obviously, the match between the observed and predicted areas is not so good here. Finally, we point out the limitations of our approach. Because we are estimating the predicted band using the endpoints of the observed intervals, our approach only works when the fuzzy regression coefficients are assumed to be symmetric triangular numbers (or intervals). Thus, non-symmetric and/or nonlinear fuzzy numbers need a more complicated treatment. Also, in Case 2 if x and y interact, e.g., the observed area is a circle instead of the rectangle in Fig. 9, our approach will not work (because the observed area does not have four corners).
References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13]
A. Bardossy, Note on fuzzy regression, Fuzzy Sets and Systems 37 (1990) 65–75. A. Bardossy, I. Bogardi, L. Duckstein, Fuzzy regression in hydrology, Water Resources Research 26 (1990) 1497–1508. A. Celmins, Least squares model fitting to fuzzy vector data, Fuzzy Sets and Systems 22 (1987) 245–269. S. Jozsef, On the effect of linear data transformations in possibilistic fuzzy linear regression, Fuzzy Sets and Systems 45 (1992) 185–188. H. Lee, H. Tanaka, Upper and lower approximation models in interval regression using regression quantile techniques, European Journal of Operational Research 116 (1999) 653–666. E.C. Ozelkan, L. Duckstein, Multi-objective fuzzy regression: A general framework, Computers and Operations Research 27 (2000) 635–652. G. Peters, Fuzzy linear regression with fuzzy intervals, Fuzzy Sets and Systems 63 (1994) 45–55. D.T. Redden, W.H. Woodall, Properties of certain fuzzy linear regression methods, Fuzzy Sets and Systems 64 (1994) 361–375. D.A. Savic, W. Pedrycz, Evaluation of fuzzy linear regression models, Fuzzy Sets and Systems 39 (1991) 51–63. M. Sawaka, H. Yano, Multiobjective fuzzy linear regression analysis for fuzzy input–output data, Fuzzy Sets and Systems 47 (1992) 173–181. H. Tanaka, I. Hayashi, J. Watada, Possibilistic linear regression analysis for fuzzy data, European Journal of Operational Research 40 (1989) 389–396. H. Tanaka, H. Ishibuchi, Identification of possibilistic linear systems by quadratic membership functions of fuzzy parameters, Fuzzy Sets and Systems 41 (1991) 145–160. H. Tanaka, S. Uejima, K. Asai, Linear regression analysis with fuzzy model, IEEE Transactions on Systems, Man, and Cybernetics 12 (1982) 903–907.
Computing, Artificial Intelligence and Computer Technology
A simple method for computation of fuzzy linear regression Mehran Hojati a
a,*
, C.R. Bector b, Kamal Smimou
b
Department of Finance and Management Science, College of Commerce, University of Saskatchewan, 25 Campus Drive, Saskatoon, Saskatchewan, Canada S7N 5A7 b Asper School of Business, University of Manitoba, Winnipeg, Manitoba, Canada R3T 2N2 Received 6 November 2001; accepted 8 January 2004 Available online 15 April 2004
Abstract We propose a new method for computation of fuzzy regression that is simple and gives good solutions. We consider two cases: First, when only the dependent variable is fuzzy, our approach is given and is compared with those suggested in the literature. Secondly, when both dependent and independent variables are fuzzy, our approach is extended and compared with those given in the literature. In each case, a simple example is used to compare the competing approaches. Ó 2004 Elsevier B.V. All rights reserved. Keywords: Fuzzy sets; Fuzzy regression; Linear programming
1. Introduction Fuzzy linear regression was proposed by Tanaka et al. [13] to determine a fuzzy linear relationship: Yb ¼ A0 x0 þ A1 x1 þ A2 x2 þ þ Ak xk ;
ð1Þ
where each regression coefficient Aj , j ¼ 0; . . . ; k, was assumed to be a symmetric triangular fuzzy number with center aj (having membership ¼ 1) and half-width cj , cj P 0 (see Fig. 1). For example, a fuzzy linear relationship (with x0 ¼ a vector of 1 s and only another x variable x1 ) can be represented by a band (the bold lines having membership ¼ 0) with a centre line (the dashed line having a membership ¼ 1) as in Fig. 2. Two cases are considered in this article: Case 1: independent variables (x) are numbers ( ¼ crisp), and response variable (y) is fuzzy. Case 2: independent variables (x) are fuzzy, and response variable (y) is also fuzzy.
*
Corresponding author. Tel.: +1-306-966-8428; fax: +1-306-966-2515. E-mail address: [email protected] (M. Hojati).
0377-2217/$ - see front matter Ó 2004 Elsevier B.V. All rights reserved. doi:10.1016/j.ejor.2004.01.039
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
173
Fig. 1. A triangular fuzzy regression coefficient.
Fig. 2. A fuzzy linear relationship.
The input data are n sets of ‘‘values’’ (yi ; xi0 ; xi1 ; . . . ; xik ), i ¼ 1; . . . ; n, n P k þ 1, where xi0 ¼ 1. The ‘‘value’’ yi is assumed to be a symmetric triangular fuzzy number with center yi and half-width ei , ei P 0. In Case 2, each independent variable value xij , i ¼ 1; . . . ; n, j ¼ 1; . . . ; k, is also assumed to be a symmetric triangular fuzzy number having a center xij and half-width fij . Given a symmetric triangular fuzzy number for yi , if we are only interested in that part of yi which has a membership value of at least H , 0 6 H 6 1, we should use the interval [yi ð1 H Þei ; yi þ ð1 H Þei ]. This interval is the bold line segment in Fig. 3. Here, H represents the minimum degree of certainty acceptable, and we will refer to this interval as H-certain observed interval. Similarly, assuming that the independent variables xj , j ¼ 1; . . . ; k, have exact (crisp) values and regression coefficient Aj , j ¼ 0; . . . ; k, are symmetric triangular fuzzy numbers, the predicted interval to a specific set of x values (xi0 ; xi1 ; . . i. ; xik ) having membership value of at least H is hcorresponding Pk Pk j¼0 ðaj ð1 H Þ cj Þ xij ; j¼0 ðaj þ ð1 H Þ cj Þ xij . We will refer to this interval as H-certain predicted interval.
2. Earlier methods For Case 1, Tanaka et al. [13] proposed the following linear programming (LP) formulation to estimate Aj , j ¼ 0; . . . ; k:
174
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
Fig. 3. An H -certain observed interval.
ðTanÞ Minimize subject to:
c0 þ c1 þ c2 þ þ ck k X ðaj þ ð1 H Þ cj Þ xij P yi þ ð1 H Þei ;
ð2Þ i ¼ 1; . . . ; n;
ð3Þ
i ¼ 1; . . . ; n;
ð4Þ
j¼0 k X
ðaj ð1 H Þ cj Þ xij 6 yi ð1 H Þei ;
j¼0
aj ¼ free;
cj P 0;
j ¼ 0; . . . ; k:
ð5Þ
Note that the above LP forces the H-certain predicted intervals (dotted vertical lines in Fig. 4) to include the H-certain observed intervals (bold vertical lines). Also, note that in the above formulation cj s are assumed to be non-negative, because the imprecision in predicted intervals usually increases for larger values of xj . There have been a few criticisms of Tanaka et al.’s approach. One shortcoming is that the solution to (Tan) is xj -scale dependent and many cj s turn out to be zero (see e.g., Jozsef [4]). To rectify this problem,
Fig. 4. Illustration of (Tan).
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
175
instead of sum of half-width of regression coefficients (2), sum of half-width of the predicted intervals can be used as the objective function (see e.g., Tanaka et al. [11] or Redden and Woodall [8]): Minimize
n X k X i¼1
0
ð2 Þ
cj xij :
j¼0
Another problem with (Tan) is that the solution is xj point-of-reference dependent, i.e., the predicted function will be very different if we first subtract the mean of the independent variables, using ðxj xj Þ instead of xj ; see e.g., Bardossy et al. [2] or Bardossy [1]. Some articles have proposed major changes to Tanaka et al.’s approach. Savic and Pedrycz [9] suggest first to find the centers, aj s, using ordinary least squares method, and then solve (Tan) with these aj s. Tanaka and Ishibuchi [12] also suggest first to determine the centers aj s using ordinary least squares method, but then to solve a quadratic programming version of (Tan) given these aj s. Celmins [3] modified the least squares method to the case when both dependent and independent variables have triangular fuzzy number values in such a way that their joint membership function is a cone. Another shortcoming of Tanaka et al.’s approach is that each H-certain predicted interval is required to contain the corresponding H-certain observed interval. This results in large coefficient half-widths cj if any response ‘value’ has large half-width ei or if there is any ‘outlier’ response. Several articles have pointed this problem and have offered remedies. Tanaka et al. [11]’s Conjunctive model relaxes this requirement, only requiring that each H-certain predicted interval to intersect the associated H-certain observed interval. Sakawa and Yano [10] considered Case 2. First, depending on the expected range of values of fuzzy regression coefficients Aj , Sakawa and Yano would classify the independent variables into three groups: J1 ¼ those variables j; j ¼ 0; . . . ; k; which will have aj ð1 H Þcj P 0; J2 ¼ those j; j ¼ 0; . . . ; k; which will have aj ð1 H Þcj < 0 and aj þ ð1 H Þcj P 0; J3 ¼ those j; j ¼ 0; . . . ; k; which will have aj þ ð1 H Þcj < 0: Then, the following conjunctive problem, re-written in our notation, will be formulated: ðSak1Þ Minimize
n X ^yiU ^yiL i¼1
subject to:
X
ðaj þ ð1 H Þcj Þðxij þ ð1 H Þfij Þ þ
j2J1 [J2
X
ðaj þ ð1 H Þcj Þ xij ð1 H Þfij ¼ ^yiU ;
j2J3
i ¼ 1; . . . ; n; ^yiU P yi ð1 H Þei ; i ¼ 1; . . . ; n; X X ðaj ð1 H Þcj Þðxij ð1 H Þfij Þ þ ðaj ð1 H Þcj Þðxij þ ð1 H Þfij Þ ¼ ^yiL ; j2J2 [J3
j2J1
i ¼ 1; . . . ; n; ^yiL 6 yi þ ð1 H Þei ; aj ¼ free;
i ¼ 1; . . . ; n;
cj P 0; j ¼ 0; . . . ; k:
This LP is illustrated in Fig. 5, where the rectangles are the H-certain observed areas. In this case, the x values are fuzzy numbers with H-certain intervals depicted by horizontal lines of the rectangles. Fig. 5 illustrates the case of positive fuzzy relationship and x1 2 J1 . In this case, only the lower right and upper left corner of each rectangle is required by (SAK1) to be within the H-certain predicted band.
176
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
Fig. 5. Illustration of (Sak1).
Sakawa and Yano also formulated the following problem, re-written in our notation, which is similar to (Tan): ðSak2Þ Minimize
n X ^yiU ^yiL i¼1
subject to:
X j2J1 [J2
þ
ðaj þ ð1 H Þcj Þ xij þ ð1 H Þfij X
ðaj þ ð1 H Þcj Þ xij ð1 H Þfij ¼ ^yiU ;
i ¼ 1; . . . ; n;
j2J3
^yiU P yi þ Hei ; i ¼ 1; . . . ; n; X ðaj ð1 H Þcj Þ xij ð1 H Þfij j2J1
þ
X
ðaj ð1 H Þcj Þ xij þ ð1 H Þfij ¼ ^yiL ;
ð6Þ
i ¼ 1; . . . ; n;
j2J2 [J3
^yiL 6 yi Hei ; aj ¼ free;
i ¼ 1; . . . ; n;
ð7Þ
cj P 0; j ¼ 0; . . . ; k:
Note that in the right-hand-side of (6) and (7), H is used instead of (1 H ) in (3) and (4), based on the concept of ‘‘necessity’’. Sakawa and Yano also proposed an interactive approach to determine the appropriate value of H by balancing the increase in H ’s value vs. the increase in the value of objective function. The problems with this approach are that (a) ^yiU and ^yiL are only approximations of predicted values’ upper and lower surface values, and (b) classifying the independent variables into three groups ahead of performing the regression is difficult (although Sakawa and Yano provide an iterative procedure for this). Peters [7] considered Case 1 and used a Conjunctive model. His LP formulation, (Pet) below, is rather complicated to explain. Let yiU , yi and yiL be the upper, center, and lower points of ith observed interval, and ^yiU and ^yiL be the upper and lower point of the ith predicted interval. The LP (Pet) allows ^yiL to be larger than yiL but smaller than yiU , and ^yiU to be smaller than yiU but larger than yiL . In fact, the average (across all observations) of deviation of ^yiU from yi , if ^yiU < yi , and ^yiL from yi , if ^yiL > yi , is minimized. This objective is balanced against the total half-width of predicted intervals Eq. (20 ) by converting (20 ) into a goal (with an
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
177
appropriately chosen maximum P0 and a minimum of 0) and representing it as a constraint. The complete formulation of Peters’ approach, simplified slightly, is: ðPetÞ Maximize subject to:
k k X
ðaj þ cj Þxij P yi ð1 ki Þei ;
i ¼ 1; . . . ; n;
ðaj cj Þxij 6 yi þ ð1 ki Þei ;
i ¼ 1; . . . ; n;
j¼0 k X j¼0
k ¼ ðk1 þ k2 þ þ kn Þ=n; n X k X cj xij 6 P0 ð1 kÞ; i¼1
j¼0
0 6 ki 6 1;
i ¼ 1; . . . ; n; k P 0;
aj ¼ free;
cj P 0; j ¼ 0; . . . ; k:
Fig. 6 shows a typical relationship of observed intervals (in bold vertical lines) to the fuzzy band in (Pet). While Peters’ formulation allows the predicted intervals to include points other than those in the observed interval, they still have to intersect the observed intervals. Thus, an outlier response ‘value’ could still force wide half-widths cj s. In addition, it is hard to guess a good value for P0 , and the solution is sensitive to this value. Lee and Tanaka [5] have proposed a complex two stage procedure, where first a group of observations ‘‘close’’ to the center of trend in data are used to determine an approximation of the predicted fuzzy regression function, and then the other observations are used to expand this solution. Ozelkan and Duckstein [6] proposed a similar formulation to (Pet), but have not required the prediction intervals to intersect the observed intervals. Ozelkan and Duckstein’s formulation #7, as illustrated in their Example 1, can be written as: ðOzelÞ Minimize
n X
ðdiU þ diL Þ
i¼1
subject to:
k X
ðaj þ ð1 H Þ cj Þxij P yi þ ð1 H Þei diU ;
i ¼ 1; . . . ; n;
j¼0 k X
ðaj ð1 H Þ cj Þxij 6 yi ð1 H Þei þ diL ;
i ¼ 1; . . . ; n;
j¼0 n X k X i¼1
cj xij 6 v;
j¼0
diL ; diU P 0; aj ¼ free;
i ¼ 1; . . . ; n; cj P 0;
j ¼ 0; . . . ; k;
where v is a parameter which should be varied over all possible values of total half-width of predicted intervals, and diU and diL , i ¼ 1; . . . ; n, are upper and lower deviation variables. The problem with this formulation is that it has to be solved several times for different values of v, and that it is not easy to
178
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
Fig. 6. Illustration of (Pet).
determine which value of v is the best. Note that because the deviation variables are required to be nonnegative, diU only measures the difference between upper values of predicted and observed intervals when the upper value of the predicted interval is smaller than upper value of the observed interval, and diL only measures the difference between lower values of predicted and observed intervals when the lower value of the predicted interval is larger than lower value of the observed interval. Thus, the width of the predicted intervals could become arbitrarily large unless they are restricted explicitly using the constraint involving v. In our paper, we propose (a) for Case 1 a formulation similar to Peters and Ozelkan & Duckstein’s but easier to use, and (b) extend this formulation to Case 2 and compare it with the approach of Sakawa and Yano.
3. Our approach 3.1. Case 1: Crisp independent variables and fuzzy dependent variable In this case, we propose the following simple goal programming-like approach. Choose the fuzzy regression coefficients so that the total deviation of upper points of H-certain predicted and associated observed intervals and deviation of lower points of H-certain predicted and associated observed intervals are minimized. This can be achieved by using the following linear program: ðHBS1Þ Minimize
n X þ diU þ diU þ diLþ þ diL i¼1
subject to:
k X
þ ðaj þ ð1 H Þ cj Þxij þ diU diU ¼ yi þ ð1 H Þei ;
i ¼ 1; . . . ; n;
ð8Þ
ðaj ð1 H Þ cj Þxij þ diLþ diL ¼ yi ð1 H Þei ;
i ¼ 1; . . . ; n;
ð9Þ
j¼0 k X j¼0 þ ; diU P 0; i ¼ 1; . . . ; n; diLþ ; diL ; diU aj ¼ free; cj P 0; j ¼ 0; . . . ; k: þ Note that for each i, i ¼ 1; . . . ; n, at most one of diU and diU will be positive and at most one of diLþ and diL þ will be positive. In fact diU diU is the distance between
upper point of H-certain predicted interval and the upper point of the H-certain observed interval, and diLþ diL is the distance between lower point of Hcertain predicted interval and the lower point of the H-certain observed interval. The objective function is equivalent to minimizing the sum of these two distances.
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
179
If the values of the dependent variable are crisp, then it can be proved (by contradiction) that the solution to (HBS1) will also be crisp and it will be close to the solution of the Least Squares method. In this case, if a fuzzy solution is desired, one can use the approach of Savic & Petrycz or Tanaka & Ishibuchi to expand this solution to have fuzzy values. 3.2. Measures of existence of fuzzy linear relationship and goodness of fit Similar to the Least Squares method for the crisp data, it is reasonable to conclude that if the fuzzy relationship band contains a horizontal line, then there may be no fuzzy linear relationship between the dependent and independent variables. If there is a fuzzy linear relationship, then it is desirable to provide a measure of goodness of fit similar to the coefficient of determination R2 . In the fuzzy case, the goodness of fit depends both on the observations and the fuzzy band. The band should both be narrow enough to be of use and wide enough to contain as many observations as possible. To measure the width of the band, we can use the sum of half-width of predicted intervals (20 ) or the sum of half-width of regression coefficients (2). To measure the overlap of observed and predicted intervals, we could use (a) the average percentage of observed intervals contained in the predicted intervals, and (b) the average percentage of predicted intervals contained in the observed intervals. In addition, we would like to define a Similarity measure as follows: Similarity of fuzzy numbers yi & ^yi ¼ ðarea of intersection of yi & ^yi Þ=ðtotal area of yi & ^yi area of intersectionÞ. In Fig. 7, the similarity of yi and ^yi ¼ hashed area=ðA þ B þ hashed area). The similarity measure ranges from 0 ( ¼ no intersection) to 1 ( ¼ perfect overlap). To measure the goodness of fit of the fuzzy band and observations, we would average the similarity measure of all pairs of yi and ^yi , i ¼ 1; . . . ; n. Example 1. Consider the data given in the Example 1 of Tanaka et al. [11], reproduced below: i
xi1
yi ¼ ðyi ; ei Þ
Observed interval
1 2 3 4 5
1 2 3 4 5
(8.0, 1.8) (6.4, 2.2) (9.5, 2.6) (13.5, 2.6) (13.0, 2.4)
(6.2, 9.8) (4.2, 8.6) (6.9, 12.1) (10.9, 16.1) (10.6, 15.4)
Fig. 7. Illustration of similarity measure of yi and ^yi .
180
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
The results of using various approaches to estimate Yb ¼ A0 þ A1 x1 , for H ¼ 0, are given below. We chose the best values we could find for Peters’ and Ozelkan & Duckstein’s methods. Tan
Pet (P0 ¼ 40)
Ozel (v ¼ 25)
Ozel (v ¼ 15)
HBS1
A0 ¼ ða0 ; c0 Þ A1 ¼ ða1 ; c1 Þ cP 0 þ cP 1 n k i¼1 j¼0 cj xij
(3.84, 3.85) (2.10, 0) 3.85 19.25
(5.18, 1.04) (1.77, 0) 1.04 5.25
(5.9, 2.5) (1.4, 0) 2.5 12.5
(5.9, 1.2) (1.5, 0.1) 1.3 7.5
(6.75, 1.65) (1.25, 0.15) 1.8 10.5
Prediction intervals: 1 2 3 4 5
(2.1, 9.8) (4.2, 11.9) (6.3, 14.0) (8.4, 16.1) (10.5, 18.2)
(5.9, 8.0) (7.7, 9.8) (9.4, 11.5) (11.2, 13.3) (13.0, 15.1)
(4.8, 9.8) (6.2, 11.2) (7.6, 12.6) (9.0, 14.0) (10.4, 15.4)
(6.2, 8.8) (7.6, 10.4) (9.1, 12.1) (10.5, 13.7) (12.0, 15.4)
(6.2, 9.8) (7.3, 11.2) (8.4, 12.6) (9.5, 14.0) (10.6, 15.4)
60%
86%
74%
85%
78%
100%
39%
80%
56%
72%
42.8%
24.4%
48.0%
35.2%
55.0%
Average percentage of observed in predicted Average percentage of predicted in observed Average similarity measure
Fig. 8 displays the observation intervals (vertical lines) and the HBS1 fuzzy band. Looking at the results, HBS1 and Ozel (v ¼ 25) provide a more balanced result, which has a relatively large value (>70%) for average percentage of observed in predicted and average percentage of predicted in observed, and relatively low values for sum of half-width of regression coefficients and sum of half-width of predicted intervals. This conclusion is confirmed by the Average Similarity Measure which is the largest for the HBS1 method. Note that observation #2 seems to be lower than the general trend in observations and observation #4 seems to be higher. Also note that the Average Similarity Measure of 55% indicates a fairly good fuzzy linear relationship, which can also be observed graphically.
Fig. 8. Illustration of performance of (HBS1).
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
181
3.3. Case 2: Fuzzy independent variables and fuzzy dependent variable We choose the fuzzy regression coefficients such that the total deviation of upper points of predicted and associated observed intervals and deviation of lower points of predicted and associated observed intervals are minimized at both lower points (‘‘left’’) and upper points (‘‘right’’) of each of the independent variable values (except x0 ). For simplicity, the following LP is formulated for the case when there is only one independent variable (in addition to x0 ): ðHBS2Þ Minimize
n X
þ þ þ þ dilU þ dilL þ dilL þ dirU þ dirU þ dirL þ dirL þ dilU
i¼1
subject to:
1 X
þ ðaj þ ð1 H Þ cj Þ xij ð1 H Þ fij þ dilU dilU ¼ yi þ ð1 H Þei ;
j¼0
i ¼ 1; . . . ; n; 1 X
ð10Þ
þ ðaj þ ð1 H Þ cj Þ xij þ ð1 H Þ fij þ dirU dirU ¼ yi þ ð1 H Þei ;
i ¼ 1; . . . ; n;
j¼0 1 X
þ ðaj ð1 H Þ cj Þ xij ð1 H Þ fij þ dilL dilL ¼ yi ð1 H Þei ;
i ¼ 1; . . . ; n;
j¼0 1 X
þ ðaj ð1 H Þ cj Þ xij þ ð1 H Þ fij þ dirL dirL ¼ yi ð1 H Þei ;
j¼0
i ¼ 1; . . . ; n; þ þ þ þ ; dilU ; dirU ; dirU ; dilL ; dilL ; dirL ; dirL dilU
aj ¼ free;
ð11Þ P 0;
i ¼ 1; . . . ; n;
cj P 0; j ¼ 0; 1;
where in the indices of the deviation variables ‘‘l’’ refers to the left (lower) point and ‘‘r’’ refers to the right (upper) point of the independent variable intervals, and ‘‘U ’’ refers to the upper points and ‘‘L’’ refers to the lower points of the observed and predicted intervals. 3.4. Measures of goodness of fit in Case 2 The Similarity measure between yi and ^yi when xi is fuzzy is similar to the case when x is crisp, except that we would like to measure the overlap of observed and predicted areas: Similarity of fuzzy numbers yi & ^yi over range of xi ¼ (volume of intersection of yi & ^yi )/(total volume of yi & ^yi volume of intersection). Because the fuzzy band changes linearly as xi value changes, we could estimate the similarity measure over the range of xi by calculating the similarity measure at the two end points of xi , xil and xir , and then averaging these two numbers. In Fig. 9, the rectangle shows the observed area, and the parallelogram (in bold lines) shows the predicted area (the membership of points are not shown). Specifically, for H ¼ 0, the x dimension of observed area for observation i ranges from xi1 fi1 to xi1 þ fi1 , and the y dimension of observed area for observation i ranges from yi ei to yi þ ei . That is, the observed area is a rectangle. However, the predicted area is a parallelogram with the same x dimension as the observed area, and y dimension at xi1 fi1 ranging from a0 c0 þ ða1 c1 Þðxi1 fi1 Þ to a0 þ c0 þ ða1 þ c1 Þðxi1 fi1 Þ, called ‘‘left’’ in the table below and y dimension
182
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
at xi1 þ fi1 ranging from a0 c0 þ ða1 c1 Þðxi1 þ fi1 Þ to a0 þ c0 þ ða1 þ c1 Þðxi1 þ fi1 Þ, called ‘‘right’’ in table below. Example 2. Consider the data given in Sakawa and Yano [10], reproduced below: i
xi1 ¼ ðxi1 ; fi1 Þ
xi1 interval
yi ¼ ðyi ; ei Þ
yi interval
1 2 3 4 5 6 7 8
(2.0, 0.5) (3.5, 0.5) (5.5, 1.0) (7.0, 0.5) (8.5, 0.5) (10.5, 1.0) (11.0, 0.5) (12.5, 0.5)
(1.5, 2.5) (3.0, 4.0) (4.5, 6.5) (6.5, 7.5) (8.0, 9.0) (9.5, 11.5) (10.5, 11.5) (12.0, 13.0)
(4.0, 0.5) (5.5, 0.5) (7.5, 1.0) (6.5, 0.5) (8.5, 0.5) (8.0, 1.0) (10.5, 0.5) (9.5, 0.5)
(3.5, 4.5) (5.0, 6.0) (6.5, 8.5) (6.0, 7.0) (8.0, 9.0) (7.0, 9.0) (10.0, 11.0) (9.0, 10.0)
The results of solving (HBS2) with H ¼ 0, and the solutions obtained by Sakawa and Yano for (Sak1) and (Sak2), for their best H value, 0.6, are given below. Sak1
Sak2
HBS2
A0 ¼ ða0 ; c0 Þ A1 ¼ ða1 ; c1 Þ cP 0 þ cP 1 n k xij i¼1 j¼0 cj
(3.37, 0.43) (0.56, 0.11) 0.54 10.10
(4.00, 2.75) (0.48, 0.08) 2.83 26.78
(3.41, 0.41) (0.52, 0.02) 0.43 4.65
Predicted intervals: 1 Left Right 2 Left Right 3 Left Right 4 Left Right 5 Left Right 6 Left Right 7 Left Right 8 Left Right
(3.61, (4.06, (4.29, (4.74, (4.96, (5.86, (5.86, (6.31, (6.53, (6.98, (7.20, (8.10, (7.65, (8.10, (8.33, (8.77,
(1.85, (2.25, (2.44, (2.84, (3.04, (3.84, (3.84, (4.24, (4.43, (4.83, (5.03, (5.83, (5.43, (5.83, (6.03, (6.42,
(3.75, (4.25, (4.50, (5.00, (5.25, (6.25, (6.25, (6.75, (7.00, (7.50, (7.75, (8.75, (8.25, (8.75, (9.00, (9.50,
Average percentage of observed area in predicted area Average percentage of predicted areain observed area Avgerage similarity measure
4.80) 5.47) 5.80) 6.47) 6.81) 8.15) 8.15) 8.82) 9.15) 9.83) 10.16) 11.50) 10.83) 11.50) 11.84) 12.51)
7.58) 8.13) 8.41) 8.97) 9.25) 10.36) 10.36) 10.92) 11.20) 11.76) 12.03) 13.15) 12.59) 13.15) 13.43) 13.99)
41%
19%
45%
83%
100%
45%
23.9%
16.8%
22.2%
4.64) 5.18) 5.45) 6.00) 6.27) 7.36) 7.36) 7.91) 8.18) 8.73) 9.00) 10.10) 9.55) 10.10) 10.36) 10.90)
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
183
Fig. 9. Illustration of similarity measure when x is fuzzy.
Fig. 10. Illustration of peformance of (HBS2).
Looking at the results, our approach (HBS2) and (SAK1) both provide fairly balanced results, with reasonable values for average percentage of observed area in the predicted area and average percentage of predicted area in the observed area, and fairly low values for sum of half-widths and half sum of predicted intervals. Note that HBS2 is easier to implement then SAK1 because it does not need to classify the x variables into three categories ahead of the computation. However, both approaches result in fairly low Average Similarity Measure (22–24%). In Fig. 10, the observations are represented by rectangles and the HBS2 fuzzy band is shown by two lines. Note that the observations (rectangles) do not generally fit inside the fuzzy band (hence the low Average Similarity Measure) but it is evident that there is a fuzzy linear relationship.
4. Conclusions In this article we have reviewed the relevant articles on fuzzy regression and provided a simple approach to determine the coefficients of fuzzy linear relationship. We have compared the approaches using two small examples. Besides simplicity, our approach gave balanced results, matching the observed and predicted values reasonably well while providing reasonably narrow fuzzy bands. We have tried our approach on several data sets and have always obtained good solutions. Three cautions in using fuzzy regression and linear programming for obtaining the results follow:
184
M. Hojati et al. / European Journal of Operational Research 166 (2005) 172–184
1. Because (a) fuzzy regression has double the number of parameters as the ordinary regression (i.e., aj and cj , j ¼ 0; 1; . . . ; k) and (b) the linear programming formulation uses sum of deviations as opposed to sum of squared deviations (used in the Least Squares method), it follows that there are alternate optimal solutions for almost all data sets and for all methods used. In order to obtain the ‘‘best’’ solution out of the alternate solutions, after problem (HBS1) or (HBS2) is solved, let the objective value be d0 . Add the following constraint to the problem: objective Pnfunction Pk ¼ d0 and replace the problem’s objective function with Minimize c0 þ þ ck or Minimize i¼1 j¼0 cj xij and re-solve. 2. Because of interaction of some of aj ’s and cj ’s, it may be necessary to add finite bounds on values of these variables in order to prevent an unbounded solution. For example, 99999 6 aj 6 99999 and 0 6 cj 6 99999. 3. Problem (HBS2) requires 2kþ1 constraints for each observation i (k ¼ number of independent variables other than intercept). This is a shortcoming of HBS2. However, all these constraints may be required. As an example, we solved a reduced version of (HBS2) with Example 2 data. We omitted constraints (10) and (11), the left upper and the right lower constraints, as suggested by Sakawa and Yano (because x1 2 J1 for Example 2). The solution was A0 ¼ ða0 ; c0 Þ ¼ ð3:94; 0:28Þ and A1 ¼ ða1 ; c1 Þ ¼ ð0:44; 0Þ, with average percentage of observed area in predicted area ¼ 48% and only average percentage of predicted area in observed area ¼ 21% (because the prediction areas are too narrow). Obviously, the match between the observed and predicted areas is not so good here. Finally, we point out the limitations of our approach. Because we are estimating the predicted band using the endpoints of the observed intervals, our approach only works when the fuzzy regression coefficients are assumed to be symmetric triangular numbers (or intervals). Thus, non-symmetric and/or nonlinear fuzzy numbers need a more complicated treatment. Also, in Case 2 if x and y interact, e.g., the observed area is a circle instead of the rectangle in Fig. 9, our approach will not work (because the observed area does not have four corners).
References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13]
A. Bardossy, Note on fuzzy regression, Fuzzy Sets and Systems 37 (1990) 65–75. A. Bardossy, I. Bogardi, L. Duckstein, Fuzzy regression in hydrology, Water Resources Research 26 (1990) 1497–1508. A. Celmins, Least squares model fitting to fuzzy vector data, Fuzzy Sets and Systems 22 (1987) 245–269. S. Jozsef, On the effect of linear data transformations in possibilistic fuzzy linear regression, Fuzzy Sets and Systems 45 (1992) 185–188. H. Lee, H. Tanaka, Upper and lower approximation models in interval regression using regression quantile techniques, European Journal of Operational Research 116 (1999) 653–666. E.C. Ozelkan, L. Duckstein, Multi-objective fuzzy regression: A general framework, Computers and Operations Research 27 (2000) 635–652. G. Peters, Fuzzy linear regression with fuzzy intervals, Fuzzy Sets and Systems 63 (1994) 45–55. D.T. Redden, W.H. Woodall, Properties of certain fuzzy linear regression methods, Fuzzy Sets and Systems 64 (1994) 361–375. D.A. Savic, W. Pedrycz, Evaluation of fuzzy linear regression models, Fuzzy Sets and Systems 39 (1991) 51–63. M. Sawaka, H. Yano, Multiobjective fuzzy linear regression analysis for fuzzy input–output data, Fuzzy Sets and Systems 47 (1992) 173–181. H. Tanaka, I. Hayashi, J. Watada, Possibilistic linear regression analysis for fuzzy data, European Journal of Operational Research 40 (1989) 389–396. H. Tanaka, H. Ishibuchi, Identification of possibilistic linear systems by quadratic membership functions of fuzzy parameters, Fuzzy Sets and Systems 41 (1991) 145–160. H. Tanaka, S. Uejima, K. Asai, Linear regression analysis with fuzzy model, IEEE Transactions on Systems, Man, and Cybernetics 12 (1982) 903–907.