- Email: [email protected]
A parametric fuzzy-set prediction model
Fuzzy Sets and Systems 17 (1985) 131-141 North-Holland
A PARAMETRIC Abraham
FUZZY-SET
BOOKSTEIN,
131
PREDICTION
MODEL
Kok Koon NG
Graduate Library School, University of Chicago, Chicago, IL 60637, USA Received April 1984 Revised November 1984 A popular prediction tool for decision making is the compensatory model based on regression analysis. In this model, the predicted performance is given by a linear combination of predictor scores, so that a reduction on the score of one variable can be compensated for by an increase in the score of a second variable. As an alternative this paper explores the predictive value of two simple Boolean models in which the predicted scores are given by predictor values connected by the OR and AND operators through the vehicle of fuzzy-set theory. Paramatized transformations relating raw scores on predictors to fuzzy-set membership are introduced and optimal values found. The model is tested on the performance scores of students in a graduate library school program. It is found that on the learning data, the disjunctive model outperforms the traditional regression model; the two models perform about the same on validation data. Keywords: Fuzzy sets, Decision making, Prediction, Conjunctive models, Disjunctive models.
1. Introduction M a n y d e c i s i o n m a k i n g s i t u a t i o n s i n v o l v e h a v i n g to p r e d i c t t h e p e r f o r m a n c e of a set o f e n t i t i e s on t h e basis of a t t r i b u t e s d e s c r i b i n g e a c h e n t i t y . T o g u i d e us, we h a v e e v a l u a t i o n s of p e r f o r m a n c e a n d a t t r i b u t e v a l u e s f o r s i m i l a r i t e m s in t h e p a s t ; h o w e v e r t h e p e r f o r m a n c e of f u t u r e s e l e c t i o n s is u n c e r t a i n . O u r t a s k is to e x p l o i t existing i n f o r m a t i o n to m a k e t h e b e s t p o s s i b l e choices. S e l e c t i n g s t u d e n t s f o r a g r a d u a t e p r o g r a m in a u n i v e r s i t y is an e x a m p l e of such a p r o b l e m . W h e n c o n s i d e r i n g a s t u d e n t for a d m i s s i o n , o n e t y p i c a l l y has a b o d y of d a t a f o r e a c h a p p l i c a n t . S o m e of this d a t a m a y b e m i n i m a l l y r e l e v a n t for t h e a d m i s s i o n d e c i s i o n , s o m e q u i t e useful, b u t all a r e i m p e r f e c t ; y e t o n t h e basis o f this d a t a , a yes o r no d e c i s i o n m u s t b e m a d e f o r e a c h a p p l i c a n t . T h e p u r p o s e o f this p a p e r is to d e s c r i b e a n d test a f u z z y - s e t b a s e d m o d e l f o r d e c i s i o n m a k i n g ; we shall use t h e p e r f o r m a n c e s c o r e s of s t u d e n t s in a G r a d u a t e L i b r a r y S c h o o l p r o g r a m as d a t a o n w h i c h to test t h e m o d e l . O v e r t h e p a s t t w e n t y y e a r s o r so, h u n d r e d s of s t u d i e s h a v e b e e n p u b l i s h e d on p r e d i c t i n g s t u d e n t p e r f o r m a n c e [ 1 - 1 0 ] . B y far, t h e m o s t c o m m o n a p p r o a c h m a k e s use o f c o r r e l a t i o n a l or, m o r e g e n e r a l l y , r e g r e s s i o n t e c h n i q u e s . In t h e s e t e c h n i q u e s , a n u m b e r of v a r i a b l e s b e l i e v e d to b e a s s o c i a t e d with f u t u r e p e r f o r m a n c e a r e c o l l e c t e d , a n d t h e b e s t p r e d i c t o r l i n e a r l y r e l a t e d to t h e s e v a r i a b l e s c o n s t r u c t e d . Such a m o d e l can b e d e s c r i b e d as ' c o m p e n s a t o r y ' in t h e s e n s e t h a t a unit 0165-0114/85/$3.30 © 1985, Elsevier Science Pubfishers B.V. (North-Holland)
132
A. Bookstein, K.K. Ng
reduction in one predictor can be compensated for by a given increase in another, so that the predicted score remains constant; if one predictor increases or decreases while the other predictors are kept constant, then the predicted variable experiences a corresponding change. Although the predictive value of the regression model has varied with research, most studies do show some validity, often moderate in size, but rarely strong. (See ref. [11-17] for discussions of factors that limit the performance of regression analyses.) Thus, while regression techniques do provide a valuable approach to prediction, attempts to improve on previous performance are clearly appropriate. One can, for example, search for additional variables that, when added to the previous, will improve prediction. In this paper, we will take a different tack. Our main purpose is to explore the possibility that approaches quite different conceptually from regression analysis may have value for prediction, and to suggest that one cause of the limited predictive capabilities of the earlier models may be the structure of the model itself. In particular, we shall test models based on Boolean logic, as generalized by the theory of fuzzy-sets, and compare the results with a traditional regression analysis. To illustrate our model, we shall test two simple Boolean models based on three prediction variables; it is not our intention in this paper to test exhaustively the Boolean combinations that are possible, but rather to provide evidence that such an approach is viable and deserving of consideration, thereby enlarging the armory of techniques available for prediction.
2. Non-regression approaches The main purpose of this paper is to test whether Boolean logic can compete effectively with the compensatory logic of regression analyses when making predictions. Continuing with the example of student performance, the regression approach might allow us to compensate for a decline in, say, quantitative aptitude by an increase in verbal aptitude. On the other hand, a Boolean model might assert, for example, that adequate scores in both abilities are required for success, and that a lower value in one score cannot be compensated for by a higher value in another; that is, performance in a program might be determined by the degree to which 'OtJAr,rr AND VERBAL' excellence is satisfied. Other and more complex Boolean models can be readily defined. The critical task in defining such a model is to quantify in a reasonable manner the degree to which the criterion is satisfied in any specific instance. Some early approaches [18] defined 'conjunctive' and 'disjunctive' decision models that attempted to capture the logic of Boolean algebra, but were in effect transformations of traditional compensatory models. An appealing mechanism to realize the most characteristic properties of Boolean logic while at the same time permitting quantification relies on fuzzy set theory [19]. In such a formulation, each attribute is represented as a fuzzy set, with degree of membership indicating the extent to which the attribute describes the entity. The rules of fuzzy set manipulation are than used to form the fuzzy set of items satisfying the criterion for acceptance. Once we accept the rules of fuzzy set theory for combining attribute values, the difficulty remains of evaluating the
A parametric fuzzy-set prediction model
133
membership function for each attribute on each entity in the decision space. In the simplest case, we may have raw scores for each attribute, and desire the optimal transformation taking that value into a membership value. For example, if verbal graduate record examination scores are used to assess verbal ability, we need a mechanism for transforming such a score into a value denoting membership in the set of applicants with excellent verbal aptitude. Bookstein discussed this problem in the context of information retrieval [20] and then more formally in [21]. In that approach, one defines a class of functions, fa, distinguished by a vector parameter, a, that transforms raw ability values into permissible m e m b e r ship function values; one then determines the values of a by fitting past data, much as one would determine the coefficients of a regression model. Thus one begins with (a) a Boolean model relating entity attributes to the predicted performance of an entity, (b) raw data estimating how well the attributes describe each entity, and (c) past data. The parameterized transformation functions, referred to as 'filter functions' in [20, 21], define a parameterized fuzzy set model, and optimal values for the parameters are sought and used for future decisions. Using different filtering functions for each of the attributes in effect allows us to change the relative impact of each attribute. The basic assumption underlying a Boolean model is that an individual's potential to succeed at some task may not be necessarily determined by the cumulative action of many attributes, as implied by the regression model. It may, for example, only depend on the level of the single best ability that the person has. In such a case, a 'disjunctive' model would be more suitable than a compensatory model. A n o t h e r possibility is that an individual's ability to succeed is determined by the extent to which he exceeds certain minimum levels in each of various significant attributes. In such a case, a 'conjunctive' model would be more appropriate. The method we are describing brings together the criteria that are important for the decision while retaining the properties commonly associated with the connectives 'AND' and 'oR'. To compare the predictive ability of these two models against the compensatory model, an analysis was done of the records of 88 students from the University of Chicago Graduate Library School, representing two classes enrolled in Autumn 1981 and Autumn 1982. The predictors used are the undergraduate grade point average (UGPA), and the G R E verbal and quantitative scores ( G R E V and G R E Q , respectively); the graduate grade point average (GGPA) was used as the success criterion. We first compared how well the various models fit the data describing one of the classes (48 students), and then used those best fit parameter values to test how well these models would have predicted performance for the second class (40 students). The main justification for using such simple models is the exploratory character of this research: to show that techniques initimately based on fuzzy-set theory can compete well with established prediction techniques. But it is also the case that variables of practical value are often difficult to predict, and that a relatively small number of predictor variables capture the bulk of the variance that can be explained at all. This is certainly true of performance in academic programs: using a large range of variables explains a modest amount of the variances and the two or three most important variables perform almost as
134
A. Bookstein, K.K. Ng
well as the full set. Thus the models being tested here, while simple, are not in fact as limited as might appear so far as exploiting the available useful information.
3. Definitions and notations
For the purpose of this study, we need a number of notations and definitions. We begin with a (traditional) set of students, each evaluated on a number of attributes believed useful in predicting each student's performance. If we have n students and N attributes, then each attribute can be represented as a vector with n components as follows:
ii,
The vector of dimension N, associated with the i-th attribute; each component is the score of a student on that attribute. The score of the j-th student is given by:
Aii
The j-th component of the i-th vector; although Aii may take an arbitrary value, we will ultimately transform Aii to a number in [0, 1]. This restriction will allow us to conceptualize each attribute as a fuzzy set of s t u d e n t s - the set of students 'succeeding' on the i-th attribute. Thus Aii will become the extent to which the j-th student has done well on (is 'in') attribute i. For Aii in [0, 1], we define:
(n,'=, Ai)i
The j-th component of the intersection of N vectors, where intersection, defined by ( ~ = 1 fi,i)i = mini {Aii}, is the extent to which the j-th student excels in all of the N attributes. This definition, and the following, is conventional in the theory of fuzzy sets and can be shown to be required for consistency with the laws of logic [23].
(U~o, A.i)~
The j-th component of the union of N vectors, where union is defined by ( U ~1 fiti)i = maxl {A~j}, again as required by the theory of fuzzy sets. A filtering function such that f:x ~ y, y e [ 0 , 1]; f is monotonically non-decreasing. Generally, scores are initially given in arbitrary units, and we must transform these scores into values of a membership function, that is, into the attributes referred to above.
We begin our analysis with raw scores for the predictors G R E V , G R E Q , and U G P A . A filtering function is then applied to produce scores between 0 and 1 as required by our model; these form the attributes used in the analysis. The filters may be defined aribtrarily. To create the filters we shall use, it is assumed that cut-off values could be defined so that students above the upper threshold 'pass' the criterion (are fully in the set represented by the attribute) and those below fail (are not members of the set). Degrees of membership are, for simplicity, assumed to vary linearly with raw scores between these thresholds.
A parametric [uzzy-set prediction model
135
4. Methodology W e found it convenient for interpretation to define the filters in two stages. First, the scores of the students were standardized to the interval [0, 1]. T h e following formulas were used for the standardization: SUGPA SGREV -
UGPA-
1.5
2.5
SGGPA =
'
G R E V - 200 600
'
SGREQ =
GGPA-
1.5
2.5 G R E Q - 200 600
The constants appearing in these formulas were based on subjective estimates of the lowest values the variables were likely to take and their range. The standardization is not essential, but useful for interpreting the results. Then, a filtering function, taking the following form, is applied to the standardized score:
I
0,
X ~ 01,
x-~
f(x) = | 0 2 - 01' 01 < x < 02, t.1,
x~02,
where x denotes one of the standardized test scores, and 01 and 02 are p a r a m e t e r s of the function that determine the lower and the u p p e r 'cut-off' points. It should be noted that 01 must be less than 02, but neither 01 nor 02, each of which is fitted from the data, is constrained to lie in the interval [0, 1]. For the conjunctive model, the predicted standardized G G P A is determined by the values of the c o m p o n e n t s of the intersection of the filtered test scores -Ai, i.e.
H e r e we are in effect defining a new fuzzy set, F, of students who are predicted to do well. This model says that students who are expected to do well are those who pass on each attribute. For the disjunctive model, prediction of success is determined by the values of the c o m p o n e n t s of the union of the filtered test scores
Ai, i.e.
that is, a student is expected to do well if he passes at least one of the predictors of success. In the a b o v e notations, Fj will ultimately be c o m p a r e d to the filtered G G P A for student j, which is taken to define the set of successful students. The O's in these filtering functions are unknown quantities. W e would like to find the combination of the values of the 0's that optimizes the association between the predicted and the actual G G P A . The software package used for this purpose is the s'mPrr routine. This is a subroutine used in conjunction with a
136
A. Bookstein, K.K. Ng
FOre-RAN program running on an IBM machine. It is designed for finding local minima of a smooth function of several parameters using a direct search method (without using derivates) [22]. The parameters for this study are the six cut-off values, two each for the three filters. In fitting our model, we used the G G P A , standardized to take values between zero and one (SGGPA), as the success criterion. The objective function, to be minimized, was the sum of squares of the difference between S G G P A , interpreted as the actual degree of membership in the set of successful students, and F, the predicted standardized G G P A . In notational form we have, for SS, the objective function, SS = ~ ( S G G P A i - Fi) 2, i=1 where n is the number of students in the study. SS depends on the parameters of the model. We also computed the correlation between S G G P A and F, using the cut-off values found above. The model was cross-validated by testing the parameter values on fresh data. Finally, for comparison purposes, a linear regression model was used, with G G P A as the criterion, and then cross-validated on the same set of data as the fuzzy set models.
5. Results Table 1 shows the mean and standard deviation for the data in group 1 and group 2. Table 2 shows the correlation coefficients between the individual predictors and G G P A . It is obvious that, for the second group, the individual predictors are more highly correlated with G G P A than the first group, although the two groups have very similar values for the means and standard deviations of the raw scores. The regression analysis on group 1 shows that, when all three predictors are used, G R E V is not a significant predictor, having a t-value of only - 0 . 7 ; this Table 1. Mean and standard deviation of the 4 test scores Standard Variable Group Mean deviation UGPA GREV GREQ GGPA
1 2 1 2 1 2 1 2
3.4 0.4 3.2 0.5 650.0 90.0 650.0 90.0 560.0 100.0 560.0 100.0 3.3 0.4 3.3 0.4
A parametric fuzzy-set prediction model
137
Table 2. Correlation between the 3 predictors and GGPA Group 1 Group UGPA 0.3 GREV 0.2 GREQ 0.3
2
0.4 0.4 0.6
result is surprising, given the generally perceived qualitative character of librarianship, but is consistent with other informal studies carried out in the School. G R E V is therefore dropped from the regression analysis. The final result is as follows: S G G P A = 0.37 + 0.25 S U G P A + 0.25 S G R E Q where S G G P A , S U G P A , and S G R E Q are defined above. The multiple correlation coefficient, R, is 0.4. If we convert both the predicted and the observed S G G P A to ranks, the S p e a r m a n coefficient is 0.5. The best result obtained for the conjunctive model yields a correlation coefficient of 0.36 between the predicted and the observed S G G P A . A detailed examination of the data reveals that the predicted value is not sensitive to the U G P A or G R E V scores for the fitted v a l u e s - w e could vary the values for U G P A and G R E V without influencing the prediction. T h a t is, using the conjunctive model, it was found that, with few exceptions, the filtered G R E Q alone determined the predicted S G G P A for the model and there is negligible improvement in using the full conjunctive model over a simple model using G R E Q alone as the predictor. The disjunctive model looks m o r e promising. The best result yields a correlation coefficient of 0.5. Cut-off values that give this result a p p e a r in Table 3, though the model is not sensitive to the lower cut-off value. Converting the predicted and the observed S G G P A to ranks, the Spearman coefficient is 0.5. A question that arises in traditional regression analysis is which variable is the most ' i m p o r t a n t ' . A similar question can be posed here for the disjunctive model, with at least as much difficulty in providing an answer. Unlike regression, in which all the variables cooperate to yield a prediction, in a Boolean model, one variable dominates and determines the prediction. This suggests our first criterion of importance: that variable is most important which will most easily dominate the other variables. Consider for example, the m e m b e r s h i p level of 0.7. This would translate into a raw score of 3.48 for U G P A , 763 for G R E V , and 620 for G R E Q . Table 3. Optimum cut-off points for the disjunctive model UGPA GREV GREQ Lower cut-off point Upper cut-off point
0.0 0.005 1 . 1 5 2 1.338
-0.330 1.146
Correlation coefficients 0.5270
138
A. Bookstein, K.K. Ng
Table 4. Comparison of the 3 models for group 1 Correlation coefficients
Regression model Disjunctive model Conjunctive model
Scores
Rank
0.44 0.53 0.36
0.51 0.52 0.46
T h a t is, in order for U G P A to have the same strength as a score of 620 for G R E Q , the U G P A score has to be 3.48; similarly, the G R E V score has to be 763. In terms of our population, 41% of the U G P A are above 3.48, but only 28% of the G R E Q are above 620, and 4% of the G R E V are above 763. This criterion emphasizes the importance of U G P A . In fact, in our population, 63% of the time it was the U G P A that determined the prediction. A n o t h e r criterion of importance is the slope of the filter, assuming the reasonableness of the standardization. For, if we select values for the predictors for which each predictor suggests the same value for the G G P A , then a unit increase in the predictor with the greatest slope will have m o r e of an impact than will a unit increase in the other variables. By this criterion, which is closest to regression case, U G P A again seems most important. In summary, Table 4 compares the correlation coefficients obtained from the three different models. T h e first column shows the correlation between the observed and the predicted S G G P A for the three models. The second column shows the S p e a r m a n coefficient if we convert both the observed and predicted values to ranks. When predicting filtered scores using the least square error criterion, the disjunctive model is clearly superior to the regression model. W h e r e predicting ranks is concerned, the regression model is nearly as good as the disjunctive model. T h e conjunctive model is much inferior to either the disjunctive or the regression model on both criteria. To test the predictive ability of the three models, the predictors from group 2 were processed using these models to find the predicted G G P A . The resultant correlation coefficients are shown in Table 5, which has the same f o r m a t as Table 4. We see that the regression model is better on both criteria. T h e conjunctive model performs about as well as the disjunctive model, which is due to the high Table 5. Comparison of the 3 models for group 2 Correlation coefficients
Regression model Disjunctive model Conjunctive model
Scores
Rank
0.64 0.62 0.58
0.69 0.62 0.61
139
A parametric [uzzy-set prediction model
Table 6. Group 1 error estimates for the 3 models Mean absolute error Regression model Disjunctive model Conjunctive model
0.11 (s.d. 0.08) 0.10 (s.d. 0.07) 0.12 (s.d. 0.08)
Mean absolute percentage error with observed score 17% (s.d. 18) 16% (s.d. 15) 19% (s.d. 18)
Mean absolute percentage error with predicted score 16% (s.d. 12) 15% (s.d. 11) 17% (s.d. 11)
c o r r e l a t i o n b e t w e e n G R E Q a n d U G P A in t h e s e c o n d g r o u p . In all cases, t h e p r e d i c t i o n s o n t h e v a l i d a t i o n d a t a is a c t u a l l y i m p r o v e d , an a r t i f a c t o f t h e h i g h e r c o r r e l a t i o n in t h e d a t a b e t w e e n t h e c r i t e r i o n v a r i a b l e a n d t h e p r e d i c t o r s . A s a f u r t h e r c o m p a r i s o n , w e h a v e c o m p u t e d , f o r t h e case of filtered scores, t h e m e a n a b s o l u t e e r r o r , t h e m e a n p e r c e n t a g e e r r o r with t h e o b s e r v e d score, a n d t h e p e r c e n t a g e e r r o r with t h e p r e d i c t e d score. T h e results a r e s h o w n in T a b l e 6 a n d T a b l e 7. A g a i n , t h e d i s j u n c t i v e m o d e l p e r f o r m e d a b o u t as well as t h e r e g r e s s i o n model.
6. Conclusion T h e c o n j u n c t i v e m o d e l , w h e n fitted, has a n e g l i g i b l e i m p r o v e m e n t o v e r a s i m p l e o n e v a r i a b l e m o d e l using G R E Q as t h e p r e d i c t o r . T h e fit w i t h g r o u p 1, a l t h o u g h i n f e r i o r to the r e g r e s s i o n m o d e l , is n o t r e a l l y t h a t b a d since G R E Q is t h e v a r i a b l e m o s t h i g h l y c o r r e l a t e d with G G P A a m o n g t h e t h r e e p r e d i c t o r s . H o w e v e r , n o t only are the mean absolute prediction error and the mean absolute percentage e r r o r b i g g e r t h a n f o r t h e d i s j u n c t i v e m o d e l , t h e y also h a v e a w i d e r s p r e a d t h a n t h e r e g r e s s i o n m o d e l . It s e e m s t h a t t h e logic t h a t for a s t u d e n t to d o well he m u s t e x c e e d s o m e t h r e s h o l d in e a c h of s e v e r a l v a r i a b l e s is i n a d e q u a t e , at l e a s t f o r the v a r i a b l e s t e s t e d ; w h e n f o r c e d to a p p l y t h a t logic, o u r p r o g r a m s i m p l y t o o k t h e single b e s t v a r i a b l e m o s t of t h e t i m e . Table 7. Group 2 error estimates for the 3 models
Regression model Disjunctive model Conjunctive model
Mean absolute error
Mean absolute percentage error with observed score
0.10 (s.d. 0.08) 0.11 (s.d. 0.08) 0.11 (s.d. 0.08)
17% (s.d. 21) 17% (s.d. 17) 19% (s.d. 22)
Mean absolute percentage error with predicted score 15% (s.d. 13) 16% (s.d. 12) 16% (s.d. 12)
A. Bookstein, K.K. Ng
140
The disjunctive model, on the other hand, fits the data of group 1 better than the regression model. The i m p r o v e m e n t is m o r e than 2 0 % if we c o m p a r e the correlation coefficients for the scores. For predicting the p e r f o r m a n c e of the students of group 2, the disjunctive model is not as good as the regression model. However, the differences are less than 5% if we c o m p a r e the correlation coefficients for scores again. C o m p a r i n g the m e a n absolute error and the m e a n absolute percentage error we find again that the two models p e r f o r m about equally well. But the variance for all three error measures are smaller for the disjunctive model than for the regression model. Thus, although the two models are quite close on the average in predicting students' p e r f o r m a n c e , the disjunctive model appears to be m o r e consistent in the prediction. The results of ttiis study look quite encouraging. It gives rather strong evidence that the disjunctive model is c o m p a r a b l e in robustness to the regression model in predicting students' p e r f o r m a n c e in graduate school. But the disjunctive model, which utilizes solely the fuzzy set operation of union, is only a special case of the general Boolean model [21]. T o take full advantage of the Boolean model, other operations, such as c o m p l e m e n t and intersection, may be included to give models such as UGPA U (GREV n GREQ) or (UGPA U GREV) n GREQ. In addition, other predictors, such as previous work experience, letter of reference, or college activities, m a y also be included to build a m o r e complex model, and more complex filter transformations m a y be tested. The important conclusion from this work is that predictive models radically different from the heavily used regression models are available, and should be taken seriously.
Acknowledgement We wish to acknowledge the use in this work of subroutine STEPIT, written by J.P. Chandler and distributed by the Q u a n t u m Chemistry P r o g r a m Exchange.
References [ 1] N.W. Burton and N.J. Turner, Effectiveness of the Graduate Record Examination for predicting first-year grades. 1981-1982 summary report of the Graduate Record Examination Validity Study Service, Educational Testing Service, Princeton, NJ, Report #209021 (1983). [2] G.E. Tully, Screening applicants for graduate study with the aptitude test of the graduate record examinations, College and University 38 (1962) 51-60. [3] G.F. Madaus and J.J. Walsh, Departmental differentials in the predictive validity of the graduate record examination aptitude test, Educational and Psychological Measurements 25 (1965) 1105-1110. [4] S. Baillie, Library school and job success, Studies in Librarianship 1 (1964) 1-176. [5] M.E. Lamson, GRE under fire-again, Journal of Education for Librarianship 12 (1972) 175-177.
A parametric fuzzy-set prediction model
141
[6] L.L. Baird, Comparative prediction of first year graduate and professional school grades in six fields, Educational and Psychological Measurement 35 (1975) 941-946. [7] J.B. Katz, Indicators of success: Queens College Department of Library Science, Journal of Education for Librarianship 19 (1978) 130-139. [8] R.M. Magrill and C. Rinehart, Success in library school: a study of admission variables, Journal of Education for Librarianship 19 (1979) 203-222. [9] R.I. Blue and J.L. Divilbiss, Optimizing selection of library school students, Journal of Education for Librarianship 21 (1981) 301-312. [10] L. Auld, GRE analytical ability as an admission factor, Library Quarterly 54 (1984). [11] R.M. Dawes, Graduate admission variables and future success, Science 30 (1975) 721-723. [12] H. Gulliksen, Harold, Theory of Mental Tests (Wiley, New York, 1950). [13] F.M. Lord and M.R. Novick, Statistical Theories of Mental Test Scores, (Addison-Wesley, MA, 1968). [14] N. Givner and K. Hynes, Admissions test validity: correcting for restriction effects, College and University 54 (1979) 119-123. [15] A. Gullickson and K. Hopkins, Interval estimation of correlation coefficients corrected for restriction of range, Educational and Psychological Measurement 36 (1976) 9-25. [16] D.W. Perrin, Admissions test validity: additional corrections for restriction effects, College and University 55 (1980) 181-185. [17] V. Srinivasan and A.G. Weinstein, Effects of curtailment of an admission model for a graduate management program, Journal of Applied Psychology 58 (1973) 339-346. [18] H.J. Einhorn, The use of nonlinear, noncompensatory models in decision making, Psychological Bulletin 73 (1970) 221-230. [19] L.A. Zadeh, Fuzzy sets, Information and Control 8 (1965) 338-353. [20] A. Bookstein, The generalized retrieval problem, ACM SIGIR Forum, 16 (1981) 4-14. [21] A. Bookstein, Fuzzy-set decision models, in: J. Kacprzyk and R.R. Yager, Eds., Management Decision Support Systems Using Fuzzy Sets and Possibility Theory (TUV-verlag, Collogne, to appear). [22] J.P. Chandler, s'rEorr-Direct search for optimization; solution of least squares problems, Quantum Chemistry Program Exchange, QCPE Program No. 307, Chemistry Department, Indiana University. [23] R. Bellman and M. Giertz, On the analytic formalism of the theory of fuzzy sets, Information Sciences 5 (1973) 149-56.
A PARAMETRIC Abraham
FUZZY-SET
BOOKSTEIN,
131
PREDICTION
MODEL
Kok Koon NG
Graduate Library School, University of Chicago, Chicago, IL 60637, USA Received April 1984 Revised November 1984 A popular prediction tool for decision making is the compensatory model based on regression analysis. In this model, the predicted performance is given by a linear combination of predictor scores, so that a reduction on the score of one variable can be compensated for by an increase in the score of a second variable. As an alternative this paper explores the predictive value of two simple Boolean models in which the predicted scores are given by predictor values connected by the OR and AND operators through the vehicle of fuzzy-set theory. Paramatized transformations relating raw scores on predictors to fuzzy-set membership are introduced and optimal values found. The model is tested on the performance scores of students in a graduate library school program. It is found that on the learning data, the disjunctive model outperforms the traditional regression model; the two models perform about the same on validation data. Keywords: Fuzzy sets, Decision making, Prediction, Conjunctive models, Disjunctive models.
1. Introduction M a n y d e c i s i o n m a k i n g s i t u a t i o n s i n v o l v e h a v i n g to p r e d i c t t h e p e r f o r m a n c e of a set o f e n t i t i e s on t h e basis of a t t r i b u t e s d e s c r i b i n g e a c h e n t i t y . T o g u i d e us, we h a v e e v a l u a t i o n s of p e r f o r m a n c e a n d a t t r i b u t e v a l u e s f o r s i m i l a r i t e m s in t h e p a s t ; h o w e v e r t h e p e r f o r m a n c e of f u t u r e s e l e c t i o n s is u n c e r t a i n . O u r t a s k is to e x p l o i t existing i n f o r m a t i o n to m a k e t h e b e s t p o s s i b l e choices. S e l e c t i n g s t u d e n t s f o r a g r a d u a t e p r o g r a m in a u n i v e r s i t y is an e x a m p l e of such a p r o b l e m . W h e n c o n s i d e r i n g a s t u d e n t for a d m i s s i o n , o n e t y p i c a l l y has a b o d y of d a t a f o r e a c h a p p l i c a n t . S o m e of this d a t a m a y b e m i n i m a l l y r e l e v a n t for t h e a d m i s s i o n d e c i s i o n , s o m e q u i t e useful, b u t all a r e i m p e r f e c t ; y e t o n t h e basis o f this d a t a , a yes o r no d e c i s i o n m u s t b e m a d e f o r e a c h a p p l i c a n t . T h e p u r p o s e o f this p a p e r is to d e s c r i b e a n d test a f u z z y - s e t b a s e d m o d e l f o r d e c i s i o n m a k i n g ; we shall use t h e p e r f o r m a n c e s c o r e s of s t u d e n t s in a G r a d u a t e L i b r a r y S c h o o l p r o g r a m as d a t a o n w h i c h to test t h e m o d e l . O v e r t h e p a s t t w e n t y y e a r s o r so, h u n d r e d s of s t u d i e s h a v e b e e n p u b l i s h e d on p r e d i c t i n g s t u d e n t p e r f o r m a n c e [ 1 - 1 0 ] . B y far, t h e m o s t c o m m o n a p p r o a c h m a k e s use o f c o r r e l a t i o n a l or, m o r e g e n e r a l l y , r e g r e s s i o n t e c h n i q u e s . In t h e s e t e c h n i q u e s , a n u m b e r of v a r i a b l e s b e l i e v e d to b e a s s o c i a t e d with f u t u r e p e r f o r m a n c e a r e c o l l e c t e d , a n d t h e b e s t p r e d i c t o r l i n e a r l y r e l a t e d to t h e s e v a r i a b l e s c o n s t r u c t e d . Such a m o d e l can b e d e s c r i b e d as ' c o m p e n s a t o r y ' in t h e s e n s e t h a t a unit 0165-0114/85/$3.30 © 1985, Elsevier Science Pubfishers B.V. (North-Holland)
132
A. Bookstein, K.K. Ng
reduction in one predictor can be compensated for by a given increase in another, so that the predicted score remains constant; if one predictor increases or decreases while the other predictors are kept constant, then the predicted variable experiences a corresponding change. Although the predictive value of the regression model has varied with research, most studies do show some validity, often moderate in size, but rarely strong. (See ref. [11-17] for discussions of factors that limit the performance of regression analyses.) Thus, while regression techniques do provide a valuable approach to prediction, attempts to improve on previous performance are clearly appropriate. One can, for example, search for additional variables that, when added to the previous, will improve prediction. In this paper, we will take a different tack. Our main purpose is to explore the possibility that approaches quite different conceptually from regression analysis may have value for prediction, and to suggest that one cause of the limited predictive capabilities of the earlier models may be the structure of the model itself. In particular, we shall test models based on Boolean logic, as generalized by the theory of fuzzy-sets, and compare the results with a traditional regression analysis. To illustrate our model, we shall test two simple Boolean models based on three prediction variables; it is not our intention in this paper to test exhaustively the Boolean combinations that are possible, but rather to provide evidence that such an approach is viable and deserving of consideration, thereby enlarging the armory of techniques available for prediction.
2. Non-regression approaches The main purpose of this paper is to test whether Boolean logic can compete effectively with the compensatory logic of regression analyses when making predictions. Continuing with the example of student performance, the regression approach might allow us to compensate for a decline in, say, quantitative aptitude by an increase in verbal aptitude. On the other hand, a Boolean model might assert, for example, that adequate scores in both abilities are required for success, and that a lower value in one score cannot be compensated for by a higher value in another; that is, performance in a program might be determined by the degree to which 'OtJAr,rr AND VERBAL' excellence is satisfied. Other and more complex Boolean models can be readily defined. The critical task in defining such a model is to quantify in a reasonable manner the degree to which the criterion is satisfied in any specific instance. Some early approaches [18] defined 'conjunctive' and 'disjunctive' decision models that attempted to capture the logic of Boolean algebra, but were in effect transformations of traditional compensatory models. An appealing mechanism to realize the most characteristic properties of Boolean logic while at the same time permitting quantification relies on fuzzy set theory [19]. In such a formulation, each attribute is represented as a fuzzy set, with degree of membership indicating the extent to which the attribute describes the entity. The rules of fuzzy set manipulation are than used to form the fuzzy set of items satisfying the criterion for acceptance. Once we accept the rules of fuzzy set theory for combining attribute values, the difficulty remains of evaluating the
A parametric fuzzy-set prediction model
133
membership function for each attribute on each entity in the decision space. In the simplest case, we may have raw scores for each attribute, and desire the optimal transformation taking that value into a membership value. For example, if verbal graduate record examination scores are used to assess verbal ability, we need a mechanism for transforming such a score into a value denoting membership in the set of applicants with excellent verbal aptitude. Bookstein discussed this problem in the context of information retrieval [20] and then more formally in [21]. In that approach, one defines a class of functions, fa, distinguished by a vector parameter, a, that transforms raw ability values into permissible m e m b e r ship function values; one then determines the values of a by fitting past data, much as one would determine the coefficients of a regression model. Thus one begins with (a) a Boolean model relating entity attributes to the predicted performance of an entity, (b) raw data estimating how well the attributes describe each entity, and (c) past data. The parameterized transformation functions, referred to as 'filter functions' in [20, 21], define a parameterized fuzzy set model, and optimal values for the parameters are sought and used for future decisions. Using different filtering functions for each of the attributes in effect allows us to change the relative impact of each attribute. The basic assumption underlying a Boolean model is that an individual's potential to succeed at some task may not be necessarily determined by the cumulative action of many attributes, as implied by the regression model. It may, for example, only depend on the level of the single best ability that the person has. In such a case, a 'disjunctive' model would be more suitable than a compensatory model. A n o t h e r possibility is that an individual's ability to succeed is determined by the extent to which he exceeds certain minimum levels in each of various significant attributes. In such a case, a 'conjunctive' model would be more appropriate. The method we are describing brings together the criteria that are important for the decision while retaining the properties commonly associated with the connectives 'AND' and 'oR'. To compare the predictive ability of these two models against the compensatory model, an analysis was done of the records of 88 students from the University of Chicago Graduate Library School, representing two classes enrolled in Autumn 1981 and Autumn 1982. The predictors used are the undergraduate grade point average (UGPA), and the G R E verbal and quantitative scores ( G R E V and G R E Q , respectively); the graduate grade point average (GGPA) was used as the success criterion. We first compared how well the various models fit the data describing one of the classes (48 students), and then used those best fit parameter values to test how well these models would have predicted performance for the second class (40 students). The main justification for using such simple models is the exploratory character of this research: to show that techniques initimately based on fuzzy-set theory can compete well with established prediction techniques. But it is also the case that variables of practical value are often difficult to predict, and that a relatively small number of predictor variables capture the bulk of the variance that can be explained at all. This is certainly true of performance in academic programs: using a large range of variables explains a modest amount of the variances and the two or three most important variables perform almost as
134
A. Bookstein, K.K. Ng
well as the full set. Thus the models being tested here, while simple, are not in fact as limited as might appear so far as exploiting the available useful information.
3. Definitions and notations
For the purpose of this study, we need a number of notations and definitions. We begin with a (traditional) set of students, each evaluated on a number of attributes believed useful in predicting each student's performance. If we have n students and N attributes, then each attribute can be represented as a vector with n components as follows:
ii,
The vector of dimension N, associated with the i-th attribute; each component is the score of a student on that attribute. The score of the j-th student is given by:
Aii
The j-th component of the i-th vector; although Aii may take an arbitrary value, we will ultimately transform Aii to a number in [0, 1]. This restriction will allow us to conceptualize each attribute as a fuzzy set of s t u d e n t s - the set of students 'succeeding' on the i-th attribute. Thus Aii will become the extent to which the j-th student has done well on (is 'in') attribute i. For Aii in [0, 1], we define:
(n,'=, Ai)i
The j-th component of the intersection of N vectors, where intersection, defined by ( ~ = 1 fi,i)i = mini {Aii}, is the extent to which the j-th student excels in all of the N attributes. This definition, and the following, is conventional in the theory of fuzzy sets and can be shown to be required for consistency with the laws of logic [23].
(U~o, A.i)~
The j-th component of the union of N vectors, where union is defined by ( U ~1 fiti)i = maxl {A~j}, again as required by the theory of fuzzy sets. A filtering function such that f:x ~ y, y e [ 0 , 1]; f is monotonically non-decreasing. Generally, scores are initially given in arbitrary units, and we must transform these scores into values of a membership function, that is, into the attributes referred to above.
We begin our analysis with raw scores for the predictors G R E V , G R E Q , and U G P A . A filtering function is then applied to produce scores between 0 and 1 as required by our model; these form the attributes used in the analysis. The filters may be defined aribtrarily. To create the filters we shall use, it is assumed that cut-off values could be defined so that students above the upper threshold 'pass' the criterion (are fully in the set represented by the attribute) and those below fail (are not members of the set). Degrees of membership are, for simplicity, assumed to vary linearly with raw scores between these thresholds.
A parametric [uzzy-set prediction model
135
4. Methodology W e found it convenient for interpretation to define the filters in two stages. First, the scores of the students were standardized to the interval [0, 1]. T h e following formulas were used for the standardization: SUGPA SGREV -
UGPA-
1.5
2.5
SGGPA =
'
G R E V - 200 600
'
SGREQ =
GGPA-
1.5
2.5 G R E Q - 200 600
The constants appearing in these formulas were based on subjective estimates of the lowest values the variables were likely to take and their range. The standardization is not essential, but useful for interpreting the results. Then, a filtering function, taking the following form, is applied to the standardized score:
I
0,
X ~ 01,
x-~
f(x) = | 0 2 - 01' 01 < x < 02, t.1,
x~02,
where x denotes one of the standardized test scores, and 01 and 02 are p a r a m e t e r s of the function that determine the lower and the u p p e r 'cut-off' points. It should be noted that 01 must be less than 02, but neither 01 nor 02, each of which is fitted from the data, is constrained to lie in the interval [0, 1]. For the conjunctive model, the predicted standardized G G P A is determined by the values of the c o m p o n e n t s of the intersection of the filtered test scores -Ai, i.e.
H e r e we are in effect defining a new fuzzy set, F, of students who are predicted to do well. This model says that students who are expected to do well are those who pass on each attribute. For the disjunctive model, prediction of success is determined by the values of the c o m p o n e n t s of the union of the filtered test scores
Ai, i.e.
that is, a student is expected to do well if he passes at least one of the predictors of success. In the a b o v e notations, Fj will ultimately be c o m p a r e d to the filtered G G P A for student j, which is taken to define the set of successful students. The O's in these filtering functions are unknown quantities. W e would like to find the combination of the values of the 0's that optimizes the association between the predicted and the actual G G P A . The software package used for this purpose is the s'mPrr routine. This is a subroutine used in conjunction with a
136
A. Bookstein, K.K. Ng
FOre-RAN program running on an IBM machine. It is designed for finding local minima of a smooth function of several parameters using a direct search method (without using derivates) [22]. The parameters for this study are the six cut-off values, two each for the three filters. In fitting our model, we used the G G P A , standardized to take values between zero and one (SGGPA), as the success criterion. The objective function, to be minimized, was the sum of squares of the difference between S G G P A , interpreted as the actual degree of membership in the set of successful students, and F, the predicted standardized G G P A . In notational form we have, for SS, the objective function, SS = ~ ( S G G P A i - Fi) 2, i=1 where n is the number of students in the study. SS depends on the parameters of the model. We also computed the correlation between S G G P A and F, using the cut-off values found above. The model was cross-validated by testing the parameter values on fresh data. Finally, for comparison purposes, a linear regression model was used, with G G P A as the criterion, and then cross-validated on the same set of data as the fuzzy set models.
5. Results Table 1 shows the mean and standard deviation for the data in group 1 and group 2. Table 2 shows the correlation coefficients between the individual predictors and G G P A . It is obvious that, for the second group, the individual predictors are more highly correlated with G G P A than the first group, although the two groups have very similar values for the means and standard deviations of the raw scores. The regression analysis on group 1 shows that, when all three predictors are used, G R E V is not a significant predictor, having a t-value of only - 0 . 7 ; this Table 1. Mean and standard deviation of the 4 test scores Standard Variable Group Mean deviation UGPA GREV GREQ GGPA
1 2 1 2 1 2 1 2
3.4 0.4 3.2 0.5 650.0 90.0 650.0 90.0 560.0 100.0 560.0 100.0 3.3 0.4 3.3 0.4
A parametric fuzzy-set prediction model
137
Table 2. Correlation between the 3 predictors and GGPA Group 1 Group UGPA 0.3 GREV 0.2 GREQ 0.3
2
0.4 0.4 0.6
result is surprising, given the generally perceived qualitative character of librarianship, but is consistent with other informal studies carried out in the School. G R E V is therefore dropped from the regression analysis. The final result is as follows: S G G P A = 0.37 + 0.25 S U G P A + 0.25 S G R E Q where S G G P A , S U G P A , and S G R E Q are defined above. The multiple correlation coefficient, R, is 0.4. If we convert both the predicted and the observed S G G P A to ranks, the S p e a r m a n coefficient is 0.5. The best result obtained for the conjunctive model yields a correlation coefficient of 0.36 between the predicted and the observed S G G P A . A detailed examination of the data reveals that the predicted value is not sensitive to the U G P A or G R E V scores for the fitted v a l u e s - w e could vary the values for U G P A and G R E V without influencing the prediction. T h a t is, using the conjunctive model, it was found that, with few exceptions, the filtered G R E Q alone determined the predicted S G G P A for the model and there is negligible improvement in using the full conjunctive model over a simple model using G R E Q alone as the predictor. The disjunctive model looks m o r e promising. The best result yields a correlation coefficient of 0.5. Cut-off values that give this result a p p e a r in Table 3, though the model is not sensitive to the lower cut-off value. Converting the predicted and the observed S G G P A to ranks, the Spearman coefficient is 0.5. A question that arises in traditional regression analysis is which variable is the most ' i m p o r t a n t ' . A similar question can be posed here for the disjunctive model, with at least as much difficulty in providing an answer. Unlike regression, in which all the variables cooperate to yield a prediction, in a Boolean model, one variable dominates and determines the prediction. This suggests our first criterion of importance: that variable is most important which will most easily dominate the other variables. Consider for example, the m e m b e r s h i p level of 0.7. This would translate into a raw score of 3.48 for U G P A , 763 for G R E V , and 620 for G R E Q . Table 3. Optimum cut-off points for the disjunctive model UGPA GREV GREQ Lower cut-off point Upper cut-off point
0.0 0.005 1 . 1 5 2 1.338
-0.330 1.146
Correlation coefficients 0.5270
138
A. Bookstein, K.K. Ng
Table 4. Comparison of the 3 models for group 1 Correlation coefficients
Regression model Disjunctive model Conjunctive model
Scores
Rank
0.44 0.53 0.36
0.51 0.52 0.46
T h a t is, in order for U G P A to have the same strength as a score of 620 for G R E Q , the U G P A score has to be 3.48; similarly, the G R E V score has to be 763. In terms of our population, 41% of the U G P A are above 3.48, but only 28% of the G R E Q are above 620, and 4% of the G R E V are above 763. This criterion emphasizes the importance of U G P A . In fact, in our population, 63% of the time it was the U G P A that determined the prediction. A n o t h e r criterion of importance is the slope of the filter, assuming the reasonableness of the standardization. For, if we select values for the predictors for which each predictor suggests the same value for the G G P A , then a unit increase in the predictor with the greatest slope will have m o r e of an impact than will a unit increase in the other variables. By this criterion, which is closest to regression case, U G P A again seems most important. In summary, Table 4 compares the correlation coefficients obtained from the three different models. T h e first column shows the correlation between the observed and the predicted S G G P A for the three models. The second column shows the S p e a r m a n coefficient if we convert both the observed and predicted values to ranks. When predicting filtered scores using the least square error criterion, the disjunctive model is clearly superior to the regression model. W h e r e predicting ranks is concerned, the regression model is nearly as good as the disjunctive model. T h e conjunctive model is much inferior to either the disjunctive or the regression model on both criteria. To test the predictive ability of the three models, the predictors from group 2 were processed using these models to find the predicted G G P A . The resultant correlation coefficients are shown in Table 5, which has the same f o r m a t as Table 4. We see that the regression model is better on both criteria. T h e conjunctive model performs about as well as the disjunctive model, which is due to the high Table 5. Comparison of the 3 models for group 2 Correlation coefficients
Regression model Disjunctive model Conjunctive model
Scores
Rank
0.64 0.62 0.58
0.69 0.62 0.61
139
A parametric [uzzy-set prediction model
Table 6. Group 1 error estimates for the 3 models Mean absolute error Regression model Disjunctive model Conjunctive model
0.11 (s.d. 0.08) 0.10 (s.d. 0.07) 0.12 (s.d. 0.08)
Mean absolute percentage error with observed score 17% (s.d. 18) 16% (s.d. 15) 19% (s.d. 18)
Mean absolute percentage error with predicted score 16% (s.d. 12) 15% (s.d. 11) 17% (s.d. 11)
c o r r e l a t i o n b e t w e e n G R E Q a n d U G P A in t h e s e c o n d g r o u p . In all cases, t h e p r e d i c t i o n s o n t h e v a l i d a t i o n d a t a is a c t u a l l y i m p r o v e d , an a r t i f a c t o f t h e h i g h e r c o r r e l a t i o n in t h e d a t a b e t w e e n t h e c r i t e r i o n v a r i a b l e a n d t h e p r e d i c t o r s . A s a f u r t h e r c o m p a r i s o n , w e h a v e c o m p u t e d , f o r t h e case of filtered scores, t h e m e a n a b s o l u t e e r r o r , t h e m e a n p e r c e n t a g e e r r o r with t h e o b s e r v e d score, a n d t h e p e r c e n t a g e e r r o r with t h e p r e d i c t e d score. T h e results a r e s h o w n in T a b l e 6 a n d T a b l e 7. A g a i n , t h e d i s j u n c t i v e m o d e l p e r f o r m e d a b o u t as well as t h e r e g r e s s i o n model.
6. Conclusion T h e c o n j u n c t i v e m o d e l , w h e n fitted, has a n e g l i g i b l e i m p r o v e m e n t o v e r a s i m p l e o n e v a r i a b l e m o d e l using G R E Q as t h e p r e d i c t o r . T h e fit w i t h g r o u p 1, a l t h o u g h i n f e r i o r to the r e g r e s s i o n m o d e l , is n o t r e a l l y t h a t b a d since G R E Q is t h e v a r i a b l e m o s t h i g h l y c o r r e l a t e d with G G P A a m o n g t h e t h r e e p r e d i c t o r s . H o w e v e r , n o t only are the mean absolute prediction error and the mean absolute percentage e r r o r b i g g e r t h a n f o r t h e d i s j u n c t i v e m o d e l , t h e y also h a v e a w i d e r s p r e a d t h a n t h e r e g r e s s i o n m o d e l . It s e e m s t h a t t h e logic t h a t for a s t u d e n t to d o well he m u s t e x c e e d s o m e t h r e s h o l d in e a c h of s e v e r a l v a r i a b l e s is i n a d e q u a t e , at l e a s t f o r the v a r i a b l e s t e s t e d ; w h e n f o r c e d to a p p l y t h a t logic, o u r p r o g r a m s i m p l y t o o k t h e single b e s t v a r i a b l e m o s t of t h e t i m e . Table 7. Group 2 error estimates for the 3 models
Regression model Disjunctive model Conjunctive model
Mean absolute error
Mean absolute percentage error with observed score
0.10 (s.d. 0.08) 0.11 (s.d. 0.08) 0.11 (s.d. 0.08)
17% (s.d. 21) 17% (s.d. 17) 19% (s.d. 22)
Mean absolute percentage error with predicted score 15% (s.d. 13) 16% (s.d. 12) 16% (s.d. 12)
A. Bookstein, K.K. Ng
140
The disjunctive model, on the other hand, fits the data of group 1 better than the regression model. The i m p r o v e m e n t is m o r e than 2 0 % if we c o m p a r e the correlation coefficients for the scores. For predicting the p e r f o r m a n c e of the students of group 2, the disjunctive model is not as good as the regression model. However, the differences are less than 5% if we c o m p a r e the correlation coefficients for scores again. C o m p a r i n g the m e a n absolute error and the m e a n absolute percentage error we find again that the two models p e r f o r m about equally well. But the variance for all three error measures are smaller for the disjunctive model than for the regression model. Thus, although the two models are quite close on the average in predicting students' p e r f o r m a n c e , the disjunctive model appears to be m o r e consistent in the prediction. The results of ttiis study look quite encouraging. It gives rather strong evidence that the disjunctive model is c o m p a r a b l e in robustness to the regression model in predicting students' p e r f o r m a n c e in graduate school. But the disjunctive model, which utilizes solely the fuzzy set operation of union, is only a special case of the general Boolean model [21]. T o take full advantage of the Boolean model, other operations, such as c o m p l e m e n t and intersection, may be included to give models such as UGPA U (GREV n GREQ) or (UGPA U GREV) n GREQ. In addition, other predictors, such as previous work experience, letter of reference, or college activities, m a y also be included to build a m o r e complex model, and more complex filter transformations m a y be tested. The important conclusion from this work is that predictive models radically different from the heavily used regression models are available, and should be taken seriously.
Acknowledgement We wish to acknowledge the use in this work of subroutine STEPIT, written by J.P. Chandler and distributed by the Q u a n t u m Chemistry P r o g r a m Exchange.
References [ 1] N.W. Burton and N.J. Turner, Effectiveness of the Graduate Record Examination for predicting first-year grades. 1981-1982 summary report of the Graduate Record Examination Validity Study Service, Educational Testing Service, Princeton, NJ, Report #209021 (1983). [2] G.E. Tully, Screening applicants for graduate study with the aptitude test of the graduate record examinations, College and University 38 (1962) 51-60. [3] G.F. Madaus and J.J. Walsh, Departmental differentials in the predictive validity of the graduate record examination aptitude test, Educational and Psychological Measurements 25 (1965) 1105-1110. [4] S. Baillie, Library school and job success, Studies in Librarianship 1 (1964) 1-176. [5] M.E. Lamson, GRE under fire-again, Journal of Education for Librarianship 12 (1972) 175-177.
A parametric fuzzy-set prediction model
141
[6] L.L. Baird, Comparative prediction of first year graduate and professional school grades in six fields, Educational and Psychological Measurement 35 (1975) 941-946. [7] J.B. Katz, Indicators of success: Queens College Department of Library Science, Journal of Education for Librarianship 19 (1978) 130-139. [8] R.M. Magrill and C. Rinehart, Success in library school: a study of admission variables, Journal of Education for Librarianship 19 (1979) 203-222. [9] R.I. Blue and J.L. Divilbiss, Optimizing selection of library school students, Journal of Education for Librarianship 21 (1981) 301-312. [10] L. Auld, GRE analytical ability as an admission factor, Library Quarterly 54 (1984). [11] R.M. Dawes, Graduate admission variables and future success, Science 30 (1975) 721-723. [12] H. Gulliksen, Harold, Theory of Mental Tests (Wiley, New York, 1950). [13] F.M. Lord and M.R. Novick, Statistical Theories of Mental Test Scores, (Addison-Wesley, MA, 1968). [14] N. Givner and K. Hynes, Admissions test validity: correcting for restriction effects, College and University 54 (1979) 119-123. [15] A. Gullickson and K. Hopkins, Interval estimation of correlation coefficients corrected for restriction of range, Educational and Psychological Measurement 36 (1976) 9-25. [16] D.W. Perrin, Admissions test validity: additional corrections for restriction effects, College and University 55 (1980) 181-185. [17] V. Srinivasan and A.G. Weinstein, Effects of curtailment of an admission model for a graduate management program, Journal of Applied Psychology 58 (1973) 339-346. [18] H.J. Einhorn, The use of nonlinear, noncompensatory models in decision making, Psychological Bulletin 73 (1970) 221-230. [19] L.A. Zadeh, Fuzzy sets, Information and Control 8 (1965) 338-353. [20] A. Bookstein, The generalized retrieval problem, ACM SIGIR Forum, 16 (1981) 4-14. [21] A. Bookstein, Fuzzy-set decision models, in: J. Kacprzyk and R.R. Yager, Eds., Management Decision Support Systems Using Fuzzy Sets and Possibility Theory (TUV-verlag, Collogne, to appear). [22] J.P. Chandler, s'rEorr-Direct search for optimization; solution of least squares problems, Quantum Chemistry Program Exchange, QCPE Program No. 307, Chemistry Department, Indiana University. [23] R. Bellman and M. Giertz, On the analytic formalism of the theory of fuzzy sets, Information Sciences 5 (1973) 149-56.