- Email: [email protected]
Evaluating students’ learning achievement based on the eigenvector method
Expert Systems with Applications 38 (2011) 8240–8250
Contents lists available at ScienceDirect
Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa
Evaluating students’ learning achievement based on the eigenvector method Shyi-Ming Chen ⇑, Ting-Kuei Li Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan, ROC
a r t i c l e
i n f o
Keywords: Eigenvector method Fuzzy sets Learning achievement index Membership functions Students’ learning achievement evaluation
a b s t r a c t In this paper, we present a new method for students’ learning achievement evaluation based on the eigenvector method. The proposed method considers the ‘‘accuracy rate’’, the ‘‘time rate’’, the ‘‘importance’’ and the ‘‘complexity’’ for evaluating students’ learning achievement. First, the proposed method transforms the attributes ‘‘accuracy rate’’ and ‘‘time rate’’ into the ‘‘effect of accuracy rate’’ and the ‘‘effect of time rate’’, respectively. Then, it generates the relative importance degrees of the attributes ‘‘effect of accuracy rate’’, ‘‘effect of time rate’’, ‘‘importance’’ and ‘‘complexity’’ based on the eigenvector method. Then, it uses the correlation coefficients between the attribute vectors and the standard deviations of the elements in the attribute vectors to calculate the fitness degrees of the attributes, where the attribute vectors represent the relationships between the attributes and the questions. Then, it generates the weights of the attributes based on the relative importance degrees of the attributes and the fitness degrees of the attributes. Then, it generates the importance degrees of the questions according to the weights of the attributes and the relation matrix representing the relationships between the questions and the attributes. Finally, based on the importance degrees vector of the questions, the grade matrix, the accuracy rate matrix, it calculates the learning achievement index of each student having the same original total score for students’ learning achievement evaluation. The proposed method provides us with a useful way for students’ learning achievement evaluation based on the eigenvector method. Ó 2011 Elsevier Ltd. All rights reserved.
1. Introduction In recent years, the fuzzy set theory (Zadeh, 1965) has been widely used in educational grading systems (Bai & Chen, 2008a, 2008b, 2008c; Biswas, 1995; Chen & Lee, 1999; Chen & Li, 2010; Echauz & Vachtsevanos, 1995; Law, 1996; Ma & Zhou, 2000; Wang & Chen, 2008; Weon & Kim, 2001; Wilson, Karr, & Freeman, 1998; Wu, 2003). Bai and Chen (2008a) presented a method for automatically constructing concept maps based on fuzzy rules and students’ testing records. It applied fuzzy rules and fuzzy reasoning techniques to automatically construct concept maps and evaluate the relevance degrees between concepts. Bai and Chen (2008b) presented a method to automatically construct the grade membership functions of lenient-type grades, strict-type grades and normal-type grades given by teachers, and then perform fuzzy reasoning to infer the scores of students. Biswas (1995) presented two methods for evaluating students’ answer-scripts based on fuzzy sets and a matching function. However, Biswas’ methods have the drawback that different fuzzy marks will be transformed into the same score. Chen and Lee (1999) presented two methods for applying fuzzy sets to evaluate students’ answer-scripts to overcome the drawback of Biswas’s methods (1995). Echauz and ⇑ Corresponding author. Tel.: +886 2 27376417; fax: +886 2 27301081. E-mail address: [email protected] (S.-M. Chen). 0957-4174/$ - see front matter Ó 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2011.01.003
Vachtsevanos (1995) presented a fuzzy grading system using students’ and instructor’s performance measures in order to modify a set of collective, approved and a priori fuzzy grades to produce a fair mark distribution. Law (1996) presented a fuzzy educational grading system, not only to assist teachers to aggregate different test scores to produce a single score or grade for students, but also to help teachers deciding whether to revise an instructional procedure for their students. Ma and Zhou (2000) presented an integrated fuzzy set approach to assess the outcomes of students-centered learning based on fuzzy set principles to represent the imprecise concepts for subjective judgment and incorporated the students’ opinions into assessments. Wang and Chen (2008) presented two methods for evaluating students’ answer-scripts using fuzzy numbers associated with the degrees of confidence, where the satisfaction levels given by the evaluator awarded to the questions of the students’ answer-scripts are represented by triangular fuzzy numbers associated with degrees of confidence between zero and one. Wilson et al. (1998) presented a flexible and adaptive grade assignment system based on fuzzy rules and genetic algorithms. Wu (2003) presented a method for evaluating learning performance of students by applying the fuzzy set theory and the item response theory to integrate the objective and the subjective information obtained from exams. Weon and Kim (2001) pointed out that an educational grading system should consider the ‘‘difficulty’’, ‘‘importance’’ and ‘‘complexity’’ of questions
S.-M. Chen, T.-K. Li / Expert Systems with Applications 38 (2011) 8240–8250
in students’ answer-scripts learning achievement evaluation. They also presented a method for students’ learning achievement using fuzzy membership functions. Bai and Chen (2008c) presented a method for evaluating students’ learning achievement using fuzzy membership functions and fuzzy rules. They pointed out that ‘‘difficulty’’ is a very subjective parameter and to adjust the scores of students during the evaluation process is not appropriate. They used the accuracy of the students’ answer-scripts and the time consumed to answer the questions to obtain the difficulty of questions based on fuzzy reasoning techniques. However, the drawback of Bai and Chen’s method (2008c) is that the weights of the attributes in Bai and Chen’s method are assigned subjectively by domain experts. Once the relationships between the attributes and the questions are changed, the weights of the attributes will still be the same as before. It is not appropriate to determine the new ranking order of the students having the same original total score using the same weights of the attributes as before. In this paper, we present a new method for students’ learning achievement evaluation based on the eigenvector method. The proposed method considers the ‘‘accuracy rate’’, the ‘‘time rate’’, the ‘‘importance’’ and the ‘‘complexity’’ for evaluating students’ learning achievement. First, the proposed method transforms the attributes ‘‘accuracy rate’’ and ‘‘time rate’’ into the ‘‘effect of accuracy rate’’ and the ‘‘effect of time rate’’, respectively. Then, it generates the relative importance degrees of the attributes ‘‘effect of accuracy rate’’, ‘‘effect of time rate’’, ‘‘importance’’ and ‘‘complexity’’ based on the eigenvector method. Then, it uses the correlation coefficients between the attribute vectors and the standard deviations of the elements in the attribute vectors to calculate the fitness degrees of the attributes, where the attribute vectors represent the relationships between the attributes and the questions. Then, it generates the weights of the attributes based on the relative importance degrees of the attributes and the fitness degrees of the attributes. Then, it generates the importance degrees of the questions according to the weights of the attributes and the relation matrix representing the relationships between the questions and the attributes. Finally, based on the importance degrees vector of the questions, the grade matrix and the accuracy rate matrix, it calculates the learning achievement index of each student having the same original total score for students’ learning achievement evaluation. The proposed method provides us with a useful way for students’ learning achievement evaluation based on the eigenvector method. The rest of this paper is organized as follows. In Section 2, we briefly review Bai and Chen’s method (2008c) for students’ learning achievement evaluation. In Section 3, we present a new method for students’ learning achievement evaluation based on the eigenvector method. In Section 4, we use an example to illustrate the proposed method and to compare the results of the proposed method with Bai and Chen’s method (2008c). The conclusions are discussed in Section 5.
where aij represents the relative importance degree of the attribute ATi to the attribute ATj, wi denotes the importance degree of the attribute ATi, wi/wj represents the relative importance degree of the attribute ATi to the attribute ATj, n denotes the number of attributes, aij P 0, aij = 1/aji, aii = 1, 1 6 i 6 n and 1 6 j 6 n. The relative importance degree aij is defined by the fuzzy scales ranging from 1 to 9, shown as follows: 1: 3: 5: 7: 9:
equally important weakly more important strongly more important demonstratively more important absolutely more important
where 2, 4, 6 and 8 are intermediate values between two adjacent judgments. After multiplying the reciprocal matrix AR by the importance degree vector WI = (w1, w2, . . . , wn), we can get
Therefore, (AR nI)WI = 0, where I is an identity matrix. Then, it can obtain the relative importance degrees of the attributes by solving the eigenvalues and the eigenvectors of the reciprocal matrix AR. Given a matrix A, a non-zero vector x is defined to be an eigenvector (Greub, 1975) of the matrix A if it satisfies the eigenvalue equation Ax = kx for some scalar k, where k is called an eigenvalue of the matrix A corresponding to the eigenvector x. The eigenvalues and the eigenvectors are calculated as follows (Greub, 1975): Step 1: Solve the equation det (A kI) = 0 and get the eigenvalues k1, k2 , . . . , where ‘‘det’’ is defined as the determinant of a square matrix and I is an identity matrix. Step 2: For each eigenvalue ki, solve the equation (A ki I) x = 0 and get the eigenvectors with respect to the eigenvalue ki, where i = 1, 2, . . .. 13 4 Example 2.1. Let us consider the matrix A ¼ . Then, we 4 7 can get
detðA kIÞ ¼ 2. A review of the eigenvector method The eigenvector method (Chen & Hwang, 1992) is useful to generate the relative importance degrees of attributes. Assume that there are n attributes AT1, AT2, . . . , and ATn. Let AR be a positive reciprocal matrix representing the relationships between the attributes, shown as follows:
8241
13 k 4 ¼ ð13 kÞð7 kÞ ð4Þð4Þ 4 7k
¼ 75 20k þ k2 ¼ 0: Because 75 20k + k2 = (5 k)(15 k) = 0, we get two eigenvalues k1 = 5 and k2 = 15. 13 5 4 x1 For k1 = 5, we have ðA 5IÞx ¼ ¼ 4 7 5 x2 8 4 x1 ¼ 0: That is, x1 = 2x2. The solutions of (A 5I)x = 0 4 2 x2 1 are x ¼ c , where c is an arbitrary value. For example, when 2 1 ; when c = 2, we can get c = 1, we can get an eigenvector v 1 ¼ 2 2 . an eigenvector v 1 ¼ 4
8242
S.-M. Chen, T.-K. Li / Expert Systems with Applications 38 (2011) 8240–8250
13 15 4 For k2 = 15, we have ðA 15IÞx ¼ 4 7 15 2 4 x1 x1 ¼ ¼ 0: That is x1 = 2x2. The solutions of 4 8 x2 x2 2 (A 15I)x = 0 are x ¼ c , where c is an arbitrary value. For 1 2 ; when example, when c = 1, we can get an eigenvector v 2 ¼ 1 4 . c = 2, we can get an eigenvector v 2 ¼ 2
3. A new method for students’ learning achievement evaluation based on the eigenvector method In this section, we present a new method for students’ learning achievement evaluation based on the eigenvector method (Chen & Hwang, 1992). Assume that there are m questions and assume that there are n students to answer these questions. Let Qi denote the ith question in the answerscript and let Sj denote the jth student, where 1 6 i 6 m and 1 6 j 6 n. Then, we can get an accuracy rate matrix A and a time rate matrix T, shown as follows:
where aij denotes the accuracy rate of the jth student Sj with respect to the ith question Qi, aij 2 [0,1], tij denotes the answer-time rate of the jth student Sj with respect to the ith question Qi, tij 2 [0, 1], 1 6 i 6 m and 1 6 j 6 n. Let G be a score matrix shown as follows:
g1
3
Q2 G¼ . ..
6g 6 2 6 6 .. 4 .
7 7 7; 7 5
Qm
gm
Q1
2
where imi1, imi2, imi3, imi4 and imi5 denote the grades of membership that the degree of importance of the ith question Qi belongs to the importance levels ‘‘low’’, ‘‘more or less low’’, ‘‘medium’’, ‘‘more or less high’’ and ‘‘high’’, respectively, imi1 2 [0,1], imi2 2 [0,1], imi3 2 [0, 1], imi4 2 [0,1], imi5 2 [0,1] and 1 6 i 6 m; ci1, ci2, ci3, ci4 and ci5 denote the grades of membership that the complexity of question Qi belongs to the complexity levels ‘‘low’’, ‘‘more or less low’’, ‘‘medium’’, ‘‘more or less high’’, and ‘‘high’’, respectively, ci1 2 [0, 1], ci2 2 [0, 1], ci3 2 [0, 1], ci4 2 [0, 1], ci5 2 [0, 1] and 1 6 i 6 m. In order to distinguish the difference degree of the accuracy rate of the students for each question and in order to distinguish the difference degree of the time rate of the students for each question, we use the standard deviation equation to transform the accuracy rate matrix A into the effect of accuracy rate vector EOA and to transform the time rate matrix T into the effect of time rate vector EOT. The larger the standard deviation of the accuracy rates of the students with respect to a question, the easier to distinguish the learning achievement of the students by means of that question; the larger the standard deviation of the time rates of the students with respect to a question, the easier to distinguish the learning achievement of students by means of that question. Therefore, we can use the attributes vectors EOA and EOT to distinguish the learning achievement of the students having the same original total score. The proposed method is now presented as follows: i of question Qi, Step 1: Calculate the average accuracy rate a shown as follows:
i ¼ a
n X
aij =n;
ð2Þ
j¼1
where gi denotes the score of the ith question Qi, gi 2 [0, 100], 1 6 i 6 m and m X
Calculate the average time rate t i of question Qi, shown as follows:
t i ¼
g i ¼ 100:
n X
t ij =n:
ð3Þ
j¼1
i¼1
Based on the accuracy rate matrix A and the score matrix G, we can calculate the total score TSj of each student Sj, shown as follows:
TSj ¼
m X
aij g i ;
ð1Þ
i¼1
where 1 6 j 6 n. The original ranking order of the students are obtained by sorting the values of TS1, TS2, . . . , and TSm in a descending order. Let IM be an importance matrix and let C be a complexity matrix given by a domain expert for each question Qi, respectively, shown as follows:
Based on the accuracy rate matrix A and the time rate matrix T, calculate the standard deviation eoai of the accuracy rates ai1, ai2, . . . , and ain of the students S1, S2, . . . , and Sn, respectively, with respect to question Qi, where 1 6 i 6 m, to obtain the effect of accuracy rate vector EOA and calculate the standard deviation eoti of the time rates ti1, ti2, . . . , and tin of the students S1, S2, . . . , and Sn, respectively, with respect to question Qi, where 1 6 i 6 m, to obtain the effect of time rate vector EOT, where
EOA ¼
Q1 ½eoa1
Q2 eoa2
Qm ; eoam
8243
S.-M. Chen, T.-K. Li / Expert Systems with Applications 38 (2011) 8240–8250
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u X u1 n i Þ2 ; eoai ¼ t ðaij a n j¼1 EOT ¼
Q1
Q2
½eot1
eot2
ð4Þ
Qm
eotm
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u X u1 n ðtij ti Þ2 ; eoti ¼ t n j¼1
;
ð5Þ
P P i ¼ nj¼1 aij =n, t i ¼ nj¼1 tij =n and 1 6 i 6 m. Transform the where a complexity matrix C into the crisp complexity vector CC and transform the importance matrix IM into the crisp importance vector CI, shown as follows:
CC ¼
cci ¼
CI ¼
cii ¼
Q1
Q2
Qm
½cc1
cc2
ccm
Step 2: Based on the relation matrix R obtained in Step 1, calculate the summation of the elements in the relation matrix R to obtain the vector SA, shown as follows:
P where saj ¼ m i¼1 r ij and 1 6 j 6 4. Normalize the vector SA to get the normalized vector SA’, shown as follows:
sai where sa0j ¼ maxðsa1 ;sa and 1 6 j 6 4. Based on the normalized 2 ;sa3 ;sa4 Þ vector SA0 and Eq. (8), transform the relation matrix R into the reciprocal matrix RE, shown as follows:
;
0:1 ci1 þ 0:3 ci2 þ 0:5 ci3 þ 0:7 ci4 þ 0:9 ci5 ; ci1 þ ci2 þ ci3 þ ci4 þ ci5 ð6Þ Q1
Q2
½ci1
ci2
cim
Qm
;
0:1 imi1 þ 0:3 imi2 þ 0:5 imi3 þ 0:7 imi4 þ 0:9 imi5 ; imi1 þ imi2 þ imi3 þ imi4 þ imi5 ð7Þ
1 6 i 6 m and 0.1, 0.3, 0.5, 0.7, 0.9 are ‘‘the right point’’, ‘‘the center point’’, ‘‘the center point’’, ‘‘the center point’’ and ‘‘the left point’’ of the fuzzy sets ‘‘low’’, ‘‘more or less low’’, ‘‘medium’’, ‘‘more or less high’’ and ‘‘high’’, respectively, as shown in Fig. 1. Based on the effect of accuracy rate matrix EOA, the effect of time rate matrix EOT, the crisp complexity vector CC and the crisp importance vector CI, construct the relation matrix R, shown as follows:
m 8 l > sa0i sa0j 10 ; if sa0i > sa0j > > < if sa0i ¼ sa0j ; reij ¼ 1; > l m > > : 1= sa0i sa0j 10 ; if sa0i < sa0j
ð8Þ
where reij = 1/reji, reii = 1, 1 6 i 6 4 and 1 6 j 6 4. Use the Matlab 7.0 to solve the reciprocal matrix RE to get the maximum eigenvalue kmax of the reciprocal matrix RE and its corresponding eigenvector E, shown as follows:
2 3 Accuracy Rate e1 7 Time Rate 6 6 e2 7 E¼ 6 7: Complexity 4 e3 5 Importance
e4
Based on the eigenvector E, calculate the percentage of the eigenvector E to obtain the percentage vector P of the eigenvector E, shown as follows:
2 3 Accuracy Rate p1 7 Time Rate 6 6 p2 7 P¼ 6 7; Complexity 4 p3 5 Importance ei
where pi ¼ P4
e j¼1 j
p4
; pi 2 ½0; 1 denotes the relative importance degree
of the ith attribute and 1 6 i 6 4. That is, the relative importance degrees of the attributes ‘‘Accuracy Rate’’, ‘‘Time Rate’’, ‘‘Complexity’’ and ‘‘Importance’’ are p1, p2, p3 and p4, respectively, where pi 2 [0, 1] and 1 6 i 6 4. Step 3: Based on the relation matrix R obtained in Step 1, get the attribute vectors R1, R2, R3 and R4, shown as follows:
3 2 Q 1 r12 7 6 Q2 Q 2 6 r22 7 7 7 7; R2 ¼ . 6 . 7; R1 ¼ . 7 7 .. .. 6 5 4 .. 5 Q m r m1 Q m r m2 3 3 2 2 Q 1 r 13 Q 1 r14 7 7 Q2 6 Q2 6 6 r 23 7 6 r24 7 7; R4 ¼ . 6 . 7; R3 ¼ . 6 . 7 7 .. 6 .. 6 4 .. 5 4 .. 5 Q1
Fig. 1. Membership functions of the fuzzy sets ‘‘low’’, ‘‘more or less low’’, ‘‘medium’’, ‘‘more or less high’’ and ‘‘high’’, respectively.
Qm
2
r 11 6r 6 21 6 . 6 . 4 .
r m3
3
Qm
r m4
ð9Þ
8244
S.-M. Chen, T.-K. Li / Expert Systems with Applications 38 (2011) 8240–8250
where the attribute vector R1 denotes the relation between the attribute ‘‘Accuracy Rate’’ and the questions, the attribute vector R2 denotes the relation between the attribute ‘‘Time Rate’’ and the questions, the attribute vector R3 denotes the relation between the attribute ‘‘Complexity’’ and the questions and the attribute vector R4 denotes the relation between the attribute ‘‘Importance’’ and the questions. Calculate the correlation coefficient Cof(Ri, Rj) between the attribute vector Ri and the attribute vector Rj, shown as follows: m P
ðrki Ri Þðr kj Rj Þ=ðm 1Þ CofðRi ; Rj Þ ¼ sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffisffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ; m m P P ðr ki Ri Þ2 =ðm 1Þ ðr kj Rj Þ2 =ðm 1Þ
the value of Fit(Ri), the easier to distinguish the ranking order of the students having the same original total score based on the attribute vector Ri, where 1 6 i 6 4. Based on the fitness degree Fit(Ri), where 1 6 j 6 4, construct the percentage vector PF, shown as follows:
where
k¼1
k¼1
k¼1
ð10Þ P Pm where jCofðRi ; Rj Þj ¼ jCofðRj ; Ri Þj, Ri ¼ m k¼1 r ki =m, Rj ¼ k¼1 r kj =m, 1 6 i 6 3 and i 6 j 6 4. For the attribute vector R1, calculate the average of the correlation coefficient soc1 of the attribute ‘‘Accuracy Rate’’ with respect to the other attributes ‘‘Time Rate’’, ‘‘Complexity’’ and ‘‘Importance’’, respectively, shown as follows:
soc1 ¼ jCofðR1 ; R2 Þj þ jCofðR1 ; R3 Þj þ jCofðR1 ; R4 Þj:
soc2 ¼ jCofðR1 ; R2 Þj þ jCofðR2 ; R3 Þj þ jCofðR2 ; R4 Þj:
ð12Þ
For the attribute vector R3, calculate the average of the correlation coefficient soc3 of the attribute ‘‘Complexity’’ with respect to other attributes ‘‘Accuracy Rate’’, ‘‘Time Rate’’ and ‘‘Importance’’, respectively, shown as follows:
soc3 ¼ jCofðR1 ; R3 Þj þ jCofðR2 ; R3 Þj þ jCofðR3 ; R4 Þj:
ð13Þ
For the attribute vector R4, calculate the average of the correlation coefficient soc4 of the attribute ‘‘Importance’’ with respect to other attributes ‘‘Accuracy Rate’’, ‘‘Time Rate’’ and ‘‘Complexity’’, respectively, shown as follows:
soc4 ¼ jCofðR1 ; R4 Þj þ jCofðR2 ; R4 Þj þ jCofðR3 ; R4 Þj:
ð14Þ
Based on the attribute vector Ri, calculate the standard deviation Std(Ri) of the attribute vector Ri, shown as follows:
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u X u1 m StdðRi Þ ¼ t ðr ji Ri Þ2 ; n j¼1
pfi ¼ 1 and 1 6 j 6 4:
i¼1
3 2 Accuracy Rate w1 7 6 Time Rate 6 w2 7 7; 6 W¼ 7 6 Complexity 4 w3 5 Importance
w4
i where wi ¼ pi þpf , wi 2 [0, 1] denotes the weight of the ith attribute 2 and 1 6 i 6 4. That is, the weights of the attributes ‘‘Accuracy Rate’’, ‘‘Time Rate’’, ‘‘Complexity’’ and ‘‘Importance’’ are w1, w2, w3 and w4, respectively, where wi 2 [0, 1] and 1 6 i 6 4. Based on the weighting vector W of the attributes and the relation matrix R, use the simple additive weighting (SAW) method (Hwang & Yoon, 1981) to obtain the importance degree vector ID of the questions, shown as follows:
where
idi ¼ ri1 w1 þ r i2 w2 þ r i3 w3 þ r i4 w4 ;
ð17Þ
and 1 6 i 6 m. Step 5: Based on the importance degree vector ID of the questions, the grade matrix G, the and accuracy rate matrix A, calculate the learning achievement index LAj of student Sj having the same original total score, shown as follows:
LAj ¼
m X
aij g i idi ;
ð18Þ
i¼1
ð15Þ
P where Ri ¼ m j¼1 r ji =n and 1 6 i 6 4. Based on the average of the correlation coefficient soci and the standard deviation Std(Ri) of the attribute vector Ri, calculate the fitness degree Fit(Ri) of the attribute vector Ri, shown as follows:
1
B soci C C B FitðRi Þ ¼ B1 4 C StdðRi Þ; A @ P sock
4 X
Step 4: Based on the percentage vector P obtained in Step 2 and the percentage vector PF obtained in Step 3, calculate the weighting vector W of the attributes, shown as follows:
ð11Þ
For the attribute vector R2, calculate the average of the correlation coefficient soc2 of the attribute ‘‘Time Rate’’ with respect to other attributes ‘‘Accuracy Rate’’, ‘‘Complexity’’ and ‘‘Importance’’, respectively, shown as follows:
0
FitðRi Þ pfi ¼ P4 ; k¼1 FitðRk Þ
ð16Þ
k¼1
where Fit(Ri) denotes the fitness degree for using the attribute vector Ri to distinguish the ranking order of the students having the same original total score, Fit(Ri) 2 [0, 1] and 1 6 j 6 4. The bigger
where 1 6 j 6 n. The larger the value of LAj, the better the ranking order of student Sj, where 1 6 j 6 n. Sort the learning achievement indices of the students having the same original total score in a descending sequence to get the new ranking order of the students. 4. An example In this section, we use the example shown in Bai and Chen (2008c) to illustrate the process of students’ learning performance evaluation of the proposed method. Assume that there are five questions Q1, Q2, Q3, Q4 and Q5 in the answerscript and assume that there are 10 students S1, S2, . . . , and S10 to answer these questions. Assume that the accuracy rate matrix A, the time rate matrix T, the score matrix G, the importance matrix IM and the complexity matrix C are shown as follows:
S.-M. Chen, T.-K. Li / Expert Systems with Applications 38 (2011) 8240–8250
8245
Because TS9 > TS1 > TS2 > TS8 > TS4 = TS5 = TS10 > TS6 > TS7 > TS3, we can see that the ranking order of the students is S9 > S1 > S2 > S8 > S4 = S5 = S10 > S6 > S7 > S3, where the students S4, S5 and S10 have the same original total score. Then, we can apply the proposed method to deal with the new ranking order of the students S4, S5 and S10, shown as follows: [Step 1] Based on the accuracy matrix A and Eq. (2), we can get the average accuracy rate vector A, shown as follows:
1 ¼ a
ð0:59 þ 0:35 þ 1 þ 0:66 þ 0:11 þ 0:08 þ 0:84 þ 0:23 þ 0:4 þ 0:24Þ ¼ 0:45; 10
2 ¼ a
ð0:01 þ 0:27 þ 0:14 þ 0:04 þ 0:88 þ 0:16 þ 0:04 þ 0:22 þ 0:81 þ 0:53Þ ¼ 0:31; 10
3 ¼ a
ð0:77 þ 0:69 þ 0:97 þ 0:71 þ 0:17 þ 0:86 þ 0:87 þ 0:42 þ 0:91 þ 0:74Þ ¼ 0:711; 10
4 ¼ a
ð0:73 þ 0:72 þ 0:18 þ 0:16 þ 0:5 þ 0:02 þ 0:32 þ 0:92 þ 0:9 þ 0:25Þ ¼ 0:47; 10
5 ¼ a
ð0:93 þ 0:49 þ 0:08 þ 0:81 þ 0:65 þ 0:93 þ 0:39 þ 0:51 þ 0:97 þ 0:61Þ ¼ 0:637; 10
Based on the time rate matrix T and Eq. (3), we can get the average time rate vector T, shown as follows: t 1 ¼ ð0:7 þ 0:4 þ 0:1 þ 1 þ 0:7 þ 0:2 þ 0:7 þ 0:6 þ 0:4 þ 0:9Þ ¼ 0:57; 10
t 2 ¼ ð1 þ 0 þ 0:9 þ 0:3 þ 1 þ 0:3 þ 0:2 þ 0:8 þ 0 þ 0:3Þ ¼ 0:48; 10
Based on the accuracy rate matrix A, the score matrix G and Eq. (1), we can get the total score of each student, where the total scores TS1, TS2, . . . , and TS10 of the students S1, S2, . . . , and S10 are calculated as follows:
t 3 ¼ ð0 þ 0:1 þ 0 þ 0:1 þ 0:9 þ 1 þ 0:2 þ 0:3 þ 0:1 þ 0:4Þ ¼ 0:31; 10
TS1 ¼ 0:59 10 þ 0:01 15 þ 0:77 20 þ 0:73 25 þ 0:93 30 ¼ 67:60; TS2 ¼ 0:35 10 þ 0:27 15 þ 0:69 20 þ 0:72 25 þ 0:49 30 ¼ 54:05; TS3 ¼ 1 10 þ 0:14 15 þ 0:97 20 þ 0:18 25 þ 0:08 30 ¼ 38:40;
t 4 ¼ ð0:2 þ 0:1 þ 0 þ 1 þ 1 þ 0:3 þ 0:4 þ 0:8 þ 0:7 þ 0:5Þ ¼ 0:5; 10
TS4 ¼ 0:66 10 þ 0:04 15 þ 0:71 20 þ 0:16 25 þ 0:81 30 ¼ 49:70; TS5 ¼ 0:11 10 þ 0:88 15 þ 0:17 20 þ 0:5 25 þ 0:65 30 ¼ 49:70;
t 5 ¼ ð0 þ 0:1 þ 1 þ 1 þ 0:6 þ 1 þ 0:8 þ 0:2 þ 0:8 þ 0:2Þ ¼ 0:57; 10
TS6 ¼ 0:08 10 þ 0:16 15 þ 0:86 20 þ 0:02 25 þ 0:93 30 ¼ 48:80; TS7 ¼ 0:84 10 þ 0:04 15 þ 0:87 20 þ 0:32 25 þ 0:39 30 ¼ 46:10; TS8 ¼ 0:23 10 þ 0:22 15 þ 0:42 20 þ 0:92 25 þ 0:51 30 ¼ 52:30; TS9 ¼ 0:4 10 þ 0:81 15 þ 0:91 20 þ 0:9 25 þ 0:97 30 ¼ 85:95; TS10 ¼ 0:24 10 þ 0:53 15 þ 0:74 20 þ 0:25 25 þ 0:61 30 ¼ 49:70:
Based on the average accuracy rate vector A and Eq. (4), we can obtain the effect of accuracy rate vector EOA, shown as follows:
8246
S.-M. Chen, T.-K. Li / Expert Systems with Applications 38 (2011) 8240–8250
eoa1
eoa2
eoa3
eoa4
eoa5
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3ffi u 2 u ð0:59 0:45Þ2 þ ð0:35 0:45Þ2 þ ð1 0:45Þ2 þ ð0:66 0:45Þ2 þ u1 6 7 2 2 2 2 ¼u t10 4 ð0:11 0:45Þ þ ð0:08 0:45Þ þ ð0:84 0:45Þ þ ð0:23 0:45Þ þ 5 ¼ 0:296; ð0:4 0:45Þ2 þ ð0:24 0:45Þ2 vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3ffi u 2 u ð0:01 0:31Þ2 þ ð0:27 0:31Þ2 þ ð0:14 0:31Þ2 þ ð0:04 0:31Þ2 þ u1 6 7 2 2 2 2 ¼u t10 4 ð0:88 0:31Þ þ ð0:16 0:31Þ þ ð0:04 0:31Þ þ ð0:22 0:31Þ þ 5 ¼ 0:303; 2 2 ð0:81 0:31Þ þ ð0:53 0:31Þ vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3ffi u 2 2 u ð0:77 0:711Þ þ ð0:69 0:711Þ2 þ ð0:97 0:711Þ2 þ ð0:71 0:711Þ2 þ u1 6 7 2 2 2 2 ¼u t10 4 ð0:17 0:711Þ þ ð0:86 0:711Þ þ ð0:87 0:711Þ þ ð0:42 0:711Þ þ 5 ¼ 0:232; 2 2 ð0:91 0:711Þ þ ð0:74 0:711Þ vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3ffi u 2 2 u ð0:73 0:47Þ þ ð0:72 0:47Þ2 þ ð0:18 0:47Þ2 þ ð0:16 0:47Þ2 þ u1 6 7 2 2 2 2 ¼u t10 4 ð0:5 0:47Þ þ ð0:02 0:47Þ þ ð0:32 0:47Þ þ ð0:92 0:47Þ þ 5 ¼ 0:312; ð0:9 0:47Þ2 þ ð0:25 0:47Þ2 vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3ffi u 2 u ð0:93 0:637Þ2 þ ð0:49 0:637Þ2 þ ð0:08 0:637Þ2 þ ð0:81 0:637Þ2 þ u1 6 7 2 2 2 2 ¼u t10 4 ð0:65 0:637Þ þ ð0:93 0:637Þ þ ð0:39 0:637Þ þ ð0:51 0:637Þ þ 5 ¼ 0:269; ð0:97 0:637Þ2 þ ð0:61 0:637Þ2
Based on the average time rate vector T and Eq. (5), we can obtain the effect of time rate vector EOT, shown as follows:
eot1
eot2
eot3
eot4
eot5
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3ffi u 2 u ð0:7 0:57Þ2 þ ð0:4 0:57Þ2 þ ð0:1 0:57Þ2 þ ð1 0:57Þ2 þ u1 6 7 2 2 2 2 ¼u t10 4 ð0:7 0:57Þ þ ð0:2 0:57Þ þ ð0:7 0:57Þ þ ð0:6 0:57Þ þ 5 ¼ 0:276; ð0:4 0:57Þ2 þ ð0:9 0:57Þ2 vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3 u 2 u ð1 0:48Þ2 þ ð0 0:48Þ2 þ ð0:9 0:48Þ2 þ ð0:3 0:48Þ2 þ u1 6 7 2 2 2 2 ¼u t10 4 ð1 0:48Þ þ ð0:3 0:48Þ þ ð0:2 0:48Þ þ ð0:8 0:48Þ þ 5 ¼ 0:382; ð0 0:48Þ2 þ ð0:3 0:48Þ2 vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3 u 2 u ð0 0:31Þ2 þ ð0:1 0:31Þ2 þ ð0 0:31Þ2 þ ð0:1 0:31Þ2 þ u1 6 7 2 2 2 2 ¼u t10 4 ð0:9 0:31Þ þ ð1 0:31Þ þ ð0:2 0:31Þ þ ð0:3 0:31Þ þ 5 ¼ 0:342; 2 2 ð0:1 0:31Þ þ ð0:4 0:31Þ vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3 u 2 2 u ð0:2 0:5Þ þ ð0:1 0:5Þ2 þ ð0 0:5Þ2 þ ð1 0:5Þ2 þ u1 6 7 2 2 2 2 ¼u t10 4 ð1 0:5Þ þ ð0:3 0:5Þ þ ð0:4 0:5Þ þ ð0:8 0:5Þ þ 5 ¼ 0:344; 2 2 ð0:7 0:5Þ þ ð0:5 0:5Þ vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3 u 2 2 u ð0 0:57Þ þ ð0:1 0:57Þ2 þ ð1 0:57Þ2 þ ð1 0:57Þ2 þ u1 6 7 2 2 2 2 ¼u t10 4 ð0:6 0:57Þ þ ð1 0:57Þ þ ð0:8 0:57Þ þ ð0:2 0:57Þ þ 5 ¼ 0:385; ð0:8 0:57Þ2 þ ð0:2 0:57Þ2
Based on the complexity matrix C and Eq. (6), we can obtain the crisp complexity vector CC, shown as follows:
0:1 0 þ 0:3 0:85 þ 0:5 0:15 þ 0:7 0 þ 0:9 0 ¼ 0:33; 0 þ 0:85 þ 0:15 þ 0 þ 0 0:1 0 þ 0:3 0 þ 0:5 0:33 þ 0:7 0:67 þ 0:9 0 ¼ 0:634; ¼ 0 þ 0 þ 0:33 þ 0:67 þ 0 0:1 0 þ 0:3 0 þ 0:5 0 þ 0:7 0:69 þ 0:9 0:31 ¼ ¼ 0:762; 0 þ 0 þ 0 þ 0:69 þ 0:31 0:1 0:56 þ 0:3 0:44 þ 0:5 0 þ 0:7 0 þ 0:9 0 ¼ ¼ 0:188; 0:56 þ 0:44 þ 0 þ 0 þ 0 0:1 0 þ 0:3 0 þ 0:5 0:7 þ 0:7 0:3 þ 0:9 0 ¼ 0:56; ¼ 0 þ 0 þ 0:7 þ 0:3 þ 0
cc1 ¼ cc2 cc3 cc4 cc5
8247
S.-M. Chen, T.-K. Li / Expert Systems with Applications 38 (2011) 8240–8250 Table 1 The eigenvalues and the eigenvectors for the reciprocal matrix RE. Eigenvalues
Eigenvectors
4.048 0.442i 0.443i 0.047
[0.130 0.209 0.512 0.823 ] [0.009 0.134i 0.201 + 0.018i [0.009 + 0.134i 0.201 0.018i [0.128 0.237 0.563 0.781 ]
0.005 + 0.486i 0.84] 0.005 0.486i 0.84]
Based on the importance matrix IM and Eq. (7), we can obtain the crisp importance vector CI, shown as follows: 0:1 0 þ 0:3 0:85 þ 0:5 0:15 þ 0:7 0 þ 0:9 0 ci1 ¼ ¼ 0:9; 0 þ 0:85 þ 0:15 þ 0 þ 0 0:1 0 þ 0:3 0 þ 0:5 0:33 þ 0:7 0:67 þ 0:9 0 ¼ 0:434; ci2 ¼ 0 þ 0 þ 0:33 þ 0:67 þ 0 0:1 0 þ 0:3 0 þ 0:5 0 þ 0:7 0:69 þ 0:9 0:31 ¼ 0:87; ci3 ¼ 0 þ 0 þ 0 þ 0:69 þ 0:31 0:1 0:56 þ 0:3 0:44 þ 0:5 0 þ 0:7 0 þ 0:9 0 ¼ 0:1; ci4 ¼ 0:56 þ 0:44 þ 0 þ 0 þ 0 0:1 0 þ 0:3 0 þ 0:5 0:7 þ 0:7 0:3 þ 0:9 0 ¼ 0:486; ci5 ¼ 0 þ 0 þ 0:7 þ 0:3 þ 0
Based on the normalized vector SA’ and Eq. (8), we can obtain the reciprocal matrix RE, shown as follows:
We use Matlab 7.0 to solve the reciprocal matrix RE to get the eigenvalues and the eigenvectors, as shown in Table 1. Based on Table 1, the maximum eigenvalue kmax of the reciprocal matrix RE is 4.048, i.e., kmax = 4.048, and its corresponding eigenvector E is shown as follows:
3 2 Accuracy Rate 0:130 7 Time Rate 6 6 0:209 7 E¼ 7: 6 4 0:512 5 Complexity Importance
0:823
Based on the eigenvector E of the reciprocal matrix RE, we can obtain the percentage vector P of the eigenvector E, shown as follows:
0:13 ¼ 0:078; 0:13 þ 0:209 þ 0:512 þ 0:823 0:209 ¼ 0:125; p2 ¼ 0:13 þ 0:209 þ 0:512 þ 0:823 0:512 ¼ 0:306; p3 ¼ 0:13 þ 0:209 þ 0:512 þ 0:823 0:823 ¼ 0:492; p4 ¼ 0:13 þ 0:209 þ 0:512 þ 0:823
p1 ¼ Based on the effect of accuracy rate matrix EOA, the effect of time rate matrix EOT, the crisp complexity vector CC and the crisp importance vector CI, construct the relation matrix R, shown as follows:
3 2 Accuracy Rate 0:078 7 Time Rate 6 6 0:125 7 P¼ 7: 6 4 Complexity 0:306 5 Importance
0:492
[Step 3] Based on the relation matrix R and Eq. (9), we can get the attribute vectors R1, R2, R3 and R4, shown as follows: [Step 2] Based on the relation matrix R, we can obtain the vector SA, shown as follows:
sa1 ¼ 0:296 þ 0:303 þ 0:232 þ 0:312 þ 0:269 ¼ 1:411; sa2 ¼ 0:276 þ 0:382 þ 0:342 þ 0:344 þ 0:385 ¼ 1:728; sa3 ¼ 0:33 þ 0:634 þ 0:762 þ 0:188 þ 0:56 ¼ 2:474; sa4 ¼ 0:9 þ 0:434 þ 0:87 þ 0:1 þ 0:486 ¼ 2:79;
Then, we normalize the vector SA to get the normalized vector SA’, shown as follows:
1:411 ¼ 0:506; sa01 ¼ maxð1:411; 1:728; 2:474; 2:79Þ sa02
1:728 ¼ 0:619; ¼ maxð1:411; 1:728; 2:474; 2:79Þ
sa03 ¼
2:474 ¼ 0:887; maxð1:411; 1:728; 2:474; 2:79Þ
sa04 ¼
2:79 ¼ 1:000; maxð1:411; 1:728; 2:474; 2:79Þ
3 3 2 0:296 Q 1 0:276 7 7 6 6 Q 2 6 0:303 7 Q 2 6 0:382 7 7 7 6 6 7 7 6 R1 ¼ Q 3 6 6 0:232 7; R2 ¼ Q 3 6 0:342 7; 7 7 6 6 Q 4 4 0:312 5 Q 4 4 0:344 5 Q1
2
Q 5 0:269 Q 5 0:385 3 3 2 2 Q 1 0:330 Q 1 0:900 7 7 6 6 Q 2 6 0:634 7 Q 2 6 0:434 7 7 7 6 6 7 7 6 R3 ¼ Q 3 6 6 0:762 7; R4 ¼ Q 3 6 0:870 7: 7 7 6 6 Q 4 4 0:188 5 Q 4 4 0:100 5 Q 5 0:560 Q 5 0:486 Then, we can get
R1 ¼ ð0:296 þ 0:303 þ 0:232 þ 0:312 þ 0:269Þ=5 ¼ 0:282; R2 ¼ ð0:276 þ 0:382 þ 0:342 þ 0:344 þ 0:385Þ=5 ¼ 0:346; R3 ¼ ð0:33 þ 0:634 þ 0:762 þ 0:188 þ 0:56Þ=5 ¼ 0:495; R4 ¼ ð0:9 þ 0:434 þ 0:87 þ 0:1 þ 0:486Þ=5 ¼ 0:558: Based on the attribute vectors R1, R2, R3, R4, the values of R1 , R2 , R3 and R4 and Eq. (10), we can calculate the correlation coefficients between attribute vectors, shown as follows:
8248
S.-M. Chen, T.-K. Li / Expert Systems with Applications 38 (2011) 8240–8250
2
ð0:296 0:282Þð0:276 0:346Þ þ ð0:303 0:282Þð0:382 0:346Þþ
1 6 ð0:232 51 4
3
7 0:282Þð0:342 0:346Þ þ ð0:312 0:282Þð0:344 0:346Þþ 5
ð0:269 0:282Þð0:385 0:346Þ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi CofðR1 ; R2 Þ ¼ vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3ffiv 3ffi ¼ 0:105; u 2 u 2 2 ð0:296 0:282Þ þ ð0:303 0:282Þ2 þ u ð0:276 0:346Þ2 þ ð0:382 0:346Þ2 þ u u 6 u 7 2 2 1 6 u 1 4 ð0:232 0:282Þ2 þ ð0:312 0:282Þ2 þ 7 5u t51 t51 4 ð0:342 0:346Þ þ ð0:344 0:346Þ þ 5 ð0:269 0:282Þ2 ð0:385 0:346Þ2 2
3 ð0:296 0:282Þð0:33 0:495Þ þ ð0:303 0:282Þð0:634 0:495Þþ 6 7 1 ð0:232 0:282Þð0:762 0:495Þ þ ð0:312 0:282Þð0:188 0:495Þþ 5 51 4 ð0:269 0:282Þð0:56 0:495Þ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi CofðR1 ; R3 Þ ¼ vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3ffiv 3ffi ¼ 0:751; u 2 u 2 ð0:296 0:282Þ2 þ ð0:303 0:282Þ2 þ u ð0:33 0:495Þ2 þ ð0:634 0:495Þ2 þ u u 6 u 6 7 2 2 1 u 1 4 ð0:232 0:282Þ2 þ ð0:312 0:282Þ2 þ 7 5u t51 t51 4 ð0:762 0:495Þ þ ð0:188 0:495Þ þ 5 ð0:269 0:282Þ2
ð0:56 0:495Þ2
2
3 ð0:296 0:282Þð0:9 0:588Þ þ ð0:303 0:282Þð0:434 0:588Þþ 7 1 6 ð0:232 0:282Þð0:87 0:588Þ þ ð0:312 0:282Þð0:1 0:588Þþ 5 51 4 ð0:269 0:282Þð0:486 0:588Þ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi CofðR1 ; R4 Þ ¼ vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3ffiv 3 ¼ 0:605; u 2 u 2 2 ð0:296 0:282Þ þ ð0:303 0:282Þ2 þ u ð0:9 0:588Þ2 þ ð0:434 0:588Þ2 þ u u 6 u 7 2 2 1 6 u 1 4 ð0:232 0:282Þ2 þ ð0:312 0:282Þ2 þ 7 5u t51 t51 4 ð0:87 0:588Þ þ ð0:1 0:588Þ þ 5 ð0:269 0:282Þ2 ð0:486 0:588Þ2 2
3 ð0:276 0:346Þð0:33 0:495Þ þ ð0:382 0:346Þð0:634 0:495Þþ 6 7 1 ð0:342 0:346Þð0:762 0:495Þ þ ð0:344 0:346Þð0:188 0:495Þþ 5 51 4 ð0:385 0:346Þð0:56 0:495Þ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi CofðR2 ; R3 Þ ¼ vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3ffiv 3ffi ¼ 0:458; u 2 u 2 ð0:276 0:346Þ2 þ ð0:382 0:346Þ2 þ u ð0:33 0:495Þ2 þ ð0:634 0:495Þ2 þ u u 6 u 6 7 2 2 1 u 1 4 ð0:342 0:346Þ2 þ ð0:344 0:346Þ2 þ 7 5u t51 t51 4 ð0:762 0:495Þ þ ð0:188 0:495Þ þ 5 ð0:385 0:346Þ2
2 1 6 51 4
ð0:56 0:495Þ2
ð0:276 0:346Þð0:9 0:588Þ þ ð0:382 0:346Þð0:434 0:588Þþ
3
7 ð0:342 0:346Þð0:87 0:588Þ þ ð0:344 0:346Þð0:1 0:588Þþ 5 ð0:385 0:346Þð0:486 0:588Þ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi CofðR2 ; R4 Þ ¼ v 3ffiv 3 ¼ 0:535; u 2 u 2 2 ð0:276 0:346Þ þ ð0:382 0:346Þ2 þ u ð0:9 0:588Þ2 þ ð0:434 0:588Þ2 þ u u 6 u 7 2 2 1 6 u 1 4 ð0:342 0:346Þ2 þ ð0:344 0:346Þ2 þ 7 5u t51 t51 4 ð0:87 0:588Þ þ ð0:1 0:588Þ þ 5 ð0:385 0:346Þ2 ð0:486 0:588Þ2 2
3 ð0:33 0:495Þð0:9 0:588Þ þ ð0:634 0:495Þð0:434 0:588Þþ 6 7 1 ð0:762 0:495Þð0:87 0:588Þ þ ð0:188 0:495Þð0:1 0:588Þþ 5 51 4 ð0:56 0:495Þð0:486 0:588Þ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi CofðR3 ; R4 Þ ¼ v 3ffiv 3 ¼ 0:469; u 2 u 2 ð0:33 0:495Þ2 þ ð0:634 0:495Þ2 þ u ð0:9 0:588Þ2 þ ð0:434 0:588Þ2 þ u u 6 u 6 7 2 2 1 u 1 4 ð0:762 0:495Þ2 þ ð0:188 0:495Þ2 þ 7 5u t51 t51 4 ð0:87 0:588Þ þ ð0:1 0:588Þ þ 5 ð0:56 0:495Þ2
ð0:486 0:588Þ2
Based on Eqs. (11)–(14), we can get
soc1 ¼ jCofðR1 ; R2 Þj þ jCofðR1 ; R3 Þj þ jCofðR1 ; R4 Þj ¼ 1:461; soc2 ¼ jCofðR1 ; R2 Þj þ jCofðR2 ; R3 Þj þ jCofðR2 ; R4 Þj ¼ 1:098; soc3 ¼ jCofðR1 ; R3 Þj þ jCofðR2 ; R3 Þj þ jCofðR3 ; R4 Þj ¼ 1:679; soc4 ¼ jCofðR1 ; R4 Þj þ jCofðR2 ; R4 Þj þ jCofðR3 ; R4 Þj ¼ 1:609; Based on the attribute vectors R1, R2, R3, R4, the values of R1 , R2 , R3 , R4 , and Eq. (15), we can calculate the standard deviations Std(R1), Std(R2), Std(R3) and Std(R4) of the attribute vectors R1, R2, R3 and R4, respectively, shown as follows:Based on the values of soc1,
soc2, soc3, soc4, the standard deviations Std(R1), Std(R2), Std(R3), Std(R4) and Eq. (16), we can calculate the fitness degrees Fit(R1), Fit(R2), Fit(R3) and Fit(R4) of the attribute vectors R1, R2, R3 and R4, respectively, shown as follows:
S.-M. Chen, T.-K. Li / Expert Systems with Applications 38 (2011) 8240–8250
8249
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi # u " u1 ð0:296 0:282Þ2 þ ð0:303 0:282Þ2 þ ð0:232 0:282Þ2 þ ¼ 0:022; StdðR1 Þ ¼ t 5 ð0:312 0:282Þ2 þ ð0:269 0:282Þ2 vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi # u " u1 ð0:276 0:346Þ2 þ ð0:382 0:346Þ2 þ ð0:342 0:346Þ2 þ t StdðR2 Þ ¼ ¼ 0:039; 5 ð0:344 0:346Þ2 þ ð0:385 0:346Þ2 vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi # u " u1 ð0:330 0:495Þ2 þ ð0:634 0:495Þ2 þ ð0:762 0:495Þ2 þ ¼ 0:208; StdðR3 Þ ¼ t 5 ð0:188 0:495Þ2 þ ð0:560 0:495Þ2 vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi # u " u1 ð0:900 0:558Þ2 þ ð0:434 0:558Þ2 þ ð0:870 0:558Þ2 þ t ¼ 0:298: StdðR4 Þ ¼ 5 ð0:100 0:558Þ2 þ ð0:486 0:558Þ2 1:461 FitðR1 Þ ¼ 1 0:029 ¼ 0:022; 1:461 þ 1:098 þ 1:679 þ 1:609 1:098 0:039 ¼ 0:032; FitðR2 Þ ¼ 1 1:461 þ 1:098 þ 1:679 þ 1:609 1:679 0:208 ¼ 0:148; FitðR3 Þ ¼ 1 1:461 þ 1:098 þ 1:679 þ 1:609 1:609 0:298 ¼ 0:216: FitðR4 Þ ¼ 1 1:461 þ 1:098 þ 0:679 þ 1:609
Based on the fitness degrees Fit(R1), Fit(R2), Fit(R3) and Fit(R4) of the attribute vectors R1, R2, R3 and R4, respectively, we calculate the percentages of the fitness degrees Fit(R1), Fit(R2), Fit(R3) and Fit(R4) of the attribute vectors R1, R2, R3 and R4, respectively, to obtain the percentage vector PF, shown as follows:
0:022 ¼ 0:052; 0:022 þ 0:032 þ 0:148 þ 0:216 0:032 ¼ 0:076; pf2 ¼ 0:022 þ 0:032 þ 0:148 þ 0:216 0:148 ¼ 0:355; pf3 ¼ 0:022 þ 0:032 þ 0:148 þ 0:216 0:216 ¼ 0:517; pf4 ¼ 0:022 þ 0:032 þ 0:148 þ 0:216
pf1 ¼
id1 ¼ 0:296 0:065 þ 0:276 0:101 þ 0:33 0:330 þ 0:9 0:504 ¼ 0:610; id2 ¼ 0:303 0:065 þ 0:382 0:101 þ 0:634 0:330 þ 0:434 0:504 ¼ 0:486; id3 ¼ 0:232 0:065 þ 0:342 0:101 þ 0:762 0:330 þ 0:87 0:504 ¼ 0:740;: id4 ¼ 0:312 0:065 þ 0:344 0:101 þ 0:188 0:330 þ 0:1 0:504 ¼ 0:167; id5 ¼ 0:269 0:065 þ 0:385 0:101 þ 0:56 0:330 þ 0:486 0:504 ¼ 0:486;
[Step 5] Based on the importance degree vector ID, the grade matrix G, the accuracy rate matrix A and Eq. (18), we can calculate the learning achievement indices LA4, LA5 and LA10 of the students S4, S5 and S10 having the same original total score, respectively, shown as follows:
LA4 ¼ 0:66 10 0:610 þ 0:04 15 0:486 þ 0:71 20 0:740 þ 0:16 25 0:167 þ 0:81 30 0:486 ¼ 27:303; LA5 ¼ 0:11 10 0:610 þ 0:88 15 0:486 þ 0:17 20 0:740 þ 0:5 25 0:167 þ 0:65 30 0:486 ¼ 21:176; LA10 ¼ 0:24 10 0:610 þ 0:53 15 0:486 þ 0:74 20 0:740 þ 0:25 25 0:167 þ 0:61 30 0:486 ¼ 26:220: Because LA4 > LA10 > LA5, the new ranking order of the 10 students is
[Step 4] Based on the percentage vector P obtained in Step 2 and the percentage vector PF obtained in Step 3, we can get the weighting vector W of the attributes, shown as follows:
0:078 þ 0:052 ¼ 0:065; 2 0:125 þ 0:076 ¼ 0:101; w2 ¼ 2 0:306 þ 0:355 ¼ 0:330; w3 ¼ 2 0:492 þ 0:517 w4 ¼ ¼ 0:504; 2
w1 ¼
Based on the weighting vector W of the attributes, the relation matrix R and Eq. (17), we can get the importance degree vector ID of the questions, shown as follows:
S9 > S1 > S2 > S8 > S4 > S10 > S5 > S6 > S7 > S3 : The new ranking order of the proposed method is the same as the one of Bai and Chen’s method (2008c). 5. Conclusions In this paper, we have presented a new method for students’ learning achievement evaluation based on the eigenvector method (Chen & Hwang, 1992). The proposed method considers the ‘‘accuracy rate’’, the ‘‘time rate’’, the ‘‘importance’’ and the ‘‘complexity’’ for evaluating students’ learning achievement. First, the proposed method transforms the attributes ‘‘accuracy rate’’ and ‘‘time rate’’ into the ‘‘effect of accuracy rate’’ and the ‘‘effect of time rate’’, respectively. Then, it generates the relative importance degrees of the attributes ‘‘effect of accuracy rate’’, ‘‘effect of time rate’’, ‘‘importance’’ and ‘‘complexity’’ based on the eigenvector method. Then, it uses the correlation coefficients between the attribute vectors and the standard deviations of the elements in the attribute vectors to calculate the fitness degrees of the attributes,
8250
S.-M. Chen, T.-K. Li / Expert Systems with Applications 38 (2011) 8240–8250
where the attribute vectors represent the relationships between the attributes and the questions. Then, it generates the weights of the attributes based on the relative importance degrees of the attributes and the fitness degrees of the attributes. Then, it generates the importance degrees of the questions according to the weights of the attributes and the relation matrix representing the relationships between the questions and the attributes. Finally, based on the importance degree vector of the questions, the grade matrix and the accuracy rate matrix, it calculates the learning achievement index of each student having the same original total score for students’ learning achievement evaluation. The proposed method provides us with a useful way for students’ learning achievement evaluation based on the eigenvector method. Acknowledgements This work was supported in part by the National Science Council, Republic of China, under Grant NSC 97–2221-E-011–107-MY3. References Bai, S. M., & Chen, S. M. (2008a). Automatically constructing concept maps based on fuzzy rules for adaptive learning systems. Expert Systems with Applications, 35(1), 41–49. Bai, S. M., & Chen, S. M. (2008b). Automatically constructing grade membership functions of fuzzy rules for students’ evaluation. Expert Systems with Applications, 35(3), 1408–1414. Bai, S. M., & Chen, S. M. (2008c). Evaluating students’ learning achievement using fuzzy membership functions and fuzzy rules. Expert Systems with Applications, 34(1), 399–410.
Biswas, R. (1995). An application of fuzzy sets in students’ evaluation. Fuzzy Sets and Systems, 74(2), 187–194. Chen, S. M., & Lee, C. H. (1999). New methods for students’ evaluating using fuzzy sets. Fuzzy Sets and Systems, 104(2), 209–218. Chen, S. M., & Li, T. K. (2010). A new method for students’ learning achievement evaluation based on the eigvenvector method. In Proceedings of the 2010 international conference on machine learning and cybernetics (pp. 2483–2488), Qingdao, Shandong, China. Chen, S. J., & Hwang, C. L. (1992). Fuzzy multiple attribute decision making. NY: Springer-Verlag. 332–333. Echauz, J. R., & Vachtsevanos, G. J. (1995). Fuzzy grading system. IEEE Transactions on Education, 38(2), 158–165. Greub, W. H. (1975). Linear algebra. New York: Springer-Verlag. Hwang, C. L., & Yoon, K. S. (1981). Multiple Attribute Decision making: Methods and Applications. Berlin: Springer-Verlag. Law, C. K. (1996). Using fuzzy numbers in education grading system. Fuzzy Sets and Systems, 83(3), 311–323. Ma, J., & Zhou, D. (2000). Fuzzy set approach to the assessment of student-centered learning. IEEE Transactions on Education, 43(2), 237–241. Wang, H. Y., & Chen, S. M. (2008). Evaluating students’ answer-scripts using fuzzy numbers associated with degrees of confidence. IEEE Transactions on Fuzzy Systems, 16(2), 403–415. Weon, S., & Kim, J. (2001). Learning achievement evaluation strategy using fuzzy membership function. In Proceedings of the 31th ASEE/IEEE frontiers in education conference (pp. 19–24), Reno, NV. Wilson, E., Karr, C. L., & Freeman, L. M. (1998). Flexible, adaptive, automatic fuzzybased grade assigning system. In Proceedings of the 1998 North American fuzzy information processing society conference (pp. 334–338). Wu, M. H. (2003). Research on applying fuzzy set theory and item response theory to evaluate learning performance. Master Thesis, Department of Information Management, Chaoyang University of Technology, Wufeng, Taichung Country, Republic of China. Zadeh, L. A. (1965). Fuzzy sets. Information and Control, 8, 338–353.
Contents lists available at ScienceDirect
Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa
Evaluating students’ learning achievement based on the eigenvector method Shyi-Ming Chen ⇑, Ting-Kuei Li Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan, ROC
a r t i c l e
i n f o
Keywords: Eigenvector method Fuzzy sets Learning achievement index Membership functions Students’ learning achievement evaluation
a b s t r a c t In this paper, we present a new method for students’ learning achievement evaluation based on the eigenvector method. The proposed method considers the ‘‘accuracy rate’’, the ‘‘time rate’’, the ‘‘importance’’ and the ‘‘complexity’’ for evaluating students’ learning achievement. First, the proposed method transforms the attributes ‘‘accuracy rate’’ and ‘‘time rate’’ into the ‘‘effect of accuracy rate’’ and the ‘‘effect of time rate’’, respectively. Then, it generates the relative importance degrees of the attributes ‘‘effect of accuracy rate’’, ‘‘effect of time rate’’, ‘‘importance’’ and ‘‘complexity’’ based on the eigenvector method. Then, it uses the correlation coefficients between the attribute vectors and the standard deviations of the elements in the attribute vectors to calculate the fitness degrees of the attributes, where the attribute vectors represent the relationships between the attributes and the questions. Then, it generates the weights of the attributes based on the relative importance degrees of the attributes and the fitness degrees of the attributes. Then, it generates the importance degrees of the questions according to the weights of the attributes and the relation matrix representing the relationships between the questions and the attributes. Finally, based on the importance degrees vector of the questions, the grade matrix, the accuracy rate matrix, it calculates the learning achievement index of each student having the same original total score for students’ learning achievement evaluation. The proposed method provides us with a useful way for students’ learning achievement evaluation based on the eigenvector method. Ó 2011 Elsevier Ltd. All rights reserved.
1. Introduction In recent years, the fuzzy set theory (Zadeh, 1965) has been widely used in educational grading systems (Bai & Chen, 2008a, 2008b, 2008c; Biswas, 1995; Chen & Lee, 1999; Chen & Li, 2010; Echauz & Vachtsevanos, 1995; Law, 1996; Ma & Zhou, 2000; Wang & Chen, 2008; Weon & Kim, 2001; Wilson, Karr, & Freeman, 1998; Wu, 2003). Bai and Chen (2008a) presented a method for automatically constructing concept maps based on fuzzy rules and students’ testing records. It applied fuzzy rules and fuzzy reasoning techniques to automatically construct concept maps and evaluate the relevance degrees between concepts. Bai and Chen (2008b) presented a method to automatically construct the grade membership functions of lenient-type grades, strict-type grades and normal-type grades given by teachers, and then perform fuzzy reasoning to infer the scores of students. Biswas (1995) presented two methods for evaluating students’ answer-scripts based on fuzzy sets and a matching function. However, Biswas’ methods have the drawback that different fuzzy marks will be transformed into the same score. Chen and Lee (1999) presented two methods for applying fuzzy sets to evaluate students’ answer-scripts to overcome the drawback of Biswas’s methods (1995). Echauz and ⇑ Corresponding author. Tel.: +886 2 27376417; fax: +886 2 27301081. E-mail address: [email protected] (S.-M. Chen). 0957-4174/$ - see front matter Ó 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2011.01.003
Vachtsevanos (1995) presented a fuzzy grading system using students’ and instructor’s performance measures in order to modify a set of collective, approved and a priori fuzzy grades to produce a fair mark distribution. Law (1996) presented a fuzzy educational grading system, not only to assist teachers to aggregate different test scores to produce a single score or grade for students, but also to help teachers deciding whether to revise an instructional procedure for their students. Ma and Zhou (2000) presented an integrated fuzzy set approach to assess the outcomes of students-centered learning based on fuzzy set principles to represent the imprecise concepts for subjective judgment and incorporated the students’ opinions into assessments. Wang and Chen (2008) presented two methods for evaluating students’ answer-scripts using fuzzy numbers associated with the degrees of confidence, where the satisfaction levels given by the evaluator awarded to the questions of the students’ answer-scripts are represented by triangular fuzzy numbers associated with degrees of confidence between zero and one. Wilson et al. (1998) presented a flexible and adaptive grade assignment system based on fuzzy rules and genetic algorithms. Wu (2003) presented a method for evaluating learning performance of students by applying the fuzzy set theory and the item response theory to integrate the objective and the subjective information obtained from exams. Weon and Kim (2001) pointed out that an educational grading system should consider the ‘‘difficulty’’, ‘‘importance’’ and ‘‘complexity’’ of questions
S.-M. Chen, T.-K. Li / Expert Systems with Applications 38 (2011) 8240–8250
in students’ answer-scripts learning achievement evaluation. They also presented a method for students’ learning achievement using fuzzy membership functions. Bai and Chen (2008c) presented a method for evaluating students’ learning achievement using fuzzy membership functions and fuzzy rules. They pointed out that ‘‘difficulty’’ is a very subjective parameter and to adjust the scores of students during the evaluation process is not appropriate. They used the accuracy of the students’ answer-scripts and the time consumed to answer the questions to obtain the difficulty of questions based on fuzzy reasoning techniques. However, the drawback of Bai and Chen’s method (2008c) is that the weights of the attributes in Bai and Chen’s method are assigned subjectively by domain experts. Once the relationships between the attributes and the questions are changed, the weights of the attributes will still be the same as before. It is not appropriate to determine the new ranking order of the students having the same original total score using the same weights of the attributes as before. In this paper, we present a new method for students’ learning achievement evaluation based on the eigenvector method. The proposed method considers the ‘‘accuracy rate’’, the ‘‘time rate’’, the ‘‘importance’’ and the ‘‘complexity’’ for evaluating students’ learning achievement. First, the proposed method transforms the attributes ‘‘accuracy rate’’ and ‘‘time rate’’ into the ‘‘effect of accuracy rate’’ and the ‘‘effect of time rate’’, respectively. Then, it generates the relative importance degrees of the attributes ‘‘effect of accuracy rate’’, ‘‘effect of time rate’’, ‘‘importance’’ and ‘‘complexity’’ based on the eigenvector method. Then, it uses the correlation coefficients between the attribute vectors and the standard deviations of the elements in the attribute vectors to calculate the fitness degrees of the attributes, where the attribute vectors represent the relationships between the attributes and the questions. Then, it generates the weights of the attributes based on the relative importance degrees of the attributes and the fitness degrees of the attributes. Then, it generates the importance degrees of the questions according to the weights of the attributes and the relation matrix representing the relationships between the questions and the attributes. Finally, based on the importance degrees vector of the questions, the grade matrix and the accuracy rate matrix, it calculates the learning achievement index of each student having the same original total score for students’ learning achievement evaluation. The proposed method provides us with a useful way for students’ learning achievement evaluation based on the eigenvector method. The rest of this paper is organized as follows. In Section 2, we briefly review Bai and Chen’s method (2008c) for students’ learning achievement evaluation. In Section 3, we present a new method for students’ learning achievement evaluation based on the eigenvector method. In Section 4, we use an example to illustrate the proposed method and to compare the results of the proposed method with Bai and Chen’s method (2008c). The conclusions are discussed in Section 5.
where aij represents the relative importance degree of the attribute ATi to the attribute ATj, wi denotes the importance degree of the attribute ATi, wi/wj represents the relative importance degree of the attribute ATi to the attribute ATj, n denotes the number of attributes, aij P 0, aij = 1/aji, aii = 1, 1 6 i 6 n and 1 6 j 6 n. The relative importance degree aij is defined by the fuzzy scales ranging from 1 to 9, shown as follows: 1: 3: 5: 7: 9:
equally important weakly more important strongly more important demonstratively more important absolutely more important
where 2, 4, 6 and 8 are intermediate values between two adjacent judgments. After multiplying the reciprocal matrix AR by the importance degree vector WI = (w1, w2, . . . , wn), we can get
Therefore, (AR nI)WI = 0, where I is an identity matrix. Then, it can obtain the relative importance degrees of the attributes by solving the eigenvalues and the eigenvectors of the reciprocal matrix AR. Given a matrix A, a non-zero vector x is defined to be an eigenvector (Greub, 1975) of the matrix A if it satisfies the eigenvalue equation Ax = kx for some scalar k, where k is called an eigenvalue of the matrix A corresponding to the eigenvector x. The eigenvalues and the eigenvectors are calculated as follows (Greub, 1975): Step 1: Solve the equation det (A kI) = 0 and get the eigenvalues k1, k2 , . . . , where ‘‘det’’ is defined as the determinant of a square matrix and I is an identity matrix. Step 2: For each eigenvalue ki, solve the equation (A ki I) x = 0 and get the eigenvectors with respect to the eigenvalue ki, where i = 1, 2, . . .. 13 4 Example 2.1. Let us consider the matrix A ¼ . Then, we 4 7 can get
detðA kIÞ ¼ 2. A review of the eigenvector method The eigenvector method (Chen & Hwang, 1992) is useful to generate the relative importance degrees of attributes. Assume that there are n attributes AT1, AT2, . . . , and ATn. Let AR be a positive reciprocal matrix representing the relationships between the attributes, shown as follows:
8241
13 k 4 ¼ ð13 kÞð7 kÞ ð4Þð4Þ 4 7k
¼ 75 20k þ k2 ¼ 0: Because 75 20k + k2 = (5 k)(15 k) = 0, we get two eigenvalues k1 = 5 and k2 = 15. 13 5 4 x1 For k1 = 5, we have ðA 5IÞx ¼ ¼ 4 7 5 x2 8 4 x1 ¼ 0: That is, x1 = 2x2. The solutions of (A 5I)x = 0 4 2 x2 1 are x ¼ c , where c is an arbitrary value. For example, when 2 1 ; when c = 2, we can get c = 1, we can get an eigenvector v 1 ¼ 2 2 . an eigenvector v 1 ¼ 4
8242
S.-M. Chen, T.-K. Li / Expert Systems with Applications 38 (2011) 8240–8250
13 15 4 For k2 = 15, we have ðA 15IÞx ¼ 4 7 15 2 4 x1 x1 ¼ ¼ 0: That is x1 = 2x2. The solutions of 4 8 x2 x2 2 (A 15I)x = 0 are x ¼ c , where c is an arbitrary value. For 1 2 ; when example, when c = 1, we can get an eigenvector v 2 ¼ 1 4 . c = 2, we can get an eigenvector v 2 ¼ 2
3. A new method for students’ learning achievement evaluation based on the eigenvector method In this section, we present a new method for students’ learning achievement evaluation based on the eigenvector method (Chen & Hwang, 1992). Assume that there are m questions and assume that there are n students to answer these questions. Let Qi denote the ith question in the answerscript and let Sj denote the jth student, where 1 6 i 6 m and 1 6 j 6 n. Then, we can get an accuracy rate matrix A and a time rate matrix T, shown as follows:
where aij denotes the accuracy rate of the jth student Sj with respect to the ith question Qi, aij 2 [0,1], tij denotes the answer-time rate of the jth student Sj with respect to the ith question Qi, tij 2 [0, 1], 1 6 i 6 m and 1 6 j 6 n. Let G be a score matrix shown as follows:
g1
3
Q2 G¼ . ..
6g 6 2 6 6 .. 4 .
7 7 7; 7 5
Qm
gm
Q1
2
where imi1, imi2, imi3, imi4 and imi5 denote the grades of membership that the degree of importance of the ith question Qi belongs to the importance levels ‘‘low’’, ‘‘more or less low’’, ‘‘medium’’, ‘‘more or less high’’ and ‘‘high’’, respectively, imi1 2 [0,1], imi2 2 [0,1], imi3 2 [0, 1], imi4 2 [0,1], imi5 2 [0,1] and 1 6 i 6 m; ci1, ci2, ci3, ci4 and ci5 denote the grades of membership that the complexity of question Qi belongs to the complexity levels ‘‘low’’, ‘‘more or less low’’, ‘‘medium’’, ‘‘more or less high’’, and ‘‘high’’, respectively, ci1 2 [0, 1], ci2 2 [0, 1], ci3 2 [0, 1], ci4 2 [0, 1], ci5 2 [0, 1] and 1 6 i 6 m. In order to distinguish the difference degree of the accuracy rate of the students for each question and in order to distinguish the difference degree of the time rate of the students for each question, we use the standard deviation equation to transform the accuracy rate matrix A into the effect of accuracy rate vector EOA and to transform the time rate matrix T into the effect of time rate vector EOT. The larger the standard deviation of the accuracy rates of the students with respect to a question, the easier to distinguish the learning achievement of the students by means of that question; the larger the standard deviation of the time rates of the students with respect to a question, the easier to distinguish the learning achievement of students by means of that question. Therefore, we can use the attributes vectors EOA and EOT to distinguish the learning achievement of the students having the same original total score. The proposed method is now presented as follows: i of question Qi, Step 1: Calculate the average accuracy rate a shown as follows:
i ¼ a
n X
aij =n;
ð2Þ
j¼1
where gi denotes the score of the ith question Qi, gi 2 [0, 100], 1 6 i 6 m and m X
Calculate the average time rate t i of question Qi, shown as follows:
t i ¼
g i ¼ 100:
n X
t ij =n:
ð3Þ
j¼1
i¼1
Based on the accuracy rate matrix A and the score matrix G, we can calculate the total score TSj of each student Sj, shown as follows:
TSj ¼
m X
aij g i ;
ð1Þ
i¼1
where 1 6 j 6 n. The original ranking order of the students are obtained by sorting the values of TS1, TS2, . . . , and TSm in a descending order. Let IM be an importance matrix and let C be a complexity matrix given by a domain expert for each question Qi, respectively, shown as follows:
Based on the accuracy rate matrix A and the time rate matrix T, calculate the standard deviation eoai of the accuracy rates ai1, ai2, . . . , and ain of the students S1, S2, . . . , and Sn, respectively, with respect to question Qi, where 1 6 i 6 m, to obtain the effect of accuracy rate vector EOA and calculate the standard deviation eoti of the time rates ti1, ti2, . . . , and tin of the students S1, S2, . . . , and Sn, respectively, with respect to question Qi, where 1 6 i 6 m, to obtain the effect of time rate vector EOT, where
EOA ¼
Q1 ½eoa1
Q2 eoa2
Qm ; eoam
8243
S.-M. Chen, T.-K. Li / Expert Systems with Applications 38 (2011) 8240–8250
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u X u1 n i Þ2 ; eoai ¼ t ðaij a n j¼1 EOT ¼
Q1
Q2
½eot1
eot2
ð4Þ
Qm
eotm
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u X u1 n ðtij ti Þ2 ; eoti ¼ t n j¼1
;
ð5Þ
P P i ¼ nj¼1 aij =n, t i ¼ nj¼1 tij =n and 1 6 i 6 m. Transform the where a complexity matrix C into the crisp complexity vector CC and transform the importance matrix IM into the crisp importance vector CI, shown as follows:
CC ¼
cci ¼
CI ¼
cii ¼
Q1
Q2
Qm
½cc1
cc2
ccm
Step 2: Based on the relation matrix R obtained in Step 1, calculate the summation of the elements in the relation matrix R to obtain the vector SA, shown as follows:
P where saj ¼ m i¼1 r ij and 1 6 j 6 4. Normalize the vector SA to get the normalized vector SA’, shown as follows:
sai where sa0j ¼ maxðsa1 ;sa and 1 6 j 6 4. Based on the normalized 2 ;sa3 ;sa4 Þ vector SA0 and Eq. (8), transform the relation matrix R into the reciprocal matrix RE, shown as follows:
;
0:1 ci1 þ 0:3 ci2 þ 0:5 ci3 þ 0:7 ci4 þ 0:9 ci5 ; ci1 þ ci2 þ ci3 þ ci4 þ ci5 ð6Þ Q1
Q2
½ci1
ci2
cim
Qm
;
0:1 imi1 þ 0:3 imi2 þ 0:5 imi3 þ 0:7 imi4 þ 0:9 imi5 ; imi1 þ imi2 þ imi3 þ imi4 þ imi5 ð7Þ
1 6 i 6 m and 0.1, 0.3, 0.5, 0.7, 0.9 are ‘‘the right point’’, ‘‘the center point’’, ‘‘the center point’’, ‘‘the center point’’ and ‘‘the left point’’ of the fuzzy sets ‘‘low’’, ‘‘more or less low’’, ‘‘medium’’, ‘‘more or less high’’ and ‘‘high’’, respectively, as shown in Fig. 1. Based on the effect of accuracy rate matrix EOA, the effect of time rate matrix EOT, the crisp complexity vector CC and the crisp importance vector CI, construct the relation matrix R, shown as follows:
m 8 l > sa0i sa0j 10 ; if sa0i > sa0j > > < if sa0i ¼ sa0j ; reij ¼ 1; > l m > > : 1= sa0i sa0j 10 ; if sa0i < sa0j
ð8Þ
where reij = 1/reji, reii = 1, 1 6 i 6 4 and 1 6 j 6 4. Use the Matlab 7.0 to solve the reciprocal matrix RE to get the maximum eigenvalue kmax of the reciprocal matrix RE and its corresponding eigenvector E, shown as follows:
2 3 Accuracy Rate e1 7 Time Rate 6 6 e2 7 E¼ 6 7: Complexity 4 e3 5 Importance
e4
Based on the eigenvector E, calculate the percentage of the eigenvector E to obtain the percentage vector P of the eigenvector E, shown as follows:
2 3 Accuracy Rate p1 7 Time Rate 6 6 p2 7 P¼ 6 7; Complexity 4 p3 5 Importance ei
where pi ¼ P4
e j¼1 j
p4
; pi 2 ½0; 1 denotes the relative importance degree
of the ith attribute and 1 6 i 6 4. That is, the relative importance degrees of the attributes ‘‘Accuracy Rate’’, ‘‘Time Rate’’, ‘‘Complexity’’ and ‘‘Importance’’ are p1, p2, p3 and p4, respectively, where pi 2 [0, 1] and 1 6 i 6 4. Step 3: Based on the relation matrix R obtained in Step 1, get the attribute vectors R1, R2, R3 and R4, shown as follows:
3 2 Q 1 r12 7 6 Q2 Q 2 6 r22 7 7 7 7; R2 ¼ . 6 . 7; R1 ¼ . 7 7 .. .. 6 5 4 .. 5 Q m r m1 Q m r m2 3 3 2 2 Q 1 r 13 Q 1 r14 7 7 Q2 6 Q2 6 6 r 23 7 6 r24 7 7; R4 ¼ . 6 . 7; R3 ¼ . 6 . 7 7 .. 6 .. 6 4 .. 5 4 .. 5 Q1
Fig. 1. Membership functions of the fuzzy sets ‘‘low’’, ‘‘more or less low’’, ‘‘medium’’, ‘‘more or less high’’ and ‘‘high’’, respectively.
Qm
2
r 11 6r 6 21 6 . 6 . 4 .
r m3
3
Qm
r m4
ð9Þ
8244
S.-M. Chen, T.-K. Li / Expert Systems with Applications 38 (2011) 8240–8250
where the attribute vector R1 denotes the relation between the attribute ‘‘Accuracy Rate’’ and the questions, the attribute vector R2 denotes the relation between the attribute ‘‘Time Rate’’ and the questions, the attribute vector R3 denotes the relation between the attribute ‘‘Complexity’’ and the questions and the attribute vector R4 denotes the relation between the attribute ‘‘Importance’’ and the questions. Calculate the correlation coefficient Cof(Ri, Rj) between the attribute vector Ri and the attribute vector Rj, shown as follows: m P
ðrki Ri Þðr kj Rj Þ=ðm 1Þ CofðRi ; Rj Þ ¼ sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffisffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ; m m P P ðr ki Ri Þ2 =ðm 1Þ ðr kj Rj Þ2 =ðm 1Þ
the value of Fit(Ri), the easier to distinguish the ranking order of the students having the same original total score based on the attribute vector Ri, where 1 6 i 6 4. Based on the fitness degree Fit(Ri), where 1 6 j 6 4, construct the percentage vector PF, shown as follows:
where
k¼1
k¼1
k¼1
ð10Þ P Pm where jCofðRi ; Rj Þj ¼ jCofðRj ; Ri Þj, Ri ¼ m k¼1 r ki =m, Rj ¼ k¼1 r kj =m, 1 6 i 6 3 and i 6 j 6 4. For the attribute vector R1, calculate the average of the correlation coefficient soc1 of the attribute ‘‘Accuracy Rate’’ with respect to the other attributes ‘‘Time Rate’’, ‘‘Complexity’’ and ‘‘Importance’’, respectively, shown as follows:
soc1 ¼ jCofðR1 ; R2 Þj þ jCofðR1 ; R3 Þj þ jCofðR1 ; R4 Þj:
soc2 ¼ jCofðR1 ; R2 Þj þ jCofðR2 ; R3 Þj þ jCofðR2 ; R4 Þj:
ð12Þ
For the attribute vector R3, calculate the average of the correlation coefficient soc3 of the attribute ‘‘Complexity’’ with respect to other attributes ‘‘Accuracy Rate’’, ‘‘Time Rate’’ and ‘‘Importance’’, respectively, shown as follows:
soc3 ¼ jCofðR1 ; R3 Þj þ jCofðR2 ; R3 Þj þ jCofðR3 ; R4 Þj:
ð13Þ
For the attribute vector R4, calculate the average of the correlation coefficient soc4 of the attribute ‘‘Importance’’ with respect to other attributes ‘‘Accuracy Rate’’, ‘‘Time Rate’’ and ‘‘Complexity’’, respectively, shown as follows:
soc4 ¼ jCofðR1 ; R4 Þj þ jCofðR2 ; R4 Þj þ jCofðR3 ; R4 Þj:
ð14Þ
Based on the attribute vector Ri, calculate the standard deviation Std(Ri) of the attribute vector Ri, shown as follows:
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u X u1 m StdðRi Þ ¼ t ðr ji Ri Þ2 ; n j¼1
pfi ¼ 1 and 1 6 j 6 4:
i¼1
3 2 Accuracy Rate w1 7 6 Time Rate 6 w2 7 7; 6 W¼ 7 6 Complexity 4 w3 5 Importance
w4
i where wi ¼ pi þpf , wi 2 [0, 1] denotes the weight of the ith attribute 2 and 1 6 i 6 4. That is, the weights of the attributes ‘‘Accuracy Rate’’, ‘‘Time Rate’’, ‘‘Complexity’’ and ‘‘Importance’’ are w1, w2, w3 and w4, respectively, where wi 2 [0, 1] and 1 6 i 6 4. Based on the weighting vector W of the attributes and the relation matrix R, use the simple additive weighting (SAW) method (Hwang & Yoon, 1981) to obtain the importance degree vector ID of the questions, shown as follows:
where
idi ¼ ri1 w1 þ r i2 w2 þ r i3 w3 þ r i4 w4 ;
ð17Þ
and 1 6 i 6 m. Step 5: Based on the importance degree vector ID of the questions, the grade matrix G, the and accuracy rate matrix A, calculate the learning achievement index LAj of student Sj having the same original total score, shown as follows:
LAj ¼
m X
aij g i idi ;
ð18Þ
i¼1
ð15Þ
P where Ri ¼ m j¼1 r ji =n and 1 6 i 6 4. Based on the average of the correlation coefficient soci and the standard deviation Std(Ri) of the attribute vector Ri, calculate the fitness degree Fit(Ri) of the attribute vector Ri, shown as follows:
1
B soci C C B FitðRi Þ ¼ B1 4 C StdðRi Þ; A @ P sock
4 X
Step 4: Based on the percentage vector P obtained in Step 2 and the percentage vector PF obtained in Step 3, calculate the weighting vector W of the attributes, shown as follows:
ð11Þ
For the attribute vector R2, calculate the average of the correlation coefficient soc2 of the attribute ‘‘Time Rate’’ with respect to other attributes ‘‘Accuracy Rate’’, ‘‘Complexity’’ and ‘‘Importance’’, respectively, shown as follows:
0
FitðRi Þ pfi ¼ P4 ; k¼1 FitðRk Þ
ð16Þ
k¼1
where Fit(Ri) denotes the fitness degree for using the attribute vector Ri to distinguish the ranking order of the students having the same original total score, Fit(Ri) 2 [0, 1] and 1 6 j 6 4. The bigger
where 1 6 j 6 n. The larger the value of LAj, the better the ranking order of student Sj, where 1 6 j 6 n. Sort the learning achievement indices of the students having the same original total score in a descending sequence to get the new ranking order of the students. 4. An example In this section, we use the example shown in Bai and Chen (2008c) to illustrate the process of students’ learning performance evaluation of the proposed method. Assume that there are five questions Q1, Q2, Q3, Q4 and Q5 in the answerscript and assume that there are 10 students S1, S2, . . . , and S10 to answer these questions. Assume that the accuracy rate matrix A, the time rate matrix T, the score matrix G, the importance matrix IM and the complexity matrix C are shown as follows:
S.-M. Chen, T.-K. Li / Expert Systems with Applications 38 (2011) 8240–8250
8245
Because TS9 > TS1 > TS2 > TS8 > TS4 = TS5 = TS10 > TS6 > TS7 > TS3, we can see that the ranking order of the students is S9 > S1 > S2 > S8 > S4 = S5 = S10 > S6 > S7 > S3, where the students S4, S5 and S10 have the same original total score. Then, we can apply the proposed method to deal with the new ranking order of the students S4, S5 and S10, shown as follows: [Step 1] Based on the accuracy matrix A and Eq. (2), we can get the average accuracy rate vector A, shown as follows:
1 ¼ a
ð0:59 þ 0:35 þ 1 þ 0:66 þ 0:11 þ 0:08 þ 0:84 þ 0:23 þ 0:4 þ 0:24Þ ¼ 0:45; 10
2 ¼ a
ð0:01 þ 0:27 þ 0:14 þ 0:04 þ 0:88 þ 0:16 þ 0:04 þ 0:22 þ 0:81 þ 0:53Þ ¼ 0:31; 10
3 ¼ a
ð0:77 þ 0:69 þ 0:97 þ 0:71 þ 0:17 þ 0:86 þ 0:87 þ 0:42 þ 0:91 þ 0:74Þ ¼ 0:711; 10
4 ¼ a
ð0:73 þ 0:72 þ 0:18 þ 0:16 þ 0:5 þ 0:02 þ 0:32 þ 0:92 þ 0:9 þ 0:25Þ ¼ 0:47; 10
5 ¼ a
ð0:93 þ 0:49 þ 0:08 þ 0:81 þ 0:65 þ 0:93 þ 0:39 þ 0:51 þ 0:97 þ 0:61Þ ¼ 0:637; 10
Based on the time rate matrix T and Eq. (3), we can get the average time rate vector T, shown as follows: t 1 ¼ ð0:7 þ 0:4 þ 0:1 þ 1 þ 0:7 þ 0:2 þ 0:7 þ 0:6 þ 0:4 þ 0:9Þ ¼ 0:57; 10
t 2 ¼ ð1 þ 0 þ 0:9 þ 0:3 þ 1 þ 0:3 þ 0:2 þ 0:8 þ 0 þ 0:3Þ ¼ 0:48; 10
Based on the accuracy rate matrix A, the score matrix G and Eq. (1), we can get the total score of each student, where the total scores TS1, TS2, . . . , and TS10 of the students S1, S2, . . . , and S10 are calculated as follows:
t 3 ¼ ð0 þ 0:1 þ 0 þ 0:1 þ 0:9 þ 1 þ 0:2 þ 0:3 þ 0:1 þ 0:4Þ ¼ 0:31; 10
TS1 ¼ 0:59 10 þ 0:01 15 þ 0:77 20 þ 0:73 25 þ 0:93 30 ¼ 67:60; TS2 ¼ 0:35 10 þ 0:27 15 þ 0:69 20 þ 0:72 25 þ 0:49 30 ¼ 54:05; TS3 ¼ 1 10 þ 0:14 15 þ 0:97 20 þ 0:18 25 þ 0:08 30 ¼ 38:40;
t 4 ¼ ð0:2 þ 0:1 þ 0 þ 1 þ 1 þ 0:3 þ 0:4 þ 0:8 þ 0:7 þ 0:5Þ ¼ 0:5; 10
TS4 ¼ 0:66 10 þ 0:04 15 þ 0:71 20 þ 0:16 25 þ 0:81 30 ¼ 49:70; TS5 ¼ 0:11 10 þ 0:88 15 þ 0:17 20 þ 0:5 25 þ 0:65 30 ¼ 49:70;
t 5 ¼ ð0 þ 0:1 þ 1 þ 1 þ 0:6 þ 1 þ 0:8 þ 0:2 þ 0:8 þ 0:2Þ ¼ 0:57; 10
TS6 ¼ 0:08 10 þ 0:16 15 þ 0:86 20 þ 0:02 25 þ 0:93 30 ¼ 48:80; TS7 ¼ 0:84 10 þ 0:04 15 þ 0:87 20 þ 0:32 25 þ 0:39 30 ¼ 46:10; TS8 ¼ 0:23 10 þ 0:22 15 þ 0:42 20 þ 0:92 25 þ 0:51 30 ¼ 52:30; TS9 ¼ 0:4 10 þ 0:81 15 þ 0:91 20 þ 0:9 25 þ 0:97 30 ¼ 85:95; TS10 ¼ 0:24 10 þ 0:53 15 þ 0:74 20 þ 0:25 25 þ 0:61 30 ¼ 49:70:
Based on the average accuracy rate vector A and Eq. (4), we can obtain the effect of accuracy rate vector EOA, shown as follows:
8246
S.-M. Chen, T.-K. Li / Expert Systems with Applications 38 (2011) 8240–8250
eoa1
eoa2
eoa3
eoa4
eoa5
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3ffi u 2 u ð0:59 0:45Þ2 þ ð0:35 0:45Þ2 þ ð1 0:45Þ2 þ ð0:66 0:45Þ2 þ u1 6 7 2 2 2 2 ¼u t10 4 ð0:11 0:45Þ þ ð0:08 0:45Þ þ ð0:84 0:45Þ þ ð0:23 0:45Þ þ 5 ¼ 0:296; ð0:4 0:45Þ2 þ ð0:24 0:45Þ2 vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3ffi u 2 u ð0:01 0:31Þ2 þ ð0:27 0:31Þ2 þ ð0:14 0:31Þ2 þ ð0:04 0:31Þ2 þ u1 6 7 2 2 2 2 ¼u t10 4 ð0:88 0:31Þ þ ð0:16 0:31Þ þ ð0:04 0:31Þ þ ð0:22 0:31Þ þ 5 ¼ 0:303; 2 2 ð0:81 0:31Þ þ ð0:53 0:31Þ vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3ffi u 2 2 u ð0:77 0:711Þ þ ð0:69 0:711Þ2 þ ð0:97 0:711Þ2 þ ð0:71 0:711Þ2 þ u1 6 7 2 2 2 2 ¼u t10 4 ð0:17 0:711Þ þ ð0:86 0:711Þ þ ð0:87 0:711Þ þ ð0:42 0:711Þ þ 5 ¼ 0:232; 2 2 ð0:91 0:711Þ þ ð0:74 0:711Þ vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3ffi u 2 2 u ð0:73 0:47Þ þ ð0:72 0:47Þ2 þ ð0:18 0:47Þ2 þ ð0:16 0:47Þ2 þ u1 6 7 2 2 2 2 ¼u t10 4 ð0:5 0:47Þ þ ð0:02 0:47Þ þ ð0:32 0:47Þ þ ð0:92 0:47Þ þ 5 ¼ 0:312; ð0:9 0:47Þ2 þ ð0:25 0:47Þ2 vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3ffi u 2 u ð0:93 0:637Þ2 þ ð0:49 0:637Þ2 þ ð0:08 0:637Þ2 þ ð0:81 0:637Þ2 þ u1 6 7 2 2 2 2 ¼u t10 4 ð0:65 0:637Þ þ ð0:93 0:637Þ þ ð0:39 0:637Þ þ ð0:51 0:637Þ þ 5 ¼ 0:269; ð0:97 0:637Þ2 þ ð0:61 0:637Þ2
Based on the average time rate vector T and Eq. (5), we can obtain the effect of time rate vector EOT, shown as follows:
eot1
eot2
eot3
eot4
eot5
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3ffi u 2 u ð0:7 0:57Þ2 þ ð0:4 0:57Þ2 þ ð0:1 0:57Þ2 þ ð1 0:57Þ2 þ u1 6 7 2 2 2 2 ¼u t10 4 ð0:7 0:57Þ þ ð0:2 0:57Þ þ ð0:7 0:57Þ þ ð0:6 0:57Þ þ 5 ¼ 0:276; ð0:4 0:57Þ2 þ ð0:9 0:57Þ2 vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3 u 2 u ð1 0:48Þ2 þ ð0 0:48Þ2 þ ð0:9 0:48Þ2 þ ð0:3 0:48Þ2 þ u1 6 7 2 2 2 2 ¼u t10 4 ð1 0:48Þ þ ð0:3 0:48Þ þ ð0:2 0:48Þ þ ð0:8 0:48Þ þ 5 ¼ 0:382; ð0 0:48Þ2 þ ð0:3 0:48Þ2 vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3 u 2 u ð0 0:31Þ2 þ ð0:1 0:31Þ2 þ ð0 0:31Þ2 þ ð0:1 0:31Þ2 þ u1 6 7 2 2 2 2 ¼u t10 4 ð0:9 0:31Þ þ ð1 0:31Þ þ ð0:2 0:31Þ þ ð0:3 0:31Þ þ 5 ¼ 0:342; 2 2 ð0:1 0:31Þ þ ð0:4 0:31Þ vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3 u 2 2 u ð0:2 0:5Þ þ ð0:1 0:5Þ2 þ ð0 0:5Þ2 þ ð1 0:5Þ2 þ u1 6 7 2 2 2 2 ¼u t10 4 ð1 0:5Þ þ ð0:3 0:5Þ þ ð0:4 0:5Þ þ ð0:8 0:5Þ þ 5 ¼ 0:344; 2 2 ð0:7 0:5Þ þ ð0:5 0:5Þ vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3 u 2 2 u ð0 0:57Þ þ ð0:1 0:57Þ2 þ ð1 0:57Þ2 þ ð1 0:57Þ2 þ u1 6 7 2 2 2 2 ¼u t10 4 ð0:6 0:57Þ þ ð1 0:57Þ þ ð0:8 0:57Þ þ ð0:2 0:57Þ þ 5 ¼ 0:385; ð0:8 0:57Þ2 þ ð0:2 0:57Þ2
Based on the complexity matrix C and Eq. (6), we can obtain the crisp complexity vector CC, shown as follows:
0:1 0 þ 0:3 0:85 þ 0:5 0:15 þ 0:7 0 þ 0:9 0 ¼ 0:33; 0 þ 0:85 þ 0:15 þ 0 þ 0 0:1 0 þ 0:3 0 þ 0:5 0:33 þ 0:7 0:67 þ 0:9 0 ¼ 0:634; ¼ 0 þ 0 þ 0:33 þ 0:67 þ 0 0:1 0 þ 0:3 0 þ 0:5 0 þ 0:7 0:69 þ 0:9 0:31 ¼ ¼ 0:762; 0 þ 0 þ 0 þ 0:69 þ 0:31 0:1 0:56 þ 0:3 0:44 þ 0:5 0 þ 0:7 0 þ 0:9 0 ¼ ¼ 0:188; 0:56 þ 0:44 þ 0 þ 0 þ 0 0:1 0 þ 0:3 0 þ 0:5 0:7 þ 0:7 0:3 þ 0:9 0 ¼ 0:56; ¼ 0 þ 0 þ 0:7 þ 0:3 þ 0
cc1 ¼ cc2 cc3 cc4 cc5
8247
S.-M. Chen, T.-K. Li / Expert Systems with Applications 38 (2011) 8240–8250 Table 1 The eigenvalues and the eigenvectors for the reciprocal matrix RE. Eigenvalues
Eigenvectors
4.048 0.442i 0.443i 0.047
[0.130 0.209 0.512 0.823 ] [0.009 0.134i 0.201 + 0.018i [0.009 + 0.134i 0.201 0.018i [0.128 0.237 0.563 0.781 ]
0.005 + 0.486i 0.84] 0.005 0.486i 0.84]
Based on the importance matrix IM and Eq. (7), we can obtain the crisp importance vector CI, shown as follows: 0:1 0 þ 0:3 0:85 þ 0:5 0:15 þ 0:7 0 þ 0:9 0 ci1 ¼ ¼ 0:9; 0 þ 0:85 þ 0:15 þ 0 þ 0 0:1 0 þ 0:3 0 þ 0:5 0:33 þ 0:7 0:67 þ 0:9 0 ¼ 0:434; ci2 ¼ 0 þ 0 þ 0:33 þ 0:67 þ 0 0:1 0 þ 0:3 0 þ 0:5 0 þ 0:7 0:69 þ 0:9 0:31 ¼ 0:87; ci3 ¼ 0 þ 0 þ 0 þ 0:69 þ 0:31 0:1 0:56 þ 0:3 0:44 þ 0:5 0 þ 0:7 0 þ 0:9 0 ¼ 0:1; ci4 ¼ 0:56 þ 0:44 þ 0 þ 0 þ 0 0:1 0 þ 0:3 0 þ 0:5 0:7 þ 0:7 0:3 þ 0:9 0 ¼ 0:486; ci5 ¼ 0 þ 0 þ 0:7 þ 0:3 þ 0
Based on the normalized vector SA’ and Eq. (8), we can obtain the reciprocal matrix RE, shown as follows:
We use Matlab 7.0 to solve the reciprocal matrix RE to get the eigenvalues and the eigenvectors, as shown in Table 1. Based on Table 1, the maximum eigenvalue kmax of the reciprocal matrix RE is 4.048, i.e., kmax = 4.048, and its corresponding eigenvector E is shown as follows:
3 2 Accuracy Rate 0:130 7 Time Rate 6 6 0:209 7 E¼ 7: 6 4 0:512 5 Complexity Importance
0:823
Based on the eigenvector E of the reciprocal matrix RE, we can obtain the percentage vector P of the eigenvector E, shown as follows:
0:13 ¼ 0:078; 0:13 þ 0:209 þ 0:512 þ 0:823 0:209 ¼ 0:125; p2 ¼ 0:13 þ 0:209 þ 0:512 þ 0:823 0:512 ¼ 0:306; p3 ¼ 0:13 þ 0:209 þ 0:512 þ 0:823 0:823 ¼ 0:492; p4 ¼ 0:13 þ 0:209 þ 0:512 þ 0:823
p1 ¼ Based on the effect of accuracy rate matrix EOA, the effect of time rate matrix EOT, the crisp complexity vector CC and the crisp importance vector CI, construct the relation matrix R, shown as follows:
3 2 Accuracy Rate 0:078 7 Time Rate 6 6 0:125 7 P¼ 7: 6 4 Complexity 0:306 5 Importance
0:492
[Step 3] Based on the relation matrix R and Eq. (9), we can get the attribute vectors R1, R2, R3 and R4, shown as follows: [Step 2] Based on the relation matrix R, we can obtain the vector SA, shown as follows:
sa1 ¼ 0:296 þ 0:303 þ 0:232 þ 0:312 þ 0:269 ¼ 1:411; sa2 ¼ 0:276 þ 0:382 þ 0:342 þ 0:344 þ 0:385 ¼ 1:728; sa3 ¼ 0:33 þ 0:634 þ 0:762 þ 0:188 þ 0:56 ¼ 2:474; sa4 ¼ 0:9 þ 0:434 þ 0:87 þ 0:1 þ 0:486 ¼ 2:79;
Then, we normalize the vector SA to get the normalized vector SA’, shown as follows:
1:411 ¼ 0:506; sa01 ¼ maxð1:411; 1:728; 2:474; 2:79Þ sa02
1:728 ¼ 0:619; ¼ maxð1:411; 1:728; 2:474; 2:79Þ
sa03 ¼
2:474 ¼ 0:887; maxð1:411; 1:728; 2:474; 2:79Þ
sa04 ¼
2:79 ¼ 1:000; maxð1:411; 1:728; 2:474; 2:79Þ
3 3 2 0:296 Q 1 0:276 7 7 6 6 Q 2 6 0:303 7 Q 2 6 0:382 7 7 7 6 6 7 7 6 R1 ¼ Q 3 6 6 0:232 7; R2 ¼ Q 3 6 0:342 7; 7 7 6 6 Q 4 4 0:312 5 Q 4 4 0:344 5 Q1
2
Q 5 0:269 Q 5 0:385 3 3 2 2 Q 1 0:330 Q 1 0:900 7 7 6 6 Q 2 6 0:634 7 Q 2 6 0:434 7 7 7 6 6 7 7 6 R3 ¼ Q 3 6 6 0:762 7; R4 ¼ Q 3 6 0:870 7: 7 7 6 6 Q 4 4 0:188 5 Q 4 4 0:100 5 Q 5 0:560 Q 5 0:486 Then, we can get
R1 ¼ ð0:296 þ 0:303 þ 0:232 þ 0:312 þ 0:269Þ=5 ¼ 0:282; R2 ¼ ð0:276 þ 0:382 þ 0:342 þ 0:344 þ 0:385Þ=5 ¼ 0:346; R3 ¼ ð0:33 þ 0:634 þ 0:762 þ 0:188 þ 0:56Þ=5 ¼ 0:495; R4 ¼ ð0:9 þ 0:434 þ 0:87 þ 0:1 þ 0:486Þ=5 ¼ 0:558: Based on the attribute vectors R1, R2, R3, R4, the values of R1 , R2 , R3 and R4 and Eq. (10), we can calculate the correlation coefficients between attribute vectors, shown as follows:
8248
S.-M. Chen, T.-K. Li / Expert Systems with Applications 38 (2011) 8240–8250
2
ð0:296 0:282Þð0:276 0:346Þ þ ð0:303 0:282Þð0:382 0:346Þþ
1 6 ð0:232 51 4
3
7 0:282Þð0:342 0:346Þ þ ð0:312 0:282Þð0:344 0:346Þþ 5
ð0:269 0:282Þð0:385 0:346Þ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi CofðR1 ; R2 Þ ¼ vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3ffiv 3ffi ¼ 0:105; u 2 u 2 2 ð0:296 0:282Þ þ ð0:303 0:282Þ2 þ u ð0:276 0:346Þ2 þ ð0:382 0:346Þ2 þ u u 6 u 7 2 2 1 6 u 1 4 ð0:232 0:282Þ2 þ ð0:312 0:282Þ2 þ 7 5u t51 t51 4 ð0:342 0:346Þ þ ð0:344 0:346Þ þ 5 ð0:269 0:282Þ2 ð0:385 0:346Þ2 2
3 ð0:296 0:282Þð0:33 0:495Þ þ ð0:303 0:282Þð0:634 0:495Þþ 6 7 1 ð0:232 0:282Þð0:762 0:495Þ þ ð0:312 0:282Þð0:188 0:495Þþ 5 51 4 ð0:269 0:282Þð0:56 0:495Þ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi CofðR1 ; R3 Þ ¼ vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3ffiv 3ffi ¼ 0:751; u 2 u 2 ð0:296 0:282Þ2 þ ð0:303 0:282Þ2 þ u ð0:33 0:495Þ2 þ ð0:634 0:495Þ2 þ u u 6 u 6 7 2 2 1 u 1 4 ð0:232 0:282Þ2 þ ð0:312 0:282Þ2 þ 7 5u t51 t51 4 ð0:762 0:495Þ þ ð0:188 0:495Þ þ 5 ð0:269 0:282Þ2
ð0:56 0:495Þ2
2
3 ð0:296 0:282Þð0:9 0:588Þ þ ð0:303 0:282Þð0:434 0:588Þþ 7 1 6 ð0:232 0:282Þð0:87 0:588Þ þ ð0:312 0:282Þð0:1 0:588Þþ 5 51 4 ð0:269 0:282Þð0:486 0:588Þ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi CofðR1 ; R4 Þ ¼ vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3ffiv 3 ¼ 0:605; u 2 u 2 2 ð0:296 0:282Þ þ ð0:303 0:282Þ2 þ u ð0:9 0:588Þ2 þ ð0:434 0:588Þ2 þ u u 6 u 7 2 2 1 6 u 1 4 ð0:232 0:282Þ2 þ ð0:312 0:282Þ2 þ 7 5u t51 t51 4 ð0:87 0:588Þ þ ð0:1 0:588Þ þ 5 ð0:269 0:282Þ2 ð0:486 0:588Þ2 2
3 ð0:276 0:346Þð0:33 0:495Þ þ ð0:382 0:346Þð0:634 0:495Þþ 6 7 1 ð0:342 0:346Þð0:762 0:495Þ þ ð0:344 0:346Þð0:188 0:495Þþ 5 51 4 ð0:385 0:346Þð0:56 0:495Þ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi CofðR2 ; R3 Þ ¼ vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 3ffiv 3ffi ¼ 0:458; u 2 u 2 ð0:276 0:346Þ2 þ ð0:382 0:346Þ2 þ u ð0:33 0:495Þ2 þ ð0:634 0:495Þ2 þ u u 6 u 6 7 2 2 1 u 1 4 ð0:342 0:346Þ2 þ ð0:344 0:346Þ2 þ 7 5u t51 t51 4 ð0:762 0:495Þ þ ð0:188 0:495Þ þ 5 ð0:385 0:346Þ2
2 1 6 51 4
ð0:56 0:495Þ2
ð0:276 0:346Þð0:9 0:588Þ þ ð0:382 0:346Þð0:434 0:588Þþ
3
7 ð0:342 0:346Þð0:87 0:588Þ þ ð0:344 0:346Þð0:1 0:588Þþ 5 ð0:385 0:346Þð0:486 0:588Þ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi CofðR2 ; R4 Þ ¼ v 3ffiv 3 ¼ 0:535; u 2 u 2 2 ð0:276 0:346Þ þ ð0:382 0:346Þ2 þ u ð0:9 0:588Þ2 þ ð0:434 0:588Þ2 þ u u 6 u 7 2 2 1 6 u 1 4 ð0:342 0:346Þ2 þ ð0:344 0:346Þ2 þ 7 5u t51 t51 4 ð0:87 0:588Þ þ ð0:1 0:588Þ þ 5 ð0:385 0:346Þ2 ð0:486 0:588Þ2 2
3 ð0:33 0:495Þð0:9 0:588Þ þ ð0:634 0:495Þð0:434 0:588Þþ 6 7 1 ð0:762 0:495Þð0:87 0:588Þ þ ð0:188 0:495Þð0:1 0:588Þþ 5 51 4 ð0:56 0:495Þð0:486 0:588Þ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi CofðR3 ; R4 Þ ¼ v 3ffiv 3 ¼ 0:469; u 2 u 2 ð0:33 0:495Þ2 þ ð0:634 0:495Þ2 þ u ð0:9 0:588Þ2 þ ð0:434 0:588Þ2 þ u u 6 u 6 7 2 2 1 u 1 4 ð0:762 0:495Þ2 þ ð0:188 0:495Þ2 þ 7 5u t51 t51 4 ð0:87 0:588Þ þ ð0:1 0:588Þ þ 5 ð0:56 0:495Þ2
ð0:486 0:588Þ2
Based on Eqs. (11)–(14), we can get
soc1 ¼ jCofðR1 ; R2 Þj þ jCofðR1 ; R3 Þj þ jCofðR1 ; R4 Þj ¼ 1:461; soc2 ¼ jCofðR1 ; R2 Þj þ jCofðR2 ; R3 Þj þ jCofðR2 ; R4 Þj ¼ 1:098; soc3 ¼ jCofðR1 ; R3 Þj þ jCofðR2 ; R3 Þj þ jCofðR3 ; R4 Þj ¼ 1:679; soc4 ¼ jCofðR1 ; R4 Þj þ jCofðR2 ; R4 Þj þ jCofðR3 ; R4 Þj ¼ 1:609; Based on the attribute vectors R1, R2, R3, R4, the values of R1 , R2 , R3 , R4 , and Eq. (15), we can calculate the standard deviations Std(R1), Std(R2), Std(R3) and Std(R4) of the attribute vectors R1, R2, R3 and R4, respectively, shown as follows:Based on the values of soc1,
soc2, soc3, soc4, the standard deviations Std(R1), Std(R2), Std(R3), Std(R4) and Eq. (16), we can calculate the fitness degrees Fit(R1), Fit(R2), Fit(R3) and Fit(R4) of the attribute vectors R1, R2, R3 and R4, respectively, shown as follows:
S.-M. Chen, T.-K. Li / Expert Systems with Applications 38 (2011) 8240–8250
8249
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi # u " u1 ð0:296 0:282Þ2 þ ð0:303 0:282Þ2 þ ð0:232 0:282Þ2 þ ¼ 0:022; StdðR1 Þ ¼ t 5 ð0:312 0:282Þ2 þ ð0:269 0:282Þ2 vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi # u " u1 ð0:276 0:346Þ2 þ ð0:382 0:346Þ2 þ ð0:342 0:346Þ2 þ t StdðR2 Þ ¼ ¼ 0:039; 5 ð0:344 0:346Þ2 þ ð0:385 0:346Þ2 vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi # u " u1 ð0:330 0:495Þ2 þ ð0:634 0:495Þ2 þ ð0:762 0:495Þ2 þ ¼ 0:208; StdðR3 Þ ¼ t 5 ð0:188 0:495Þ2 þ ð0:560 0:495Þ2 vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi # u " u1 ð0:900 0:558Þ2 þ ð0:434 0:558Þ2 þ ð0:870 0:558Þ2 þ t ¼ 0:298: StdðR4 Þ ¼ 5 ð0:100 0:558Þ2 þ ð0:486 0:558Þ2 1:461 FitðR1 Þ ¼ 1 0:029 ¼ 0:022; 1:461 þ 1:098 þ 1:679 þ 1:609 1:098 0:039 ¼ 0:032; FitðR2 Þ ¼ 1 1:461 þ 1:098 þ 1:679 þ 1:609 1:679 0:208 ¼ 0:148; FitðR3 Þ ¼ 1 1:461 þ 1:098 þ 1:679 þ 1:609 1:609 0:298 ¼ 0:216: FitðR4 Þ ¼ 1 1:461 þ 1:098 þ 0:679 þ 1:609
Based on the fitness degrees Fit(R1), Fit(R2), Fit(R3) and Fit(R4) of the attribute vectors R1, R2, R3 and R4, respectively, we calculate the percentages of the fitness degrees Fit(R1), Fit(R2), Fit(R3) and Fit(R4) of the attribute vectors R1, R2, R3 and R4, respectively, to obtain the percentage vector PF, shown as follows:
0:022 ¼ 0:052; 0:022 þ 0:032 þ 0:148 þ 0:216 0:032 ¼ 0:076; pf2 ¼ 0:022 þ 0:032 þ 0:148 þ 0:216 0:148 ¼ 0:355; pf3 ¼ 0:022 þ 0:032 þ 0:148 þ 0:216 0:216 ¼ 0:517; pf4 ¼ 0:022 þ 0:032 þ 0:148 þ 0:216
pf1 ¼
id1 ¼ 0:296 0:065 þ 0:276 0:101 þ 0:33 0:330 þ 0:9 0:504 ¼ 0:610; id2 ¼ 0:303 0:065 þ 0:382 0:101 þ 0:634 0:330 þ 0:434 0:504 ¼ 0:486; id3 ¼ 0:232 0:065 þ 0:342 0:101 þ 0:762 0:330 þ 0:87 0:504 ¼ 0:740;: id4 ¼ 0:312 0:065 þ 0:344 0:101 þ 0:188 0:330 þ 0:1 0:504 ¼ 0:167; id5 ¼ 0:269 0:065 þ 0:385 0:101 þ 0:56 0:330 þ 0:486 0:504 ¼ 0:486;
[Step 5] Based on the importance degree vector ID, the grade matrix G, the accuracy rate matrix A and Eq. (18), we can calculate the learning achievement indices LA4, LA5 and LA10 of the students S4, S5 and S10 having the same original total score, respectively, shown as follows:
LA4 ¼ 0:66 10 0:610 þ 0:04 15 0:486 þ 0:71 20 0:740 þ 0:16 25 0:167 þ 0:81 30 0:486 ¼ 27:303; LA5 ¼ 0:11 10 0:610 þ 0:88 15 0:486 þ 0:17 20 0:740 þ 0:5 25 0:167 þ 0:65 30 0:486 ¼ 21:176; LA10 ¼ 0:24 10 0:610 þ 0:53 15 0:486 þ 0:74 20 0:740 þ 0:25 25 0:167 þ 0:61 30 0:486 ¼ 26:220: Because LA4 > LA10 > LA5, the new ranking order of the 10 students is
[Step 4] Based on the percentage vector P obtained in Step 2 and the percentage vector PF obtained in Step 3, we can get the weighting vector W of the attributes, shown as follows:
0:078 þ 0:052 ¼ 0:065; 2 0:125 þ 0:076 ¼ 0:101; w2 ¼ 2 0:306 þ 0:355 ¼ 0:330; w3 ¼ 2 0:492 þ 0:517 w4 ¼ ¼ 0:504; 2
w1 ¼
Based on the weighting vector W of the attributes, the relation matrix R and Eq. (17), we can get the importance degree vector ID of the questions, shown as follows:
S9 > S1 > S2 > S8 > S4 > S10 > S5 > S6 > S7 > S3 : The new ranking order of the proposed method is the same as the one of Bai and Chen’s method (2008c). 5. Conclusions In this paper, we have presented a new method for students’ learning achievement evaluation based on the eigenvector method (Chen & Hwang, 1992). The proposed method considers the ‘‘accuracy rate’’, the ‘‘time rate’’, the ‘‘importance’’ and the ‘‘complexity’’ for evaluating students’ learning achievement. First, the proposed method transforms the attributes ‘‘accuracy rate’’ and ‘‘time rate’’ into the ‘‘effect of accuracy rate’’ and the ‘‘effect of time rate’’, respectively. Then, it generates the relative importance degrees of the attributes ‘‘effect of accuracy rate’’, ‘‘effect of time rate’’, ‘‘importance’’ and ‘‘complexity’’ based on the eigenvector method. Then, it uses the correlation coefficients between the attribute vectors and the standard deviations of the elements in the attribute vectors to calculate the fitness degrees of the attributes,
8250
S.-M. Chen, T.-K. Li / Expert Systems with Applications 38 (2011) 8240–8250
where the attribute vectors represent the relationships between the attributes and the questions. Then, it generates the weights of the attributes based on the relative importance degrees of the attributes and the fitness degrees of the attributes. Then, it generates the importance degrees of the questions according to the weights of the attributes and the relation matrix representing the relationships between the questions and the attributes. Finally, based on the importance degree vector of the questions, the grade matrix and the accuracy rate matrix, it calculates the learning achievement index of each student having the same original total score for students’ learning achievement evaluation. The proposed method provides us with a useful way for students’ learning achievement evaluation based on the eigenvector method. Acknowledgements This work was supported in part by the National Science Council, Republic of China, under Grant NSC 97–2221-E-011–107-MY3. References Bai, S. M., & Chen, S. M. (2008a). Automatically constructing concept maps based on fuzzy rules for adaptive learning systems. Expert Systems with Applications, 35(1), 41–49. Bai, S. M., & Chen, S. M. (2008b). Automatically constructing grade membership functions of fuzzy rules for students’ evaluation. Expert Systems with Applications, 35(3), 1408–1414. Bai, S. M., & Chen, S. M. (2008c). Evaluating students’ learning achievement using fuzzy membership functions and fuzzy rules. Expert Systems with Applications, 34(1), 399–410.
Biswas, R. (1995). An application of fuzzy sets in students’ evaluation. Fuzzy Sets and Systems, 74(2), 187–194. Chen, S. M., & Lee, C. H. (1999). New methods for students’ evaluating using fuzzy sets. Fuzzy Sets and Systems, 104(2), 209–218. Chen, S. M., & Li, T. K. (2010). A new method for students’ learning achievement evaluation based on the eigvenvector method. In Proceedings of the 2010 international conference on machine learning and cybernetics (pp. 2483–2488), Qingdao, Shandong, China. Chen, S. J., & Hwang, C. L. (1992). Fuzzy multiple attribute decision making. NY: Springer-Verlag. 332–333. Echauz, J. R., & Vachtsevanos, G. J. (1995). Fuzzy grading system. IEEE Transactions on Education, 38(2), 158–165. Greub, W. H. (1975). Linear algebra. New York: Springer-Verlag. Hwang, C. L., & Yoon, K. S. (1981). Multiple Attribute Decision making: Methods and Applications. Berlin: Springer-Verlag. Law, C. K. (1996). Using fuzzy numbers in education grading system. Fuzzy Sets and Systems, 83(3), 311–323. Ma, J., & Zhou, D. (2000). Fuzzy set approach to the assessment of student-centered learning. IEEE Transactions on Education, 43(2), 237–241. Wang, H. Y., & Chen, S. M. (2008). Evaluating students’ answer-scripts using fuzzy numbers associated with degrees of confidence. IEEE Transactions on Fuzzy Systems, 16(2), 403–415. Weon, S., & Kim, J. (2001). Learning achievement evaluation strategy using fuzzy membership function. In Proceedings of the 31th ASEE/IEEE frontiers in education conference (pp. 19–24), Reno, NV. Wilson, E., Karr, C. L., & Freeman, L. M. (1998). Flexible, adaptive, automatic fuzzybased grade assigning system. In Proceedings of the 1998 North American fuzzy information processing society conference (pp. 334–338). Wu, M. H. (2003). Research on applying fuzzy set theory and item response theory to evaluate learning performance. Master Thesis, Department of Information Management, Chaoyang University of Technology, Wufeng, Taichung Country, Republic of China. Zadeh, L. A. (1965). Fuzzy sets. Information and Control, 8, 338–353.