- Email: [email protected]
A mining-based approach on discovering courses pattern for constructing suitable learning path
Expert Systems with Applications 37 (2010) 4156–4167
Contents lists available at ScienceDirect
Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa
A mining-based approach on discovering courses pattern for constructing suitable learning path Tung-Cheng Hsieh *, Tzone-I Wang Department of Engineering Science, National Cheng Kung University, No. 1, Ta-Hsueh Road, Tainan 701, Taiwan, ROC
a r t i c l e
i n f o
Keywords: Self-directed learner Data mining Formal Concept Analysis (FCA) Concept Lattice Learning path
a b s t r a c t In recent years, browser has become one of the most popular tools for searching information on the Internet. Although a person can conveniently find and download specific learning materials to gain fragmented knowledge, most of the materials are imperfect and have no particular order in the content. Therefore, most of the self-directed learners spend most of time in surveying and choosing the right learning materials collected from the Internet. This paper develops a web-based learning support system that harnesses two approaches, the learning path constructing approach and the learning object recommending approach. With collected documents and a learning subject from a learner, the system first discovers some candidate courses by using a data mining approach based on the Apriori algorithm. Next, the leaning path constructing approach, based on the Formal Concept Analysis, builds a Concept Lattice, using keywords extracted from some selected documents, to form a relationship hierarchy of all the concepts represented by the keywords. It then uses FCA to further compute mutual relationships among documents to decide a suitable learning path. For a chosen learning path, the support system uses both the preference-based and the correlation-based algorithms for recommending the most suitable learning objects or documents for each unit of the courses in order to facilitate more efficient learning for the learner. This e-learning support system can be embedded in any information retrieval system for surfers to do more efficient learning on the Internet. Ó 2009 Elsevier Ltd. All rights reserved.
1. Introduction With a faster, more accessible Internet, nowadays people tend to search and learn from Internet for fragmented knowledge. Educationists or enthusiasts, seeing the benefit of on-line material, feverishly set up their web sites to share their knowledge of specific domains (Rafaeli, Barak, Dan-Gur, & Toch, 2004). Though being versatile, these web sites generally follow no standards for content organization and presentation order. They might be just web pages or some learning objects embedded in web pages. When posted on to the Internet, collected and indexed by robots using keywords, and returned by powerful search engines, usually, a vast amount of them, homepages or learning objects, is returned directly to a user with no particular order. Even if they might really be related, a user still has to move forward and backward among the material trying to figure out which page to read first because the user might has had little or no experience in a specific domain. Although a user might have some intuitions about the domain but these intuitions are yet to be connected. Thus, an effective way of organizing col-
* Corresponding author. E-mail address: [email protected] (T.-C. Hsieh). 0957-4174/$ - see front matter Ó 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2009.11.007
lected material into systematic and sequenced learning contents would be a great help for users who are novel to a specific domain. Furthermore, Dietinger (2003) has pointed out many advantages of e-learning activities. One of them is the importance of ‘‘Adaptive Learning”. The learning content can be adapted according to each learner’s strengths and weaknesses to achieve the most efficient learning experience. That is, by judging the initial knowledge of a learner on a topic and the learner’s preferred learning style, an e-learning system can decide what learning content should be offered next (Papanikolaou, Grigoriadou, Magoulas, & Kornilakis, 2002). Many researchers have been devoting themselves into adaptive e-learning methodologies and platforms works (Tseng, Chu, Hwang, & Tsai, 2008). Some consider that the importance of a learner’s ability and the difficulty of courses are the key issues for developing the e-learning system (Chen & Chung, 2008; Huang, Huang, & Chen, 2007; Liu & Yang, 2005). Several of the studies emphasized their proposed system’s ability to construct curriculum sequencing to help self-learning. Chen, Liu, and Chang (2006) presented a Personalized Web-based Instruction System (PWIS) to construct suitable learning pathway based on a modified item response theory for helping learning. Colace, De Santo, and Iacone (2005) presented an approach, which can obtain the learning style and capabilities of each learner, to arrange
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167
the learning path adaptively with most suitable teaching contents. Chen (2008), in the mean time, developed a genetic-based personalized e-learning system to provide a learning path according to the courseware difficulty and the result of pre-test for individual learner. During last decade, many e-learning systems have been developed for assisting learners in learning in a variety of areas. Nevertheless, many of these systems are trying hard in considering the needs of particular learners and just in time learning in the course (Ismail, 2002). One methodology that is based on an ontological approach to aggregate and infer users’ intentions for learning objects retrieval is proposed in our previous study (Lee, Tsai, & Wang, 2008). However, this methodology lacks a part for efficiently making an appropriate learning sequence among retrieved learning objects for learners. Fig. 1 depicts, as an example, a scenario of a self-directed learner learning some materials for a subject. The self-directed learner issues a subject on the Internet, and then immediately retrieves some learning materials, in which with no particular order, based on his/her intentions by search engine. After that, learners need to spend most time in organizing and skipping the learning materials they have searched. Each node, cyclic, square, or triangle, represents a concept, which the learner should comprehend after learning the material. All of the cyclic nodes, colored in gray, are the concepts that the learner has realized before reading. A square node, colored in green, represents a concept that a learner still cannot realize after reading the contents in the material. A triangle node, colored in red, represents a main concept in the material. After reading material 1, there may be some concepts the learner cannot understand so that she/he has to try reading other materials to support her/his learning by her/ himself, in Fig. 1’s case material 2 and so on. It would be very helpful for self-directed learners if there is a mechanism that automatically construct learning paths and recommend learning objects according to a learner’s intentions. Furthermore, in e-learning paradigm, researchers have been focused on sorting, automatically or semi-automatically, collected learning contents of a domain into predefined classes of a platform’s, which are drawn up by experts of the domain. These platforms can construct and recommend learning path base on the predefined classes and the hierarchical organization of learning contents, but automatic learning path construction for collected learning contents without human expert interference is current
4157
missing and would be of great help for users if it is built and recommended the material. For such self-directed learners, this paper develops a web-based learning support system, which harnesses two main approaches, the learning path constructing approach and the learning object recommending approach. The learning path constructing approach finds out a set of candidate courses first by using the data mining Apriori algorithm. It then uses Formal Concept Analysis to constructs a Concept Lattice by using keywords extracted from the collected documents to generate a relationship hierarchy between all the concepts represented by the keywords. It also uses FCA to further compute mutual relationships among documents to decide a suitable learning path. For each course units in a chosen learning path, the support system will select the most suitable documents or learning objects, using algorithms based on both learner preference and learner correlation, in order to facilitate more efficient learning for a learner. The rest of this paper is organized as follows. Section 2 draws the architecture of the proposed approach and algorithms in detail, Section 3 presents some experimental results and discussions, and the conclusion is in Section 4. 2. System architecture In this paper, several approaches are used to automatically construct a suitable learning path and recommend suitable contents from collected materials. The architecture of the proposed system is shown in Fig. 2. The major components carries out the procedure are the Learning Interface, the Candidate Course Generator, the Learning Object Content Preprocessor, the Learning Object Correlative Weighting Generator and the Learning Object Recommendation Module. 2.1. System description A learner logins to the Learner Interface to give a subject (a learning goal or a short query) he/she intends to learn. If it is a new subject that is never queried before, the Learner Interface will send out retrieval agents to collect learning materials from the Internet based on the learner’s learning subject and stores them into the learning objects repository. Domain experts can also
Fig. 1. A learning behavior example of self-directed learners.
4158
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167
Fig. 2. The system architecture.
plug-in a file which describes specific knowledge, such as course units. Base on a subject, the Candidate Course Generator discover several courseware unit patterns by data mining on the collected learning resources using the well-known Apriori algorithm. Then the Learning Object Content Preprocessor is notified to find out all the significant keywords of every learning object, using an Adapted TF–IDF (ATF–IDF) algorithm. After that, the Learning Object Correlative Weighting Generator constructs a hierarchy of relationships between all the concepts represented by the keywords and uses FCA to further compute the mutual relationships among all the learning objects or documents to decide a suitable learning path. After a learning paths is selected, the Learning Object Recommendation Module uses both the preference-based and the correlationbased algorithms, which ranks the degree of relevance of learning objects or documents to a learner’s intension and preference, for recommending the most suitable learning objects or documents for each unit of the courses in order to facilitate more efficient learning for all the learners. The details of all the system components are described as follows.
2.2. System components 2.2.1. Learning interface To develop a secure and stable learning environment, the open source and web-based course management system Moodle is employed. Moodle, is acronym for Modular Object-Oriented Dynamic Learning Environment, which allows easy learner management, courseware construction, as well as rich teaching activity via browsers (Moodle). To the developed learning environment, a learner gives a subject to learn via the Learner Interface. When a learner login, the system sets up or fetches and updates the learner’s profile, i.e. her/his preference, feedback and portfolio, and then stores the profile into the learner’s profile database. With these profile data the proposed approach will decide an appropriate learning path and choose learning objects in accordance with the learner’s learning subject. From this Learner Interface, some retrieval agents are issued, when necessary, to collect learning documents, web pages or learning objects, from the Internet. The retrieved learning materials then are put into the learning object repository. Fig. 3 displays a generated suitable learning path and the learning objects recommended.
2.2.2. Candidate course generator The major function of the Candidate Course Generator is to find out a course pattern, which contains a set of course units according to a learner’s learning subject, issued by a learner, and based on which, the Generator retrieves some learning objects from the Learning Object Repository. Fig. 4 illustrates the processes of candidate course unit generator. Before find out a course pattern, a material is processed to have all its meaningful keywords filtered and tagged (CKIP AutoTag, 1998). Each learning material will record a set of course unit themes, which is called a themeset (set of themes). The Generator then collects a set of course unit themes from these learning objects, and based on which, finds other related candidate course units in order to be more accordant with the learner’s learning subject. A data mining approach, the Apriori algorithm (Agrawal, Imielinski, & Swami, 1993; Agrawal & Srikant, 1994), is adopted to discover the association rules among all collected course unit themes. Assume an itemset I ¼ fi1 ; i2 ; i3 ; . . . ; im g is a set of items. An itemset is called a large-itemset if its support value is greater or equal to the user-specified support threshold (called minSupport). An association rule is an expression X ) Y where X and Y are disjoint itemsets, which represents possibility when X appears that Y will also appear. The support of an association rule is the support of X [ Y, and the confidence of such a rule is the fraction of all transactions containing X that also contain Y (Hidber, 1999). The following is the detail of the Apriori algorithm. Apriori algorithm Input: Learning objects repository (LOR), Threshold of minimum support value (minSupport). Output: Large itemsets in learning objects repository (LI). Procedure: 1: LI1 = find large 1-itemsets in LOR. 2: For (k=2; LIk-1–/;k++){ Ck = apriori-gen(LIk-1); // New candidates 3: for all of records r 2 LOR { Ct = subset(Ck, r) // Candidates contained in r for all of candidates c 2 Ct c.count++;} LIk ¼ fc 2 C k jc:count P min Supportg} 4: Return LI ¼ [k LIk ; This paper uses JAVA programming language as an example. Domain experts have defined some course units for this specific do-
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167
4159
Fig. 3. The interface of developed learning environment.
Fig. 4. Processes of candidate course generator.
main and put them into the system. An example of candidate course unit generation by using the Apriori algorithm is shown in Fig. 5, which assumes that a learner intends to learn the ‘‘For-statements” subject in Java. The learning object repository (LOR) is searched to find five learning object that are associated with the ‘‘For-statements” theme and contain contents for among Break-statement, For-statement, Variables, Arrays, Data type, and If-statement themes
respectively. For simplicity, these themes are symbolized into six letters, A, B, C, D, E, and F in the example. The threshold values of minSupport and minConfidence are set to 3 and 0.5, respectively. After the operation of the Apriori algorithm, the large itemset {For-statement, Variables, and Data type} is found and the association rule {For-statement}){Variables, Data type} can be also generated. The result of this association rule indicates that ‘‘60%” of the
4160
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167
Fig. 5. An example of candidate courses generated by using the Apriori algorithm.
learning objects that are designated for the ‘‘For-statements” theme also mentioned both the subject themes of ‘‘Variables” and ‘‘Data type”. After that, the generator finds out all of the learning objects which are designated for the ‘‘For-statements”, ‘‘Variables” and ‘‘Data type” course themes, and forwards them to the next stage to find an appropriate learning path.
2.2.3. Learning object content preprocessor After discovering a set of candidate course units and their associated learning materials by the Apriori algorithm, the learning object content preprocessor extract all the important keywords from the materials. A usual way of doing such keyword extraction is to use TF–IDF (Term Frequency/Inverse Document Frequency) that finds out for a document those specific keywords that distinguishes the document from the others. The less a keyword of a document appears in other documents, the more specific the keyword is for the document and the higher the TF–IDF value is (Aizawa, 2003; Koprinska, Poon, Clark, & Chan, 2007). The formula of TF– IDF is shown in following,
.P k ðnij l¼1 nil Þ logðN=nj Þ r ij ¼ rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi . Pk Pt 2 2 j¼1 ðnij l¼1 nil Þ ½logðN=nj Þ
2.2.4. Learning object correlative weighting generator After significant keywords of each material are extracted, several steps are involved in computing mutual relationships. In this paper, the Formal Concept Analysis approach can be used to derive implicit relationships between concepts using learning objects and attributes associated with these concepts (Chi, 2007; Ganter & Wille, 1999; Wille, 1982). Three steps are required in this paper in using FCA to construct inter-relationships between significant keywords of learning objects. Furthermore, these relationships are used to calculate the correlative weighting that represent the coherences between learning objects in the final step. 2.2.4.1. Step 1: generate all the concepts. In this paper, the term (O, A, X) denotes a binary relation X between a set of objects O and a set of attributes A. That is X # O A, where O is the set of all the description statements of all the learning objects derived from the association rule and A is the set of all the significant keywords of all the learning objects derived from the same association rule. To generate a concept set C, assume S is a partial set of O, Q is a partial set of A, and both S # O and Q # A holds, the set of all the attributes of S is:
rðSÞ ¼ fa 2 Aj8o 2 S : ða; oÞ 2 Xg; ð1Þ
and the set of all the significant keywords of all the learning objects of Q is:
sðQ Þ ¼ fo 2 Oj8a 2 Q : ða; oÞ 2 Xg: where rij is the importance of keyword j in document i with its value between 0 and 1, nij is the number of appearance of keyword j in P document i, kl¼1 nil is the total meaningful term frequency in document i, N is the total number of documents, and nj is the number of document in which keyword j appears. The TF–IDF algorithm can filter out insignificant terms in a document; nevertheless, it will also drop off some repeatedly appearing keywords of a document and these keywords are considered very important for the approach used in this paper. This paper modifies the normal TF–IDF by first lowering its threshold to let more keywords come in, hoping to retain keywords considered significant for the proposed approach. But the side effect is that more insignificant keywords also remain because of the lowered threshold. For this sake, in addition to the standard threshold of the TF– IDF, a second threshold, the Continuation Coefficient-CC, is introduced and the new TF–IDF is called ATF–IDF. Experiments will show some of results of comparison between TF–IDF and ATF–IDF.
A concept is then a pair of (o, a), i.e. a pair of a description statement and a significant keyword, i.e. the set of concepts C is the set of (S, Q) where Q = r(S) and S = s(Q) (Davey & Priestley, 2002; Weng, Tsai, Liu, & Hsu, 2006). 2.2.4.2. Step 2: generate the whole hierarchy-relationships between concepts. After generate a set of concepts, the hierarchy-relationships among the concepts can be established. A concept (A0, B0) that is a sub-concept of a concept (A1, B1) is denoted as (A0, B0) # (A1, B1), i.e. c0 = (A0, B0) is a sub-concept of c1 = (A1, B1). Given two elements (I1, J1) and (I2, J2) in the concept hierarchy, their infimum is defined as:
ðI1 ; J1 Þ \ ðI2 ; J 2 Þ ¼ ðI1 \ I2 ; rðI1 \ I2 ÞÞ: and their supremum is defined as:
ðI1 ; J1 Þ [ ðI2 ; J 2 Þ ¼ ðsðJ 1 \ J 2 Þ; J 1 \ J 2 Þ:
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167
The whole hierarchy relationship of concepts is constructed by using the infimum and supremum relationships. This step determines the parent and the child concepts of a specific concept. 2.2.4.3. Step 3: generate the inter-relationships between concepts. In addition to the hierarchy-relationships, there are attribute-associated inter-relationships between concepts, which cannot be identified by FCA. This step identifies the inter-relationships among established concepts using the method derived in (Weng et al., 2006). Assume there are two concepts c0 = (A0, B0) and c1 = (A1, B1), if there exist a set Setx and a set Sety, where SetxB0 and SetyB1, and if Setx = Sety, then a inter-relationship between c0 and c1 can be identified. All the inter-relationships between all the concepts can be identified in the same way. 2.2.4.4. Step 4: calculate the correlative weighting. When recommending a course to a learner with sequenced learning objects, the continuity between the contents of these learning objects is very important for the learner to successfully understand what are given in the course. To be able to automatically construct suitable learning paths with better content continuity, coherences between learning objects have to be built, which are termed as Correlative Weighting and computed from the generated Concept Lattice by formula (2) below. All the correlative weightings are further normalized by formula (3) and are arranged in a Correlative Weighting Matrix, the CWM, as in Eq. (4)
CW p;q ¼
m X n X jK pi \2p K qj j þ jK pi \2q K qj j W pi;qj jK pi j i¼1 j¼1
W pi;qj
W pi;qj ¼ 0:7; W pi;qj ¼ 0:3;
cwi;j ¼
CW i;j MinðCW i;n Þ ; MaxðCW i;n Þ MinðCW i;n Þ 2
Hierarchy-Relationship Inter-Relationship n ¼ 1; . . . ; v
ð3Þ
3
cw1;1
cw1;2
cw1;v
6 cw2;1 6 CWM ¼ 6 6 .. 4 .
cw2;2 .. . cwu;2
cw2;v 7 7 7 .. .. 7 . . 5 cwu;v
cwu;1
ð2Þ
ð4Þ
Among the three formulas, CWp,q is the coherence between learning objects p and q, Kpi is the ith significant keyword of p and Kqj is the jth significant keyword of q, |Kpi| is the number of appearances of Kpi in p and jK pi \2p K qj j is number of co-appearances of Kpi and Kqj in p and vice verse of jK qj \2q K qi j, and Wpi,qj is a given weight of hierarchy relationship between Kpi and Kqj. 2.2.5. Learning path generation To fulfill a goal issued by a learner, e.g. to satisfy a learner’s desire to understand a specific terminology in a domain, usually requires prerequisite knowledge for the terminology. Learning objects associated to the prerequisite knowledge must be prepared for such a learning activity and they must be taught in the front side of the learning path, if necessarily. This paper builds a suitable learning path for a learner according to the Correlative Weighting Matrix of learning objects collected. To build a learning path satisfying a specific goal with better continuity, the learning object most matches the goal is chosen as the main subject of learning, the learning object has the highest weight of correlation to the goal is selected for the prerequisite knowledge, and all the others are found and arranged subsequently according to their correlative weighting. The learning path is built within a screening window size. The following algorithm briefs the learning path building procedure.
4161
Learning Path Generation Algorithm Input: Learning goal (LG), large themeset size (LTS) and correlative weighting matrix (CWMij). Output: A suitable learning path (LP). Procedure: 1: LP = {0} 2: Find a leaning object (LOgoal) that meets the LG from Learning Object set. Add the LOgoal into LP, i.e. LP = {? LOgoal}. 3: While (length of LP < LTS){ Find the LOX with the highest correlative weighting to the Head of LP in CWMij If (LOX 2 LP) Do { Find the LOX with the next highest correlative weighting to the Head of LP in CWMij } until (LOX R LP)} else Add the LOX into LP 4: Return LP.
2.2.6. Learning object recommendation module Every course unit may be associated with several learning objects in the repository. These learning objects may have different degree of difficulties and various content types for different learning styles. Some of the learning objects are able to supply more efficient learning for a learner while others are not. Therefore, how to recommend the most suitable learning objects for each course unit of a learning path is an important task of a system. In order to facilitate this process, a methodology that find the most suitable learning objects is proposed, and is implemented as the learning object recommendation shown as in Fig. 6. The methodology uses a hybrid method that recommends learning objects in several phases. The Recommendation phase uses two algorithms, i.e. the preference-based and the correlation-based evaluation approaches, which use a learner’s personal preference as well as his/ her neighbors’ suggestions to calculate the recommendation scores for ranking all the learning objects chosen from the repository for each course unit of a learning path in order to select most suitable ones for the learning path. Finally, the learner feedbacks phase deals with the feedbacks of learners and updates their profiles that, as a result, will affect the next recommendation. The detail of this methodology can also be found in one of the previous studies (Wang, Tsai, Lee, & Chiu, 2007). 3. Experiments Several experiments have been conducted to show merits of the approach proposed in this paper. Some of them are presented as follows. 3.1. ATF–IDF vs. TF–IDF This experiment is conducted in order to compare the precision of TF–IDF and ATF–IDF on extracting significant keywords for the
Fig. 6. Phases of the learning object recommendation module.
4162
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167
proposed approach. Data used are learning objects taken from the Java Learning Object Ontology (JLOO), which was introduced in Lee, Ye, and Wang (2005). Several human experts are invited to pick out from these learning objects all the significant keywords that should be chosen by the approach and gather them into a relevant keyword set. The two significant keyword sets built by TF–
IDF and ATF–IDF respectively are compared with the base set by two criteria, the precision and the recall rates. The precision rate represents the percentage of the picked keywords that are in the relevant keyword set and the recall rate shows the percentage of relevant keywords that are correctly picked out in both methods; they are formulated as follows:
Fig. 7. Precision and recall rates of TF–IDF.
Fig. 8. Precision and recall curves of ATF–IDF.
Table 1 Total of association rule list for the ‘‘If” and ‘‘For” subjects with different setting. Goal
Association rule
minSupport(%)/ minConfidence(%)
For
For){Variable}, For){Comparison and Conditional}, For){While} For){Variable, Comparison and Conditional}, For){Variable, While}, For){Comparison and Conditional, If} For){Variable, Comparison and Conditional, If}, For){Variable, Comparison and Conditional, While}, For){Variable, Data Type, Break and Continue}, For){Variable, Comparison and Conditional, If, While} For){Variable, Comparison and Conditional, While}, For){Variable, Data Type, Break and Continue} For){Variable, Data Type, Break and Continue} For){Data Type, Variable, Comparison and Conditional, If, Break and Continue}
0.9/0.4 0.8/0.4
If
If){Variable} If){Variable} If){Variable, Comparison and Conditional}, If){Data Type, Variable} If){Data Type, Logical and Bitwise, Comparison and Conditional}, If){Variable, Logical and Bitwise, Break and Continue} If){Data Type, Logical and Bitwise, Comparison and Conditional} If){Data Type, Variable, Comparison and Conditional, Break and Continue}
0.7/0.4
0.6/0.4 0.5/0.4 0.4/0.4 0.9/0.4 0.8/0.4 0.7/0.4 0.6/0.4 0.5/0.4 0.4/0.4
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167
jfRelevant Keywordsg \ fRetrieved Keywordsgj jfRetrieved Keywordsgj jfRelevant Keywordsg \ fRetrieved Keywordsgj Recall ¼ jfRelevant Keywordsgj Precision ¼
ð5Þ ð6Þ
Fig. 7 shows how TF–IDF performs in the experiment. No matter how the threshold is adjusted, the precision rate remains almost flat and is low for the proposed approach even when the recall rate has dropped under the precision rate. While as shown in Fig. 8, ATF–IDF performs much better in finding significant keywords for the proposed approach. With the Continuation Coefficient-CC set at 0.02 and threshold of TF–IDF varied between the ranges shown in the figure, as can be seen, the precision rate keeps rising and is just over 0.5 when TF–IDF threshold is set at 0.055. 3.2. Learning path construction
4163
learning subject, which is further divided into ‘‘If” and ‘‘For” subjects. Topics for experiments are set to Java programming language and documents for the topics are collected by retrieval agents from the Internet. By using the Apriori algorithm, the association rules for each of learning subject can be discovered. In this experiment, the minConfidence is set at 0.4 and the minSupport is set from 0.9 to 0.4. Table 1 shows the mining results for the ‘‘If” and ‘‘For” subjects with different setting. Three performance measures, Precision, Recall and F-measure, are adopted for this experiment and their general definitions are shown in formula (7), formula (8) and formula (9), respectively
Precision ¼
jfRelevant Course Unitsg \ fRetrieved Course Unitsgj jfRetrieved Course Unitsgj ð7Þ
jfRelevant Course Unitsg \ fRetrieved Course Unitsgj ð8Þ jfRelevant Course Unitsgj 2 Precision Recall F¼ ð9Þ Precision þ Recall
Recall ¼
Several experiments have being conducted to evaluate the proposed approach in this section, two will be discussed here. The 10 experts on Java programming language also participate in evaluating the results of all the experiments from different perspectives. Course units of collected documents are first automatically choose by the system. The first experiment is focused on how to identify course units from collected learning objects for a specific learning subject. For illustrating, ‘‘Control Flow Statements” is taken as the
Figs. 9 and 10 show the average Precision and average Recall of the ‘‘For” and ‘‘If” subject respectively evaluated by the 10 domain experts on the experiment results of the proposed approach. When the minSupport threshold is set to 0.5, the experiment yields a much better performance than that of other settings. The higher the min-
Fig. 9. Average precision curves of the ‘‘For” and ‘‘If” subjects evaluated by 10 domain experts.
Fig. 10. Average recall curves of the ‘‘For” and ‘‘If” subjects evaluated by 10 domain experts.
4164
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167
Fig. 11. F-measure of the ‘‘For” and ‘‘If” subjects by 10 domain experts.
Support threshold is, the less the themesets (to the extreme, only 1 themeset) are retrieved by the system from the repository. In such cases, a high precision can be reached, but the recall can be very low. On the other hand, when the value of the minSupport threshold decreases, the recall increases until threshold reaches 0.5. Fig. 11 shows the F-measure of both the ‘‘For” and ‘‘If” subjects evaluated by the same 10 domain experts on the experiment results of the proposed approach. The same phenomenon can be observed that when the minSupport threshold is set to 0.5, the proposed approach operates more efficiently. Table 2 shows the discovered association rules for the ‘‘If” and ‘‘For” subjects. By using Formal Concept Analysis, the correlative weighting matrix is calculated and listed in Tables 3 and 4. After the correlations are found, according to the CWM, the suitable learning paths can be generated. The second experiment is conducted for evaluating both the correlations between learning objects and the learning paths constructed by the proposed approach. Using the same sources as experiment 1, the two generated learning paths are list in Table 5 with the third column showing the result of the evaluation from the same ten domain experts. Every segment of the 2 generated learning paths and the results are shown in Figs. 12 and 13. For example, if the approach finds that subject {Variable} is most correlated to subject {Data Type}, and more than half of the experts think the combination is correct, then it is given a 100% mark; otherwise 0% mark. An evaluation factor Hit Ratio is used to summarize and average all the marks. For example in Fig. 12 the final Hit Ratio value is 100. Except for the learning path for ‘‘If”, which has the Hit Ratio of 100, all others have Hit Ratio of 66.7. From this experiment, the proposed approach can efficiently offer a correct learning path to learners can be also confirmed. Table 6 shows the evaluation results for the ‘‘Control Flow Statements”, ‘‘Classes and Objects”, ‘‘Interface and Inheritance”, and ‘‘Advance Application” categories of the Java programming language. The minSupport and minConfidence are set to 0.5 and 0.4, respectively. We found that the averages of precision and recall were more than 50% in finding candidate courses. Moreover, the average of Hit Ratio for the four categories was 73.6% in constructing suitable learning paths. 3.3. Questionnaire analysis We also estimated the utility of using the developed system to help a student learn. In this section, in order to estimate the practicability of the system for learners, a questionnaire that contains 10 questions was adapted; it is shown in Table 7. Each question has five levels of answer: ‘Strongly Agree’, ‘Agree’, ‘Neutral’, ‘Dis-
Table 2 The discovered association rules for the ‘‘If” and ‘‘For” subjects. Goal Large themeset
Association rule
Support(%)/ Confidence (%)
If
{Data Type, Comparison and Conditional Operators, Logical and Bitwise Operators, If Statements}
For
{Data Type, Variables, Break and Continue Statements, For Statements}
{If Statements}){Data 0.732/0.641 Type, Comparison and Conditional Operators, Logical and Bitwise Operators} {For Statements}){Data 0.680/0.521 Type, Variables, Break and Continue Statements}
minSupport=0.5, minConfidence=0.4.
Table 3 Correlative weighting matrix for ‘‘If” unit. CM
Data type
Comparison and conditional operators
Logical and bitwise operators
If statements
Data type Comparison and conditional operators Logical and bitwise operators If statements
1 0.394
0.432 1
0.444 0.402
0.318 0.339
0.771
0.447
1
0
0.306
0.656
0
1
Table 4 Correlative weighting matrix for ‘‘For” unit. CM
Data type
Break and continue statements
Variables
For statements
Data type Break and continue statements Variables For statements
1 0.89
0.319 1
0.444 0.363
0 0.454
0.475 0
0.303 0.484
1 0.368
0.524 1
agree’, and ‘Strongly Disagree’. Several learners (n = 31) who had used the developed system to learn were invited to fill out the questionnaire and attended the ‘‘Programming Design” course. The evaluation results indicate that most learners agreed that the
4165
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167 Table 5 Learning paths for different goal. Goal
Learning path
If
{Data Type}?{Logical and Bitwise Operators} ?{Comparison and Conditional Operators} ?{If Statements} {Variables}?{Data Type}?{Break and Continue Statements} ?{For Statements}
For
Correlative weighting
Hit ratio (%)
0:656
fIf Satementsg ! fComparison and Conditional Operatorsg 0:402
1
0:771
! fLogical and Bitwise Operatorsg ! fData Typeg 0:484
fFor Satementsg ! fBreak and Continue Satementsg 0:475
0.667
0:444
! fData Typeg ! fVariablesg
Fig. 12. Evaluation on ‘‘If” learning path.
Fig. 13. Evaluation on ‘‘For” learning path.
Table 6 The results for different subjects on Java programming language. Category
Subject
Average of precision (%)
Average of recall (%)
Fmeasure
Hit ratio (%)
Average of hit ratio for each category
Control flow statements
If For While Switch
0.62 0.72 0.77 0.81
0.66 0.74 0.71 0.78
0.64 0.73 0.74 0.79
1 0.667 0.667 0.667
0.75
Classes and objects
Class and object Method Constructor
0.65 0.62 0.66
0.68 0.65 0.65
0.66 0.63 0.65
1 0.667 1
0.889
Interface and inheritance
Interface Inheritance Abstract methods and classes
0.56 0.64 0.5
0.61 0.66 0.53
0.58 0.65 0.51
0.667 0.667 0.333
0.556
Advance application
Package Thread Applet Input/output
0.63 0.66 0.51 0.71
0.61 0.60 0.46 0.72
0.62 0.63 0.48 0.71
1 0.667 0.333 1
0.75
minSupport (%) = 0.5, minConfidence (%) = 0.4
Average hit of ratio (%)
0.736
4166
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167
Table 7 A questionnaire. #
#1
Question description
Level
#3
I agree that using the proposed e-learning system to provide a suitable learning path is very convenient to me on self-learning I agree that those learning objects decided by the proposed e-learning system are helpful to me I agree that I can certainly solve problems after using the proposed e-learning system
#4
I agree that I will study hard every time in the proposed e-learning system
#5
I think that using the proposed e-learning system can promote my learning motivation and interest I think that the proposed e-learning system offers a friendly user interface and I will still use the platform for learning I think that using the proposed e-learning system more often will decrease my interaction with teachers or peers I think that using the proposed e-learning system can promote personal learning ability on of each subject When I login the proposed e-learning system for learning, I can clearly realize the learning procedure of the platform By using the proposed e-learning system, I feel that I can efficiently decrease time in searching Internet for the materials which facilitate learning
#2
#6 #7 #8 #9 #10
Strongly agree
Agree
Neutral
Disagree
Strongly disagree
16 (51.6%) 20 (64.5%) 5 (16.1%) 2 (6.7%) 15 (48.6%) 18 (58.2%) 3 (10%) 8 (25.8%) 16 (51.6%) 15 (48.3%)
8 (25.8%) 8 (25.8%) 23 (74.2%) 25 (80.3%) 10 (32.3%) 4 (13%) 7 (22.6%) 9 (28.5%) 10 (32.3%) 8 (25.8%)
4 (12.9%) 2 (6.7%) 2 (6.7%) 3 (10%) 5 (16.1%) 8 (25.8%) 12 (38%) 10 (32.3%) 5 (16.1%) 4 (12.9%)
2 (6.7%) 1 (3%) 1 (3%) 1 (3%) 0 (0%) 1 (3%) 3 (10%) 2 (6.7%) 0 (0%) 3 (10%)
1 (3%) 0 (0%) 0 (0%) 0 (0%) 1 (3%) 0 (0%) 6 (19.4%) 2 (6.7%) 0 (0%) 1 (3%)
developed system helped them learn efficiently. Over 77% of learners agreed that the appropriate learning path, which the proposed system constructed, helped them learn the Java programming language. Moreover, 90% of learners agreed that the recommended learning materials helped them because they did not need to spend a lot of time surveying and choosing the right learning materials.
Acknowledgement
4. Conclusions
Agrawal, R., & Srikant, R. (1994). Fast algorithms for mining association rules. In Proceedings of 20th international conference on very large data bases (pp. 487– 499). Agrawal, R., Imielinski, T., & Swami, A. (1993). Mining association rules between sets of items in large databases. In Proceedings of the ACM SIGMOD conference on management of data (pp. 207–216). Aizawa, A. (2003). An information-theoretic perspective of TF–IDF measures. Information Processing & Management, 39(1), 45–65. Chen, C.-M. (2008). Intelligent web-based learning system with personalized learning path guidance. Computers & Education, 51, 787–814. Chen, C.-M., & Chung, C.-J. (2008). Personalized mobile English vocabulary learning system based on item response theory and learning memory cycle. Computers & Education, 51, 624–645. Chen, C.-M., Liu, C.-Y., & Chang, M.-H. (2006). Personalized curriculum sequencing utilizing modified item response theory for web-based instruction. Expert Systems with Applications, 30(2), 378–396. Chi, Y.-L. (2007). Elicitation synergy of extracting conceptual tags and hierarchies in textual document. Expert Systems with Applications, 32(2), 349–357. CKIP AutoTag (1998). Chinese knowledge information processing group. Academic sinica, Taiwan. Colace, F., De Santo, M., & Iacone, M. (2005). Work in progress – ASAP: An automatic student adapted learning path generator. In Proceedings of 35th annual conference frontiers in education (pp. 13–14). Davey, B. A., & Priestley, H. A. (2002). Introduction to lattices and order (pp. 65–84). Cambridge University Press. Dietinger, T. (2003). Aspects of e-learning environments. Institute for information processing and computer supported new media (IICM). Graz University of Technology A-8010 Graz, Austria. Ganter, B., & Wille, R. (1999). Formal concept analysis: Mathematical foundations. Berlin, Heidelberg: Springer-Verlag. Hidber, C. (1999). Online association rule mining. In Proceedings of the ACM SIGMOD conference on management of data (pp. 145–156). Huang, M.-J., Huang, H.-S., & Chen, M.-Y. (2007). Constructing a personalized elearning system based on genetic algorithm and case-based reasoning approach. Expert Systems with Applications, 33(3), 551–564. Ismail, J. (2002). The design of an e-learning system beyond the hype. Internet and Higher Education, 4, 329–336. Koprinska, I., Poon, J., Clark, J., & Chan, J. (2007). Learning to classify e-mail. Information Sciences, 177(10), 2167–2187. Lee, M.-C., Ye, D.-Y., & Wang, T.-I (2005). Java learning object ontology. In Proceedings of the fifth IEEE international conference on advanced learning technologies (pp. 538–542).
Most research on e-learning has used a pre-constructed category or some manual information for semi-automatic learning object classification and learning path construction in specific domains. Few platforms have focused on automatically constructing learning paths for occasional knowledge on unspecific domains from open environments, such as fragmented knowledge from the Internet. In this paper, a support system with two main approaches, the learning path construction approach and the learning object recommendation approach, is proposed. The system first finds a set of candidate course subjects based on the well-known Apriori algorithm. Then, the learning path construction approach use Formal Concept Analysis to build a Concept Lattice using keywords extracted from collected documents to form a relationship hierarchy between all the concepts represented by the keywords. It uses FCA to further compute the mutual relationships among documents to determine the appropriate learning path. Moreover, in order to assist a learner to study efficiently, the support system employs both preferencebased and correlation-based algorithms to find the most suitable learning objects collected from the Internet by retrieval agents. The experiments show that the proposed approaches achieve more than 50% accuracy in finding candidate courses, and at least a 66.7% suitability in constructed learning paths. Over 77% of learners agreed that the appropriate learning path, which the proposed system constructed, helped them learn the Java programming language. Moreover, 90% of learners agreed that the recommended learning materials also helped them. The proposed approach helps learners browse and read collected learning objects or documents in the correct order to understand the fragmented knowledge.
This work is supported by the Nation Science Council of Taiwan under the contract NSC95-2221-E-006-158-MY3.
References
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167 Lee, M.-C., Tsai, K.-H., & Wang, T.-I. (2008). A practical ontology query expansion algorithm for semantic-aware learning objects retrieval. Computers & Education, 50, 1240–1257. Liu, H.-I., & Yang, M.-N. (2005). QoL guaranteed adaptation and personalization in elearning systems. IEEE Transactions on Education, 48(4), 676–687. Moodle. Modular object-oriented dynamic learning environment.
4167
Tseng, Judy. C.-R., Chu, H.-C., Hwang, G.-J., & Tsai, C.-C. (2008). Development of an adaptive learning system with two sources of personalization information. Computers & Education, 51, 776–786. Wang, T.-I., Tsai, K.-H., Lee, M.-C., & Chiu, T.-K. (2007). Personalized learning objects recommendation based on semantic-aware discovery and a learners preference pattern. Educational Technology & Society, 10(3), 84–105. Weng, S.-S., Tsai, H.-J., Liu, S.-C., & Hsu, C.-H. (2006). Ontology construction for information classification. Expert Systems with Applications, 31(1), 1–12. Wille, R. (1982). Restructuring lattice theory: An approach based on hierarchies of concepts. Ordered Sets, 445, 470.
Contents lists available at ScienceDirect
Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa
A mining-based approach on discovering courses pattern for constructing suitable learning path Tung-Cheng Hsieh *, Tzone-I Wang Department of Engineering Science, National Cheng Kung University, No. 1, Ta-Hsueh Road, Tainan 701, Taiwan, ROC
a r t i c l e
i n f o
Keywords: Self-directed learner Data mining Formal Concept Analysis (FCA) Concept Lattice Learning path
a b s t r a c t In recent years, browser has become one of the most popular tools for searching information on the Internet. Although a person can conveniently find and download specific learning materials to gain fragmented knowledge, most of the materials are imperfect and have no particular order in the content. Therefore, most of the self-directed learners spend most of time in surveying and choosing the right learning materials collected from the Internet. This paper develops a web-based learning support system that harnesses two approaches, the learning path constructing approach and the learning object recommending approach. With collected documents and a learning subject from a learner, the system first discovers some candidate courses by using a data mining approach based on the Apriori algorithm. Next, the leaning path constructing approach, based on the Formal Concept Analysis, builds a Concept Lattice, using keywords extracted from some selected documents, to form a relationship hierarchy of all the concepts represented by the keywords. It then uses FCA to further compute mutual relationships among documents to decide a suitable learning path. For a chosen learning path, the support system uses both the preference-based and the correlation-based algorithms for recommending the most suitable learning objects or documents for each unit of the courses in order to facilitate more efficient learning for the learner. This e-learning support system can be embedded in any information retrieval system for surfers to do more efficient learning on the Internet. Ó 2009 Elsevier Ltd. All rights reserved.
1. Introduction With a faster, more accessible Internet, nowadays people tend to search and learn from Internet for fragmented knowledge. Educationists or enthusiasts, seeing the benefit of on-line material, feverishly set up their web sites to share their knowledge of specific domains (Rafaeli, Barak, Dan-Gur, & Toch, 2004). Though being versatile, these web sites generally follow no standards for content organization and presentation order. They might be just web pages or some learning objects embedded in web pages. When posted on to the Internet, collected and indexed by robots using keywords, and returned by powerful search engines, usually, a vast amount of them, homepages or learning objects, is returned directly to a user with no particular order. Even if they might really be related, a user still has to move forward and backward among the material trying to figure out which page to read first because the user might has had little or no experience in a specific domain. Although a user might have some intuitions about the domain but these intuitions are yet to be connected. Thus, an effective way of organizing col-
* Corresponding author. E-mail address: [email protected] (T.-C. Hsieh). 0957-4174/$ - see front matter Ó 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2009.11.007
lected material into systematic and sequenced learning contents would be a great help for users who are novel to a specific domain. Furthermore, Dietinger (2003) has pointed out many advantages of e-learning activities. One of them is the importance of ‘‘Adaptive Learning”. The learning content can be adapted according to each learner’s strengths and weaknesses to achieve the most efficient learning experience. That is, by judging the initial knowledge of a learner on a topic and the learner’s preferred learning style, an e-learning system can decide what learning content should be offered next (Papanikolaou, Grigoriadou, Magoulas, & Kornilakis, 2002). Many researchers have been devoting themselves into adaptive e-learning methodologies and platforms works (Tseng, Chu, Hwang, & Tsai, 2008). Some consider that the importance of a learner’s ability and the difficulty of courses are the key issues for developing the e-learning system (Chen & Chung, 2008; Huang, Huang, & Chen, 2007; Liu & Yang, 2005). Several of the studies emphasized their proposed system’s ability to construct curriculum sequencing to help self-learning. Chen, Liu, and Chang (2006) presented a Personalized Web-based Instruction System (PWIS) to construct suitable learning pathway based on a modified item response theory for helping learning. Colace, De Santo, and Iacone (2005) presented an approach, which can obtain the learning style and capabilities of each learner, to arrange
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167
the learning path adaptively with most suitable teaching contents. Chen (2008), in the mean time, developed a genetic-based personalized e-learning system to provide a learning path according to the courseware difficulty and the result of pre-test for individual learner. During last decade, many e-learning systems have been developed for assisting learners in learning in a variety of areas. Nevertheless, many of these systems are trying hard in considering the needs of particular learners and just in time learning in the course (Ismail, 2002). One methodology that is based on an ontological approach to aggregate and infer users’ intentions for learning objects retrieval is proposed in our previous study (Lee, Tsai, & Wang, 2008). However, this methodology lacks a part for efficiently making an appropriate learning sequence among retrieved learning objects for learners. Fig. 1 depicts, as an example, a scenario of a self-directed learner learning some materials for a subject. The self-directed learner issues a subject on the Internet, and then immediately retrieves some learning materials, in which with no particular order, based on his/her intentions by search engine. After that, learners need to spend most time in organizing and skipping the learning materials they have searched. Each node, cyclic, square, or triangle, represents a concept, which the learner should comprehend after learning the material. All of the cyclic nodes, colored in gray, are the concepts that the learner has realized before reading. A square node, colored in green, represents a concept that a learner still cannot realize after reading the contents in the material. A triangle node, colored in red, represents a main concept in the material. After reading material 1, there may be some concepts the learner cannot understand so that she/he has to try reading other materials to support her/his learning by her/ himself, in Fig. 1’s case material 2 and so on. It would be very helpful for self-directed learners if there is a mechanism that automatically construct learning paths and recommend learning objects according to a learner’s intentions. Furthermore, in e-learning paradigm, researchers have been focused on sorting, automatically or semi-automatically, collected learning contents of a domain into predefined classes of a platform’s, which are drawn up by experts of the domain. These platforms can construct and recommend learning path base on the predefined classes and the hierarchical organization of learning contents, but automatic learning path construction for collected learning contents without human expert interference is current
4157
missing and would be of great help for users if it is built and recommended the material. For such self-directed learners, this paper develops a web-based learning support system, which harnesses two main approaches, the learning path constructing approach and the learning object recommending approach. The learning path constructing approach finds out a set of candidate courses first by using the data mining Apriori algorithm. It then uses Formal Concept Analysis to constructs a Concept Lattice by using keywords extracted from the collected documents to generate a relationship hierarchy between all the concepts represented by the keywords. It also uses FCA to further compute mutual relationships among documents to decide a suitable learning path. For each course units in a chosen learning path, the support system will select the most suitable documents or learning objects, using algorithms based on both learner preference and learner correlation, in order to facilitate more efficient learning for a learner. The rest of this paper is organized as follows. Section 2 draws the architecture of the proposed approach and algorithms in detail, Section 3 presents some experimental results and discussions, and the conclusion is in Section 4. 2. System architecture In this paper, several approaches are used to automatically construct a suitable learning path and recommend suitable contents from collected materials. The architecture of the proposed system is shown in Fig. 2. The major components carries out the procedure are the Learning Interface, the Candidate Course Generator, the Learning Object Content Preprocessor, the Learning Object Correlative Weighting Generator and the Learning Object Recommendation Module. 2.1. System description A learner logins to the Learner Interface to give a subject (a learning goal or a short query) he/she intends to learn. If it is a new subject that is never queried before, the Learner Interface will send out retrieval agents to collect learning materials from the Internet based on the learner’s learning subject and stores them into the learning objects repository. Domain experts can also
Fig. 1. A learning behavior example of self-directed learners.
4158
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167
Fig. 2. The system architecture.
plug-in a file which describes specific knowledge, such as course units. Base on a subject, the Candidate Course Generator discover several courseware unit patterns by data mining on the collected learning resources using the well-known Apriori algorithm. Then the Learning Object Content Preprocessor is notified to find out all the significant keywords of every learning object, using an Adapted TF–IDF (ATF–IDF) algorithm. After that, the Learning Object Correlative Weighting Generator constructs a hierarchy of relationships between all the concepts represented by the keywords and uses FCA to further compute the mutual relationships among all the learning objects or documents to decide a suitable learning path. After a learning paths is selected, the Learning Object Recommendation Module uses both the preference-based and the correlationbased algorithms, which ranks the degree of relevance of learning objects or documents to a learner’s intension and preference, for recommending the most suitable learning objects or documents for each unit of the courses in order to facilitate more efficient learning for all the learners. The details of all the system components are described as follows.
2.2. System components 2.2.1. Learning interface To develop a secure and stable learning environment, the open source and web-based course management system Moodle is employed. Moodle, is acronym for Modular Object-Oriented Dynamic Learning Environment, which allows easy learner management, courseware construction, as well as rich teaching activity via browsers (Moodle). To the developed learning environment, a learner gives a subject to learn via the Learner Interface. When a learner login, the system sets up or fetches and updates the learner’s profile, i.e. her/his preference, feedback and portfolio, and then stores the profile into the learner’s profile database. With these profile data the proposed approach will decide an appropriate learning path and choose learning objects in accordance with the learner’s learning subject. From this Learner Interface, some retrieval agents are issued, when necessary, to collect learning documents, web pages or learning objects, from the Internet. The retrieved learning materials then are put into the learning object repository. Fig. 3 displays a generated suitable learning path and the learning objects recommended.
2.2.2. Candidate course generator The major function of the Candidate Course Generator is to find out a course pattern, which contains a set of course units according to a learner’s learning subject, issued by a learner, and based on which, the Generator retrieves some learning objects from the Learning Object Repository. Fig. 4 illustrates the processes of candidate course unit generator. Before find out a course pattern, a material is processed to have all its meaningful keywords filtered and tagged (CKIP AutoTag, 1998). Each learning material will record a set of course unit themes, which is called a themeset (set of themes). The Generator then collects a set of course unit themes from these learning objects, and based on which, finds other related candidate course units in order to be more accordant with the learner’s learning subject. A data mining approach, the Apriori algorithm (Agrawal, Imielinski, & Swami, 1993; Agrawal & Srikant, 1994), is adopted to discover the association rules among all collected course unit themes. Assume an itemset I ¼ fi1 ; i2 ; i3 ; . . . ; im g is a set of items. An itemset is called a large-itemset if its support value is greater or equal to the user-specified support threshold (called minSupport). An association rule is an expression X ) Y where X and Y are disjoint itemsets, which represents possibility when X appears that Y will also appear. The support of an association rule is the support of X [ Y, and the confidence of such a rule is the fraction of all transactions containing X that also contain Y (Hidber, 1999). The following is the detail of the Apriori algorithm. Apriori algorithm Input: Learning objects repository (LOR), Threshold of minimum support value (minSupport). Output: Large itemsets in learning objects repository (LI). Procedure: 1: LI1 = find large 1-itemsets in LOR. 2: For (k=2; LIk-1–/;k++){ Ck = apriori-gen(LIk-1); // New candidates 3: for all of records r 2 LOR { Ct = subset(Ck, r) // Candidates contained in r for all of candidates c 2 Ct c.count++;} LIk ¼ fc 2 C k jc:count P min Supportg} 4: Return LI ¼ [k LIk ; This paper uses JAVA programming language as an example. Domain experts have defined some course units for this specific do-
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167
4159
Fig. 3. The interface of developed learning environment.
Fig. 4. Processes of candidate course generator.
main and put them into the system. An example of candidate course unit generation by using the Apriori algorithm is shown in Fig. 5, which assumes that a learner intends to learn the ‘‘For-statements” subject in Java. The learning object repository (LOR) is searched to find five learning object that are associated with the ‘‘For-statements” theme and contain contents for among Break-statement, For-statement, Variables, Arrays, Data type, and If-statement themes
respectively. For simplicity, these themes are symbolized into six letters, A, B, C, D, E, and F in the example. The threshold values of minSupport and minConfidence are set to 3 and 0.5, respectively. After the operation of the Apriori algorithm, the large itemset {For-statement, Variables, and Data type} is found and the association rule {For-statement}){Variables, Data type} can be also generated. The result of this association rule indicates that ‘‘60%” of the
4160
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167
Fig. 5. An example of candidate courses generated by using the Apriori algorithm.
learning objects that are designated for the ‘‘For-statements” theme also mentioned both the subject themes of ‘‘Variables” and ‘‘Data type”. After that, the generator finds out all of the learning objects which are designated for the ‘‘For-statements”, ‘‘Variables” and ‘‘Data type” course themes, and forwards them to the next stage to find an appropriate learning path.
2.2.3. Learning object content preprocessor After discovering a set of candidate course units and their associated learning materials by the Apriori algorithm, the learning object content preprocessor extract all the important keywords from the materials. A usual way of doing such keyword extraction is to use TF–IDF (Term Frequency/Inverse Document Frequency) that finds out for a document those specific keywords that distinguishes the document from the others. The less a keyword of a document appears in other documents, the more specific the keyword is for the document and the higher the TF–IDF value is (Aizawa, 2003; Koprinska, Poon, Clark, & Chan, 2007). The formula of TF– IDF is shown in following,
.P k ðnij l¼1 nil Þ logðN=nj Þ r ij ¼ rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi . Pk Pt 2 2 j¼1 ðnij l¼1 nil Þ ½logðN=nj Þ
2.2.4. Learning object correlative weighting generator After significant keywords of each material are extracted, several steps are involved in computing mutual relationships. In this paper, the Formal Concept Analysis approach can be used to derive implicit relationships between concepts using learning objects and attributes associated with these concepts (Chi, 2007; Ganter & Wille, 1999; Wille, 1982). Three steps are required in this paper in using FCA to construct inter-relationships between significant keywords of learning objects. Furthermore, these relationships are used to calculate the correlative weighting that represent the coherences between learning objects in the final step. 2.2.4.1. Step 1: generate all the concepts. In this paper, the term (O, A, X) denotes a binary relation X between a set of objects O and a set of attributes A. That is X # O A, where O is the set of all the description statements of all the learning objects derived from the association rule and A is the set of all the significant keywords of all the learning objects derived from the same association rule. To generate a concept set C, assume S is a partial set of O, Q is a partial set of A, and both S # O and Q # A holds, the set of all the attributes of S is:
rðSÞ ¼ fa 2 Aj8o 2 S : ða; oÞ 2 Xg; ð1Þ
and the set of all the significant keywords of all the learning objects of Q is:
sðQ Þ ¼ fo 2 Oj8a 2 Q : ða; oÞ 2 Xg: where rij is the importance of keyword j in document i with its value between 0 and 1, nij is the number of appearance of keyword j in P document i, kl¼1 nil is the total meaningful term frequency in document i, N is the total number of documents, and nj is the number of document in which keyword j appears. The TF–IDF algorithm can filter out insignificant terms in a document; nevertheless, it will also drop off some repeatedly appearing keywords of a document and these keywords are considered very important for the approach used in this paper. This paper modifies the normal TF–IDF by first lowering its threshold to let more keywords come in, hoping to retain keywords considered significant for the proposed approach. But the side effect is that more insignificant keywords also remain because of the lowered threshold. For this sake, in addition to the standard threshold of the TF– IDF, a second threshold, the Continuation Coefficient-CC, is introduced and the new TF–IDF is called ATF–IDF. Experiments will show some of results of comparison between TF–IDF and ATF–IDF.
A concept is then a pair of (o, a), i.e. a pair of a description statement and a significant keyword, i.e. the set of concepts C is the set of (S, Q) where Q = r(S) and S = s(Q) (Davey & Priestley, 2002; Weng, Tsai, Liu, & Hsu, 2006). 2.2.4.2. Step 2: generate the whole hierarchy-relationships between concepts. After generate a set of concepts, the hierarchy-relationships among the concepts can be established. A concept (A0, B0) that is a sub-concept of a concept (A1, B1) is denoted as (A0, B0) # (A1, B1), i.e. c0 = (A0, B0) is a sub-concept of c1 = (A1, B1). Given two elements (I1, J1) and (I2, J2) in the concept hierarchy, their infimum is defined as:
ðI1 ; J1 Þ \ ðI2 ; J 2 Þ ¼ ðI1 \ I2 ; rðI1 \ I2 ÞÞ: and their supremum is defined as:
ðI1 ; J1 Þ [ ðI2 ; J 2 Þ ¼ ðsðJ 1 \ J 2 Þ; J 1 \ J 2 Þ:
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167
The whole hierarchy relationship of concepts is constructed by using the infimum and supremum relationships. This step determines the parent and the child concepts of a specific concept. 2.2.4.3. Step 3: generate the inter-relationships between concepts. In addition to the hierarchy-relationships, there are attribute-associated inter-relationships between concepts, which cannot be identified by FCA. This step identifies the inter-relationships among established concepts using the method derived in (Weng et al., 2006). Assume there are two concepts c0 = (A0, B0) and c1 = (A1, B1), if there exist a set Setx and a set Sety, where SetxB0 and SetyB1, and if Setx = Sety, then a inter-relationship between c0 and c1 can be identified. All the inter-relationships between all the concepts can be identified in the same way. 2.2.4.4. Step 4: calculate the correlative weighting. When recommending a course to a learner with sequenced learning objects, the continuity between the contents of these learning objects is very important for the learner to successfully understand what are given in the course. To be able to automatically construct suitable learning paths with better content continuity, coherences between learning objects have to be built, which are termed as Correlative Weighting and computed from the generated Concept Lattice by formula (2) below. All the correlative weightings are further normalized by formula (3) and are arranged in a Correlative Weighting Matrix, the CWM, as in Eq. (4)
CW p;q ¼
m X n X jK pi \2p K qj j þ jK pi \2q K qj j W pi;qj jK pi j i¼1 j¼1
W pi;qj
W pi;qj ¼ 0:7; W pi;qj ¼ 0:3;
cwi;j ¼
CW i;j MinðCW i;n Þ ; MaxðCW i;n Þ MinðCW i;n Þ 2
Hierarchy-Relationship Inter-Relationship n ¼ 1; . . . ; v
ð3Þ
3
cw1;1
cw1;2
cw1;v
6 cw2;1 6 CWM ¼ 6 6 .. 4 .
cw2;2 .. . cwu;2
cw2;v 7 7 7 .. .. 7 . . 5 cwu;v
cwu;1
ð2Þ
ð4Þ
Among the three formulas, CWp,q is the coherence between learning objects p and q, Kpi is the ith significant keyword of p and Kqj is the jth significant keyword of q, |Kpi| is the number of appearances of Kpi in p and jK pi \2p K qj j is number of co-appearances of Kpi and Kqj in p and vice verse of jK qj \2q K qi j, and Wpi,qj is a given weight of hierarchy relationship between Kpi and Kqj. 2.2.5. Learning path generation To fulfill a goal issued by a learner, e.g. to satisfy a learner’s desire to understand a specific terminology in a domain, usually requires prerequisite knowledge for the terminology. Learning objects associated to the prerequisite knowledge must be prepared for such a learning activity and they must be taught in the front side of the learning path, if necessarily. This paper builds a suitable learning path for a learner according to the Correlative Weighting Matrix of learning objects collected. To build a learning path satisfying a specific goal with better continuity, the learning object most matches the goal is chosen as the main subject of learning, the learning object has the highest weight of correlation to the goal is selected for the prerequisite knowledge, and all the others are found and arranged subsequently according to their correlative weighting. The learning path is built within a screening window size. The following algorithm briefs the learning path building procedure.
4161
Learning Path Generation Algorithm Input: Learning goal (LG), large themeset size (LTS) and correlative weighting matrix (CWMij). Output: A suitable learning path (LP). Procedure: 1: LP = {0} 2: Find a leaning object (LOgoal) that meets the LG from Learning Object set. Add the LOgoal into LP, i.e. LP = {? LOgoal}. 3: While (length of LP < LTS){ Find the LOX with the highest correlative weighting to the Head of LP in CWMij If (LOX 2 LP) Do { Find the LOX with the next highest correlative weighting to the Head of LP in CWMij } until (LOX R LP)} else Add the LOX into LP 4: Return LP.
2.2.6. Learning object recommendation module Every course unit may be associated with several learning objects in the repository. These learning objects may have different degree of difficulties and various content types for different learning styles. Some of the learning objects are able to supply more efficient learning for a learner while others are not. Therefore, how to recommend the most suitable learning objects for each course unit of a learning path is an important task of a system. In order to facilitate this process, a methodology that find the most suitable learning objects is proposed, and is implemented as the learning object recommendation shown as in Fig. 6. The methodology uses a hybrid method that recommends learning objects in several phases. The Recommendation phase uses two algorithms, i.e. the preference-based and the correlation-based evaluation approaches, which use a learner’s personal preference as well as his/ her neighbors’ suggestions to calculate the recommendation scores for ranking all the learning objects chosen from the repository for each course unit of a learning path in order to select most suitable ones for the learning path. Finally, the learner feedbacks phase deals with the feedbacks of learners and updates their profiles that, as a result, will affect the next recommendation. The detail of this methodology can also be found in one of the previous studies (Wang, Tsai, Lee, & Chiu, 2007). 3. Experiments Several experiments have been conducted to show merits of the approach proposed in this paper. Some of them are presented as follows. 3.1. ATF–IDF vs. TF–IDF This experiment is conducted in order to compare the precision of TF–IDF and ATF–IDF on extracting significant keywords for the
Fig. 6. Phases of the learning object recommendation module.
4162
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167
proposed approach. Data used are learning objects taken from the Java Learning Object Ontology (JLOO), which was introduced in Lee, Ye, and Wang (2005). Several human experts are invited to pick out from these learning objects all the significant keywords that should be chosen by the approach and gather them into a relevant keyword set. The two significant keyword sets built by TF–
IDF and ATF–IDF respectively are compared with the base set by two criteria, the precision and the recall rates. The precision rate represents the percentage of the picked keywords that are in the relevant keyword set and the recall rate shows the percentage of relevant keywords that are correctly picked out in both methods; they are formulated as follows:
Fig. 7. Precision and recall rates of TF–IDF.
Fig. 8. Precision and recall curves of ATF–IDF.
Table 1 Total of association rule list for the ‘‘If” and ‘‘For” subjects with different setting. Goal
Association rule
minSupport(%)/ minConfidence(%)
For
For){Variable}, For){Comparison and Conditional}, For){While} For){Variable, Comparison and Conditional}, For){Variable, While}, For){Comparison and Conditional, If} For){Variable, Comparison and Conditional, If}, For){Variable, Comparison and Conditional, While}, For){Variable, Data Type, Break and Continue}, For){Variable, Comparison and Conditional, If, While} For){Variable, Comparison and Conditional, While}, For){Variable, Data Type, Break and Continue} For){Variable, Data Type, Break and Continue} For){Data Type, Variable, Comparison and Conditional, If, Break and Continue}
0.9/0.4 0.8/0.4
If
If){Variable} If){Variable} If){Variable, Comparison and Conditional}, If){Data Type, Variable} If){Data Type, Logical and Bitwise, Comparison and Conditional}, If){Variable, Logical and Bitwise, Break and Continue} If){Data Type, Logical and Bitwise, Comparison and Conditional} If){Data Type, Variable, Comparison and Conditional, Break and Continue}
0.7/0.4
0.6/0.4 0.5/0.4 0.4/0.4 0.9/0.4 0.8/0.4 0.7/0.4 0.6/0.4 0.5/0.4 0.4/0.4
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167
jfRelevant Keywordsg \ fRetrieved Keywordsgj jfRetrieved Keywordsgj jfRelevant Keywordsg \ fRetrieved Keywordsgj Recall ¼ jfRelevant Keywordsgj Precision ¼
ð5Þ ð6Þ
Fig. 7 shows how TF–IDF performs in the experiment. No matter how the threshold is adjusted, the precision rate remains almost flat and is low for the proposed approach even when the recall rate has dropped under the precision rate. While as shown in Fig. 8, ATF–IDF performs much better in finding significant keywords for the proposed approach. With the Continuation Coefficient-CC set at 0.02 and threshold of TF–IDF varied between the ranges shown in the figure, as can be seen, the precision rate keeps rising and is just over 0.5 when TF–IDF threshold is set at 0.055. 3.2. Learning path construction
4163
learning subject, which is further divided into ‘‘If” and ‘‘For” subjects. Topics for experiments are set to Java programming language and documents for the topics are collected by retrieval agents from the Internet. By using the Apriori algorithm, the association rules for each of learning subject can be discovered. In this experiment, the minConfidence is set at 0.4 and the minSupport is set from 0.9 to 0.4. Table 1 shows the mining results for the ‘‘If” and ‘‘For” subjects with different setting. Three performance measures, Precision, Recall and F-measure, are adopted for this experiment and their general definitions are shown in formula (7), formula (8) and formula (9), respectively
Precision ¼
jfRelevant Course Unitsg \ fRetrieved Course Unitsgj jfRetrieved Course Unitsgj ð7Þ
jfRelevant Course Unitsg \ fRetrieved Course Unitsgj ð8Þ jfRelevant Course Unitsgj 2 Precision Recall F¼ ð9Þ Precision þ Recall
Recall ¼
Several experiments have being conducted to evaluate the proposed approach in this section, two will be discussed here. The 10 experts on Java programming language also participate in evaluating the results of all the experiments from different perspectives. Course units of collected documents are first automatically choose by the system. The first experiment is focused on how to identify course units from collected learning objects for a specific learning subject. For illustrating, ‘‘Control Flow Statements” is taken as the
Figs. 9 and 10 show the average Precision and average Recall of the ‘‘For” and ‘‘If” subject respectively evaluated by the 10 domain experts on the experiment results of the proposed approach. When the minSupport threshold is set to 0.5, the experiment yields a much better performance than that of other settings. The higher the min-
Fig. 9. Average precision curves of the ‘‘For” and ‘‘If” subjects evaluated by 10 domain experts.
Fig. 10. Average recall curves of the ‘‘For” and ‘‘If” subjects evaluated by 10 domain experts.
4164
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167
Fig. 11. F-measure of the ‘‘For” and ‘‘If” subjects by 10 domain experts.
Support threshold is, the less the themesets (to the extreme, only 1 themeset) are retrieved by the system from the repository. In such cases, a high precision can be reached, but the recall can be very low. On the other hand, when the value of the minSupport threshold decreases, the recall increases until threshold reaches 0.5. Fig. 11 shows the F-measure of both the ‘‘For” and ‘‘If” subjects evaluated by the same 10 domain experts on the experiment results of the proposed approach. The same phenomenon can be observed that when the minSupport threshold is set to 0.5, the proposed approach operates more efficiently. Table 2 shows the discovered association rules for the ‘‘If” and ‘‘For” subjects. By using Formal Concept Analysis, the correlative weighting matrix is calculated and listed in Tables 3 and 4. After the correlations are found, according to the CWM, the suitable learning paths can be generated. The second experiment is conducted for evaluating both the correlations between learning objects and the learning paths constructed by the proposed approach. Using the same sources as experiment 1, the two generated learning paths are list in Table 5 with the third column showing the result of the evaluation from the same ten domain experts. Every segment of the 2 generated learning paths and the results are shown in Figs. 12 and 13. For example, if the approach finds that subject {Variable} is most correlated to subject {Data Type}, and more than half of the experts think the combination is correct, then it is given a 100% mark; otherwise 0% mark. An evaluation factor Hit Ratio is used to summarize and average all the marks. For example in Fig. 12 the final Hit Ratio value is 100. Except for the learning path for ‘‘If”, which has the Hit Ratio of 100, all others have Hit Ratio of 66.7. From this experiment, the proposed approach can efficiently offer a correct learning path to learners can be also confirmed. Table 6 shows the evaluation results for the ‘‘Control Flow Statements”, ‘‘Classes and Objects”, ‘‘Interface and Inheritance”, and ‘‘Advance Application” categories of the Java programming language. The minSupport and minConfidence are set to 0.5 and 0.4, respectively. We found that the averages of precision and recall were more than 50% in finding candidate courses. Moreover, the average of Hit Ratio for the four categories was 73.6% in constructing suitable learning paths. 3.3. Questionnaire analysis We also estimated the utility of using the developed system to help a student learn. In this section, in order to estimate the practicability of the system for learners, a questionnaire that contains 10 questions was adapted; it is shown in Table 7. Each question has five levels of answer: ‘Strongly Agree’, ‘Agree’, ‘Neutral’, ‘Dis-
Table 2 The discovered association rules for the ‘‘If” and ‘‘For” subjects. Goal Large themeset
Association rule
Support(%)/ Confidence (%)
If
{Data Type, Comparison and Conditional Operators, Logical and Bitwise Operators, If Statements}
For
{Data Type, Variables, Break and Continue Statements, For Statements}
{If Statements}){Data 0.732/0.641 Type, Comparison and Conditional Operators, Logical and Bitwise Operators} {For Statements}){Data 0.680/0.521 Type, Variables, Break and Continue Statements}
minSupport=0.5, minConfidence=0.4.
Table 3 Correlative weighting matrix for ‘‘If” unit. CM
Data type
Comparison and conditional operators
Logical and bitwise operators
If statements
Data type Comparison and conditional operators Logical and bitwise operators If statements
1 0.394
0.432 1
0.444 0.402
0.318 0.339
0.771
0.447
1
0
0.306
0.656
0
1
Table 4 Correlative weighting matrix for ‘‘For” unit. CM
Data type
Break and continue statements
Variables
For statements
Data type Break and continue statements Variables For statements
1 0.89
0.319 1
0.444 0.363
0 0.454
0.475 0
0.303 0.484
1 0.368
0.524 1
agree’, and ‘Strongly Disagree’. Several learners (n = 31) who had used the developed system to learn were invited to fill out the questionnaire and attended the ‘‘Programming Design” course. The evaluation results indicate that most learners agreed that the
4165
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167 Table 5 Learning paths for different goal. Goal
Learning path
If
{Data Type}?{Logical and Bitwise Operators} ?{Comparison and Conditional Operators} ?{If Statements} {Variables}?{Data Type}?{Break and Continue Statements} ?{For Statements}
For
Correlative weighting
Hit ratio (%)
0:656
fIf Satementsg ! fComparison and Conditional Operatorsg 0:402
1
0:771
! fLogical and Bitwise Operatorsg ! fData Typeg 0:484
fFor Satementsg ! fBreak and Continue Satementsg 0:475
0.667
0:444
! fData Typeg ! fVariablesg
Fig. 12. Evaluation on ‘‘If” learning path.
Fig. 13. Evaluation on ‘‘For” learning path.
Table 6 The results for different subjects on Java programming language. Category
Subject
Average of precision (%)
Average of recall (%)
Fmeasure
Hit ratio (%)
Average of hit ratio for each category
Control flow statements
If For While Switch
0.62 0.72 0.77 0.81
0.66 0.74 0.71 0.78
0.64 0.73 0.74 0.79
1 0.667 0.667 0.667
0.75
Classes and objects
Class and object Method Constructor
0.65 0.62 0.66
0.68 0.65 0.65
0.66 0.63 0.65
1 0.667 1
0.889
Interface and inheritance
Interface Inheritance Abstract methods and classes
0.56 0.64 0.5
0.61 0.66 0.53
0.58 0.65 0.51
0.667 0.667 0.333
0.556
Advance application
Package Thread Applet Input/output
0.63 0.66 0.51 0.71
0.61 0.60 0.46 0.72
0.62 0.63 0.48 0.71
1 0.667 0.333 1
0.75
minSupport (%) = 0.5, minConfidence (%) = 0.4
Average hit of ratio (%)
0.736
4166
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167
Table 7 A questionnaire. #
#1
Question description
Level
#3
I agree that using the proposed e-learning system to provide a suitable learning path is very convenient to me on self-learning I agree that those learning objects decided by the proposed e-learning system are helpful to me I agree that I can certainly solve problems after using the proposed e-learning system
#4
I agree that I will study hard every time in the proposed e-learning system
#5
I think that using the proposed e-learning system can promote my learning motivation and interest I think that the proposed e-learning system offers a friendly user interface and I will still use the platform for learning I think that using the proposed e-learning system more often will decrease my interaction with teachers or peers I think that using the proposed e-learning system can promote personal learning ability on of each subject When I login the proposed e-learning system for learning, I can clearly realize the learning procedure of the platform By using the proposed e-learning system, I feel that I can efficiently decrease time in searching Internet for the materials which facilitate learning
#2
#6 #7 #8 #9 #10
Strongly agree
Agree
Neutral
Disagree
Strongly disagree
16 (51.6%) 20 (64.5%) 5 (16.1%) 2 (6.7%) 15 (48.6%) 18 (58.2%) 3 (10%) 8 (25.8%) 16 (51.6%) 15 (48.3%)
8 (25.8%) 8 (25.8%) 23 (74.2%) 25 (80.3%) 10 (32.3%) 4 (13%) 7 (22.6%) 9 (28.5%) 10 (32.3%) 8 (25.8%)
4 (12.9%) 2 (6.7%) 2 (6.7%) 3 (10%) 5 (16.1%) 8 (25.8%) 12 (38%) 10 (32.3%) 5 (16.1%) 4 (12.9%)
2 (6.7%) 1 (3%) 1 (3%) 1 (3%) 0 (0%) 1 (3%) 3 (10%) 2 (6.7%) 0 (0%) 3 (10%)
1 (3%) 0 (0%) 0 (0%) 0 (0%) 1 (3%) 0 (0%) 6 (19.4%) 2 (6.7%) 0 (0%) 1 (3%)
developed system helped them learn efficiently. Over 77% of learners agreed that the appropriate learning path, which the proposed system constructed, helped them learn the Java programming language. Moreover, 90% of learners agreed that the recommended learning materials helped them because they did not need to spend a lot of time surveying and choosing the right learning materials.
Acknowledgement
4. Conclusions
Agrawal, R., & Srikant, R. (1994). Fast algorithms for mining association rules. In Proceedings of 20th international conference on very large data bases (pp. 487– 499). Agrawal, R., Imielinski, T., & Swami, A. (1993). Mining association rules between sets of items in large databases. In Proceedings of the ACM SIGMOD conference on management of data (pp. 207–216). Aizawa, A. (2003). An information-theoretic perspective of TF–IDF measures. Information Processing & Management, 39(1), 45–65. Chen, C.-M. (2008). Intelligent web-based learning system with personalized learning path guidance. Computers & Education, 51, 787–814. Chen, C.-M., & Chung, C.-J. (2008). Personalized mobile English vocabulary learning system based on item response theory and learning memory cycle. Computers & Education, 51, 624–645. Chen, C.-M., Liu, C.-Y., & Chang, M.-H. (2006). Personalized curriculum sequencing utilizing modified item response theory for web-based instruction. Expert Systems with Applications, 30(2), 378–396. Chi, Y.-L. (2007). Elicitation synergy of extracting conceptual tags and hierarchies in textual document. Expert Systems with Applications, 32(2), 349–357. CKIP AutoTag (1998). Chinese knowledge information processing group. Academic sinica, Taiwan. Colace, F., De Santo, M., & Iacone, M. (2005). Work in progress – ASAP: An automatic student adapted learning path generator. In Proceedings of 35th annual conference frontiers in education (pp. 13–14). Davey, B. A., & Priestley, H. A. (2002). Introduction to lattices and order (pp. 65–84). Cambridge University Press. Dietinger, T. (2003). Aspects of e-learning environments. Institute for information processing and computer supported new media (IICM). Graz University of Technology A-8010 Graz, Austria. Ganter, B., & Wille, R. (1999). Formal concept analysis: Mathematical foundations. Berlin, Heidelberg: Springer-Verlag. Hidber, C. (1999). Online association rule mining. In Proceedings of the ACM SIGMOD conference on management of data (pp. 145–156). Huang, M.-J., Huang, H.-S., & Chen, M.-Y. (2007). Constructing a personalized elearning system based on genetic algorithm and case-based reasoning approach. Expert Systems with Applications, 33(3), 551–564. Ismail, J. (2002). The design of an e-learning system beyond the hype. Internet and Higher Education, 4, 329–336. Koprinska, I., Poon, J., Clark, J., & Chan, J. (2007). Learning to classify e-mail. Information Sciences, 177(10), 2167–2187. Lee, M.-C., Ye, D.-Y., & Wang, T.-I (2005). Java learning object ontology. In Proceedings of the fifth IEEE international conference on advanced learning technologies (pp. 538–542).
Most research on e-learning has used a pre-constructed category or some manual information for semi-automatic learning object classification and learning path construction in specific domains. Few platforms have focused on automatically constructing learning paths for occasional knowledge on unspecific domains from open environments, such as fragmented knowledge from the Internet. In this paper, a support system with two main approaches, the learning path construction approach and the learning object recommendation approach, is proposed. The system first finds a set of candidate course subjects based on the well-known Apriori algorithm. Then, the learning path construction approach use Formal Concept Analysis to build a Concept Lattice using keywords extracted from collected documents to form a relationship hierarchy between all the concepts represented by the keywords. It uses FCA to further compute the mutual relationships among documents to determine the appropriate learning path. Moreover, in order to assist a learner to study efficiently, the support system employs both preferencebased and correlation-based algorithms to find the most suitable learning objects collected from the Internet by retrieval agents. The experiments show that the proposed approaches achieve more than 50% accuracy in finding candidate courses, and at least a 66.7% suitability in constructed learning paths. Over 77% of learners agreed that the appropriate learning path, which the proposed system constructed, helped them learn the Java programming language. Moreover, 90% of learners agreed that the recommended learning materials also helped them. The proposed approach helps learners browse and read collected learning objects or documents in the correct order to understand the fragmented knowledge.
This work is supported by the Nation Science Council of Taiwan under the contract NSC95-2221-E-006-158-MY3.
References
T.-C. Hsieh, T.-I. Wang / Expert Systems with Applications 37 (2010) 4156–4167 Lee, M.-C., Tsai, K.-H., & Wang, T.-I. (2008). A practical ontology query expansion algorithm for semantic-aware learning objects retrieval. Computers & Education, 50, 1240–1257. Liu, H.-I., & Yang, M.-N. (2005). QoL guaranteed adaptation and personalization in elearning systems. IEEE Transactions on Education, 48(4), 676–687. Moodle. Modular object-oriented dynamic learning environment.
4167
Tseng, Judy. C.-R., Chu, H.-C., Hwang, G.-J., & Tsai, C.-C. (2008). Development of an adaptive learning system with two sources of personalization information. Computers & Education, 51, 776–786. Wang, T.-I., Tsai, K.-H., Lee, M.-C., & Chiu, T.-K. (2007). Personalized learning objects recommendation based on semantic-aware discovery and a learners preference pattern. Educational Technology & Society, 10(3), 84–105. Weng, S.-S., Tsai, H.-J., Liu, S.-C., & Hsu, C.-H. (2006). Ontology construction for information classification. Expert Systems with Applications, 31(1), 1–12. Wille, R. (1982). Restructuring lattice theory: An approach based on hierarchies of concepts. Ordered Sets, 445, 470.