Learning objects and objectives towards automatic learning construction

Available online at www.sciencedirect.com

European Journal of Operational Research 187 (2008) 1449–1458 www.elsevier.com/locate/ejor

Learning objects and objectives towards automatic learning construction George Mavrommatis

*

Department of Informatics, University of Piraeus, 185 34 Piraeus, Greece Available online 13 November 2006

Abstract This paper presents a method that creates instructionally sound learning experiences by means of learning objects. The method uses a mathematical model, distinguishes two kinds of Learning Objects Properties and proceeds in two major steps: first, the Course Creation is transformed into Set Covering under specific requirements derived from Learning Theories and practice; second, the Alternative Learning Sources are selected by using a similarity measure specially defined for this purpose.  2006 Elsevier B.V. All rights reserved. Keywords: Education; Distance learning; Instructional design; Information retrieval

1. Elements of e-learning Communication between teachers and learners with print documents via classic postal service is considered to be the first generation of distance learning. This was first introduced in the late 1800s. Today, we are within the fourth major evolution of distance learning, relying upon two-way communication via multimedia desktop computers, which in general, is what we understand as e-learning (Burgess and Russell, 2003). In COM (2001) e-learning was defined as ‘‘the use of new multimedia technologies and the Internet to improve the quality of learning by facilitating access to resources and services as well as remote exchanges and collaboration’’.

*

Tel.: +30 2104142263; fax: +30 2104142264. E-mail address: [email protected]

From any point of view one approaches the subject, one conclusion can be certainly drawn: distance education is now increasingly conducted via the Internet (web-based) and this trend will keep growing at even greater rates (McGorry, 2003). The Internet, either as sole communication media or as part of a hybrid-type (e.g. ISDN video terminals and networked PCs) is becoming the standard platform for e-learning environments (Kodama, 2001). Even more, it fits perfectly the transformation that is taking place from ‘‘broadcast’’ to ‘‘interactive’’ learning (Tapscott, 1998). Learning objects (LO) is a relatively recent trend in Web-based courseware authoring. They are based on the idea that an instructor creates small learning components that can be combined and reused in different contexts. Learning Objects Metadata (LOM) is the information used to describe a LO. Recent work (IEEE LTSC, 2002) provides standardization of LO metadata: the LOM framework is already

0377-2217/$ - see front matter  2006 Elsevier B.V. All rights reserved. doi:10.1016/j.ejor.2006.09.024

1450

G. Mavrommatis / European Journal of Operational Research 187 (2008) 1449–1458

part of the Sharable Content Object Reference Model (SCORM, 2004) and all this effort, despite of some criticism, is expected to speed up the creation of learning objects repositories, that is large pools containing retrievable LO and metadata indexes based on the standard (Neven and Duval, 2002). The platform and the building blocks, therefore, are present. Rehak (2004) states two major forthcoming challenges, in order to fully integrate online learning to the educational systems: first, create a very large volume of easily accessible learning content, and second, create a user-friendly environment, that will get the right content to the learner at the right time. The design of such an environment is an open question. This environment has to be based on a proper search model, which obviously will be the heart of the system. This paper seeks to combine Learning Objects with Instructional Design principles as the connecting glue, in order to create personalized learning situations over the Internet. Remainder is organized as follows: Section 2 derives from instructional design theories and existing literature to present an LO definition, classifies their properties and forms a simple mathematical model as framework; Section 3 presents an approach to solving the problem of LO sequencing and course creation; an example is presented in Section 4; in order to support a constructivist learning environment, Section 5 presents a method of selecting and ranking LO similar to the ones selected in the course; finally, some conclusions are drawn in Section 6.

2. Setting the objectives Learning Theories supply the principles of Instructional Design. There are three major Learning Schools met in the literature (Ertmer and Newby, 1993): Behaviorism, Cognitivism and Constructivism. Both Behaviorism and Cognitivism are also referred as Objectivism, whereas learning is considered more subjective in Constructivism. Objectivism seeks to efficiently transfer knowledge to learners, sets goals and creates educational objectives that instruction tries to achieve. Constructivism believes that each learner builds his/her personal view of the world based on experiences and, compared to Objectivism, is more open-ended, more subject-free and learner-oriented, trying to

facilitate knowledge construction rather than communicating knowledge (Mergel, 1998). Constructivism has not described a single instructional design model. However, one can state a few principles that link theory with practice (Honebein, 1996; Tait, 1997). A Learner among others: • Sets his/her personal goals for learning and builds on already existing knowledge. • Constructs his/her own personal knowledge base and must be provided with multiple views of a subject area and modes of representation. A major resultant one can easily deduce from the above listed principles is Adaptability. Along the four dimensions that characterize a typical e-learning environment (Pahl, 2002), Adaptability is an attribute of the pedagogy dimension. In recent years lots of adaptive hypermedia systems (Wang, 2003) have been developed (e.g. Brusilovsky and Vassileva, 2003). Also see (Brusilovsky, 2003) for an attempt to exhibit the similarities among various known types of adaptive hypermedia systems. Under the light of Learning Objects, Instructional Principles and the Web, the problem can go much further than adaptation within a (mostly) predefined web page or even a course. It is a matter of adaptation within the educational WWW (Fig. 1) which forms a huge Megaworld. A user sets his/her learning goals and the system has to support these selections by providing the proper material of Learning Objects. In order to achieve this, each Learning Object contained in the Megaworld has to be indexed either by proper use of metadata or an extension, that will be added to the schema. Discussion and criticism about LO focuses on two major points (Wiley, 2000; Friesen, 2004) which are not independent: • How should a Learning Object be defined? • How shall it be related to pedagogy and learning theories? It is Objectivism that uses Task Analysis Methods as a tool in instructional design process. Task analysis is based on the assumption that every knowledge field or complex cognitive skill to be taught consists of a number of elementary skills. Therefore, the complex item can be broken down into constituent skills finally leading to construction of some kind of learning hierarchy and create sequence of instruction (Jonassen et al., 1999).

G. Mavrommatis / European Journal of Operational Research 187 (2008) 1449–1458

Learning Objects

Objectivist Learning

Learning Objects Definition

Task Analysis

ConceptIndexed Learning Objects Repositories

Information Retrieval

1451

Web-based Constructivist Oriented Learning Environment

Fig. 1. The megaworld evolution.

Among numerous definitions of LO found in the literature, some are extremely broad (IEEE LTSC, 2002; Wiley, 2000), others are narrower and more specific (e.g. Hamel and Ryan-Jones, 2002; Himes and Wagner, 2002; Smith, 2004). Deriving from a careful study of the existing literature and viewed mainly from the educational perspective, the following definition of a LO is given in this paper: ‘‘A Learning Object is a standalone, reusable, digital resource that aims at teaching one or more instructional objectives or concepts’’. By using a Task Analysis method such as Learning Hierarchy Analysis or Learning Contingency Analysis, a certain knowledge field may be broken down to its constituent parts. Of course, the decision of whether the skills’ analysis has proceeded at a low enough level of granularity is up to the designer. Let S be the Information Space consisting of all nodes that were created by Task Analysis. Information Space S contains all component parts composing the knowledge field: S ¼ fsi g;

i 2 M ¼ f1; 2; . . . ; mg:

The Skills Relative Position Table R = (rij), representing the learning hierarchy, is then constructed as follows: 8 1; if node si is parent of node sj ; > < rij ¼ 2; if node si is child of node sj ; > : 0; otherwise:

Definition 1. A node sj 2 S is a leaf node iff rij 5 2 "i 2 M. Definition 2. For each node si0 2 S; i0 2 M we define set Family(i0) containing all its immediate children: Familyði0 Þ ¼ fsj 2 S 8j 2 M : ri0 j ¼ 1g: Obviously, for all leaf properties it is Family(i0) = B.

Definition 3. If Family(i0) 5 B then node si0 is called the pivot node of Family(i0). A pivot node si0 together with its corresponding Family(i0) form a Cluster (SCORM, 2004). Definition 4. For each pivot si0 , we define Leaves(i0) as the set of pivot’s children that are also leaf nodes: Leavesði0 Þ ¼ fsj 2 S : rij 6¼ 2 8i 2 M and ri0 j ¼ 1g: It is clear that the condition Leaves(i0)  Family(i0) holds for every pivot, while the condition Leaves (i0) = Family(i0) holds for every pivot that is father of leaf properties only. Let L be the set of available Learning Objects dealing with a certain knowledge field, where L ¼ fkp g;

p ¼ 1; 2; . . . ; t:

• Each kp presents (different) approaches to one or more knowledge items, skills or concepts from S. We call them functional properties of the LO, by analogy to software engineering terminology. Therefore, for each kp 2 L, a content set P is defined P ¼ fsi1 ; si2 ; . . . ; sik g;

k 6 m;

and consequently a LO can be viewed as subset of Information Space S. • Each kp has certain characteristics, its non-functional properties: required studying time, level of difficulty, etc. Each non-functional property can be represented by a Cost that is assigned to the Learning Objects. Let C ¼ fcp g; p ¼ 1; 2; . . . ; t be the cost for each kp respectively. Additionally, a learner’s knowledge on the certain field can be represented by a similar set: usr ¼ fui1 ; ui2 ; . . . ; uik g  S: Although Objectivism and Constructivism are considered as the two extreme ends of the Learning theories continuum, it is often proposed that a weighted use of principles from both could be done

1452

G. Mavrommatis / European Journal of Operational Research 187 (2008) 1449–1458

simultaneously (Vrasidas, 2000). As stated in (Wilson, 1997) ‘‘. . .an instructional strategy that imposes structure may actually help learners make constructions needed for learning’’. There are two major steps involved into the Megaworld Exploration. The first one builds a Learning Backbone by sequencing the proper Learning Objects, while the second selects sets of LO as Alternatives to the ones selected in the first place. Next sections present a methodology of how information manipulation and retrieval techniques (Baeza-Yates and Ribeiro-Neto, 1999) can be applied to the above simple model in order to support knowledge construction. 3. The Learning Backbone Creation of the Learning Backbone is done by sequencing the proper Learning Objects. This has to be done in a way that satisfies the following requirements: • • • •

preserves learning hierarchy, respects user’s previous knowledge, fulfils user’s targets, and satisfies user’s preferences (any non-functional properties plus learning pace).

Algorithm LB, presented in current section, outputs a sequence Ci, i = 1, . . ., d of sets of Learning Objects (Fig. 2). In order to preserve Learning Hierarchy, sequencing is achieved by traversing the hierarchy tree using the Skills Relative Position Table. The Algorithm at each iteration creates set Candidates containing all Leaves(i) such that LeavesðiÞ ¼ FamilyðiÞ: Any properties in Candidates that belong to usr too are being removed. Selection of proper Learning Objects for each Candidates instance (done by procedure Candidates_covering) can be expressed as follows: Given a set Candidates, containing all ‘‘valid’’ properties (concepts a learner is capable of learning) and a collection L (Learning Objects) of sets of these properties, find a subset C of L so that each property in Candidates belongs to at least one of the subsets in C. This is the well-known Set Covering Problem (SCP) which is NP-Hard (Tovey, 2002): no polynomial time algorithm has yet been discovered to solve it exactly. In general, SCP may be subject to a variety of simplifications (Coudert and Madre, 1995) like essentiality, dominance, etc. For example, we may limit our search for LO in a set L 0  L of

Learner’s Preferences

Learning Hierarchy

Set Covering

Cost

d=1 Candidate Properties

Chapter C(d)

Learning Objects Repositories

Learner’s Knowledge

All Properties Covered? Yes

d=d+1

No

Chapters C(1) C(2) . . . C(d)

COURSE Fig. 2. Learning Backbone (LB).

G. Mavrommatis / European Journal of Operational Research 187 (2008) 1449–1458

Learning Objects such that P  Candidates, " kp 2 L 0 , but, if no solutions found, we may extend the search in a set as wide as Candidates [ usr. Although SCP is NP-Hard, there exist many algorithmic approaches that very efficiently yield near optimal solutions. One of the best polynomial time algorithms for approximating set cover is the greedy algorithm (Gue´ret et al., 2003). In the weighted version of SCP, Chvatal’s greedy heuristic consists at each iteration in selecting the subset that optimizes some criterion. The determination of such a criterion in our case affects: • LO density: One may ask for a small number of LO that teach more concepts per each, while another may ask for a larger number of LO, that each one teaches less concepts. • Satisfaction of a Learner’s preferences on nonfunctional properties: By choosing LO that optimize (minimize or maximize) a corresponding Cost, or even a composite function that is composed of more than one Costs. • Both the above can be combined to a single criterion. For example, let us assume that the nonfunctional property of interest is the level of difficulty which can be measured by a Cost function. During each iteration of the Candidates_covering procedure, one may select the set P (corresponding to a LO kp) having the minimum ratio jPcpj. Then, one will be seeking for LO each one introducing the less possible new concepts (low density) but with high level of difficulty. It is clear that Learning pace is regulated by LO density. The learning pace may also be regulated by adjusting the ‘‘tree traversal rate’’, that is, controlling the addition rate of next properties to cover. In Algorithm LB, new properties are added into the Candidates instance each time the Leaves(i) = Family(i) reaches. In order to somehow speed up the hierarchy traversal, one may add all new Leaves(i) at each iteration, without waiting for this condition to reach. Each time Candidates_covering finishes, all Leaves(i) that have composed current instance of Candidates are removed and not considered for further computation. Every Leaves(i) corresponds to a pivot node, namely si. This node either belongs to Family(k) for some k, so we add this pivot to the corresponding Leaves(k), or does not belong to any Family(k) which means that si is the root (top) node

1453

of Hierarchy. In the first case, a new instance of Candidates is composed, while in the latter, after covering si the algorithm ends. Algorithm LB follows: Algorithm LB Input: Number of properties m, Skills Relative Position Table R for i = 1 to m create Leaves(i), Family(i) //for Leaves(i) that can be defined, according to Definition 4// d=0 do I=B for all i in [1, m] that Leaves(i) is defined if (i NOT marked) if Leaves(i) = Family(i) { mark i I = I [ {i} } d=d+1 candidates(d) = B for each i 2 I candidates(d) = candidates(d) [ Leaves(i) candidates(d) = candidates(d)  usr Call Candidates_covering(C(d)) top = true for each i 2 I for k = 1 to m if si 2 Family(k) { Leaves(k) = Leaves(k) [ {si} top = false } while NOT top d=d+1 candidates(d) = {si:i 2 I} Call Candidates_covering(C(d)) // top (root) node Output: Learning Backbone C(i), i = 1, . . . , d End LB 4. Putting it together: An example For the example’s sake, let us consider a certain knowledge field where Task Analysis yielded the result shown in Fig. 3. As we can see, the Information Space in question consists of 10 properties (concepts, skills and so on).

1454

G. Mavrommatis / European Journal of Operational Research 187 (2008) 1449–1458

s1

s2

s3

s5

s4

s6

s7

s8

s9

s10

Fig. 3. Hierarchical task analysis.

Table 1 Skills Relative Position Table Properties

s1

s2

s3

s4

s5

s6

s7

s8

s9

s10

s1 s2 s3 s4 s5 s6 s7 s8 s9 s10

0 2 2 2 0 0 0 0 0 0

1 0 0 0 2 2 0 0 0 0

1 0 0 0 0 0 0 0 0 0

1 0 0 0 0 0 2 2 0 0

0 1 0 0 0 0 0 0 0 0

0 1 0 0 0 0 0 0 0 0

0 0 0 1 0 0 0 0 2 2

0 0 0 1 0 0 0 0 0 0

0 0 0 0 0 0 1 0 0 0

0 0 0 0 0 0 1 0 0 0

Therefore it is m = 10, S = {si}, M = {1, 2, . . . , 10}. Let us also assume that usr = {s6, s10}, which means that the learner already knows these properties. By utilizing information in Fig. 3, we construct the Skills Relative Position Table, shown in Table 1. For each node (property) si we construct, if it can be defined, the corresponding sets Leaves(i) and Family(i). Results are shown in Table 2. Based on Table 2, Algorithm LB follows the steps described below, leading to the creation of a four Chapters’ Course:

Table 2 Families and Leaves Node i

Family(i)

Leaves(i)

1 2 3 4 5 6 7 8 9 10

{s2, s3, s4} {s5, s6} B {s7, s8} B B {s9, s10} B B B

{s3} {s5, s6} Not defined {s8} Not defined Not defined {s9, s10} Not defined Not defined Not defined

Step 1 • I = {unmarked i 2 M:Leaves(i) = Family(i)} = {2, 7}. • Mark nodes s2 and s7. • Candidates = [ i2ILeaves(i)  usr = {s5, s6, s9, s10}  {s6, s10} = {s5, s9}. • d = 1. • Candidates_covering(C(1)) creates Chapter 1 that presents properties s5, s9. • The pivots of all Leaves(i) covered above are s2, s7. • s7 belongs to Family(4), therefore we add it to Leaves(4) = {s8} [ {s7} = {s7, s8}. • s2 belongs to Family(1), therefore we add it to Leaves(1) = {s3} [ {s2} = {s2, s3}. Step 2 • I = {unmarked i 2 M:Leaves(i) = Family(i)} = {4}. • Mark node s4. • Candidates = [ i2ILeaves(i)  usr = {s7, s8} {s6, s10} = {s7, s8}. • d = d + 1 = 2. • Candidates_covering(C(2)) creates Chapter 2, that presents properties s7 and s8. • The pivots of Leaves(i) covered above: s4. • s4 2 Family(1), therefore we add the node to Leaves(1) = {s2, s3} [ {s4} = {s2, s3, s4}. Step 3 • I = {unmarked i 2 M:Leaves(i) = Family(i)} = {1}. • Mark node s1. • Candidates = [ i 2 ILeaves(i)  usr = {s2,s3,s4}  {s5, s6, s9} = {s2, s3, s4}. • d = d + 1 = 3. • Candidates_covering(C(3)) generates Chapter 3 presenting properties s2,s3 and s4. • The pivots of Leaves(i) found in previous steps: s1. Final step • s1 does not belong to a Family(i). • Candidates = {s1}. • d = d + 1 = 4. • Candidates_covering(C(4)) generates Chapter 4 presenting property s1. Output: four sets of Learning Objects C(1), C(2), C(3), C(4), the Chapters, composing a Learning Backbone.

G. Mavrommatis / European Journal of Operational Research 187 (2008) 1449–1458

After a user goes through the Learning Backbone, it is guaranteed that he/she has been presented all the concepts in the knowledge field, proper sequenced and compatible with learner’s preferences. Therefore we may assume that the learner has been presented a full Course on the field in question. 5. Exploration to construct learning It is an essential part of a constructivist-oriented learning environment to present the user with different views and approaches of the subject in question. To achieve this, a user has to be presented with a variety of Learning Objects from the same Information Space S, still preserving the requirements mentioned in Section 3. These LO contain already known properties and the user may choose one or more to go through. The system may select kp 2 S, such that: (a1) P  Candidates(d), presenting a view of properties contained in the corresponding Chapter C(d). (a2) P  [di¼1 Candidates(i), presenting a view of properties in all already previously taught Chapters. (a3) P  S, presenting LO retrieved from the Megaworld regardless of which Candidates instance they belong. Let kp, kq 2 S. In any case of the above, selection, evaluation and, eventually ranking of the LO, has to be done by using a proper similarity measure s(kp, kq) which takes into account and allows a weighted participation of both their Functional and Non-Functional Properties. Regarding the Functional properties, we need a measure of how close these two sets are. We propose the Overlap Coefficient (Mitchel, 2000), a wellknown set similarities measure: jP \ Qj FPðkp ; kq Þ ¼ 2 ½0; 1: minfjP j; jQjg Regarding the non-functional properties we define the following normalized Euclidean distance metric: NFPðkp ; kq Þ ¼ 1 

jcp  cq j 2 ½0; 1: maxfci g i2M

By combining both measures, s(kp, kq) is defined as follows:

1455

sðkp ; kq Þ ¼ D  FPðkp ; kq Þ þ ð1  DÞ  NFPðkp ; kq Þ; D 2 ½0; 1; which leads us to define equality (equivalence) between LO: 8 > < P ¼ Q and cp ¼ cq ; if D 2 ð0; 1Þ; kp ¼ kq () P ¼ Q; if D ¼ 1; > : cp ¼ cq ; if D ¼ 0: Measure s verifies the properties (a)–(d) of a similarity measure (Nunez-Garcia and Wolkenhauer, 1999): (a) s(kp, kq) 2 [0, 1]. (b) s(kp, kp) = 1. (c) s(kp, kq) = s(kq, kp). Proof (a)–(c). All by definition of FP(kp, kq), NFP(kp, kq), s(kp, kq). [(d)] s(kp, kq) = 1 ) kp = kq. Proof sðkp ; kq Þ ¼ 1 ) D  ðNFPðkp ; kq Þ  FPðkp ; kq ÞÞ ¼ NFPðkp ; kq Þ  1: (d1) If NFP(kp,kq)  FP(kp,kq) = 0 then NFP(kp,kq)  1 = 0 and therefore FP(kp, kq) = NFP(kp,kq) = 1, which leads us to P = Q and cp = cq and finally kp = kq. (d2) If NFP(kp, kq)  FP(kp, kq) 5 0, then D ¼ NFPðkp ; kq Þ  1 . NFPðkp ; kq Þ  FPðkp ; kq Þ D 2 ½0; 1 )

NFPðkp ; kq Þ  1 NFPðkp ; kq Þ  FPðkp ; kq Þ

P 0;

ð1Þ

and FPðkp ; kq Þ  1 6 0: NFPðkp ; kq Þ  FPðkp ; kq Þ

ð2Þ

We distinguish two sub-cases: (d2.1) NFP(kp,kq)  FP(kp,kq) < 0. Then (2) ) FP(kp,kq)  1 P 0 ) FP(kp,kq) = 1 ) P = Q, which leads us to kp = kq, if D = 1. In the case that D 5 1, s(kp,kq) = 1 ) NFP(kp,kq) = 1 ) cp = cq and we deduce kp = kq, as well.

1456

G. Mavrommatis / European Journal of Operational Research 187 (2008) 1449–1458 Information Space S

Information Space S’

p1

s1 Identical property

s2

p2

s4

s3

p4 s5

p3 p5

s6 p6

p7

Fig. 4. Overlapping domains.

(d2.2) NFP(kp, kq)  FP(kp, kq) > 0. Then (1) ) NFP(kp, kq)  1 P 0 which finally leads us to the same conclusion kp = kq. h The steps described in Sections 3 and 5 may be used to retrieve LO that are subsets of another Information Space S 0 and still relevant to the subject in question, thus presenting further more views of the subject area. Domains are often not complete nor unique. For example, Physics overlaps with Mathematics, Chemistry, History, and so on. This holds for almost every traditionally defined learning domain. Let S 0 be an Information Space such that S \ S 0 5 B. For example, let us consider the case illustrated in Fig. 4, where properties s4 2 S and p2 2 S 0 are identical. While traversing S, and after property s4 has been covered, the system switches to S 0 , and follows the same procedure considering the sub-space of S 0 that has p2 as its root node. 6. Concluding remarks The advent of the Internet has already influenced education. Learning Objects and standardization of their metadata is expected to add more value to distance learning and fully integrate it to the educational systems. Moreover, the purpose of metadata standard is to facilitate indexing, search, and use of learning objects as building blocks of lessons, including automatic and dynamic creation of adaptive personalized ones. In the near future, it is expected that Learning Objects Repositories will increase their numbers, and that a user will be able to retrieve from a very large number of available LO presenting various knowledge fields with different approaches and characteristics.

In any case, all this effort cannot lack instructional design. The future of educational technology is calling for revisiting of traditional instructional models (Hamel and Ryan-Jones, 2002). On the other hand, much work on educational technology is focused on implementation, with less attention paid to analysis, design and modeling. This paper tried to outline how information retrieval techniques combined with instructional design principles could be used on Learning Objects, to present an adaptive, instructionally sound environment. Utilizing Task Analysis resulted in a definition of Learning Objects focusing on an mostly educational aspect. Task Analysis also provided the starting point for a distinction between two classes of Learning Objects properties. In order to approach a constructivist-oriented environment, two major steps were presented: Learning Backbone was generated by transforming the core of the problem to the weighted version of Set Covering, while pluralism of constructivist environments was preserved by defining a similarity measure and retrieving Learning Objects similar to the ones initially contained in the Learning Backbone. Finally, the similarity measure resulted in defining equality (equivalence) between Learning Objects. There are still open questions in defining LO properties and combining them in the retrieval process. In present paper level of difficulty was used as a quantitative factor. Yet, a more detailed analysis of what characterizes a LO and how it can be represented is needed. Other characteristics like physical size, download time, etc., are of some importance and should somehow influence the selection. Learning pace regulation is discussed in present paper but, no doubt, the subject needs more

G. Mavrommatis / European Journal of Operational Research 187 (2008) 1449–1458

attention. For example, during Learning Backbone generation, one may allow selection of LO that contain concepts already known to the learner. Selection of a method that solves or approximates the SCP efficiently and in a way that fits the special requirements of instruction needs also careful consideration. Another point under research, is link creation among LO that the system selects as a Learning Backbone and, more important, among the Alternatives retrieved during step 2. Both classes of properties could be used for this purpose. For example, functional properties may be used for conceptual relation and linking and non-functional for other kinds of relation, like progressive difficulty. Yet another open issue is selection, combination and testing of a Learning Objects similarity measure. Finally, incorporation of other instructional design theories into the described model, or a relevant to be designed, has to be researched too. The use of Learning Objects in education is under development and therefore there are many issues still open. This paper tries to aim towards this general target, but there is still much more to be done. As Wiley (2000) puts it, ‘‘more theorists are needed’’. References Baeza-Yates, R., Ribeiro-Neto, B., 1999. Modern Information Retrieval. ACM Press. Brusilovsky, P., 2003. Developing adaptive educational hypermedia systems: From design models to authoring tools. In: Murray, T., Blessing, S., Ainsworth, S. (Eds.), Authoring Tools for Advanced Technology Learning Environment. Kluwer Academic Publishers, Dordrecht. Brusilovsky, P., Vassileva, J., 2003. Course sequencing techniques for large-scale web-based education. International Journal of Continuing Engineering Education and Lifelong Learning 13 (1/2), 75–94. Burgess, J., Russell, J., 2003. The effectiveness of distance learning initiatives in organizations. Journal of Vocational Behavior, Available on line from: . COM, 2001. The eLearning Action Plan. Commission of the European Communities, 172 final. Coudert, O., Madre, J.-C., 1995. New ideas for solving covering problems. In: 32nd ACM/IEEE Design Automation Conference, 1995. Ertmer, P.A., Newby, T.J., 1993. Behaviorism, cognitivism, constructivism: Comparing critical features from an instructional design perspective. Performance Improvement Quarterly 6 (4), 50–72. Friesen, N., 2004. Three objections to learning objects. In: McGreal, R. (Ed.), Online Education Using Learning Objects. Routledge/Falmer, London.

1457

Gue´ret, C., Jussien, N., Lhomme, O., Pavageau, C., Prins, C., 2003. Loading aircraft for military operations. Journal of the Operational Research Society 54 (5), 458–465. Hamel, C., Ryan-Jones, D., 2002. Designing instruction with learning objects. International Journal of Educational Technology, 2002 Double Issue-v3, n1. Himes, F., Wagner, E., 2002. Macromedia MX: Empowering Enterprise eLearning, Macromedia White Paper, September 2002. Honebein, P., 1996. Seven goals for the design of constructivist learning environments. In: Wilson, B. (Ed.), Constructivist Learning Environments: Case Studies in Instructional Design. Educational Technology Publications, New Jersey, pp. 11– 24. IEEE LTSC, 2002. Learning Technology Standards Committee, IEEE 1484.12.1-2002 Learning Object Metadata Working Group, Draft Standard for Learning Object Metadata, 15 July, 2002. Jonassen, D., Tessmer, M., Hannum, W., 1999. Task Analysis Methods for Instructional Design. Lawrence Erlbaum Associates, London. Kodama, M., 2001. Distance learning using video terminals—an empirical study. International Journal of Information Management. 21 (3), 227–243. McGorry, S., 2003. Measuring quality in online programs. Internet and Higher Education 6 (2003), 159–177. Mergel, B., 1998. Instructional design and learning theories. Online Article, available from: . Mitchel, C., 2000. Cardinal, nominal or ordinal similarity measures in comparative evaluation of information retrieval process. In: Second International Conference on Language Resources and Evaluation, LREC-2000, pp. 1509– 1513. Neven, F., Duval, E., 2002. Reusable Learning Objects: A Survey of LOM-Based Repositories. Multimedia’02, December 1–6, 2002, Juan-les-Pins, France, 2002 ACM 1-58113-620-X/02/ 0012. Nunez-Garcia, J., Wolkenhauer, O., 1999. Random-sets clustering based on set-similarities, Control Systems Centre, UMIST, Report No. 878. Pahl, C., 2002. Managing evolution and change in web-based teaching and learning environments. Computers & Education 40 (2003), 99–114. Rehak, D., 2004. Good & Plenty, Googlezon, Your Grandmother and Nike: Challenges for Ubiquitous Learning & Learning Technology. In: Proceedings, The Partnership in Global Learning: Workshop On E-learning Objects and Systems, June 2004. SCORM, 2004. Advanced Distributed Learning (ADL), Sharable Content Object Reference Model (SCORM) 2004 Overview, 2004. Smith, R., 2004. Guidelines for Authors of Learning Objects. The New Media Consortium, NMC. Tait, B., 1997. Constructive Internet based learning. Active Learning 7. Tapscott, D., 1998. Growing Up Digital: The Rise of the Net Generation. McGraw-Hill, New York. Tovey, C., 2002. Tutorial on computational complexity. Interfaces 32 (3), 30–61, INFORMS 2002. Vrasidas, C., 2000. Constructivism versus objectivism: Implications for interaction, course design, and evaluation in distance

1458

G. Mavrommatis / European Journal of Operational Research 187 (2008) 1449–1458

education. International Journal of Educational Telecommunications 6 (4), 339–362. Wang, H., 2003. Hypermedia: A brief literature review. Journal of Educational Computing, Design & Online Learning 4, Fall 2003. Wiley, D.A., 2000. Connecting learning objects to instructional design theory: A definition, a metaphor, and a taxonomy. In:

D.A. Wiley (Ed.), The Instructional Use of Learning Objects: Online Version. Retrieved Aug-01-2003 from the World Wide Web: . Wilson, B., 1997. Reflections on constructivism and instructional design. In: Dills, C.R., Romiszowski, A.A. (Eds.), Instructional Development Paradigms. Educational Technology Publications, Englewood Cliffs, NJ.