- Email: [email protected]
Using process-oriented, sequencing educational technologies: Some important pedagogical issues
Computers in Human Behavior Computers in Human Behavior 23 (2007) 2742–2759 www.elsevier.com/locate/comphumbeh
Using process-oriented, sequencing educational technologies: Some important pedagogical issues Olivera Marjanovic
*
School of Business, The University of Sydney, Sydney, NSW 2006, Australia Available online 6 October 2006
Abstract In recent times, sequencing technologies are becoming increasingly used, both by the university and the industry sectors. In essence, these complex systems support sequencing of, and navigation through units of content. They come in two different types: SCORM-based and workflow-based systems. Current research efforts related to sequencing technologies concentrate mostly on technical issues, while the associated pedagogical issues remain unexplored. The main objective of this paper is to describe the most important pedagogical issues that need to be taken into account when implementing any type of sequencing educational technologies (either SCORM- or workflow-based). These issues were identified during an action-learning project related to the practical implementation of a workflow-based educational system. During the reflection phase of this project, these issues were then genarised, so they could be applied to any type of sequencing technology and in any application domain (teaching discipline). The paper describes how sequencing technology can be used to enable a more flexible learning experience (especially in terms of time and flexible learning pathways) and then proceeds with the detailed analysis of the associated pedagogical issues. Ó 2006 Elsevier Ltd. All rights reserved. Keywords: Pedagogical issues; Learning Designs; Workflow technology
1. Introduction These days, Learning Management Systems (LMS) are becoming increasingly popular among many universities from all around the world. Examples include technologies such Lotus Learning Space and recently merged WebCT and BlackBoard. More than a decade *
Tel.: +61 2 9351 8477; fax: +61 2 9662 6638. E-mail address: [email protected].
0747-5632/$ - see front matter Ó 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.chb.2006.08.005
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
2743
of development and deployment of these systems have resulted in the substantial accumulated knowledge and experience related to both technical and pedagogical issues. Consequently, educators have gained a much better understanding of the opportunities created by these technologies as well as their major limitations. It is now clear that one of the major limitations of the current LMS is certainly their task-orientation. In other words, they are oriented towards supporting individual tasks while their process-support is still very limited. At the same time, the relevant educational literature confirms that going through a carefully designed teaching/learning process creates much better teaching outcomes than going through isolated learning tasks. The need to support teaching/learning processes, rather than simple tasks, in recent times has resulted in a new type of process-oriented, educational technology. These complex systems also called sequencing technologies because they support sequencing of, and navigation through educational content and/or activities. Currently there are two different types of sequencing technologies: SCORM-based and workflow-based systems. SCORM is one of the latest educational standards (ADL, 2004a). Among other features, it enables sequencing of educational content that is organised in a form of an activity tree. SCORM’s sequencing and navigation mechanism (ADL, 2004b) is then used to deliver ‘‘chunks’’ (units) of content to a single learner based on their progress. The other type of sequencing technology includes the so-called workflow-based systems. These systems aim to support teaching and learning process that is represented as a sequence of individual tasks (activities) that involve multiple learners. In this paper, we distinguish between two different categories of workflow-based systems. The first category includes workflow technology systems. In essence, these are various applications of workflow-management systems (WFMS, 1999), in the education domain to support highly structured teaching/learning processes. The other emerging category of workflow-based systems includes systems that are based on the very recent learning design theory (Koper & Tattersall, 2005) and the associated learning design standard (IMS-LD, 2003). Although these systems do not incorporate workflow management systems, they use workflows to model learning designs, in particular sequencing and coordination of individual tasks in a learning process. IMS-LD standard focuses on formal representation of learning designs that are executable by computers. Current research efforts in the areas of sequencing technologies (both SCORM-based and workflow-based systems) concentrate more on technical issues (such as formal representations that are directly executable by computers) than the associated pedagogical issues. ‘‘. . .Unless all learning specification turn the focus from infrastructure to pedagogical soundness, they are in danger of becoming instructionally irrelevant. . .Despite the progress being made on the interoperability front, that doesn’t necessarily guarantee that what actually runs on SCORM systems will be worthwhile instructionally’’ (Welsch, 2004, p. 2). The main objective of this paper is to describe the main pedagogical issues that need to be taken into account when implementing sequencing educational technologies (either SCORM- or workflow-based). These specific issues were identified during an action-learning project undertaken by the author. The main objective of this project was to observe pedagogical issues related to the implementation of a proprietary workflow-based educational system used to enable a more flexible learning experience in a postgraduate course. Although this particular learning experience was made possible by sequencing technology, the paper will not focus on technical details. Instead, it will describe the main generic pedagogical issues, applicable to any implementation of sequencing educational technologies.
2744
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
The paper is organized as follows. Section 2 offers an overview of the sequencing technologies including SCORM-based systems as well as both categories of workflow-based systems. Section 3 illustrates how sequencing technologies can be used to enable more flexible learning experience, especially in terms of time and provision of more flexible learning pathways. Section 4 describes an example of a practical implementation of workflowbased sequencing technology in a postgraduate course. Details of the action-learning research method are described in Section 5. Finally Section 6 offers a summary of the important pedagogical issues related to any implementation of sequencing technologies irrespectively of their type (SCORM- or workflow-based systems). 2. An overview of sequencing technologies This section will give a brief overview of current sequencing technologies including their main limitations. 2.1. SCORM-based systems Similarly to the other emerging educational technology standards such as LOM (Learning Object Metadata) standard (LOM, 2002), SCORM also aims to enable interoperability of reusable, sharable objects (instructional components). However, it goes one step further towards process support. It enables sequencing of a learning content in a form of the socalled ‘‘activity tree’’ to suit the needs of a particular learner. SCORM supports the notion of learning content composed from relatively small, reusable content objects aggregated together to form units of instruction such as courses, modules, etc. Thus, a LMS based on the SCORM standard, offers a set of functionalities designed to deliver, track, report and manage learning content as the learner moves through the assigned activity tree. This dynamic presentation of the learning content is enabled by the ‘‘Sequencing and Navigation Model’’ (see ADL, 2004b). Most importantly, the instructional designer can specify sequencing rules and navigation behaviour independently from the content objects. 2.2. Workflow-based systems In this paper, we distinguish between two different categories of workflow-based systems: workflow technology systems and learning design systems. 2.2.1. Workflow technology systems This category includes various applications of workflow management systems in the education domain. Workflow management systems are process-technologies designed to specify, execute, manage, monitor and streamline processes by allocating the right task to the right person at the right point of time along with the resources needed to perform the assigned task. Workflow technology enables coordination of different tasks as well as the integration of tools and technologies used to support individual tasks. Because of these features, workflow technology is considered to be the leading coordination and integration technology (WFMS, 1999). Workflow models are designed by workflow analysts and stored in the workflow repository during design (or build) time. Then, during run-time, a process instance is created and
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
2745
enacted (executed) based on the stored, pre-defined model. The actual sequencing is achieved by the workflow engine that uses a pre-defined model to deliver individual tasks to different users on the basis of different events (e.g. task completion event). For example, when one task is completed for a particular instance, workflow engine will ‘‘read’’ the corresponding workflow model and activate the subsequent tasks if their activation conditions are satisfied. Workflow management systems are, in essence, business technology. However, they have been also adopted as education technology designed to support teaching and learning processes, especially their coordination aspect. There are many notable examples of workflow technology systems in education. For example, Van der Veen, Jones, and Collis (1998) developed a workflow-based solution to support management of student projects. In another example by Marjanovic and Orlowska (2000), the tool called Flex-eL uses a predefined process model to guide students through a set of learning modules. Holden, Kay, Poon, and Yacef (2005) use workflow engine to enable personalised delivery of content to students based on their needs. Peter and Vantroys (2005) have implemented a flexible workflow engine to support learning design process. However, when it comes to supporting learning/teaching process, workflow management systems are still limited. First of all, this technology requires a complete process model to be pre-defined and stored in the workflow repository. Furthermore, this technology offers very limited flexibility. On the other hand, learning designs need to be flexible to allow possible ad-hoc changes during teaching/learning activity. In other words, at any point of time, teacher should have full control to easily change learning activities in any way that is required to suit the needs of a particular group of learners. Workflow technology is yet to provide this level of flexibility. Compared to SCORM’s Sequencing and Navigation model (ADL, 2004b), workflow management systems are more advanced in terms of their coordination functionality. They enable sequencing of both content and/or individual tasks for multiple participants in the same educational activity. Also, from the modeling perspective, workflow models are much richer models in terms of possible coordination and sequencing constructs and options. Therefore, they could be also used to express a relatively simple sequencing structure as in SCORM-based systems. 2.2.2. Learning design systems This category of sequencing systems includes systems that are based or inspired by very recent developments in the area of learning designs. Although these systems do not use workflow management systems, they use workflow-based concepts to describe sequencing and coordination of individual tasks in a teaching/learning process. This is why, in this paper, they are also classified as workflow-based systems. The emerging learning design theory, described by Koper and Tattersall (2005), aims to address the evident problem of current educational technologies designed to deliver content rather than enable and support innovative process-oriented learning/teaching activities. This theory is very significant for the future developments in the area of educational technologies as it promotes the top-down rather than bottom-up approach to design of learning experiences. More precisely, it starts from the proven pedagogical models rather than the available technology.
2746
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
A new term Learning Design (LD) is used to describe, at the conceptual level, a learning experience in the form of a teaching/learning process. More precisely, LD defines ‘‘under which conditions, which activities have to be performed by learners and teachers to enable learners to attain the desired learning objectives’’ (Koper & Tattersall, 2005). In order to design a teaching/learning process, teachers start from the learning objectives and then identify all activities (tasks) that different participants (students and their teacher(s)) need to do in order to achieve these objectives. Then for each task, they identify a set of learning resources as well as possible educational technology tools that could be used to support it. Note that some tasks may not be supported by technology at all (e.g. face-to-face discussion). Thus, LD can be used to describe teaching/learning processes in both e-learning and in the mixed (blended) mode of learning. Adoption of the principles of the learning design theory ensures that all educational activities are guided by pedagogy, rather than the available educational resources or technology. An example of LD is a problem-solving process that is widely used in many disciplines. In fact, it forms a foundation of problem-based learning. Although, the actual implementation of problem-based learning can vary, in general, this LD incorporates the following activities: identification of possible alternatives, analysis of the identified alternatives and selection of a possible solution (that can be followed by the plan of intended actions). Shortly after its introduction, this theory was followed by an emerging educational technology standard called IMS Learning Design Specification (IMS-LD, 2003). This standard promotes digital representation of learning designs that are executable by computers. They are usually referred to as IMS-LD. The current version of IMS-LD consists of three components: (i) an information model, (ii) a best-practice and implementation guide and (iii) an XML binding. For more details see (LD-IMS, 2003). At this point, it is important to observe that one of the main objectives for development of the IMS-LD standard is to make digital representations consistent and consequently, reusable by different software packages. Even though the theory and practice of learning designs is relatively new, there are already notable developments of various systems that either directly implement learning designs or follow the underlying principles. Examples include Coppercore, EduBox, Eduplone, LAMS, Lobster, Reload Software. Table 1 offers a partial summary of the main features of these systems. For more detailed comparison see Britain (2004). In addition to authoring and runtime environments, the emerging directions of research and developments in this area include applications of artificial intelligence. For example, Sicilia, Sanchez-Alonso, and Garcia-Barriocanal (2006) use AI techniques to support the process of design of IMS-LD. Furthermore, Ullrich (2005) proposes a formal model of hierarchical task network planner to describe course generation out of learning objects residing in one or several repositories. Although, this approach does not deal directly with IMS-LD, the assembled course structure corresponds to the organisation element of the IMS Content Package (IMS-LD, 2003). Another interesting research direction include knowledge management related to learning designs, in particular sharing and reuse of learning designs among teachers. (see Marjanovic, 2005). Having in mind the increased number of learning design applications as well as the associated open research problems, new research directions are likely to appear in the near future.
Coppercore
EduBox
Eduplone
LAMS
Lobster
Reload Software
Description/ Purpose/ Scope
A runtime engine to allow developers to incorporate IMSLD into their applications
EML authoring and run-time environment IMSLD version currently in development
A Zope/Plone based System that implements basic support for learning activity sequences using IMS-LD
Both an authoring and a runtime environment for learning activity sequences consisting of LAMS learning activity tools
An Authoring environment for sequences of learning objects
A learning design editor, implementing IMS-LD and a player are currently in development
Who is it for
Application Developers
Teachers and Learners
Teachers
Teachers and Learners
Teachers
The editor is for teachers with knowledge of IMSLD
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
Table 1 A partial summary of the main features of the major LD systems adapted from Britain (2004)
2747
2748
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
3. Sequencing technology and flexible learning Increased diversity in our classes creates the need for a more flexible learning experience. For example, a recent study from US illustrates that ‘‘only 16% of the student population may be described as ‘‘traditional’’ – that is, ages 18–22, attending college full-time and living on campus. Many now attend college part-time. More than 70% work, and 41% are over age 25’’ (Shapiro & Levine, 2003, p. 6). This is very similar to the current situation in Australia as confirmed by McInnis (2001). Consequently, many students are choosing flexible learning courses because they are unable to fit in with a conventional study regime with its rigid scheduling and inflexible business hours. Furthermore, the word outside the university is increasingly using flexible learning in the form of ‘‘just-in-time’’ training materials and self-managed organisational learning. The ability to manage his/her own learning has been widely recognised as an important work-related skill for any employee. There are many possible implementations of flexible learning and even more definitions that one can find in the literature. This paper adopts the following definition: ‘‘. . .Flexible teaching and learning is that mixture of educational philosophy, pedagogical strategies, delivery modalities and administrative structures which allows maximum choice for differences in student learning needs, styles and circumstances’’ (Lunding, 1997, p. 3). According to this definition, flexible learning is not necessarily web-based learning and does not necessarily involve information technology to support or deliver it. It is about engaging students into more active and productive learning experience designed to better suit their individual learning styles. In many cases that one could find in the literature (see for example Maguire & Matejka, 2000), and observe in practice, ‘‘flexible’’ usually means any time/any place access to the learning resources, usually on the web. Obviously, having flexible access to study material is only one possible aspect or dimension of flexibility. According to Brown (1999), flexible learning can be analysed along the following dimensions: access to the learning experience, teaching and learning methods, course structure, interaction, course content, use of www, delivery medium, assessment and delivery mix. Table 2 summarises the most important characteristics of these 11 dimensions of flexibility that could be used individually or combined in a number of ways to create innovative learning scenarios for both classroom setting as well as on-line learning. It is possible to observe that the above table does not fully explore the time dimension (except as the access time to a particular learning experience). However, in order to make flexible learning more flexible in terms of time, it is necessary to use the time in new and more productive ways. As pointed out in the Report of the US National Commission on Time and Learning (1994), time should be a factor supporting the learning process rather than a boundary marking its limits. The idea that time should be used as a flexible resource, opens profoundly different opportunities for the new approach to learning and major educational change. Furthermore, curriculum is almost always organised in a strict linear mode (i.e. ‘‘production line’’), where the sequence of learning topics (and the associated resources) is predetermined by the lecturer at the beginning of the semester. There is no provision for the alternative learning paths that would suit the needs and learning styles of individual students. This is another dimension of flexibility not covered by the previous table.
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
2749
Table 2 Dimensions of flexible learning (Brown, 1999) Dimension
Flexibility Less ! More
Access to Learning Experience Teaching and Learning Methods Course structure Interaction Course content Delivery Medium Use of www Assessment
Fixed time/place Traditional Lecture and Tutorial All modules are compulsory Passive listening Teacher decides Face to face Transmit content Teacher directed
Delivery mix
Use of 1 medium
Some self choice Problem-based learning
Some restrictions Self-directed
Core + Options
Alternative choices
Traditional tutorials Negotiated projects Online print Bulletin boards Mixture of assessment methods More than one medium
High Interaction Learning contracts Online Interactive activities Negotiated (Personalised) Resource based delivery
For the first time, sequencing technology (both SCORM- and workflow-based systems) makes practical implementation of these two additional dimensions of flexibility possible and even relatively simple. First of all it is possible to introduce flexible start and finish times for each task (unit of content) based on user’s needs and progress. Technically, this is made possible by the coordination mechanism used by both technologies. Furthermore, it is possible to increase flexibility of the curriculum. This means the introduction of flexible learning pathways so students can progress through the content in a variety of ways based on their needs and preferences. Table 3 illustrates these two additional dimensions of flexibility, made possible by sequencing technology. It is important to observe that the flexible start and finish times for each task enable the time dimension to be used in a completely different way than even before. Thus, it enables flexible deadlines for various assessment items and consequently flexible duration of a semester/course or even, the overall study experience. However, introduction of any new dimension of flexibility creates additional requirements for student management and coordination. Without sequencing technology this task would be very tedious and error prone, thus impossible to implement in practice (especially with a large number of students). In addition to coordination, sequencing technology also enables monitoring of student progress and delivery of the next unit or task at the right point of time. Table 3 Additional dimensions of flexibility made possible by sequencing technology Dimension
Flexibility
Time dimension
Teacher defines deadlines for start and end of each task/content module Linear ‘‘productionline’’ curriculum
Less ! More
Learning pathways
Some deadlines
Students decide on their own deadlines
Most students follow the same learning pathway
Students choose their own learning paths
2750
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
At the same time, introduction of any new dimension of flexibility has a profound effect on the overall student’s learning experience and possible consequences for the staff’s workload. This is why it is crucial to better understand pedagogical issues related to increased flexibility made possible by sequencing technology (both SCORM- and workflow-based systems). The following section describes, in more details, an example of practical implementation of flexible time and learning pathways with a group of students. 4. An example: Using sequencing technology to enable more flexibility This section describes a practical implementation of workflow-based sequencing technology in a postgraduate course at author’s previous university. This particular course was an introductory Information System (IS) course, specially designed for postgraduate students with a prior degree in other (non-IS) disciplines. This course was offered during the first semester of their study and was a pre-requisite for most other courses within the program. A typical size of this class was about 60 students. Most students doing this course were already in full time employment and had to fit their studies around their work. Previous intakes of students doing this course, pointed out difficulties in meeting the strict deadlines for assignments and exams due to their work and family commitments. Consequently, around 10–15% of each new student intake would cancel their enrolment by week 4 of the semester. In order to enhance flexibility of student learning experience in this course, a proprietary workflow technology system was introduced to enable implementation of flexible deadlines and flexible learning pathways. This course was accompanied by an action-learning project, undertaken by the author to investigate the associated pedagogical issues – in particular, the effectiveness of time flexibility and flexible learning pathways on student learning. Instead of the pre-defined, linear sequence of modules, curriculum design was based on the concept of flexible learning pathways. Although all learning modules had to be completed, students could choose the order of individual modules. Fig. 1 illustrates the concept. For example, after completing Module 1, a student could study Module 2 and Module 3 in any order. Then, after completing both Quiz 1 and Quiz 2, students could proceed in several different ways. For example, they could study Modules 4 and 5 and attempt Quiz 3, study Modules 6 and 7 and attempt Quiz 4 or study Module 8 and take the practical assignment. Only after completing all required modules and quizzes, students could proceed with the final exam. The flexible time dimension was implemented through flexible deadlines. This means that students were able to choose their own deadlines for each quiz, the assignment as well Module 2
Quiz 1
Module 4
Module 5
Quiz 3
Module 6
Module 7
Quiz 4
Module 8
Practical Assignment
Module 1 Module 3
Quiz 2
Fig. 1. Flexible learning pathways.
Final Exam
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
2751
as the final exam. The only hard deadline was the completion date for the whole course, to ensure that students could enroll in subsequent courses. This means that students could choose the preferred date and book a quiz, at least one week in advance. Quizzes were scheduled two times per week at different times to give more flexibility to students. All quizzes were fully supervised (face-to-face). A special care was taken to ensure that all quizzes had the same level of difficulty. A practical assignment was designed to help students combine theory and practical IT skills. Students could choose when to take it. However, once taken, it had to be completed within 4 weeks. After completing all other assessment parts, students could choose when to do the final exam. Face-to-face consultations were offered every working day of the week (for 2 h) by different staff members and the weekly face-to-face group learning sessions were organised once per week in place of the traditional lecture. During weekly learning sessions students were involved in various interactive learning activities such as problem-solving exercises, design exercises etc. These sessions were not made compulsory. However, if students elected to come they were expected to come prepared. Finally, students also had fully supervised practical, self-paced learning sessions in the computer labs and again they could choose if, and when they wanted to come. In order to help students plan their learning pathways and deadlines students were given the recommended study time (duration) for each module. They also had the recommended deadlines for individual assessment components (quizzes, the practical assignment and the final exam). Students could use them to plan their study in terms of duration. In other words, they could use the recommended deadlines as guidelines, if they wanted to accelerate their study or even take the longest possible time but still complete the course on time. The workflow application enabled sequencing of content (modules) based on student’s progress. As all quizzes were done face-to-face, technology enabled students to select and book the preferred date for a quiz (if they completed the required content modules). Teachers (lecturer and tutors) could use this technology to monitor student progress through the content (at the level of individual modules) and also to find out how many students are coming to do which quiz and when. They could also generate various reports about students’ progress and their marks. Although the actual details of the workflow technology system are out of the scope of this paper, it is possible to observe that, in essence, it supported two features: sequencing of content and management of assessment (i.e. booking of quizzes and assignments). Consequently, it would be possible to use the other types sequencing technologies (both SCORM- and workflow-based) to implement the same or very similar solution. This is why the main pedagogical issues observed during this project, are also applicable to educational applications of other types of sequencing technologies. 5. Action learning project The section describes the action-learning project in more details, including the research method and the main outcomes. 5.1. Research method The main objective of the action-learning project was to investigate the main pedagogical issues related to the implementation of this particular sequencing technology. In par-
2752
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
ticular, we were interested to investigate and understand whether the introduced flexibility actually improved students’ learning experience. The study used the action learning research method because it enabled the researcher not only to observe, but also to actively participate in the study implementation through data collection, analysis, recommendation and implementation of changes as well as critical reflection. Fig. 2 depicts the action learning method as used in this project. 5.2. The initial assumptions Based on the feedback obtained from the previous student intakes, the following assumptions were made about the students at the beginning of this project: Postgraduate students, having at least one prior degree, are likely to have very good time management skills. Some, if not majority of the students, would be motivated to use flexible deadline and complete this course faster, thus leaving more time for other courses. Postgraduate students would prefer flexibility to study off-campus and set their own deadlines rather than come to university at predefined times. This was confirmed by the survey of previous intakes, as well as the initial survey of this group of students. Preparation for the flexible mode of learning
Introduction of flexible mode of learning to students
Implementation of recommended changes
Reflection and recommendation for ongoing changes (improvements) during semester
Data collection
Data Analysis
Reflection on the overall teaching/learning experience
Identification of the main pedagogical issues Fig. 2. The action learning research method used in this project.
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
2753
Postgraduate students in this class should be experienced learners, having at least one previous degree, and would be able to effectively apply different learning strategies. Students from different professional/educational backgrounds may choose different sequences of modules. For example, students from technical backgrounds may prefer to start from more technical (mathematically-oriented) modules. Students would continue to form and work in study groups and support each other. 5.3. Main objectives of the action-learning project The main objectives of the action learning research were to better understand the following pedagogical aspects of flexible learning: The overall students’ learning experience. The importance of various time constraints in the learning process and the required time management skills. The importance and structure of the face-to-face component in flexible learning and how it could be used more effectively. Quantity and nature of the workload of the teaching staff involved in this type of flexible learning. 5.4. Data collection The following data collection techniques were used: Observation of teaching staff recorded during weekly review sessions. Informal interviews with students – a number of students who attended weekly learning sessions, practical sessions and consultation sessions were interviewed and their feedback recorded. Focus groups Analysis of students’ progress throughout the semester (including time analysis). This analysis was based on the recorded dates when students completed various assessment items. Survey of students – all students were surveyed at the end of the semester. Analysis of the final results – obtained at the end of the semester. Data was collected throughout semester and teaching staff had meetings every week to analyse the data, reflect and propose new improvements and actions how to put them in place. The lecturer in charge also kept a reflective journal. 5.5. Major outcomes After the course was completed and detailed analysis of various aspects of student learning experience performed, the following major outcomes were observed: All students extended the duration of the course by taking the longest possible option available.
2754
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
All students followed the same learning pathway. At the end of the course, their results were comparable to all previous intakes. The fact that they spent more time doing this course than any other intake did not result in better results. Additional flexibility, although enabled and supported by sequencing technology, created an enormous workload for all teaching staff members. They had to prepare and manage many different versions of the same assessment items to cater for different groups of students who could take quizzes and assignments at different point in time. The accumulated experience was carefully analysed to identify the main pedagogical issues and possible improvements of this particular sequencing technology. More importantly, during the final reflection phase of this action-learning project, these issues were then further generalised, as described in the next section. 6. The main pedagogical issues This section analyses the main pedagogical issues related to the implementation of sequencing technologies irrespectively of their type (workflow- or SCORM-based systems) and the application domain (i.e. teaching discipline). These generic issues can be described as follows: 6.1. Content-based versus process-based educational experience Current applications of sequencing technologies often focus too much on delivery of educational content. However, the content-based view of educational pedagogy is very limited. It can be summarised as follows: ‘‘In order to learn, a single learner has to work through a sequence of learning objects. The underlying assumption is that learning is process of consuming content. Teaching is envisioned as the art of selecting and offering content in a structured, sequenced way, and of tracking the learner’s progress and assessing the acquired knowledge. Current educational practice is more complex and advanced than this.’’ (Koper & Olivier, 2004, p. 97). This was also confirmed in this project. Contentbased modules were combined with quizzes designed to test content. However, from the pedagogical point of view, this is not sufficient. Learning/teaching activities should also include collaborative activities where the main emphasis is not on consumption of educational content. Examples include debates, problem-solving and role-play activities. Therefore, technology should be used to support these collaborative activities in a more innovative way rather than just ‘‘feed’’ the content to students. 6.2. Change of traditional roles Irrespectively of the type of technology used to enable or support it, flexible learning requires the change of traditional roles of teachers and students including their responsibilities and expectations. Most importantly, in the case of flexible deadlines, it requires students to take greater responsibility for their learning than in traditional transmissive mode of teaching. ‘‘Perhaps the most important outcome of higher education should be the development of the ability to manage one’s own learning. The need for this is made more urgent by greater mix of ability of students in a mass higher education system. Flexible learning pathways develop this ability. Being taught often inhibits such development’’
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
2755
(Race, 1999). This project confirmed that, it is crucial to carefully introduce students to this mode of learning and clarify their roles and responsibilities as well as their expectations. 6.3. The importance of group experience Sequencing technology enables personalised solutions that often focus on individual progress through a process. At the same time, it is very important to enable students to progress as a cohort. For example, in this particular project students were able to choose and follow their own flexible learning pathways. However, very soon after completing the first set of modules, they ended up following the same pathway. In many instances, they were even waiting for each other to start new modules. In this way, they could compare their learning experiences, support each other, but also, find out how their own results compared to those of the other students in the same cohort. Some students indicated that this comparison to the other students was a very important motivational factor for their learning. 6.4. Finding a good balance between flexibility and control It is very simple to use sequencing technologies to set up various time constraints i.e. to impose strict deadlines or to offer the ultimate time flexibility by allowing students to set up their own deadlines. In this particular project, students were only given the recommended deadlines. However, due to the other time pressures, the whole group ended up doing quizzes at the end of the course. Not a single student took the offered option to complete this course earlier. Students’ interviews and surveys confirmed that the actual time they spent studying each module and doing the whole course was much shorter than the recommended time. Towards the end of the course, they were rushing to complete one module after another without allowing enough time to reflect upon their learning experience (i.e. allow the content to ‘‘sink in’’ – as one student pointed out). Furthermore, some students had very long breaks between completing the initial modules (1, 2 and 3) and starting the subsequent modules. These ‘‘study breaks’’ coincide with the major assessment items and the mid-session exams in other courses they were doing in parallel. To be effective, learning has to be a continuous and progressive endeavor. Obviously, long breaks, especially when studying topics in a new area, will have a deteriorating effect on student learning, as they are very likely to forget what they had done in previous modules. In essence, students were using increased flexibility in this course to study other courses with strict deadlines. A similar point was also confirmed by McInnis (2001) in their Australia-wide research on the study experience of first year students. They found out that when given the additional flexibility in a course or program, most students (if not all) used this additional time for work or other activities. One could argue that postgraduate students should be different from first years, however when it comes to flexible deadlines this may not be the case, as confirmed in this project. At the same time, one has to stay realistic. Even in the traditional study mode, a majority of (if not all) students complete their assignments just before they are due. Many bad time management habits may remain undiscovered by students simply because of the strict deadlines in the traditional mode of study. However, the same habits would cause much more problems in the flexible mode of learning.
2756
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
Sequencing technology can be easily ‘‘configured’’ to create very different flexible learning experiences (especially related to temporal constraints). For example, inflexible deadlines managed by technology can create extra pressure so students will feel as if they are on ‘‘some kind of production line’’. On the other hand, if time is too flexible, this will not necessarily create more time for learning. So it is very important for teachers to organise individual modules in a way that will allow enough time for learning, prevent long breaks between modules but also allow enough time for reflection on what has been learnt. Therefore, when using sequencing technologies teachers need to find a fine balance between flexibility and control. However, as this project confirmed, this is not an easy task. 6.5. Students’ ability to handle the increased flexibility Another important pedagogical issue, closely related to the previous one, is students’ ability to handle the increased flexibility enabled by sequencing technology. In this particular project a large number of students confirmed that it was very hard for them to handle the increased flexibility, especially flexible deadlines. More than 95% of all students indicated that they would have preferred the strict deadlines because the recommended deadlines were not strict enough to motivate them to learn. However, the reasons for this apparent inability to handle deadlines are likely to be much more complex than what may look like on the surface (i.e. extrinsically motivated students who did not take full responsibility for their own progressive learning). In his work on situational leadership theory and its application to student learning, Grow (1996) explains how students develop as learners from dependent via interested, involved to self-directed learners. Dependent students are highly reliant on teacher’s directions and instructions and need structure. They cannot handle choices very well. On the other hand, self-directed learners are able to set their own learning goals and standards. They have the required skills in time management, project management, goal setting and self-evaluation. As Grow pointed out, dependent students will ‘‘not be able to make use of the ‘‘freedom to learn’’ because they lack skills such as goal-setting, self-evaluation, project management, critical thinking, group participation, learning strategies, information resources, and self-esteem, which makes self-directed learning possible’’. The theory of situational leadership confirms that when the followers (in this case dependent learners) do not receive the required level of instructions and guidance from their leader, they are unable to perform the required task. Hersey (1983) described the resulting experience of the followers: ‘‘lacking the ability to perform the task, they tend to feel that leader has little interest in what they are doing and does not care about their progress or them personally’’. This is exactly how some students felt about their learning experience in this course. On the other hand, recall that in this particular case, all students had completed at least an undergraduate course. One could argue that majority of post-graduate students are or should be self-directed learners. This may not always be the case. Grow (1996) argues that even self-directed learners can become (or ‘‘switch back’’ to) temporary dependent if they are learning something from a completely new area. This is very applicable to this particular group of students. They were probably self-directed or at least independent learners in their own discipline. But in this particular case, they were doing a course from a completely new discipline and as dependent learners were unable to handle the increased flexibility. This is probably one more reason why the face-to-face discussion sessions were, as they indicated, the most valuable part of this course.
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
2757
Therefore, when using sequencing technologies to provide additional flexibility it is necessary to carefully consider the current level of student development as learners but also to take into account if they are new to the given discipline. Only then one can decide how much flexibility and, most importantly, what kind of flexibility they can handle. 6.6. Teacher-driven versus technology-driven educational experience Another very important issue is the problem of control that teachers have when using sequencing technologies. As all educators know, the teaching/learning process is highly dynamic and relies on teachers’ experience and expertise to determine what is best for a particular group of students at the particular point of time. So sometimes students need to progress through some tasks individually and then through some other tasks as a group (e.g. collaborative learning tasks). Students may need to go back and revise and re-do some activities from the previous modules. Teachers may need to incorporate new interesting modules if they become available. In essence, the experienced teachers concentrate on students’ learning and very often on-the-fly modifications of the intended activities and lesson plans need to be implemented to better suit a particular group. The whole experience has to be teacher driven rather than technology driven. Unfortunately, currently available sequencing technologies (both SCORM- and workflow-based) are still quite inflexible to support on-the-fly modifications as they are normally done in the face-to-face activities. 6.7. The importance of face-to-face learning experience This project also re-confirmed the importance of face-to-face activities, especially in the blended mode of learning. Applications of sequencing technologies enable teachers to supplement face-to-face activities with online activities and extend students’ learning experiences beyond the time they spend in the classrooms. In this way, teachers could ensure continuity of the learning process. But at the same time, this gave the teacher a real opportunity to make face-to-face activities even more effective and more valuable to their students. Thus, when designing the face-to-face activities to complement the application of sequencing technology, less emphasis needs to be placed on the content and more on critical thinking, innovation, sharing of teacher’s experiential knowledge in the given discipline. Then, these activities become something that cannot be easily replaced by technology. At the same time, these face-to-face sessions do not only help students to learn about a particular topic, but also motivate, inspire and reinforce the ‘‘team spirit’’. This was very clear in this particular course, were face-to-face group learning sessions were unanimously voted as the best part of student learning experience. Finally the main argument is that any application of process-oriented, sequencing technology has to be guided by the intended learning objectives and pedagogy rather than technology. This particular technology makes it relatively simple for teachers to increase the flexibility just by changing several parameters (e.g. deadlines). However, this type of change that can be easily handled by technology has a profound effect on student learning experience. This is why it is necessary to fully understand capabilities as well as possible limitations of these technologies as they can easily create technology-centered learning experience where the main emphasis is on (more or less flexible) progression through the sequence of content rather than learning.
2758
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
7. Conclusion Sequencing technologies (such as SCORM- and workflow-based systems) support sequencing of units of educational content and tasks (in the case of workflows). At the same time, these technologies can be used to enable new types of learning experiences never before possible. This paper illustrates how workflow-based sequencing technology can be used to make possible the implementation of two additional dimension of flexibility: flexible time (implemented via flexible deadlines) and flexible learning pathways. However, as this paper illustrates through an example of an action-learning project in this domain, the introduction of any new dimension of flexibility has a profound effect on students’ learning experience. This is why the associated pedagogical issues have to be identified and fully understood before one can start to experience the benefits made possible by this technology. Finally it is important to point out again that the content-oriented pedagogy is quite limited. The very recent theory of Learning designs (Koper & Tattersall, 2005) confirms that learning/teaching activities are very complex processes that include much more than delivery of content. Consequently, our current and future work in this area include further investigation of the pedagogical issues related to the implementation of various processoriented educational technologies, especially learning design systems. References ADL (2004a). SCORM – 2nd Ed. overview. Available from http://www.adlnet.org. ADL (2004b). SCORM – sequencing and navigation (SN), Version 1.3.1. Available from http://www.adlnet.org. Britain, S. (2004). A review of learning design: concept, specifications and tools. A report for the JISC E-Learning Pedagogy Programme, May 2004. Available from http://dspace.learningnetworks.org/handle/1820/267. Brown, A. (1999). ‘‘Dimensions of flexible learning’’. Learning Research and Development Unit (LRDU). University of Queensland, Australia. Grow, G. O. (1996). Teaching learners to be self-directed. Adult Education Quarterly, 41(3), 125–149. Available fromhttp://www.longleaf.net/ggrow. Hersey, P. (1983). Leader effectiveness and adaptability description, LEAD matrix: Directions for matrix scoring and analysis. San Diego: University Associates, Inc. Holden, S., Kay, J., Poon, J., & Yacef, K. (2005). Workflow-based personalised document delivery. International Journal of E-Learning, 4(2), 131–148. IMS-LD, (2003). IMS Learning Design. Information Model, Best Practices and Implementation Guide, XML Binding, Schemas. Version 1.0 Final Specification, IMS Global Learning Consortium, Inc. Available from http://www.imsglobal.org/content/learningdesign/. Koper, R., & Olivier, B. (2004). Representing the learning design of units of learning. Educational Technology and Society, 7(3), 97–111. Koper, R., & Tattersall (Eds.). (2005). Learning design: A handbook on modeling and delivering networked education and training. Berlin: Springer. LOM (2002). Standard for Learning Object Metadata. Learning Technologies Standards Committee of the IEEE 148.41.21. Lunding, R., (1997). Flexible delivery: an international perspective. UQ Teaching and Educational Development Institute Conference, Initiatives in Flexible Delivery, September, 1997. Available from http://www.tedi.uq.edu.au/tei/flex_delivery/Lundin.html. Maguire, M., Matejka, D. (2000). Online delivery: Making the rough road smooth. In Flexible learning for a flexible society, proceedings of ASET-HERDSA 2000 conference, Toowoomba, Qld, 2–5 July. ASET and HERDSA. Marjanovic, O. (2005). Towards Web-based handbook of process oriented learning designs. Educational Technologies and Society Journal, IEEE Learning Technology Task Force, 8(2), 66–82.
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
2759
Marjanovic, O., Orlowska, M. (2000). Making the flexible learning more flexible. In Proceedings of IEEE IWALT 2000, International Workshop on advanced Learning technologies, New Zealand. McInnis, C. (2001). Signs of disengagement the changing undergraduate experience in Australian universities. Centre for the Study of Higher Education. Peter, Y., & Vantroys, T. (2005). Platform support for pedagogical scenarios. Educational Technology and Society, 8(3), 122–137. Race, P. (1999). Practical pointers to flexible learning. Available from http://www.lgu.ac.uk/delibrations/ flex_learning/race_content.html. Shapiro, N. S., & Levine, J. H. (2003). Creating learning communities – a practical guide to winning support, organising for change and implementing programs. USA: Jossey-Bass Inc.. Sicilia, M.-A., Sanchez-Alonso, S., & Garcia-Barriocanal, E. (2006). On supporting the process of learning design through planners. In Proceedings of the virtual campus workshop, Barcelona, in press. Available from http:// www.cc.uah.es/ie/elsem/papers/Sicilia_vc06.pdf. Ullrich, C. (2005). Course generation based on HTN planning. In Proceedings of the 13th annual workshop of the SIG adaptibility and user modeling in interactive systems, pp. 74–79. Available from: http://www.activemath.org/publications/Ullrich-CourseGenerationHTN-ABIS-2005.pdf. US National Commission on Time and Learning (1994). Report: Prisoners of Time. Van der Veen, J., Jones, V., & Collis, B. (1998). Workflow applied to projects in higher education. In Proceedings of the international conference on cooperative information systems, COOPIS’98. Welsch, E. (2004). SCORM: clarity or calamity. Online Learning Magazine, 29(Nov). WFMS (1999). Workflow management coalition: workflow process model. Available from http://www.wfmc.org.
Using process-oriented, sequencing educational technologies: Some important pedagogical issues Olivera Marjanovic
*
School of Business, The University of Sydney, Sydney, NSW 2006, Australia Available online 6 October 2006
Abstract In recent times, sequencing technologies are becoming increasingly used, both by the university and the industry sectors. In essence, these complex systems support sequencing of, and navigation through units of content. They come in two different types: SCORM-based and workflow-based systems. Current research efforts related to sequencing technologies concentrate mostly on technical issues, while the associated pedagogical issues remain unexplored. The main objective of this paper is to describe the most important pedagogical issues that need to be taken into account when implementing any type of sequencing educational technologies (either SCORM- or workflow-based). These issues were identified during an action-learning project related to the practical implementation of a workflow-based educational system. During the reflection phase of this project, these issues were then genarised, so they could be applied to any type of sequencing technology and in any application domain (teaching discipline). The paper describes how sequencing technology can be used to enable a more flexible learning experience (especially in terms of time and flexible learning pathways) and then proceeds with the detailed analysis of the associated pedagogical issues. Ó 2006 Elsevier Ltd. All rights reserved. Keywords: Pedagogical issues; Learning Designs; Workflow technology
1. Introduction These days, Learning Management Systems (LMS) are becoming increasingly popular among many universities from all around the world. Examples include technologies such Lotus Learning Space and recently merged WebCT and BlackBoard. More than a decade *
Tel.: +61 2 9351 8477; fax: +61 2 9662 6638. E-mail address: [email protected].
0747-5632/$ - see front matter Ó 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.chb.2006.08.005
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
2743
of development and deployment of these systems have resulted in the substantial accumulated knowledge and experience related to both technical and pedagogical issues. Consequently, educators have gained a much better understanding of the opportunities created by these technologies as well as their major limitations. It is now clear that one of the major limitations of the current LMS is certainly their task-orientation. In other words, they are oriented towards supporting individual tasks while their process-support is still very limited. At the same time, the relevant educational literature confirms that going through a carefully designed teaching/learning process creates much better teaching outcomes than going through isolated learning tasks. The need to support teaching/learning processes, rather than simple tasks, in recent times has resulted in a new type of process-oriented, educational technology. These complex systems also called sequencing technologies because they support sequencing of, and navigation through educational content and/or activities. Currently there are two different types of sequencing technologies: SCORM-based and workflow-based systems. SCORM is one of the latest educational standards (ADL, 2004a). Among other features, it enables sequencing of educational content that is organised in a form of an activity tree. SCORM’s sequencing and navigation mechanism (ADL, 2004b) is then used to deliver ‘‘chunks’’ (units) of content to a single learner based on their progress. The other type of sequencing technology includes the so-called workflow-based systems. These systems aim to support teaching and learning process that is represented as a sequence of individual tasks (activities) that involve multiple learners. In this paper, we distinguish between two different categories of workflow-based systems. The first category includes workflow technology systems. In essence, these are various applications of workflow-management systems (WFMS, 1999), in the education domain to support highly structured teaching/learning processes. The other emerging category of workflow-based systems includes systems that are based on the very recent learning design theory (Koper & Tattersall, 2005) and the associated learning design standard (IMS-LD, 2003). Although these systems do not incorporate workflow management systems, they use workflows to model learning designs, in particular sequencing and coordination of individual tasks in a learning process. IMS-LD standard focuses on formal representation of learning designs that are executable by computers. Current research efforts in the areas of sequencing technologies (both SCORM-based and workflow-based systems) concentrate more on technical issues (such as formal representations that are directly executable by computers) than the associated pedagogical issues. ‘‘. . .Unless all learning specification turn the focus from infrastructure to pedagogical soundness, they are in danger of becoming instructionally irrelevant. . .Despite the progress being made on the interoperability front, that doesn’t necessarily guarantee that what actually runs on SCORM systems will be worthwhile instructionally’’ (Welsch, 2004, p. 2). The main objective of this paper is to describe the main pedagogical issues that need to be taken into account when implementing sequencing educational technologies (either SCORM- or workflow-based). These specific issues were identified during an action-learning project undertaken by the author. The main objective of this project was to observe pedagogical issues related to the implementation of a proprietary workflow-based educational system used to enable a more flexible learning experience in a postgraduate course. Although this particular learning experience was made possible by sequencing technology, the paper will not focus on technical details. Instead, it will describe the main generic pedagogical issues, applicable to any implementation of sequencing educational technologies.
2744
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
The paper is organized as follows. Section 2 offers an overview of the sequencing technologies including SCORM-based systems as well as both categories of workflow-based systems. Section 3 illustrates how sequencing technologies can be used to enable more flexible learning experience, especially in terms of time and provision of more flexible learning pathways. Section 4 describes an example of a practical implementation of workflowbased sequencing technology in a postgraduate course. Details of the action-learning research method are described in Section 5. Finally Section 6 offers a summary of the important pedagogical issues related to any implementation of sequencing technologies irrespectively of their type (SCORM- or workflow-based systems). 2. An overview of sequencing technologies This section will give a brief overview of current sequencing technologies including their main limitations. 2.1. SCORM-based systems Similarly to the other emerging educational technology standards such as LOM (Learning Object Metadata) standard (LOM, 2002), SCORM also aims to enable interoperability of reusable, sharable objects (instructional components). However, it goes one step further towards process support. It enables sequencing of a learning content in a form of the socalled ‘‘activity tree’’ to suit the needs of a particular learner. SCORM supports the notion of learning content composed from relatively small, reusable content objects aggregated together to form units of instruction such as courses, modules, etc. Thus, a LMS based on the SCORM standard, offers a set of functionalities designed to deliver, track, report and manage learning content as the learner moves through the assigned activity tree. This dynamic presentation of the learning content is enabled by the ‘‘Sequencing and Navigation Model’’ (see ADL, 2004b). Most importantly, the instructional designer can specify sequencing rules and navigation behaviour independently from the content objects. 2.2. Workflow-based systems In this paper, we distinguish between two different categories of workflow-based systems: workflow technology systems and learning design systems. 2.2.1. Workflow technology systems This category includes various applications of workflow management systems in the education domain. Workflow management systems are process-technologies designed to specify, execute, manage, monitor and streamline processes by allocating the right task to the right person at the right point of time along with the resources needed to perform the assigned task. Workflow technology enables coordination of different tasks as well as the integration of tools and technologies used to support individual tasks. Because of these features, workflow technology is considered to be the leading coordination and integration technology (WFMS, 1999). Workflow models are designed by workflow analysts and stored in the workflow repository during design (or build) time. Then, during run-time, a process instance is created and
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
2745
enacted (executed) based on the stored, pre-defined model. The actual sequencing is achieved by the workflow engine that uses a pre-defined model to deliver individual tasks to different users on the basis of different events (e.g. task completion event). For example, when one task is completed for a particular instance, workflow engine will ‘‘read’’ the corresponding workflow model and activate the subsequent tasks if their activation conditions are satisfied. Workflow management systems are, in essence, business technology. However, they have been also adopted as education technology designed to support teaching and learning processes, especially their coordination aspect. There are many notable examples of workflow technology systems in education. For example, Van der Veen, Jones, and Collis (1998) developed a workflow-based solution to support management of student projects. In another example by Marjanovic and Orlowska (2000), the tool called Flex-eL uses a predefined process model to guide students through a set of learning modules. Holden, Kay, Poon, and Yacef (2005) use workflow engine to enable personalised delivery of content to students based on their needs. Peter and Vantroys (2005) have implemented a flexible workflow engine to support learning design process. However, when it comes to supporting learning/teaching process, workflow management systems are still limited. First of all, this technology requires a complete process model to be pre-defined and stored in the workflow repository. Furthermore, this technology offers very limited flexibility. On the other hand, learning designs need to be flexible to allow possible ad-hoc changes during teaching/learning activity. In other words, at any point of time, teacher should have full control to easily change learning activities in any way that is required to suit the needs of a particular group of learners. Workflow technology is yet to provide this level of flexibility. Compared to SCORM’s Sequencing and Navigation model (ADL, 2004b), workflow management systems are more advanced in terms of their coordination functionality. They enable sequencing of both content and/or individual tasks for multiple participants in the same educational activity. Also, from the modeling perspective, workflow models are much richer models in terms of possible coordination and sequencing constructs and options. Therefore, they could be also used to express a relatively simple sequencing structure as in SCORM-based systems. 2.2.2. Learning design systems This category of sequencing systems includes systems that are based or inspired by very recent developments in the area of learning designs. Although these systems do not use workflow management systems, they use workflow-based concepts to describe sequencing and coordination of individual tasks in a teaching/learning process. This is why, in this paper, they are also classified as workflow-based systems. The emerging learning design theory, described by Koper and Tattersall (2005), aims to address the evident problem of current educational technologies designed to deliver content rather than enable and support innovative process-oriented learning/teaching activities. This theory is very significant for the future developments in the area of educational technologies as it promotes the top-down rather than bottom-up approach to design of learning experiences. More precisely, it starts from the proven pedagogical models rather than the available technology.
2746
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
A new term Learning Design (LD) is used to describe, at the conceptual level, a learning experience in the form of a teaching/learning process. More precisely, LD defines ‘‘under which conditions, which activities have to be performed by learners and teachers to enable learners to attain the desired learning objectives’’ (Koper & Tattersall, 2005). In order to design a teaching/learning process, teachers start from the learning objectives and then identify all activities (tasks) that different participants (students and their teacher(s)) need to do in order to achieve these objectives. Then for each task, they identify a set of learning resources as well as possible educational technology tools that could be used to support it. Note that some tasks may not be supported by technology at all (e.g. face-to-face discussion). Thus, LD can be used to describe teaching/learning processes in both e-learning and in the mixed (blended) mode of learning. Adoption of the principles of the learning design theory ensures that all educational activities are guided by pedagogy, rather than the available educational resources or technology. An example of LD is a problem-solving process that is widely used in many disciplines. In fact, it forms a foundation of problem-based learning. Although, the actual implementation of problem-based learning can vary, in general, this LD incorporates the following activities: identification of possible alternatives, analysis of the identified alternatives and selection of a possible solution (that can be followed by the plan of intended actions). Shortly after its introduction, this theory was followed by an emerging educational technology standard called IMS Learning Design Specification (IMS-LD, 2003). This standard promotes digital representation of learning designs that are executable by computers. They are usually referred to as IMS-LD. The current version of IMS-LD consists of three components: (i) an information model, (ii) a best-practice and implementation guide and (iii) an XML binding. For more details see (LD-IMS, 2003). At this point, it is important to observe that one of the main objectives for development of the IMS-LD standard is to make digital representations consistent and consequently, reusable by different software packages. Even though the theory and practice of learning designs is relatively new, there are already notable developments of various systems that either directly implement learning designs or follow the underlying principles. Examples include Coppercore, EduBox, Eduplone, LAMS, Lobster, Reload Software. Table 1 offers a partial summary of the main features of these systems. For more detailed comparison see Britain (2004). In addition to authoring and runtime environments, the emerging directions of research and developments in this area include applications of artificial intelligence. For example, Sicilia, Sanchez-Alonso, and Garcia-Barriocanal (2006) use AI techniques to support the process of design of IMS-LD. Furthermore, Ullrich (2005) proposes a formal model of hierarchical task network planner to describe course generation out of learning objects residing in one or several repositories. Although, this approach does not deal directly with IMS-LD, the assembled course structure corresponds to the organisation element of the IMS Content Package (IMS-LD, 2003). Another interesting research direction include knowledge management related to learning designs, in particular sharing and reuse of learning designs among teachers. (see Marjanovic, 2005). Having in mind the increased number of learning design applications as well as the associated open research problems, new research directions are likely to appear in the near future.
Coppercore
EduBox
Eduplone
LAMS
Lobster
Reload Software
Description/ Purpose/ Scope
A runtime engine to allow developers to incorporate IMSLD into their applications
EML authoring and run-time environment IMSLD version currently in development
A Zope/Plone based System that implements basic support for learning activity sequences using IMS-LD
Both an authoring and a runtime environment for learning activity sequences consisting of LAMS learning activity tools
An Authoring environment for sequences of learning objects
A learning design editor, implementing IMS-LD and a player are currently in development
Who is it for
Application Developers
Teachers and Learners
Teachers
Teachers and Learners
Teachers
The editor is for teachers with knowledge of IMSLD
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
Table 1 A partial summary of the main features of the major LD systems adapted from Britain (2004)
2747
2748
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
3. Sequencing technology and flexible learning Increased diversity in our classes creates the need for a more flexible learning experience. For example, a recent study from US illustrates that ‘‘only 16% of the student population may be described as ‘‘traditional’’ – that is, ages 18–22, attending college full-time and living on campus. Many now attend college part-time. More than 70% work, and 41% are over age 25’’ (Shapiro & Levine, 2003, p. 6). This is very similar to the current situation in Australia as confirmed by McInnis (2001). Consequently, many students are choosing flexible learning courses because they are unable to fit in with a conventional study regime with its rigid scheduling and inflexible business hours. Furthermore, the word outside the university is increasingly using flexible learning in the form of ‘‘just-in-time’’ training materials and self-managed organisational learning. The ability to manage his/her own learning has been widely recognised as an important work-related skill for any employee. There are many possible implementations of flexible learning and even more definitions that one can find in the literature. This paper adopts the following definition: ‘‘. . .Flexible teaching and learning is that mixture of educational philosophy, pedagogical strategies, delivery modalities and administrative structures which allows maximum choice for differences in student learning needs, styles and circumstances’’ (Lunding, 1997, p. 3). According to this definition, flexible learning is not necessarily web-based learning and does not necessarily involve information technology to support or deliver it. It is about engaging students into more active and productive learning experience designed to better suit their individual learning styles. In many cases that one could find in the literature (see for example Maguire & Matejka, 2000), and observe in practice, ‘‘flexible’’ usually means any time/any place access to the learning resources, usually on the web. Obviously, having flexible access to study material is only one possible aspect or dimension of flexibility. According to Brown (1999), flexible learning can be analysed along the following dimensions: access to the learning experience, teaching and learning methods, course structure, interaction, course content, use of www, delivery medium, assessment and delivery mix. Table 2 summarises the most important characteristics of these 11 dimensions of flexibility that could be used individually or combined in a number of ways to create innovative learning scenarios for both classroom setting as well as on-line learning. It is possible to observe that the above table does not fully explore the time dimension (except as the access time to a particular learning experience). However, in order to make flexible learning more flexible in terms of time, it is necessary to use the time in new and more productive ways. As pointed out in the Report of the US National Commission on Time and Learning (1994), time should be a factor supporting the learning process rather than a boundary marking its limits. The idea that time should be used as a flexible resource, opens profoundly different opportunities for the new approach to learning and major educational change. Furthermore, curriculum is almost always organised in a strict linear mode (i.e. ‘‘production line’’), where the sequence of learning topics (and the associated resources) is predetermined by the lecturer at the beginning of the semester. There is no provision for the alternative learning paths that would suit the needs and learning styles of individual students. This is another dimension of flexibility not covered by the previous table.
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
2749
Table 2 Dimensions of flexible learning (Brown, 1999) Dimension
Flexibility Less ! More
Access to Learning Experience Teaching and Learning Methods Course structure Interaction Course content Delivery Medium Use of www Assessment
Fixed time/place Traditional Lecture and Tutorial All modules are compulsory Passive listening Teacher decides Face to face Transmit content Teacher directed
Delivery mix
Use of 1 medium
Some self choice Problem-based learning
Some restrictions Self-directed
Core + Options
Alternative choices
Traditional tutorials Negotiated projects Online print Bulletin boards Mixture of assessment methods More than one medium
High Interaction Learning contracts Online Interactive activities Negotiated (Personalised) Resource based delivery
For the first time, sequencing technology (both SCORM- and workflow-based systems) makes practical implementation of these two additional dimensions of flexibility possible and even relatively simple. First of all it is possible to introduce flexible start and finish times for each task (unit of content) based on user’s needs and progress. Technically, this is made possible by the coordination mechanism used by both technologies. Furthermore, it is possible to increase flexibility of the curriculum. This means the introduction of flexible learning pathways so students can progress through the content in a variety of ways based on their needs and preferences. Table 3 illustrates these two additional dimensions of flexibility, made possible by sequencing technology. It is important to observe that the flexible start and finish times for each task enable the time dimension to be used in a completely different way than even before. Thus, it enables flexible deadlines for various assessment items and consequently flexible duration of a semester/course or even, the overall study experience. However, introduction of any new dimension of flexibility creates additional requirements for student management and coordination. Without sequencing technology this task would be very tedious and error prone, thus impossible to implement in practice (especially with a large number of students). In addition to coordination, sequencing technology also enables monitoring of student progress and delivery of the next unit or task at the right point of time. Table 3 Additional dimensions of flexibility made possible by sequencing technology Dimension
Flexibility
Time dimension
Teacher defines deadlines for start and end of each task/content module Linear ‘‘productionline’’ curriculum
Less ! More
Learning pathways
Some deadlines
Students decide on their own deadlines
Most students follow the same learning pathway
Students choose their own learning paths
2750
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
At the same time, introduction of any new dimension of flexibility has a profound effect on the overall student’s learning experience and possible consequences for the staff’s workload. This is why it is crucial to better understand pedagogical issues related to increased flexibility made possible by sequencing technology (both SCORM- and workflow-based systems). The following section describes, in more details, an example of practical implementation of flexible time and learning pathways with a group of students. 4. An example: Using sequencing technology to enable more flexibility This section describes a practical implementation of workflow-based sequencing technology in a postgraduate course at author’s previous university. This particular course was an introductory Information System (IS) course, specially designed for postgraduate students with a prior degree in other (non-IS) disciplines. This course was offered during the first semester of their study and was a pre-requisite for most other courses within the program. A typical size of this class was about 60 students. Most students doing this course were already in full time employment and had to fit their studies around their work. Previous intakes of students doing this course, pointed out difficulties in meeting the strict deadlines for assignments and exams due to their work and family commitments. Consequently, around 10–15% of each new student intake would cancel their enrolment by week 4 of the semester. In order to enhance flexibility of student learning experience in this course, a proprietary workflow technology system was introduced to enable implementation of flexible deadlines and flexible learning pathways. This course was accompanied by an action-learning project, undertaken by the author to investigate the associated pedagogical issues – in particular, the effectiveness of time flexibility and flexible learning pathways on student learning. Instead of the pre-defined, linear sequence of modules, curriculum design was based on the concept of flexible learning pathways. Although all learning modules had to be completed, students could choose the order of individual modules. Fig. 1 illustrates the concept. For example, after completing Module 1, a student could study Module 2 and Module 3 in any order. Then, after completing both Quiz 1 and Quiz 2, students could proceed in several different ways. For example, they could study Modules 4 and 5 and attempt Quiz 3, study Modules 6 and 7 and attempt Quiz 4 or study Module 8 and take the practical assignment. Only after completing all required modules and quizzes, students could proceed with the final exam. The flexible time dimension was implemented through flexible deadlines. This means that students were able to choose their own deadlines for each quiz, the assignment as well Module 2
Quiz 1
Module 4
Module 5
Quiz 3
Module 6
Module 7
Quiz 4
Module 8
Practical Assignment
Module 1 Module 3
Quiz 2
Fig. 1. Flexible learning pathways.
Final Exam
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
2751
as the final exam. The only hard deadline was the completion date for the whole course, to ensure that students could enroll in subsequent courses. This means that students could choose the preferred date and book a quiz, at least one week in advance. Quizzes were scheduled two times per week at different times to give more flexibility to students. All quizzes were fully supervised (face-to-face). A special care was taken to ensure that all quizzes had the same level of difficulty. A practical assignment was designed to help students combine theory and practical IT skills. Students could choose when to take it. However, once taken, it had to be completed within 4 weeks. After completing all other assessment parts, students could choose when to do the final exam. Face-to-face consultations were offered every working day of the week (for 2 h) by different staff members and the weekly face-to-face group learning sessions were organised once per week in place of the traditional lecture. During weekly learning sessions students were involved in various interactive learning activities such as problem-solving exercises, design exercises etc. These sessions were not made compulsory. However, if students elected to come they were expected to come prepared. Finally, students also had fully supervised practical, self-paced learning sessions in the computer labs and again they could choose if, and when they wanted to come. In order to help students plan their learning pathways and deadlines students were given the recommended study time (duration) for each module. They also had the recommended deadlines for individual assessment components (quizzes, the practical assignment and the final exam). Students could use them to plan their study in terms of duration. In other words, they could use the recommended deadlines as guidelines, if they wanted to accelerate their study or even take the longest possible time but still complete the course on time. The workflow application enabled sequencing of content (modules) based on student’s progress. As all quizzes were done face-to-face, technology enabled students to select and book the preferred date for a quiz (if they completed the required content modules). Teachers (lecturer and tutors) could use this technology to monitor student progress through the content (at the level of individual modules) and also to find out how many students are coming to do which quiz and when. They could also generate various reports about students’ progress and their marks. Although the actual details of the workflow technology system are out of the scope of this paper, it is possible to observe that, in essence, it supported two features: sequencing of content and management of assessment (i.e. booking of quizzes and assignments). Consequently, it would be possible to use the other types sequencing technologies (both SCORM- and workflow-based) to implement the same or very similar solution. This is why the main pedagogical issues observed during this project, are also applicable to educational applications of other types of sequencing technologies. 5. Action learning project The section describes the action-learning project in more details, including the research method and the main outcomes. 5.1. Research method The main objective of the action-learning project was to investigate the main pedagogical issues related to the implementation of this particular sequencing technology. In par-
2752
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
ticular, we were interested to investigate and understand whether the introduced flexibility actually improved students’ learning experience. The study used the action learning research method because it enabled the researcher not only to observe, but also to actively participate in the study implementation through data collection, analysis, recommendation and implementation of changes as well as critical reflection. Fig. 2 depicts the action learning method as used in this project. 5.2. The initial assumptions Based on the feedback obtained from the previous student intakes, the following assumptions were made about the students at the beginning of this project: Postgraduate students, having at least one prior degree, are likely to have very good time management skills. Some, if not majority of the students, would be motivated to use flexible deadline and complete this course faster, thus leaving more time for other courses. Postgraduate students would prefer flexibility to study off-campus and set their own deadlines rather than come to university at predefined times. This was confirmed by the survey of previous intakes, as well as the initial survey of this group of students. Preparation for the flexible mode of learning
Introduction of flexible mode of learning to students
Implementation of recommended changes
Reflection and recommendation for ongoing changes (improvements) during semester
Data collection
Data Analysis
Reflection on the overall teaching/learning experience
Identification of the main pedagogical issues Fig. 2. The action learning research method used in this project.
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
2753
Postgraduate students in this class should be experienced learners, having at least one previous degree, and would be able to effectively apply different learning strategies. Students from different professional/educational backgrounds may choose different sequences of modules. For example, students from technical backgrounds may prefer to start from more technical (mathematically-oriented) modules. Students would continue to form and work in study groups and support each other. 5.3. Main objectives of the action-learning project The main objectives of the action learning research were to better understand the following pedagogical aspects of flexible learning: The overall students’ learning experience. The importance of various time constraints in the learning process and the required time management skills. The importance and structure of the face-to-face component in flexible learning and how it could be used more effectively. Quantity and nature of the workload of the teaching staff involved in this type of flexible learning. 5.4. Data collection The following data collection techniques were used: Observation of teaching staff recorded during weekly review sessions. Informal interviews with students – a number of students who attended weekly learning sessions, practical sessions and consultation sessions were interviewed and their feedback recorded. Focus groups Analysis of students’ progress throughout the semester (including time analysis). This analysis was based on the recorded dates when students completed various assessment items. Survey of students – all students were surveyed at the end of the semester. Analysis of the final results – obtained at the end of the semester. Data was collected throughout semester and teaching staff had meetings every week to analyse the data, reflect and propose new improvements and actions how to put them in place. The lecturer in charge also kept a reflective journal. 5.5. Major outcomes After the course was completed and detailed analysis of various aspects of student learning experience performed, the following major outcomes were observed: All students extended the duration of the course by taking the longest possible option available.
2754
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
All students followed the same learning pathway. At the end of the course, their results were comparable to all previous intakes. The fact that they spent more time doing this course than any other intake did not result in better results. Additional flexibility, although enabled and supported by sequencing technology, created an enormous workload for all teaching staff members. They had to prepare and manage many different versions of the same assessment items to cater for different groups of students who could take quizzes and assignments at different point in time. The accumulated experience was carefully analysed to identify the main pedagogical issues and possible improvements of this particular sequencing technology. More importantly, during the final reflection phase of this action-learning project, these issues were then further generalised, as described in the next section. 6. The main pedagogical issues This section analyses the main pedagogical issues related to the implementation of sequencing technologies irrespectively of their type (workflow- or SCORM-based systems) and the application domain (i.e. teaching discipline). These generic issues can be described as follows: 6.1. Content-based versus process-based educational experience Current applications of sequencing technologies often focus too much on delivery of educational content. However, the content-based view of educational pedagogy is very limited. It can be summarised as follows: ‘‘In order to learn, a single learner has to work through a sequence of learning objects. The underlying assumption is that learning is process of consuming content. Teaching is envisioned as the art of selecting and offering content in a structured, sequenced way, and of tracking the learner’s progress and assessing the acquired knowledge. Current educational practice is more complex and advanced than this.’’ (Koper & Olivier, 2004, p. 97). This was also confirmed in this project. Contentbased modules were combined with quizzes designed to test content. However, from the pedagogical point of view, this is not sufficient. Learning/teaching activities should also include collaborative activities where the main emphasis is not on consumption of educational content. Examples include debates, problem-solving and role-play activities. Therefore, technology should be used to support these collaborative activities in a more innovative way rather than just ‘‘feed’’ the content to students. 6.2. Change of traditional roles Irrespectively of the type of technology used to enable or support it, flexible learning requires the change of traditional roles of teachers and students including their responsibilities and expectations. Most importantly, in the case of flexible deadlines, it requires students to take greater responsibility for their learning than in traditional transmissive mode of teaching. ‘‘Perhaps the most important outcome of higher education should be the development of the ability to manage one’s own learning. The need for this is made more urgent by greater mix of ability of students in a mass higher education system. Flexible learning pathways develop this ability. Being taught often inhibits such development’’
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
2755
(Race, 1999). This project confirmed that, it is crucial to carefully introduce students to this mode of learning and clarify their roles and responsibilities as well as their expectations. 6.3. The importance of group experience Sequencing technology enables personalised solutions that often focus on individual progress through a process. At the same time, it is very important to enable students to progress as a cohort. For example, in this particular project students were able to choose and follow their own flexible learning pathways. However, very soon after completing the first set of modules, they ended up following the same pathway. In many instances, they were even waiting for each other to start new modules. In this way, they could compare their learning experiences, support each other, but also, find out how their own results compared to those of the other students in the same cohort. Some students indicated that this comparison to the other students was a very important motivational factor for their learning. 6.4. Finding a good balance between flexibility and control It is very simple to use sequencing technologies to set up various time constraints i.e. to impose strict deadlines or to offer the ultimate time flexibility by allowing students to set up their own deadlines. In this particular project, students were only given the recommended deadlines. However, due to the other time pressures, the whole group ended up doing quizzes at the end of the course. Not a single student took the offered option to complete this course earlier. Students’ interviews and surveys confirmed that the actual time they spent studying each module and doing the whole course was much shorter than the recommended time. Towards the end of the course, they were rushing to complete one module after another without allowing enough time to reflect upon their learning experience (i.e. allow the content to ‘‘sink in’’ – as one student pointed out). Furthermore, some students had very long breaks between completing the initial modules (1, 2 and 3) and starting the subsequent modules. These ‘‘study breaks’’ coincide with the major assessment items and the mid-session exams in other courses they were doing in parallel. To be effective, learning has to be a continuous and progressive endeavor. Obviously, long breaks, especially when studying topics in a new area, will have a deteriorating effect on student learning, as they are very likely to forget what they had done in previous modules. In essence, students were using increased flexibility in this course to study other courses with strict deadlines. A similar point was also confirmed by McInnis (2001) in their Australia-wide research on the study experience of first year students. They found out that when given the additional flexibility in a course or program, most students (if not all) used this additional time for work or other activities. One could argue that postgraduate students should be different from first years, however when it comes to flexible deadlines this may not be the case, as confirmed in this project. At the same time, one has to stay realistic. Even in the traditional study mode, a majority of (if not all) students complete their assignments just before they are due. Many bad time management habits may remain undiscovered by students simply because of the strict deadlines in the traditional mode of study. However, the same habits would cause much more problems in the flexible mode of learning.
2756
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
Sequencing technology can be easily ‘‘configured’’ to create very different flexible learning experiences (especially related to temporal constraints). For example, inflexible deadlines managed by technology can create extra pressure so students will feel as if they are on ‘‘some kind of production line’’. On the other hand, if time is too flexible, this will not necessarily create more time for learning. So it is very important for teachers to organise individual modules in a way that will allow enough time for learning, prevent long breaks between modules but also allow enough time for reflection on what has been learnt. Therefore, when using sequencing technologies teachers need to find a fine balance between flexibility and control. However, as this project confirmed, this is not an easy task. 6.5. Students’ ability to handle the increased flexibility Another important pedagogical issue, closely related to the previous one, is students’ ability to handle the increased flexibility enabled by sequencing technology. In this particular project a large number of students confirmed that it was very hard for them to handle the increased flexibility, especially flexible deadlines. More than 95% of all students indicated that they would have preferred the strict deadlines because the recommended deadlines were not strict enough to motivate them to learn. However, the reasons for this apparent inability to handle deadlines are likely to be much more complex than what may look like on the surface (i.e. extrinsically motivated students who did not take full responsibility for their own progressive learning). In his work on situational leadership theory and its application to student learning, Grow (1996) explains how students develop as learners from dependent via interested, involved to self-directed learners. Dependent students are highly reliant on teacher’s directions and instructions and need structure. They cannot handle choices very well. On the other hand, self-directed learners are able to set their own learning goals and standards. They have the required skills in time management, project management, goal setting and self-evaluation. As Grow pointed out, dependent students will ‘‘not be able to make use of the ‘‘freedom to learn’’ because they lack skills such as goal-setting, self-evaluation, project management, critical thinking, group participation, learning strategies, information resources, and self-esteem, which makes self-directed learning possible’’. The theory of situational leadership confirms that when the followers (in this case dependent learners) do not receive the required level of instructions and guidance from their leader, they are unable to perform the required task. Hersey (1983) described the resulting experience of the followers: ‘‘lacking the ability to perform the task, they tend to feel that leader has little interest in what they are doing and does not care about their progress or them personally’’. This is exactly how some students felt about their learning experience in this course. On the other hand, recall that in this particular case, all students had completed at least an undergraduate course. One could argue that majority of post-graduate students are or should be self-directed learners. This may not always be the case. Grow (1996) argues that even self-directed learners can become (or ‘‘switch back’’ to) temporary dependent if they are learning something from a completely new area. This is very applicable to this particular group of students. They were probably self-directed or at least independent learners in their own discipline. But in this particular case, they were doing a course from a completely new discipline and as dependent learners were unable to handle the increased flexibility. This is probably one more reason why the face-to-face discussion sessions were, as they indicated, the most valuable part of this course.
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
2757
Therefore, when using sequencing technologies to provide additional flexibility it is necessary to carefully consider the current level of student development as learners but also to take into account if they are new to the given discipline. Only then one can decide how much flexibility and, most importantly, what kind of flexibility they can handle. 6.6. Teacher-driven versus technology-driven educational experience Another very important issue is the problem of control that teachers have when using sequencing technologies. As all educators know, the teaching/learning process is highly dynamic and relies on teachers’ experience and expertise to determine what is best for a particular group of students at the particular point of time. So sometimes students need to progress through some tasks individually and then through some other tasks as a group (e.g. collaborative learning tasks). Students may need to go back and revise and re-do some activities from the previous modules. Teachers may need to incorporate new interesting modules if they become available. In essence, the experienced teachers concentrate on students’ learning and very often on-the-fly modifications of the intended activities and lesson plans need to be implemented to better suit a particular group. The whole experience has to be teacher driven rather than technology driven. Unfortunately, currently available sequencing technologies (both SCORM- and workflow-based) are still quite inflexible to support on-the-fly modifications as they are normally done in the face-to-face activities. 6.7. The importance of face-to-face learning experience This project also re-confirmed the importance of face-to-face activities, especially in the blended mode of learning. Applications of sequencing technologies enable teachers to supplement face-to-face activities with online activities and extend students’ learning experiences beyond the time they spend in the classrooms. In this way, teachers could ensure continuity of the learning process. But at the same time, this gave the teacher a real opportunity to make face-to-face activities even more effective and more valuable to their students. Thus, when designing the face-to-face activities to complement the application of sequencing technology, less emphasis needs to be placed on the content and more on critical thinking, innovation, sharing of teacher’s experiential knowledge in the given discipline. Then, these activities become something that cannot be easily replaced by technology. At the same time, these face-to-face sessions do not only help students to learn about a particular topic, but also motivate, inspire and reinforce the ‘‘team spirit’’. This was very clear in this particular course, were face-to-face group learning sessions were unanimously voted as the best part of student learning experience. Finally the main argument is that any application of process-oriented, sequencing technology has to be guided by the intended learning objectives and pedagogy rather than technology. This particular technology makes it relatively simple for teachers to increase the flexibility just by changing several parameters (e.g. deadlines). However, this type of change that can be easily handled by technology has a profound effect on student learning experience. This is why it is necessary to fully understand capabilities as well as possible limitations of these technologies as they can easily create technology-centered learning experience where the main emphasis is on (more or less flexible) progression through the sequence of content rather than learning.
2758
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
7. Conclusion Sequencing technologies (such as SCORM- and workflow-based systems) support sequencing of units of educational content and tasks (in the case of workflows). At the same time, these technologies can be used to enable new types of learning experiences never before possible. This paper illustrates how workflow-based sequencing technology can be used to make possible the implementation of two additional dimension of flexibility: flexible time (implemented via flexible deadlines) and flexible learning pathways. However, as this paper illustrates through an example of an action-learning project in this domain, the introduction of any new dimension of flexibility has a profound effect on students’ learning experience. This is why the associated pedagogical issues have to be identified and fully understood before one can start to experience the benefits made possible by this technology. Finally it is important to point out again that the content-oriented pedagogy is quite limited. The very recent theory of Learning designs (Koper & Tattersall, 2005) confirms that learning/teaching activities are very complex processes that include much more than delivery of content. Consequently, our current and future work in this area include further investigation of the pedagogical issues related to the implementation of various processoriented educational technologies, especially learning design systems. References ADL (2004a). SCORM – 2nd Ed. overview. Available from http://www.adlnet.org. ADL (2004b). SCORM – sequencing and navigation (SN), Version 1.3.1. Available from http://www.adlnet.org. Britain, S. (2004). A review of learning design: concept, specifications and tools. A report for the JISC E-Learning Pedagogy Programme, May 2004. Available from http://dspace.learningnetworks.org/handle/1820/267. Brown, A. (1999). ‘‘Dimensions of flexible learning’’. Learning Research and Development Unit (LRDU). University of Queensland, Australia. Grow, G. O. (1996). Teaching learners to be self-directed. Adult Education Quarterly, 41(3), 125–149. Available fromhttp://www.longleaf.net/ggrow. Hersey, P. (1983). Leader effectiveness and adaptability description, LEAD matrix: Directions for matrix scoring and analysis. San Diego: University Associates, Inc. Holden, S., Kay, J., Poon, J., & Yacef, K. (2005). Workflow-based personalised document delivery. International Journal of E-Learning, 4(2), 131–148. IMS-LD, (2003). IMS Learning Design. Information Model, Best Practices and Implementation Guide, XML Binding, Schemas. Version 1.0 Final Specification, IMS Global Learning Consortium, Inc. Available from http://www.imsglobal.org/content/learningdesign/. Koper, R., & Olivier, B. (2004). Representing the learning design of units of learning. Educational Technology and Society, 7(3), 97–111. Koper, R., & Tattersall (Eds.). (2005). Learning design: A handbook on modeling and delivering networked education and training. Berlin: Springer. LOM (2002). Standard for Learning Object Metadata. Learning Technologies Standards Committee of the IEEE 148.41.21. Lunding, R., (1997). Flexible delivery: an international perspective. UQ Teaching and Educational Development Institute Conference, Initiatives in Flexible Delivery, September, 1997. Available from http://www.tedi.uq.edu.au/tei/flex_delivery/Lundin.html. Maguire, M., Matejka, D. (2000). Online delivery: Making the rough road smooth. In Flexible learning for a flexible society, proceedings of ASET-HERDSA 2000 conference, Toowoomba, Qld, 2–5 July. ASET and HERDSA. Marjanovic, O. (2005). Towards Web-based handbook of process oriented learning designs. Educational Technologies and Society Journal, IEEE Learning Technology Task Force, 8(2), 66–82.
O. Marjanovic / Computers in Human Behavior 23 (2007) 2742–2759
2759
Marjanovic, O., Orlowska, M. (2000). Making the flexible learning more flexible. In Proceedings of IEEE IWALT 2000, International Workshop on advanced Learning technologies, New Zealand. McInnis, C. (2001). Signs of disengagement the changing undergraduate experience in Australian universities. Centre for the Study of Higher Education. Peter, Y., & Vantroys, T. (2005). Platform support for pedagogical scenarios. Educational Technology and Society, 8(3), 122–137. Race, P. (1999). Practical pointers to flexible learning. Available from http://www.lgu.ac.uk/delibrations/ flex_learning/race_content.html. Shapiro, N. S., & Levine, J. H. (2003). Creating learning communities – a practical guide to winning support, organising for change and implementing programs. USA: Jossey-Bass Inc.. Sicilia, M.-A., Sanchez-Alonso, S., & Garcia-Barriocanal, E. (2006). On supporting the process of learning design through planners. In Proceedings of the virtual campus workshop, Barcelona, in press. Available from http:// www.cc.uah.es/ie/elsem/papers/Sicilia_vc06.pdf. Ullrich, C. (2005). Course generation based on HTN planning. In Proceedings of the 13th annual workshop of the SIG adaptibility and user modeling in interactive systems, pp. 74–79. Available from: http://www.activemath.org/publications/Ullrich-CourseGenerationHTN-ABIS-2005.pdf. US National Commission on Time and Learning (1994). Report: Prisoners of Time. Van der Veen, J., Jones, V., & Collis, B. (1998). Workflow applied to projects in higher education. In Proceedings of the international conference on cooperative information systems, COOPIS’98. Welsch, E. (2004). SCORM: clarity or calamity. Online Learning Magazine, 29(Nov). WFMS (1999). Workflow management coalition: workflow process model. Available from http://www.wfmc.org.