- Email: [email protected]
An empirical model for tutoring strategy selection in multimedia tutoring systems
Decision Support Systems 29 Ž2000. 31–45 www.elsevier.comrlocaterdsw
An empirical model for tutoring strategy selection in multimedia tutoring systems Amelia K.Y. Tong, Marios C. Angelides ) Department of Information Systems and Computing, Brunel UniÕersity, Uxbridge, Middlesex UB8 3PH, UK
Abstract This paper proposes an empirical model for tutoring strategy selection in multimedia tutoring systems based on factors that influence human tutoring strategy selection. It also demonstrates how this model is used in a multimedia tutoring system and assesses the benefits through the comparative evaluation of two multimedia tutoring systems, one that includes the model and one that does not. q 2000 Elsevier Science B.V. All rights reserved. Keywords: Multimedia tutoring systems; Tutoring strategy selection; Multiple tutoring strategies
1. Introduction In empirical studies of human teaching in the 1990s, the use of different tutoring strategies has been found to form a substantial part of the overall teaching interaction. It is certainly important for a tutoring system to provide more than one tutoring strategy because if the system has only one tutoring strategy to use, it may not be able to adapt promptly to the changing cognitive needs of the student w21x. Researchers in tutoring systems are looking for the knowledge underpinning demonstrably effective ways of tutoring w18x. The pressing need is not just for knowledge on optimal ways of tutoring, but for knowledge about how to tutor in an effective way. As with overall tutoring, the same applies to the use of tutoring strategies. This paper argues that the focus of knowledge on the use of multiple tutoring )
Corresponding author. E-mail address: [email protected] ŽM.C. Angelides..
strategies is on when, why and which one to use. PEPE w31x is a planning framework that allows the designer of an intelligent tutoring system ŽITS. to incorporate and to encode a variety of tutoring strategies in an ITS. The main focus of PEPE, however, is on content planning as opposed to delivery planning. This separation refers to the distinction between the subject matter and the formats in which it can be presented. Content planning entails the generation, ordering and selection of content goals, and the monitoring of the execution of the content plan. Delivery planning involves choosing the actual activities and interactions that help the student achieve his goals. PEPE focuses on the content issues alone. Although PEPE sets out to accommodate the use of more than one tutoring strategies, it does not describe how the selection should proceed, nor does it describe how content planning can be used for tutoring strategy selection. Eon w17x is a set of authoring tools for ITSs designed to be used by human tutors and instruc-
0167-9236r00r$ - see front matter q 2000 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 7 - 9 2 3 6 Ž 0 0 . 0 0 0 6 1 - 0
32
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
tional designers to represent their own tutoring strategies. The primary focus of Eon is on the use of multiple tutoring strategies. This ‘‘meta-strategic’’ knowledge base is distinct from the tutoring strategy knowledge base. Eon provides tools for describing the subject domain as network-related topics, creating reusable interactive presentation screens, and authoring the procedures, which tell the tutoring system how and when to interact with the student. In terms of tutoring strategy selection, Eon uses a parameterised approach that allows authors of ITSs to define a number of tutoring strategy parameters, e.g. ‘degree of hinting’ and ‘degree of interruption’. Eon currently offers a general mechanism for representing tutoring strategies, but does not, in itself, contain any specific tutoring strategies. The success of the tutoring strategies used has, therefore, not been taken into account. Tutoring strategies have subsequently not been linked with tutorial goals. One of the most significant contributions in this area is the ‘‘dynamic method selection’’ in GTE w30x. GTE is a Generic Tutoring Environment for developing courseware, based upon a generic instructional task-based approach. The selection takes into account instructional tasks, instructional methods and instructional objects. Instructional tasks are activities accomplished by the tutor in a tutorial. Instructional methods are procedures to carry out tasks, which are taken as equivalent to tutoring strategies in the context of this paper. Instructional objects are means that methods employ. Instructional methods are chosen according to a numerical applicability value for each method. These values are calculated according to a list of conditions attached to the individual methods. While GTE has comprehensively represented the instructional processes from the tutor’s viewpoint, the learner’s perspective, which is also important in the tutoring process, has not been sufficiently addressed w6x. GTE’s selection could be improved if it could couple a student’s learning results with the selection mechanism. There is no separate knowledge base for the selection procedure in GTE. While having such knowledge embedded in the tutoring strategies themselves makes their context more explicit, it obscures the relationship between the various elements within the selection mechanism, thus lessening the transparency between the different aspects, which affect selection w17x.
These examples provide evidence that although many systems use more than one tutoring strategy and that they all incorporate some kind of support for tutoring strategy selection, they lack a common basis on which the rules governing the selection can be applied. The lack of common basis for tutoring strategy selection supports the fact that what is missing is a formalisation of such a process. The benefits of formalising tutoring strategy selection in tutoring systems are to ensure that tutoring systems consider each tutoring strategy according to Refs. w7,12,13,22x: what the system aims to teach; the individual student; the particular situation or application the tutoring strategy is to be used for; the previous failures and successes of a tutoring strategy in similar situations; what a human tutor would have done; and to ensure in general that tutoring strategy selection is provided for in an adaptive and yet coherent manner. This paper does not propose new research in computational tutoring strategies. Its purpose is to model the process of tutoring strategy selection in multimedia tutoring systems. Consequently, the paper is organised as follows. First, it constructs a model for tutoring strategy selection based on factors that influence human tutoring strategy selection, followed by a comparative evaluation of SONATA and ARISTOTLE, two multimedia tutoring systems that use multiple tutoring strategies, but only ARISTOTLE incorporates the model for tutoring strategy selection in its architecture.
2. An empirical model for tutoring strategy selection Characteristics that constitute satisfactory tutoring strategy selection have been studied by various researchers. When considering how to select a tutoring strategy for a given point in time, different tutoring strategies are needed for the different pieces of tutorial material within the domain, and for different cognitive levels among students w14x. This implies that tutoring strategy selection is needed when there is a change in the nature of tutorial material, a change in cognitive needs among different students who interact with the system, and a change in cognitive needs within the same student as he advances
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
along the tutorial discourse. Their discussion suggests that subject matter difference and cognitive difference should be taken into account when making tutoring strategy selection. One question all researchers try to answer is ‘‘when’’ specifically tutoring strategy selection should be triggered. One answer is when a tutoring strategy appears to have failed. A student’s test results on the tutorial material may reflect a degree of tutoring strategy success or failure w11x. Test scores may also be used to decide whether the student should revise the same material before moving onto new material, which, in turn, determine the scope of application of the next tutoring strategy, that is, whether it is to be used for teaching Žand re-teaching. or testing what has been taught w32x. This characteristic implies that tutoring strategy selection is needed to check whether the tutoring strategy selected is appropriate when the test scores of a student show extreme patterns, i.e. perfect or poor. The scope of application also affects tutoring strategy selection. It is obvious that checks should be made when test scores are poor. Checks should also be made when the student is showing excellent progress, to make sure that the tutoring strategies are working properly, and not that the student finds the material too easy. The student’s own tutoring strategy preference should be taken into account w9x. Research in intrinsic motivation has consistently identified a sense of personal control as an important motivational factor in learning w16,19x. Tutoring strategy selection is needed when motivating a student to learn. The choice of tutoring strategy may involve the student expressing his own preference to give a sense of learner’s control. However, the learner may not always know what is best for himself w24x. The system should, therefore, be able to provide an environment, in which the student has the impression of control, with the amount of control carefully monitored by the tutor. The nature of the tutorial material affects the tutorial environment in which tutorial is to take place. The tutorial environment in turn governs the type of communication channel between the system and the student. Different tutoring strategies that are used to teach different material within the same domain require a different tutorial environment. For example, tutoring procedural knowledge may call for
33
the animated display to demonstrate each process to the student, whereas tutoring factual knowledge may only need a textual display. Tutoring strategy selection is, therefore, needed when there is a change in the presentation method for a different piece of tutorial material. The choice if which tutoring strategy to use may depend on which tutorial environment are required since these environments facilitate the delivery of the tutoring strategies. Tutoring systems should be viewed as an automated tool to assist the human tutor, and not to replace them because human tutors are considered to be an effective but expensive resource w3x. Tutoring strategy selection should, therefore, allow for human tutor intervention in the process to ensure system efficiency and to explain to the student about a change of tutoring strategy to lessen confusion when necessary. The process of tutoring strategy selection in a model should include all the above characteristics w29x. The model for tutoring strategy selection consists of the following aspects: tutorial goal, use of tutoring strategy, domain knowledge, user knowledge, tutor knowledge, tutoring strategy delivery environment and external human tutor support. These aspects are shown in Table 1. Individually, these aspects are advances about tutoring on the macro level, each contributing to the
Table 1 The model for tutoring strategy selection Tutoring strategy selection Aspects Tutorial goal Use of tutoring strategy Domain knowledge User knowledge
Tutor knowledge
Tutoring strategy delivery environment External human tutor support
Definitives Teachrre-teach, Testrre-test Type context Advancement level: novicer advanced; Individual tutoring strategy preference: implicitr explicit Tutoringstrategies, Tutoring strategies general success rates Text, Graphics, Audiorvisual, Animation, Simulation, Combination of environments
34
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
structure of tutoring, the process of tutoring and learning. These aspects may appear to compete with, and, even occasionally, contradict each other in isolation. All aspects in the model for tutoring strategy selection should, therefore, be considered in synergy. The model for tutoring strategy selection is domainindependent. It begins its operation by addressing any detected tutoring strategy failure. 2.1. Tutorial goal Tutorial goals affect the tutoring strategy selection because they define the objectives of a tutorial session w19,20x. For example, if the tutorial goal of a session is to help the student understand how mouthto-mouth resuscitation saves lives, the objectives of the session can range from learning the facts of how the human respiratory system works, to the procedures for carrying out resuscitation. To match a tutoring strategy with a goal of the tutorial discourse means that the aim of the tutoring strategy must be the same as the aim of the objectives and in turn of the tutorial goal. Therefore, a tutorial goal can in fact be viewed as a tutoring strategy goal. A tutorial goal can be matched by more than one tutoring strategy goal, because a tutorial goal often has more than one objective. For example, to match the objective of learning about the respiratory system, tutoring strategies with goals aimed for tutoring factual events may be used. To match the objective of how to carry out resuscitation, tutoring strategies with goals aimed for tutoring procedural events may be used. The model for tutoring strategy selection thereby tries to match tutoring strategy goals inherent in tutoring strategies with the tutorial goal at a particular moment in time. 2.2. Use of tutoring strategy The model for tutoring strategy selection distinguishes two types of scope of application alongside with the objectives defined by the tutorial goal. They are teachingrre-teaching and testingrre-testing. Before deciding which tutoring strategy is to be used next, the model has to take into account what the next tutoring strategy is to be used for. Student–system interaction so far can suggest which tutoring strategies are more appropriate for the different scopes of application.
2.3. Domain knowledge The content of the tutorial material affects the design of instructional procedure w5x. For example, tutoring within the domain of science usually involves a lot of problem-solving. However, mathematical problems can often be solved by paper and pencil, whereas for the empirical sciences, such as chemistry, the student may need special devices and a sample of matter to study physical or chemical properties. The content of the tutorial material, therefore, affects the environment in which the tutorial is to be carried out. The type of tutorial material refers to whether the material to be imparted is procedural or declarative knowledge. 2.4. User knowledge Two areas of user knowledge are distinguished: advancement level, and the student’s individual preferences in tutoring strategies, either implicit or explicit. Advancement level in the model for tutoring strategy selection represents two stages of the student’s cognitive progress: novice and advanced. The student’s current level of knowledge and skills indicates the amount of tutorial material imparted that the student has learnt. The advancement level represents the level of expertise Žwhat he knows., and how far the particular expertise level the student is at Žhow much he knows.. In the model for tutoring strategy selection, a student’s preference is represented either as implicit or explicit. The act of asking the student results in the explicit preference. Different tutoring strategies’ prior successes with that individual student constitute the implicit preference. The system should know which tutoring strategy is suitable for that individual student at a given time, demonstrated implicitly by the prior success rates of different tutoring strategies with the student. It is important to note, however, there might be noises in this prior success deduction, e.g. repetition of the same piece of tutorial material to the student may lead to ‘reproductive’ learning due to memorising and rehearsing w20x. The number of times a piece of tutorial material has been represented to the student is, therefore, taken into account in the implicit preference. Implicit preference is considered first because the student himself may be
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
inconsistent in his own choices of tutoring strategy. In addition, the model for tutoring strategy selection also checks the student’s advancement level before the student’s explicit preference is asked for. For instance, it may be inappropriate for a beginner student to judge for himself what tutoring strategy is best at the start of the tutorial discourse. This means that when the implicit preference does not show a favourite tutoring strategy, the model for tutoring strategy will only ask for an explicit preference when the student has shown sufficient progress in learning the tutorial material, i.e. when he has progressed to advanced level. 2.5. Tutor knowledge Before deciding which tutoring strategy is to be used next, the model for tutoring strategy selection needs to know which tutoring strategies are offered in the system. Therefore, the availability of tutoring strategies is a necessity for tutoring strategy selection. Suitability of a tutoring strategy for an individual student in a particular situation is fundamental in tutoring strategy selection. Nevertheless, the model for tutoring strategy selection should also know which tutoring strategies are appropriate for students in general. For this purpose, the tutor knowledge aspect of the model for tutoring strategy selection provides information on the tutoring strategy general success rates with all the students. This information is obtained by taking into account all the interactions between the system and all the students who are registered to have had at least a session with the system. This information gives the overall success rates with all the students, as opposed to the successes with each individual student. The system ‘‘learns’’ what is a generally appropriate tutoring strategy for certain situations as it builds up its tutoring experience through increasing interactions with increasing number of students. 2.6. Tutoring strategy deliÕery enÕironment The tutoring strategy delivery environment provides the student with the scenario in which the tutorial material is introduced w10x. This aspect, together with the content of the tutorial material, af-
35
fects the model for tutoring strategy selection. If the content of the domain suggests the use of audio and video in the delivery environment, then the tutoring strategy to be used has to be able to tutor with audio and video. 2.7. External human tutor support One of the situations where human tutor could step in is when the tutoring strategy selected by the system fails. The student’s control over the tutorial session only increases as he advances, human intervention may be more frequent for novices and slower learners. Once the human tutor is called, he can look into the student model of the system, as well as eliciting information from the student directly on how the student feels about the tutorial session, in particular, how he feels about the tutoring strategies used. The investigation into the student model would give the human tutor an insight into the system’s inferences on the successes and failures of various tutoring strategies. The human tutor can query the student directly to see if there is any mismatch between the system’s deductions and the student’s own judgement. The human tutor can then decide which tutoring strategy is to be used next, based on his own experience and knowledge about the student. If the human tutor, after consultation with the student, finds that there is a discrepancy between the system’s perception of the tutorial discourse and reality, he could then also amend or add new data on the system. This section has described the individual aspects of the model for tutoring strategy selection, as well as how they complement each other. Section 3 describes the rules employed by the model during the selection by the model.
3. An algorithm for the process of tutoring strategy selection Embedded in the model for tutoring strategy selection are certain rules to ensure consistent decisions. These rules serve to govern tutoring strategy weights and the selection. There are four types of rules in the model for tutoring strategy selection. They are suitability rules, proficiency rules, block-
36
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
age rules and sequence rules. Suitability and proficiency rules both measure a tutoring strategy according to the user knowledge. Suitability rules also consider how suitable a tutoring strategy in terms of the scope of application. Blockage rules assess a tutoring strategy according to its previous successes and failures in similar situations. Sequence rules measure a tutoring strategy according to the order of the use of tutoring strategies. Suitability rules alter the priorities of tutoring strategies based on their suitability for the current student and the scope of
application. For instance, tutoring strategies are favoured when they match the student’s individual style. Suitability rules favour tutoring strategies that have had successes previously with the student. Proficiency rules govern tutoring strategy selection according to the student’s proficiency in the domain, according to his advancement level. An example of such rules is increasing the student’s control when he has reached advanced level. Blockage rules lower the priorities of those tutoring strategies that have failed previously. For example, if a tutoring strategy has
Fig. 1. The selection process in the model for tutoring strategy selection.
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
appeared to have failed, blockage rules will lower its priority in the subsequent selection. Sequence rules adjust priorities to favour tutoring strategies that are similar to the one previously selected. Sequence rules are consistent with the idea of the model that the use of tutoring strategies should be coherent. There are two exceptions to the sequence rules. One is when similar tutoring strategies have been used and effective learning still has not appeared to have taken place. Then, a completely different tutoring strategy may have a higher priority compared to the ones similar to the previous ones used. The other one is when the human tutor decides the sequence rules are to be broken, for example, when a student has perfect progress with one particular tutoring strategy for a considerable length of time. This is to check whether the student still finds it challenging to learn with that particular tutoring strategy to avoid boredom. These rules are interpreted in the sequence and alternatives the model for tutoring strategy selection goes through when selecting a tutoring strategy. This sequence of the selection operation, as shown in Fig. 1, summarises the discussion of the various aspects in Section 2. The model for tutoring strategy selection has been developed as an ‘add-on’ component to the architecture of any full-scale, knowledge-based tutoring system which uses multiple tutoring strategies. Such a system contains separate knowledge-based representations of the subject matter, the student, the tutorial goals and strategies and separate knowledge processes for each one of the three knowledge-based representations. Section 4 demonstrates the benefits of the model for tutoring strategy selection through the comparative evaluation of two multimedia tutoring systems, ARISTOTLE and SONATA. Although they both use several tutoring strategies, only ARISTOTLE incorporates the model for tutoring strategy selection.
4. Comparing
ARISTOTLE
with
SONATA
The functions and operability of the model for tutoring strategy selection are demonstrated by the deployment of the model in ARISTOTLE w29x. ARISTOTLE is a multimedia tutoring system for zoology and
37
was originally implemented using a method for developing interactive multimedia tutoring systems w1x. SONATA is a multimedia tutoring system for music theory and serves the purpose of demonstrating the incorporation of multiple tutoring strategies w4x. This section starts with a description of SONATA, and then of ARISTOTLE. The differences in the use of tutoring strategies between the two systems are then presented. Finally, the benefits of formalising tutoring strategy selection, as illustrated by ARISTOTLE are discussed. 4.1.
SONATA
SONATA tutors knowledge covering the syllabus of the grade 1 theory of music examination set by the Associated Board of the Royal School of Music. The domain is divided into four areas. These areas are basic terms used in music theory, keys and tonic triads, rhythms, and Italian terms and musical signs. Tutoring in each area involves SONATA initially teaching the student about the area, followed by the student applying his acquired knowledge through 12 successive problem-solving activities, before moving onto a new area. SONATA was developed using Hypercard. It uses a semantic network of ‘cards’ for storing information. These cards depict information nodes. These nodes are linked via information links. These nodes can be linked sequentially, hierarchically or non-hierarchically. Multimedia material such as audio and video are not represented directly in the card structure. Instead, they are annotated to a card via an ‘annotation’ link. The name of a link represents the semantic content of the annotation. In SONATA, the domain, tutoring and student knowledge are kept together in a ‘stack’ of cards. A stack is a collection of related cards of information, which logically belong together. Links are installed to integrate the stored knowledge, both within and between individual knowledge model. Knowledge in SONATA’s domain model includes the domain knowledge and a multimedia base. Domain knowledge contains information on the four aforementioned areas in music theory. The multimedia base contains information about the annotation of multimedia material to the cards, as well as the content of the material itself.
38
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
Knowledge in the student model contains the student overlay knowledge. Student overlay knowledge represents the current level of understanding of the domain. It contains information about the student’s knowledge on the four areas of the domain. This is represented as overlay scores for the problem-solving activities the student has to engage in during the tutorial discourse. The value of an overlay score is determined by whether the student’s answer is correct, and the number of times the student has attempted the same problem-solving activity. The tutoring strategy used for the problem-solving activity that yielded a score is also represented in the student overlay knowledge. Hence, the student’s prior success with the different tutoring strategies can be deduced from the student overlay knowledge. The student’s advancement level Žnovice or advanced. is deduced from the amount of tutorial material the student has gone through. Knowledge in SONATA’s tutor model includes tutorial goals, and the different tutoring strategies SONATA uses. Tutorial goals determines what is to be taught in a tutorial session. SONATA employs both multimedia-based and non-multimedia-based tutor-
ing strategies. Non-multimedia-based tutoring strategies are used when tutoring material, which is not possible to be taught through the use of multimedia, such as the notation of a note in terms of its position on the staff in music scores. Fig. 2 illustrates SONATA using a multimedia-based tutoring strategy. Due to the fact that SONATA does not incorporate a model for tutoring strategy selection, its use of tutoring strategy can only be explained in terms of a description of the process of tutorial interaction. The same tutoring strategy is always used when an area in the domain is introduced for the first time. This tutoring strategy is learning through exploration. The student then needs to apply his knowledge from learning through exploration in a series of problem-solving activities in the same area. The student is only allowed to explore the tutorial material in an area for a certain number of times. This is to prevent the student from ‘dwelling’ in a scenario for too long without having to apply the knowledge he acquired from his exploration experience. After a new area has been introduced, the student then has to go through 12 successive problem-solving activities in that area, before moving onto the
Fig. 2. SONATA using a multimedia-based tutoring strategy.
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
next area. The tutoring strategies to be used depend on how successful each tutoring strategy has been with that student. A tutoring strategy is regarded as successful if the student provides the correct solution in a problem-solving activity which employs that tutoring strategy. When the data recorded for the student–system interaction is still small, this success condition immediately affects the use of the next tutoring strategy. For example, if Strategy X fails in problem-solving activity three of area one, and that Strategy Y is successful as the second tutoring strategy used for the same activity, then the first strategy to be used in problem-solving activity four in area one is Strategy Y. If Strategy Y fails, SONATA will use Strategy Z and tutor the same activity again. Strategy X is used first because it was successful in the previous activity. Strategy Z is preferred to Strategy X as a second choice because the former has not been attempted in the last activity whereas the latter failed. When the student–system interaction has built up, information on tutoring strategy successes will be gathered and interrupted on an overall basis, rather than on problem by problem basis. Fig. 3 shows a sample student record information on the student overlay knowledge, and the results of the use of the different tutoring strategies.
4.2.
39
ARISTOTLE
ARISTOTLE tutors a basic knowledge of zoology. The entities of interest to the system are animals. Three animals were chosen and were grouped into classes. Animals were divided into vertebrates and invertebrates. Vertebrates were further divided into mammals and reptiles. Arthropods were chosen as a type of invertebrate. Students learn about the physical properties of the entities Ždescription., such as the different body parts they have, what the entities do Ževents., such as hunting, and the activities that constitute the events Žactions., such as observing prey. ARISTOTLE was developed using Asymetrix Multimedia ToolBook. It uses multimedia frames Žmframes. to model the content of audio and video and to integrate this information with information about the entities of interest to the system. Two types of m-frames are used in ARISTOTLE: syntactic m-frames ŽSYMs. and semantic m-frames ŽSEMs.. SYMs contain information about the syntactic co-ordinates of objects and the spatial relationships among objects within a frame of a video clip. SEMs contain information which represents the semantic content of a sequence of video or audio frames. A SEM also
Fig. 3. A sample student record in SONATA.
40
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
provides semantic information that is not related to a media segment within ARISTOTLE. In this way, all information related to an entity of interest is kept together. Knowledge in ARISTOTLE’s domain model consists of a multimedia resource, and domain knowledge. The multimedia resource contains information on the co-ordinates and spatial relationships among objects depicted within a video frame of a clip, together with the associated raw audio and video frames. Domain knowledge contains information about the entities of interest. ARISTOTLE’s student model includes the student overlay knowledge. It contains information about how strongly the student has been judged to know the acquired knowledge, the tutoring strategy, which yielded that knowledge acquisition, and any shots, i.e. any sets of continuous video andror audio frames, that were used with that tutoring strategy.
The knowledge on student’s misconceptions contains information about the seriousness of the misconception. The strength of acquired knowledge are determined by the number of attempts made in solving the problem, the number of times assistance is requested, and how well the student has provided the answer, i.e. whether there are any spelling mistakes, and whether there are too many words provided in the answer. The tutor model contains tutorial goals and tutoring strategies. Tutorial goals determine what the student is to be taught. Two main types of tutoring strategies are used in ARISTOTLE: non-multimediabased tutoring strategies and multimedia-based tutoring strategies. Non-multimedia-based tutoring strategies are used to tutor facts about an animal or animal class that cannot be taught through the use of multimedia. Fig. 4 shows ARISTOTLE using a multimediabased multiple choice tutoring strategy.
Fig. 4. ARISTOTLE using a multimedia-based tutoring strategy.
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
Tutoring strategy selection in ARISTOTLE begins with retrieving from the tutor model all the appropriate tutoring strategies available for the sub-goals specified by the tutorial goal. The resulting tutoring strategies are the tutoring strategies associated with the goal. The model for tutoring strategy selection then refers to ARISTOTLE domain model to determine the tutoring strategy delivery environment required, either multimedia based or non-multimedia-based, and match it with the relevant tutoring strategies. The model for tutoring strategy selection then determines what the next tutoring strategy is to be used for, i.e. to teach or to test, by referring to the student–system interaction so far. The system goes into testingrre-testing mode after teaching has been completed. The model for tutoring strategy selection then retrieves the weights for teaching and testing attached to each tutoring strategy in the student
41
model in order to pick the tutoring strategy with the highest weight. If the highest weight is shared by several tutoring strategies, then the general success rates calculated over all previous student interactions and stored in the tutor model are checked, to pick the one with the highest weight among these tutoring strategies. If the highest general success rate is still shared by several tutoring strategies, then the model for tutoring strategy selection checks the student model to determine the student’s advancement level. If the student is novice, the model for tutoring strategy selection prompts the human tutor to select. If the student is advanced, the model for tutoring strategy selection prompts the student to make a selection. The selection in either case will be from among the relevant tutoring strategies that share the highest general success rate. The model for tutoring strategy
Fig. 5. ARISTOTLE prompts for the human to pick a tutoring strategy.
42
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
selection offers the human tutor the choice of viewing the student model. Fig. 5 illustrates ARISTOTLE prompting the human tutor to pick a tutoring strategy. In those cases where the advanced student has made a choice and the tutoring strategy fails, the human tutor is called to make a choice. In addition, when a tutoring strategy has worked ‘too well’ for the student for a number of consecutive tutorial goals, the model for tutoring strategy selection calls for the human tutor to determine whether a different tutoring strategy should be used. The human tutor can also amendrenter new data to the student model by clicking on the view student model button on that screen. 4.3. Benefits of the formalisation of tutoring strategy selection Formalising tutoring strategy selection ensures that the tutoring strategy selected matches what the system teaches. The model for tutoring strategy selection matches the tutor strategies in ARISTOTLE’s tutor model to the relevant knowledge in the domain model via the tutorial goal. SONATA’s tutor model also includes knowledge on tutorial goals. However, without the model for tutoring strategy selection, the tutorial goal is not matched with the tutoring strategies. Formalising tutoring strategy selection ensures that the tutoring strategy selected matches the particular student. The model for tutoring strategy selection utilises ARISTOTLE’s knowledge on student’s advancement level to match tutoring strategies with the cognitive needs of students at different levels. It also allows the student, when appropriate, to express his preference in selection. SONATA distinguishes the student’s advancement level as well. However, it does not match the advancement level with the use of tutoring strategies. SONATA does not allow the student to intervene with its use of tutoring strategies. If the student has difficulty in a problem-solving activity with one tutoring strategy, SONATA will tutor the same material with another tutoring strategy, hoping that the student will grasp the point. The system may go round and round for a long period of time without achieving its tutorial purpose, and the student frustrated.
Ensuring that tutoring strategies are selected according to the particular scope of application is another benefit arising from tutoring strategy selection formalisation. ARISTOTLE’s tutor model includes knowledge on the scope of application of a tutoring strategy that distinguishes between teachingrreteaching and testingrre-testing. ARISTOTLE’s tutor model refers to the student model to establish if all teaching should be done before any testing, if material needs re-teaching before any testing is done, and if the student requires assistance from ARISTOTLE before testing commences. ARISTOTLE records information on student performance while being taught or tested in order to calculate the student’s advancement level. SONATA addresses the difference between learning through exploration and the problem-solving activities. However, without the model for tutoring strategy selection, it currently does not relate the different scopes of Žre.teaching and Žre.testing with the use of tutoring strategies. Formalisation of tutoring strategy selection ensures that tutoring strategy selection invokes human tutor intervention when necessary. When the human tutor is called to make a decision, ARISTOTLE presents the tutoring strategies from which the system failed to make a choice. The human tutor is given the choice to examine the student model before making a choice. The conditions under which the call for the human tutor was necessary are recorded along with the decision that the human tutor has made so that if these conditions arise again the system can take the same decision itself. SONATA allows neither the human tutor nor the student to intervene with the use of tutoring strategies. Table 2 summarises the benefits of the model for tutoring strategy selection in multimedia tutoring systems in the comparative evaluation of ARISTOTLE and SONATA. The evaluation is carried out according to the criteria based on the work of Siemer and Angelides w26x, who propose a system-independent, comprehensive evaluation method that can be applied to the evaluation on any ITSs. Their evaluation is based on desirable behaviour properties expected from a tutor in any tutorial environment. Both ARISTOTLE and SONATA are both multimedia tutoring systems. The only architectural difference between them is the model for tutoring strategy selection, which is only incorporated in ARISTOTLE, and not in SONATA. The
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45 Table 2 A summary of the benefits of the model for tutoring strategy selection demonstrated through the comparative evaluation of
43
ARISTOTLE
and
SONATA
Evaluation criteria ŽSiemer and Angelides w26x. Tutorial goals clearly defined and adopt appropriate tutoring strategy accordingly
Issues addressed by SONATA
ARISTOTLE
Tutoring goals defined but not matched with tutoring strategies
U
Tutoring strategies tailored to the cognitive needs of students at different levels Student actively take part in the tutoring process System intervention when the student appears to be in difficulty
Past learning experiences and learning preferences recorded and adapted to accordingly System monitoring changes proposed by the student and comment if they seem to be unwise
U U U
U
summary of the comparative evaluation in Table 2 shows that without the model for tutoring strategy selection, SONATA does not address all the issues and the subsequent implications of these issues in multimedia tutoring systems, therefore. undermining the potential pedagogical effects of multimedia tutoring systems.
5. Concluding discussion Several issues arise from the research described in this paper. These issues relate to future work on the model for tutoring strategy selection, as well as the research in the deepening of the understanding of tutoring and learning. The derived model attaches weights to the different tutoring strategies to determine the order in which the tutoring strategies are picked. The weighting method is governed by sequence, suitability, proficiency and blockage rules. Further research into
U U
Improvements contributed by the model for tutoring strategy selection Matching tutoring strategies with tutorial goal. Tutoring strategies selected subsequently match the subgoals defined by the tutorial goal. Matching tutoring strategies with the student’s advancement level The student is invited to take part in selecting a tutoring strategy to suit his own learning style Differentiation in the scope of applications requires the system to provide assistance, re-teaching and re-testing when a student appears to have difficulty
The system is required to monitor the changes induced by the explicit preferences suggested by the student. The human tutor is called for when there is a discrepancy between the system’s deduction and the student’s own judgement.
how exact the indices used in the weighting are, may result in a finer grained prioritisation, which, in turn, will enhance the functionality of the overall architecture. Quantitative studies such as multiple regression analysis may help to study indices used to assess the veracity of knowledge of the student w23x. The derived model is able to cope with an infinite number of tutoring strategies made available by the system. The model for tutoring strategy selection ensures that more than just two tutoring strategies are used in a system. The collection of tutoring strategies should be as extensive as possible to increase the system’s adaptiveness. The more tutoring strategies available the greater the benefits of tutoring strategy selection. A feature that could improve on the use of multiple tutoring strategies in tutoring systems is the system’s ability to allow the human tutor to build more tutoring strategies into the system themselves, by means of an authoring tool w15x or from ‘parts’ available in the system. The derived model allows the student to pick a tutoring strategy himself in order to increase motivation and to express his preference at times when the
44
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
system could not decide which tutoring strategy to use. However, the proposed architecture does not detect when the student needs motivating and how this freedom of expression of preference may affect him from a cognitive perspective. Asking the student to intervene tutoring strategy selection involves the student’s self-regulation in learning, e.g. monitoring the effects of the tutoring strategies, adjusting his study approach in order to reach his learning goals w27x. Students differ in the degree to which they operate these regulatory measures themselves. Many may expect the tutor to control these measures for them. Further research on the student’s intervention of tutoring strategy selection is needed such that this self-regulated approach in tutoring can be exercised with minimal discrepancy between the student’s expectations and experience, to prevent the student from disorientation w2x. The human tutor plays an important part in the efficient operation of the model for tutoring strategy selection in the proposed architecture. The role of the human tutor in computer-based tutoring interaction has been acknowledged by many, but addressed by only a few w28x. Research in the human tutor intervention in computer-based tutoring is not only scarce, almost all of them concentrate on the benefits to the student w25,28x. By studying the behaviour and learning processes of students using tutoring systems that focus on tutoring strategy selection, human tutors may also benefit from interacting with such systems, by being ‘forced’ to reflect on their own selection of tutoring strategies. The difficulties in analysing the use of tutoring strategy in professional tutors have been emphasised w8x. Involving human tutor formally in a computer-based tutorial will open a gateway to the understanding of the expertise in human tutors. This line of research will provide valuable information about learning episodes, and about the process of knowledge elicitation by expert human tutors.
w3x
w4x
w5x
w6x
w7x
w8x
w9x
w10x
w11x
w12x w13x
w14x
w15x w16x
w17x
References w1x H. Agius, M.C. Angelides, Integrating logical video and audio segments with content-related information in instructional multimedia systems, Information and Software Technology 39 Ž10. Ž1997. 679–694. w2x G.S. Akerlind, A.C. Trevitt, Enhancing self-directed learning
w18x
w19x w20x
through educational technology: when students resist the change, Innovation in Education and Training International 36 Ž2. Ž1999. 96–105. J.R. Anderson, C.F. Boyle, A.T. Corbett, M.W. Lewis, Cognitive modelling and intelligent tutoring, in: W.J. Clancey, E. Soloway ŽEds.., Artificial Intelligence and Learning Environment, Elsevier, Amsterdam, 1990, pp. 7–49. M.C. Angelides, A.K.Y. Tong, Implementing multiple tutoring strategies in an intelligent tutoring system for music learning, Journal of Information Technology 10 Ž3. Ž1995. 52–62. S. Dijkstra, The integration of instructional systems design models and constructive design principles, Instructional Science 25 Ž1. Ž1997. 1–13. J. Elen, Automating ID: the impact of theoretical knowledge bases and reference systems, Instructional Science 26 Ž3–4. Ž1998. 281–297. H. Ellington, Educational innovation — where are we now? A critical review of recent developments, Innovation and Learning in Education 1 Ž1. Ž1995. 15–20. M. Elshout-Mohr, B. van Hont-Wolters, H. Broekkamp, Mapping situations in classroom and research: eight types of instructional-learning episodes, Learning and Instruction 9 Ž1. Ž1999. 57–75. C. Frasson, E. Aimeur, A comparison of three learning strategies in intelligent tutoring systems, Journal of Educational Computing Research 14 Ž4. Ž1996. 371–383. H.M. Halff, Curriculum and instruction in automated tutors, in: M.C. Polson, J.J. Richardson ŽEds.., Foundations of Intelligent Tutoring Systems, Erlbaum, New Jersey, 1988, pp. 79–108. M.J. Hannafin, S.M. Land, The foundations and assumptions of technology-enhanced student-centred learning environments, Instructional Science 25 Ž3. Ž1997. 167–202. D. Jonassen, Computers in Classroom: Mindtools for Critical Thinking, Englewood Cliffs, NJ Ž1996.. S. Kennewell, Computer modelling and the curriculum — integrated or loosely attached? Education and Information Technologies 2 Ž3. Ž1997. 247–260. T. Khan, Y.J. Yip, pedagogical principles on case-based CAL, Journal of Computer Assisted Learning 12 Ž1996. 172–192. N. Major, Modelling teaching strategies, Journal of Artificial Intelligence in Education 6 Ž2–3. Ž1995. 117–152. ID2 Research Group 1998, M.D. Merrill, ID expert-intelligent computer-based multimedia interactive instructional development and delivery system, Instructional Science 26 Ž3–4. Ž1998. 243–262. T. Murray, Authoring instructional expertise in knowledge based tutors, Instructional Science 26 Ž3–4. Ž1998. 263–280. S. Naidu, M. Oliver, Critical incident-based computer supported collaborative learning, Instructional Science 27 Ž5. Ž1999. 329–354. A.C. Orstein, Strategies For Effective Teaching, Dubuque, Brown and Benchmark, 1995. C.M. Reigeluth, Instructional design: what is it and why is it? in: C.M. Reigeluth ŽEd.., Instructional Design Theories And
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
w21x w22x
w23x
w24x
w25x
w26x
w27x
w28x w29x
w30x
w31x
w32x
Models: An Overview of Their Current Status, Erlbaum, New Jersey, 1983, pp. 3–36. E. Sadler-Smith, R. Riding, Cognitive style and instructional preferences, Instructional Science 27 Ž5. Ž1999. 355–371. T.J. Shuell, Designing instructional computing systems for meaningful learning, in: M. Jones, P.H. Winne ŽEds.., Adaptive Learning Environment: Foundations and Frontier, Springer-Verlag, Berlin, 1992, pp. 19–54. V.J. Shute, J.W. Regian, Principles for evaluating intelligent tutoring systems, Journal of Artificial Intelligence and Education 4 Ž2–3. Ž1993. 245–271. H.S. Shyu, S.W. Brown, Learner control: the effects on learning a procedural task during computer-based videodisc instruction, International Journal of Instructional Media 22 Ž3. Ž1995. 217–231. J. Siemer, INTUITION — applying intelligent tutoring to gaming simulation, Journal of Computing and Information Technology 3 Ž1. Ž1995. 35–43. J. Siemer, M.C. Angelides, A comprehensive method for the evaluation of complete intelligent tutoring systems, Decision Support Systems 22 Ž1. Ž1998. 85–102. A. Slaats, H.G. Lodewijks, J.M. van der Sanden, Learning styles in secondary vocational education disciplinary differences, Learning and Instruction 9 Ž5. Ž1999. 475–492. J.M. Spector, The role of epistemology in instructional design, Instructional Science 26 Ž3–4. Ž1998. 193–203. A.K.Y. Tong, M.C. Angelides, Formalising tutoring strategy selection in multimedia tutoring systems, Information and Software Technology 41 Ž2. Ž1999. 67–90. K. Van Marcke, GTE: an epistemological approach to instructional modelling, Instructional Science 26 Ž3–4. Ž1998. 147– 191. B. Wasson, Facilitating dynamic pedagogical decision making: PEPE and GTE, Instructional Science 26 Ž3–4. Ž1998. 299–316. B.P. Woolf, W. Hall, Multimedia pedagogues — interactive
45
systems for teaching and learning, IEEE Computing Ž1995. 74–80, ŽMay.. Marios C. Angelides is a Professor of Computing in the Department of Information Systems and Computing at Brunel University. He holds a BSc in Computing and a PhD in Information Systems, both from the London School of Economics where he was a lecturer in information systems for several years. He has over 10 years of research experience in Multimedia Information Systems and Superhighways where he has published extensively in journal and book format. His most recent books are Multimedia Information Systems ŽKluwer Academic Publishers, 1997. and Multimedia Information Superhighways Žforthcoming.. He is a member of the management committee of the UK Multimedia Special Interest Group, the British Computer Society, the IEEE Computer Society, the ACM, the Information Resources Management Association, the UK Academy for Information Systems and the Engineering Professors’ Council. Amelia K.Y. Tong is a Lecturer in Computing in the Department of Information Systems and Computing at Brunel University. She holds a BSc in Computing, an MSc and a PhD in Information Systems, all from the London School of Economics. Her research interests are in the areas of Multimedia Information Systems and Intelligent Tutoring Systems. She is a member of the IEEE Computer Society, the ACM, and the British Computer Society.
An empirical model for tutoring strategy selection in multimedia tutoring systems Amelia K.Y. Tong, Marios C. Angelides ) Department of Information Systems and Computing, Brunel UniÕersity, Uxbridge, Middlesex UB8 3PH, UK
Abstract This paper proposes an empirical model for tutoring strategy selection in multimedia tutoring systems based on factors that influence human tutoring strategy selection. It also demonstrates how this model is used in a multimedia tutoring system and assesses the benefits through the comparative evaluation of two multimedia tutoring systems, one that includes the model and one that does not. q 2000 Elsevier Science B.V. All rights reserved. Keywords: Multimedia tutoring systems; Tutoring strategy selection; Multiple tutoring strategies
1. Introduction In empirical studies of human teaching in the 1990s, the use of different tutoring strategies has been found to form a substantial part of the overall teaching interaction. It is certainly important for a tutoring system to provide more than one tutoring strategy because if the system has only one tutoring strategy to use, it may not be able to adapt promptly to the changing cognitive needs of the student w21x. Researchers in tutoring systems are looking for the knowledge underpinning demonstrably effective ways of tutoring w18x. The pressing need is not just for knowledge on optimal ways of tutoring, but for knowledge about how to tutor in an effective way. As with overall tutoring, the same applies to the use of tutoring strategies. This paper argues that the focus of knowledge on the use of multiple tutoring )
Corresponding author. E-mail address: [email protected] ŽM.C. Angelides..
strategies is on when, why and which one to use. PEPE w31x is a planning framework that allows the designer of an intelligent tutoring system ŽITS. to incorporate and to encode a variety of tutoring strategies in an ITS. The main focus of PEPE, however, is on content planning as opposed to delivery planning. This separation refers to the distinction between the subject matter and the formats in which it can be presented. Content planning entails the generation, ordering and selection of content goals, and the monitoring of the execution of the content plan. Delivery planning involves choosing the actual activities and interactions that help the student achieve his goals. PEPE focuses on the content issues alone. Although PEPE sets out to accommodate the use of more than one tutoring strategies, it does not describe how the selection should proceed, nor does it describe how content planning can be used for tutoring strategy selection. Eon w17x is a set of authoring tools for ITSs designed to be used by human tutors and instruc-
0167-9236r00r$ - see front matter q 2000 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 7 - 9 2 3 6 Ž 0 0 . 0 0 0 6 1 - 0
32
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
tional designers to represent their own tutoring strategies. The primary focus of Eon is on the use of multiple tutoring strategies. This ‘‘meta-strategic’’ knowledge base is distinct from the tutoring strategy knowledge base. Eon provides tools for describing the subject domain as network-related topics, creating reusable interactive presentation screens, and authoring the procedures, which tell the tutoring system how and when to interact with the student. In terms of tutoring strategy selection, Eon uses a parameterised approach that allows authors of ITSs to define a number of tutoring strategy parameters, e.g. ‘degree of hinting’ and ‘degree of interruption’. Eon currently offers a general mechanism for representing tutoring strategies, but does not, in itself, contain any specific tutoring strategies. The success of the tutoring strategies used has, therefore, not been taken into account. Tutoring strategies have subsequently not been linked with tutorial goals. One of the most significant contributions in this area is the ‘‘dynamic method selection’’ in GTE w30x. GTE is a Generic Tutoring Environment for developing courseware, based upon a generic instructional task-based approach. The selection takes into account instructional tasks, instructional methods and instructional objects. Instructional tasks are activities accomplished by the tutor in a tutorial. Instructional methods are procedures to carry out tasks, which are taken as equivalent to tutoring strategies in the context of this paper. Instructional objects are means that methods employ. Instructional methods are chosen according to a numerical applicability value for each method. These values are calculated according to a list of conditions attached to the individual methods. While GTE has comprehensively represented the instructional processes from the tutor’s viewpoint, the learner’s perspective, which is also important in the tutoring process, has not been sufficiently addressed w6x. GTE’s selection could be improved if it could couple a student’s learning results with the selection mechanism. There is no separate knowledge base for the selection procedure in GTE. While having such knowledge embedded in the tutoring strategies themselves makes their context more explicit, it obscures the relationship between the various elements within the selection mechanism, thus lessening the transparency between the different aspects, which affect selection w17x.
These examples provide evidence that although many systems use more than one tutoring strategy and that they all incorporate some kind of support for tutoring strategy selection, they lack a common basis on which the rules governing the selection can be applied. The lack of common basis for tutoring strategy selection supports the fact that what is missing is a formalisation of such a process. The benefits of formalising tutoring strategy selection in tutoring systems are to ensure that tutoring systems consider each tutoring strategy according to Refs. w7,12,13,22x: what the system aims to teach; the individual student; the particular situation or application the tutoring strategy is to be used for; the previous failures and successes of a tutoring strategy in similar situations; what a human tutor would have done; and to ensure in general that tutoring strategy selection is provided for in an adaptive and yet coherent manner. This paper does not propose new research in computational tutoring strategies. Its purpose is to model the process of tutoring strategy selection in multimedia tutoring systems. Consequently, the paper is organised as follows. First, it constructs a model for tutoring strategy selection based on factors that influence human tutoring strategy selection, followed by a comparative evaluation of SONATA and ARISTOTLE, two multimedia tutoring systems that use multiple tutoring strategies, but only ARISTOTLE incorporates the model for tutoring strategy selection in its architecture.
2. An empirical model for tutoring strategy selection Characteristics that constitute satisfactory tutoring strategy selection have been studied by various researchers. When considering how to select a tutoring strategy for a given point in time, different tutoring strategies are needed for the different pieces of tutorial material within the domain, and for different cognitive levels among students w14x. This implies that tutoring strategy selection is needed when there is a change in the nature of tutorial material, a change in cognitive needs among different students who interact with the system, and a change in cognitive needs within the same student as he advances
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
along the tutorial discourse. Their discussion suggests that subject matter difference and cognitive difference should be taken into account when making tutoring strategy selection. One question all researchers try to answer is ‘‘when’’ specifically tutoring strategy selection should be triggered. One answer is when a tutoring strategy appears to have failed. A student’s test results on the tutorial material may reflect a degree of tutoring strategy success or failure w11x. Test scores may also be used to decide whether the student should revise the same material before moving onto new material, which, in turn, determine the scope of application of the next tutoring strategy, that is, whether it is to be used for teaching Žand re-teaching. or testing what has been taught w32x. This characteristic implies that tutoring strategy selection is needed to check whether the tutoring strategy selected is appropriate when the test scores of a student show extreme patterns, i.e. perfect or poor. The scope of application also affects tutoring strategy selection. It is obvious that checks should be made when test scores are poor. Checks should also be made when the student is showing excellent progress, to make sure that the tutoring strategies are working properly, and not that the student finds the material too easy. The student’s own tutoring strategy preference should be taken into account w9x. Research in intrinsic motivation has consistently identified a sense of personal control as an important motivational factor in learning w16,19x. Tutoring strategy selection is needed when motivating a student to learn. The choice of tutoring strategy may involve the student expressing his own preference to give a sense of learner’s control. However, the learner may not always know what is best for himself w24x. The system should, therefore, be able to provide an environment, in which the student has the impression of control, with the amount of control carefully monitored by the tutor. The nature of the tutorial material affects the tutorial environment in which tutorial is to take place. The tutorial environment in turn governs the type of communication channel between the system and the student. Different tutoring strategies that are used to teach different material within the same domain require a different tutorial environment. For example, tutoring procedural knowledge may call for
33
the animated display to demonstrate each process to the student, whereas tutoring factual knowledge may only need a textual display. Tutoring strategy selection is, therefore, needed when there is a change in the presentation method for a different piece of tutorial material. The choice if which tutoring strategy to use may depend on which tutorial environment are required since these environments facilitate the delivery of the tutoring strategies. Tutoring systems should be viewed as an automated tool to assist the human tutor, and not to replace them because human tutors are considered to be an effective but expensive resource w3x. Tutoring strategy selection should, therefore, allow for human tutor intervention in the process to ensure system efficiency and to explain to the student about a change of tutoring strategy to lessen confusion when necessary. The process of tutoring strategy selection in a model should include all the above characteristics w29x. The model for tutoring strategy selection consists of the following aspects: tutorial goal, use of tutoring strategy, domain knowledge, user knowledge, tutor knowledge, tutoring strategy delivery environment and external human tutor support. These aspects are shown in Table 1. Individually, these aspects are advances about tutoring on the macro level, each contributing to the
Table 1 The model for tutoring strategy selection Tutoring strategy selection Aspects Tutorial goal Use of tutoring strategy Domain knowledge User knowledge
Tutor knowledge
Tutoring strategy delivery environment External human tutor support
Definitives Teachrre-teach, Testrre-test Type context Advancement level: novicer advanced; Individual tutoring strategy preference: implicitr explicit Tutoringstrategies, Tutoring strategies general success rates Text, Graphics, Audiorvisual, Animation, Simulation, Combination of environments
34
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
structure of tutoring, the process of tutoring and learning. These aspects may appear to compete with, and, even occasionally, contradict each other in isolation. All aspects in the model for tutoring strategy selection should, therefore, be considered in synergy. The model for tutoring strategy selection is domainindependent. It begins its operation by addressing any detected tutoring strategy failure. 2.1. Tutorial goal Tutorial goals affect the tutoring strategy selection because they define the objectives of a tutorial session w19,20x. For example, if the tutorial goal of a session is to help the student understand how mouthto-mouth resuscitation saves lives, the objectives of the session can range from learning the facts of how the human respiratory system works, to the procedures for carrying out resuscitation. To match a tutoring strategy with a goal of the tutorial discourse means that the aim of the tutoring strategy must be the same as the aim of the objectives and in turn of the tutorial goal. Therefore, a tutorial goal can in fact be viewed as a tutoring strategy goal. A tutorial goal can be matched by more than one tutoring strategy goal, because a tutorial goal often has more than one objective. For example, to match the objective of learning about the respiratory system, tutoring strategies with goals aimed for tutoring factual events may be used. To match the objective of how to carry out resuscitation, tutoring strategies with goals aimed for tutoring procedural events may be used. The model for tutoring strategy selection thereby tries to match tutoring strategy goals inherent in tutoring strategies with the tutorial goal at a particular moment in time. 2.2. Use of tutoring strategy The model for tutoring strategy selection distinguishes two types of scope of application alongside with the objectives defined by the tutorial goal. They are teachingrre-teaching and testingrre-testing. Before deciding which tutoring strategy is to be used next, the model has to take into account what the next tutoring strategy is to be used for. Student–system interaction so far can suggest which tutoring strategies are more appropriate for the different scopes of application.
2.3. Domain knowledge The content of the tutorial material affects the design of instructional procedure w5x. For example, tutoring within the domain of science usually involves a lot of problem-solving. However, mathematical problems can often be solved by paper and pencil, whereas for the empirical sciences, such as chemistry, the student may need special devices and a sample of matter to study physical or chemical properties. The content of the tutorial material, therefore, affects the environment in which the tutorial is to be carried out. The type of tutorial material refers to whether the material to be imparted is procedural or declarative knowledge. 2.4. User knowledge Two areas of user knowledge are distinguished: advancement level, and the student’s individual preferences in tutoring strategies, either implicit or explicit. Advancement level in the model for tutoring strategy selection represents two stages of the student’s cognitive progress: novice and advanced. The student’s current level of knowledge and skills indicates the amount of tutorial material imparted that the student has learnt. The advancement level represents the level of expertise Žwhat he knows., and how far the particular expertise level the student is at Žhow much he knows.. In the model for tutoring strategy selection, a student’s preference is represented either as implicit or explicit. The act of asking the student results in the explicit preference. Different tutoring strategies’ prior successes with that individual student constitute the implicit preference. The system should know which tutoring strategy is suitable for that individual student at a given time, demonstrated implicitly by the prior success rates of different tutoring strategies with the student. It is important to note, however, there might be noises in this prior success deduction, e.g. repetition of the same piece of tutorial material to the student may lead to ‘reproductive’ learning due to memorising and rehearsing w20x. The number of times a piece of tutorial material has been represented to the student is, therefore, taken into account in the implicit preference. Implicit preference is considered first because the student himself may be
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
inconsistent in his own choices of tutoring strategy. In addition, the model for tutoring strategy selection also checks the student’s advancement level before the student’s explicit preference is asked for. For instance, it may be inappropriate for a beginner student to judge for himself what tutoring strategy is best at the start of the tutorial discourse. This means that when the implicit preference does not show a favourite tutoring strategy, the model for tutoring strategy will only ask for an explicit preference when the student has shown sufficient progress in learning the tutorial material, i.e. when he has progressed to advanced level. 2.5. Tutor knowledge Before deciding which tutoring strategy is to be used next, the model for tutoring strategy selection needs to know which tutoring strategies are offered in the system. Therefore, the availability of tutoring strategies is a necessity for tutoring strategy selection. Suitability of a tutoring strategy for an individual student in a particular situation is fundamental in tutoring strategy selection. Nevertheless, the model for tutoring strategy selection should also know which tutoring strategies are appropriate for students in general. For this purpose, the tutor knowledge aspect of the model for tutoring strategy selection provides information on the tutoring strategy general success rates with all the students. This information is obtained by taking into account all the interactions between the system and all the students who are registered to have had at least a session with the system. This information gives the overall success rates with all the students, as opposed to the successes with each individual student. The system ‘‘learns’’ what is a generally appropriate tutoring strategy for certain situations as it builds up its tutoring experience through increasing interactions with increasing number of students. 2.6. Tutoring strategy deliÕery enÕironment The tutoring strategy delivery environment provides the student with the scenario in which the tutorial material is introduced w10x. This aspect, together with the content of the tutorial material, af-
35
fects the model for tutoring strategy selection. If the content of the domain suggests the use of audio and video in the delivery environment, then the tutoring strategy to be used has to be able to tutor with audio and video. 2.7. External human tutor support One of the situations where human tutor could step in is when the tutoring strategy selected by the system fails. The student’s control over the tutorial session only increases as he advances, human intervention may be more frequent for novices and slower learners. Once the human tutor is called, he can look into the student model of the system, as well as eliciting information from the student directly on how the student feels about the tutorial session, in particular, how he feels about the tutoring strategies used. The investigation into the student model would give the human tutor an insight into the system’s inferences on the successes and failures of various tutoring strategies. The human tutor can query the student directly to see if there is any mismatch between the system’s deductions and the student’s own judgement. The human tutor can then decide which tutoring strategy is to be used next, based on his own experience and knowledge about the student. If the human tutor, after consultation with the student, finds that there is a discrepancy between the system’s perception of the tutorial discourse and reality, he could then also amend or add new data on the system. This section has described the individual aspects of the model for tutoring strategy selection, as well as how they complement each other. Section 3 describes the rules employed by the model during the selection by the model.
3. An algorithm for the process of tutoring strategy selection Embedded in the model for tutoring strategy selection are certain rules to ensure consistent decisions. These rules serve to govern tutoring strategy weights and the selection. There are four types of rules in the model for tutoring strategy selection. They are suitability rules, proficiency rules, block-
36
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
age rules and sequence rules. Suitability and proficiency rules both measure a tutoring strategy according to the user knowledge. Suitability rules also consider how suitable a tutoring strategy in terms of the scope of application. Blockage rules assess a tutoring strategy according to its previous successes and failures in similar situations. Sequence rules measure a tutoring strategy according to the order of the use of tutoring strategies. Suitability rules alter the priorities of tutoring strategies based on their suitability for the current student and the scope of
application. For instance, tutoring strategies are favoured when they match the student’s individual style. Suitability rules favour tutoring strategies that have had successes previously with the student. Proficiency rules govern tutoring strategy selection according to the student’s proficiency in the domain, according to his advancement level. An example of such rules is increasing the student’s control when he has reached advanced level. Blockage rules lower the priorities of those tutoring strategies that have failed previously. For example, if a tutoring strategy has
Fig. 1. The selection process in the model for tutoring strategy selection.
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
appeared to have failed, blockage rules will lower its priority in the subsequent selection. Sequence rules adjust priorities to favour tutoring strategies that are similar to the one previously selected. Sequence rules are consistent with the idea of the model that the use of tutoring strategies should be coherent. There are two exceptions to the sequence rules. One is when similar tutoring strategies have been used and effective learning still has not appeared to have taken place. Then, a completely different tutoring strategy may have a higher priority compared to the ones similar to the previous ones used. The other one is when the human tutor decides the sequence rules are to be broken, for example, when a student has perfect progress with one particular tutoring strategy for a considerable length of time. This is to check whether the student still finds it challenging to learn with that particular tutoring strategy to avoid boredom. These rules are interpreted in the sequence and alternatives the model for tutoring strategy selection goes through when selecting a tutoring strategy. This sequence of the selection operation, as shown in Fig. 1, summarises the discussion of the various aspects in Section 2. The model for tutoring strategy selection has been developed as an ‘add-on’ component to the architecture of any full-scale, knowledge-based tutoring system which uses multiple tutoring strategies. Such a system contains separate knowledge-based representations of the subject matter, the student, the tutorial goals and strategies and separate knowledge processes for each one of the three knowledge-based representations. Section 4 demonstrates the benefits of the model for tutoring strategy selection through the comparative evaluation of two multimedia tutoring systems, ARISTOTLE and SONATA. Although they both use several tutoring strategies, only ARISTOTLE incorporates the model for tutoring strategy selection.
4. Comparing
ARISTOTLE
with
SONATA
The functions and operability of the model for tutoring strategy selection are demonstrated by the deployment of the model in ARISTOTLE w29x. ARISTOTLE is a multimedia tutoring system for zoology and
37
was originally implemented using a method for developing interactive multimedia tutoring systems w1x. SONATA is a multimedia tutoring system for music theory and serves the purpose of demonstrating the incorporation of multiple tutoring strategies w4x. This section starts with a description of SONATA, and then of ARISTOTLE. The differences in the use of tutoring strategies between the two systems are then presented. Finally, the benefits of formalising tutoring strategy selection, as illustrated by ARISTOTLE are discussed. 4.1.
SONATA
SONATA tutors knowledge covering the syllabus of the grade 1 theory of music examination set by the Associated Board of the Royal School of Music. The domain is divided into four areas. These areas are basic terms used in music theory, keys and tonic triads, rhythms, and Italian terms and musical signs. Tutoring in each area involves SONATA initially teaching the student about the area, followed by the student applying his acquired knowledge through 12 successive problem-solving activities, before moving onto a new area. SONATA was developed using Hypercard. It uses a semantic network of ‘cards’ for storing information. These cards depict information nodes. These nodes are linked via information links. These nodes can be linked sequentially, hierarchically or non-hierarchically. Multimedia material such as audio and video are not represented directly in the card structure. Instead, they are annotated to a card via an ‘annotation’ link. The name of a link represents the semantic content of the annotation. In SONATA, the domain, tutoring and student knowledge are kept together in a ‘stack’ of cards. A stack is a collection of related cards of information, which logically belong together. Links are installed to integrate the stored knowledge, both within and between individual knowledge model. Knowledge in SONATA’s domain model includes the domain knowledge and a multimedia base. Domain knowledge contains information on the four aforementioned areas in music theory. The multimedia base contains information about the annotation of multimedia material to the cards, as well as the content of the material itself.
38
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
Knowledge in the student model contains the student overlay knowledge. Student overlay knowledge represents the current level of understanding of the domain. It contains information about the student’s knowledge on the four areas of the domain. This is represented as overlay scores for the problem-solving activities the student has to engage in during the tutorial discourse. The value of an overlay score is determined by whether the student’s answer is correct, and the number of times the student has attempted the same problem-solving activity. The tutoring strategy used for the problem-solving activity that yielded a score is also represented in the student overlay knowledge. Hence, the student’s prior success with the different tutoring strategies can be deduced from the student overlay knowledge. The student’s advancement level Žnovice or advanced. is deduced from the amount of tutorial material the student has gone through. Knowledge in SONATA’s tutor model includes tutorial goals, and the different tutoring strategies SONATA uses. Tutorial goals determines what is to be taught in a tutorial session. SONATA employs both multimedia-based and non-multimedia-based tutor-
ing strategies. Non-multimedia-based tutoring strategies are used when tutoring material, which is not possible to be taught through the use of multimedia, such as the notation of a note in terms of its position on the staff in music scores. Fig. 2 illustrates SONATA using a multimedia-based tutoring strategy. Due to the fact that SONATA does not incorporate a model for tutoring strategy selection, its use of tutoring strategy can only be explained in terms of a description of the process of tutorial interaction. The same tutoring strategy is always used when an area in the domain is introduced for the first time. This tutoring strategy is learning through exploration. The student then needs to apply his knowledge from learning through exploration in a series of problem-solving activities in the same area. The student is only allowed to explore the tutorial material in an area for a certain number of times. This is to prevent the student from ‘dwelling’ in a scenario for too long without having to apply the knowledge he acquired from his exploration experience. After a new area has been introduced, the student then has to go through 12 successive problem-solving activities in that area, before moving onto the
Fig. 2. SONATA using a multimedia-based tutoring strategy.
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
next area. The tutoring strategies to be used depend on how successful each tutoring strategy has been with that student. A tutoring strategy is regarded as successful if the student provides the correct solution in a problem-solving activity which employs that tutoring strategy. When the data recorded for the student–system interaction is still small, this success condition immediately affects the use of the next tutoring strategy. For example, if Strategy X fails in problem-solving activity three of area one, and that Strategy Y is successful as the second tutoring strategy used for the same activity, then the first strategy to be used in problem-solving activity four in area one is Strategy Y. If Strategy Y fails, SONATA will use Strategy Z and tutor the same activity again. Strategy X is used first because it was successful in the previous activity. Strategy Z is preferred to Strategy X as a second choice because the former has not been attempted in the last activity whereas the latter failed. When the student–system interaction has built up, information on tutoring strategy successes will be gathered and interrupted on an overall basis, rather than on problem by problem basis. Fig. 3 shows a sample student record information on the student overlay knowledge, and the results of the use of the different tutoring strategies.
4.2.
39
ARISTOTLE
ARISTOTLE tutors a basic knowledge of zoology. The entities of interest to the system are animals. Three animals were chosen and were grouped into classes. Animals were divided into vertebrates and invertebrates. Vertebrates were further divided into mammals and reptiles. Arthropods were chosen as a type of invertebrate. Students learn about the physical properties of the entities Ždescription., such as the different body parts they have, what the entities do Ževents., such as hunting, and the activities that constitute the events Žactions., such as observing prey. ARISTOTLE was developed using Asymetrix Multimedia ToolBook. It uses multimedia frames Žmframes. to model the content of audio and video and to integrate this information with information about the entities of interest to the system. Two types of m-frames are used in ARISTOTLE: syntactic m-frames ŽSYMs. and semantic m-frames ŽSEMs.. SYMs contain information about the syntactic co-ordinates of objects and the spatial relationships among objects within a frame of a video clip. SEMs contain information which represents the semantic content of a sequence of video or audio frames. A SEM also
Fig. 3. A sample student record in SONATA.
40
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
provides semantic information that is not related to a media segment within ARISTOTLE. In this way, all information related to an entity of interest is kept together. Knowledge in ARISTOTLE’s domain model consists of a multimedia resource, and domain knowledge. The multimedia resource contains information on the co-ordinates and spatial relationships among objects depicted within a video frame of a clip, together with the associated raw audio and video frames. Domain knowledge contains information about the entities of interest. ARISTOTLE’s student model includes the student overlay knowledge. It contains information about how strongly the student has been judged to know the acquired knowledge, the tutoring strategy, which yielded that knowledge acquisition, and any shots, i.e. any sets of continuous video andror audio frames, that were used with that tutoring strategy.
The knowledge on student’s misconceptions contains information about the seriousness of the misconception. The strength of acquired knowledge are determined by the number of attempts made in solving the problem, the number of times assistance is requested, and how well the student has provided the answer, i.e. whether there are any spelling mistakes, and whether there are too many words provided in the answer. The tutor model contains tutorial goals and tutoring strategies. Tutorial goals determine what the student is to be taught. Two main types of tutoring strategies are used in ARISTOTLE: non-multimediabased tutoring strategies and multimedia-based tutoring strategies. Non-multimedia-based tutoring strategies are used to tutor facts about an animal or animal class that cannot be taught through the use of multimedia. Fig. 4 shows ARISTOTLE using a multimediabased multiple choice tutoring strategy.
Fig. 4. ARISTOTLE using a multimedia-based tutoring strategy.
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
Tutoring strategy selection in ARISTOTLE begins with retrieving from the tutor model all the appropriate tutoring strategies available for the sub-goals specified by the tutorial goal. The resulting tutoring strategies are the tutoring strategies associated with the goal. The model for tutoring strategy selection then refers to ARISTOTLE domain model to determine the tutoring strategy delivery environment required, either multimedia based or non-multimedia-based, and match it with the relevant tutoring strategies. The model for tutoring strategy selection then determines what the next tutoring strategy is to be used for, i.e. to teach or to test, by referring to the student–system interaction so far. The system goes into testingrre-testing mode after teaching has been completed. The model for tutoring strategy selection then retrieves the weights for teaching and testing attached to each tutoring strategy in the student
41
model in order to pick the tutoring strategy with the highest weight. If the highest weight is shared by several tutoring strategies, then the general success rates calculated over all previous student interactions and stored in the tutor model are checked, to pick the one with the highest weight among these tutoring strategies. If the highest general success rate is still shared by several tutoring strategies, then the model for tutoring strategy selection checks the student model to determine the student’s advancement level. If the student is novice, the model for tutoring strategy selection prompts the human tutor to select. If the student is advanced, the model for tutoring strategy selection prompts the student to make a selection. The selection in either case will be from among the relevant tutoring strategies that share the highest general success rate. The model for tutoring strategy
Fig. 5. ARISTOTLE prompts for the human to pick a tutoring strategy.
42
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
selection offers the human tutor the choice of viewing the student model. Fig. 5 illustrates ARISTOTLE prompting the human tutor to pick a tutoring strategy. In those cases where the advanced student has made a choice and the tutoring strategy fails, the human tutor is called to make a choice. In addition, when a tutoring strategy has worked ‘too well’ for the student for a number of consecutive tutorial goals, the model for tutoring strategy selection calls for the human tutor to determine whether a different tutoring strategy should be used. The human tutor can also amendrenter new data to the student model by clicking on the view student model button on that screen. 4.3. Benefits of the formalisation of tutoring strategy selection Formalising tutoring strategy selection ensures that the tutoring strategy selected matches what the system teaches. The model for tutoring strategy selection matches the tutor strategies in ARISTOTLE’s tutor model to the relevant knowledge in the domain model via the tutorial goal. SONATA’s tutor model also includes knowledge on tutorial goals. However, without the model for tutoring strategy selection, the tutorial goal is not matched with the tutoring strategies. Formalising tutoring strategy selection ensures that the tutoring strategy selected matches the particular student. The model for tutoring strategy selection utilises ARISTOTLE’s knowledge on student’s advancement level to match tutoring strategies with the cognitive needs of students at different levels. It also allows the student, when appropriate, to express his preference in selection. SONATA distinguishes the student’s advancement level as well. However, it does not match the advancement level with the use of tutoring strategies. SONATA does not allow the student to intervene with its use of tutoring strategies. If the student has difficulty in a problem-solving activity with one tutoring strategy, SONATA will tutor the same material with another tutoring strategy, hoping that the student will grasp the point. The system may go round and round for a long period of time without achieving its tutorial purpose, and the student frustrated.
Ensuring that tutoring strategies are selected according to the particular scope of application is another benefit arising from tutoring strategy selection formalisation. ARISTOTLE’s tutor model includes knowledge on the scope of application of a tutoring strategy that distinguishes between teachingrreteaching and testingrre-testing. ARISTOTLE’s tutor model refers to the student model to establish if all teaching should be done before any testing, if material needs re-teaching before any testing is done, and if the student requires assistance from ARISTOTLE before testing commences. ARISTOTLE records information on student performance while being taught or tested in order to calculate the student’s advancement level. SONATA addresses the difference between learning through exploration and the problem-solving activities. However, without the model for tutoring strategy selection, it currently does not relate the different scopes of Žre.teaching and Žre.testing with the use of tutoring strategies. Formalisation of tutoring strategy selection ensures that tutoring strategy selection invokes human tutor intervention when necessary. When the human tutor is called to make a decision, ARISTOTLE presents the tutoring strategies from which the system failed to make a choice. The human tutor is given the choice to examine the student model before making a choice. The conditions under which the call for the human tutor was necessary are recorded along with the decision that the human tutor has made so that if these conditions arise again the system can take the same decision itself. SONATA allows neither the human tutor nor the student to intervene with the use of tutoring strategies. Table 2 summarises the benefits of the model for tutoring strategy selection in multimedia tutoring systems in the comparative evaluation of ARISTOTLE and SONATA. The evaluation is carried out according to the criteria based on the work of Siemer and Angelides w26x, who propose a system-independent, comprehensive evaluation method that can be applied to the evaluation on any ITSs. Their evaluation is based on desirable behaviour properties expected from a tutor in any tutorial environment. Both ARISTOTLE and SONATA are both multimedia tutoring systems. The only architectural difference between them is the model for tutoring strategy selection, which is only incorporated in ARISTOTLE, and not in SONATA. The
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45 Table 2 A summary of the benefits of the model for tutoring strategy selection demonstrated through the comparative evaluation of
43
ARISTOTLE
and
SONATA
Evaluation criteria ŽSiemer and Angelides w26x. Tutorial goals clearly defined and adopt appropriate tutoring strategy accordingly
Issues addressed by SONATA
ARISTOTLE
Tutoring goals defined but not matched with tutoring strategies
U
Tutoring strategies tailored to the cognitive needs of students at different levels Student actively take part in the tutoring process System intervention when the student appears to be in difficulty
Past learning experiences and learning preferences recorded and adapted to accordingly System monitoring changes proposed by the student and comment if they seem to be unwise
U U U
U
summary of the comparative evaluation in Table 2 shows that without the model for tutoring strategy selection, SONATA does not address all the issues and the subsequent implications of these issues in multimedia tutoring systems, therefore. undermining the potential pedagogical effects of multimedia tutoring systems.
5. Concluding discussion Several issues arise from the research described in this paper. These issues relate to future work on the model for tutoring strategy selection, as well as the research in the deepening of the understanding of tutoring and learning. The derived model attaches weights to the different tutoring strategies to determine the order in which the tutoring strategies are picked. The weighting method is governed by sequence, suitability, proficiency and blockage rules. Further research into
U U
Improvements contributed by the model for tutoring strategy selection Matching tutoring strategies with tutorial goal. Tutoring strategies selected subsequently match the subgoals defined by the tutorial goal. Matching tutoring strategies with the student’s advancement level The student is invited to take part in selecting a tutoring strategy to suit his own learning style Differentiation in the scope of applications requires the system to provide assistance, re-teaching and re-testing when a student appears to have difficulty
The system is required to monitor the changes induced by the explicit preferences suggested by the student. The human tutor is called for when there is a discrepancy between the system’s deduction and the student’s own judgement.
how exact the indices used in the weighting are, may result in a finer grained prioritisation, which, in turn, will enhance the functionality of the overall architecture. Quantitative studies such as multiple regression analysis may help to study indices used to assess the veracity of knowledge of the student w23x. The derived model is able to cope with an infinite number of tutoring strategies made available by the system. The model for tutoring strategy selection ensures that more than just two tutoring strategies are used in a system. The collection of tutoring strategies should be as extensive as possible to increase the system’s adaptiveness. The more tutoring strategies available the greater the benefits of tutoring strategy selection. A feature that could improve on the use of multiple tutoring strategies in tutoring systems is the system’s ability to allow the human tutor to build more tutoring strategies into the system themselves, by means of an authoring tool w15x or from ‘parts’ available in the system. The derived model allows the student to pick a tutoring strategy himself in order to increase motivation and to express his preference at times when the
44
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
system could not decide which tutoring strategy to use. However, the proposed architecture does not detect when the student needs motivating and how this freedom of expression of preference may affect him from a cognitive perspective. Asking the student to intervene tutoring strategy selection involves the student’s self-regulation in learning, e.g. monitoring the effects of the tutoring strategies, adjusting his study approach in order to reach his learning goals w27x. Students differ in the degree to which they operate these regulatory measures themselves. Many may expect the tutor to control these measures for them. Further research on the student’s intervention of tutoring strategy selection is needed such that this self-regulated approach in tutoring can be exercised with minimal discrepancy between the student’s expectations and experience, to prevent the student from disorientation w2x. The human tutor plays an important part in the efficient operation of the model for tutoring strategy selection in the proposed architecture. The role of the human tutor in computer-based tutoring interaction has been acknowledged by many, but addressed by only a few w28x. Research in the human tutor intervention in computer-based tutoring is not only scarce, almost all of them concentrate on the benefits to the student w25,28x. By studying the behaviour and learning processes of students using tutoring systems that focus on tutoring strategy selection, human tutors may also benefit from interacting with such systems, by being ‘forced’ to reflect on their own selection of tutoring strategies. The difficulties in analysing the use of tutoring strategy in professional tutors have been emphasised w8x. Involving human tutor formally in a computer-based tutorial will open a gateway to the understanding of the expertise in human tutors. This line of research will provide valuable information about learning episodes, and about the process of knowledge elicitation by expert human tutors.
w3x
w4x
w5x
w6x
w7x
w8x
w9x
w10x
w11x
w12x w13x
w14x
w15x w16x
w17x
References w1x H. Agius, M.C. Angelides, Integrating logical video and audio segments with content-related information in instructional multimedia systems, Information and Software Technology 39 Ž10. Ž1997. 679–694. w2x G.S. Akerlind, A.C. Trevitt, Enhancing self-directed learning
w18x
w19x w20x
through educational technology: when students resist the change, Innovation in Education and Training International 36 Ž2. Ž1999. 96–105. J.R. Anderson, C.F. Boyle, A.T. Corbett, M.W. Lewis, Cognitive modelling and intelligent tutoring, in: W.J. Clancey, E. Soloway ŽEds.., Artificial Intelligence and Learning Environment, Elsevier, Amsterdam, 1990, pp. 7–49. M.C. Angelides, A.K.Y. Tong, Implementing multiple tutoring strategies in an intelligent tutoring system for music learning, Journal of Information Technology 10 Ž3. Ž1995. 52–62. S. Dijkstra, The integration of instructional systems design models and constructive design principles, Instructional Science 25 Ž1. Ž1997. 1–13. J. Elen, Automating ID: the impact of theoretical knowledge bases and reference systems, Instructional Science 26 Ž3–4. Ž1998. 281–297. H. Ellington, Educational innovation — where are we now? A critical review of recent developments, Innovation and Learning in Education 1 Ž1. Ž1995. 15–20. M. Elshout-Mohr, B. van Hont-Wolters, H. Broekkamp, Mapping situations in classroom and research: eight types of instructional-learning episodes, Learning and Instruction 9 Ž1. Ž1999. 57–75. C. Frasson, E. Aimeur, A comparison of three learning strategies in intelligent tutoring systems, Journal of Educational Computing Research 14 Ž4. Ž1996. 371–383. H.M. Halff, Curriculum and instruction in automated tutors, in: M.C. Polson, J.J. Richardson ŽEds.., Foundations of Intelligent Tutoring Systems, Erlbaum, New Jersey, 1988, pp. 79–108. M.J. Hannafin, S.M. Land, The foundations and assumptions of technology-enhanced student-centred learning environments, Instructional Science 25 Ž3. Ž1997. 167–202. D. Jonassen, Computers in Classroom: Mindtools for Critical Thinking, Englewood Cliffs, NJ Ž1996.. S. Kennewell, Computer modelling and the curriculum — integrated or loosely attached? Education and Information Technologies 2 Ž3. Ž1997. 247–260. T. Khan, Y.J. Yip, pedagogical principles on case-based CAL, Journal of Computer Assisted Learning 12 Ž1996. 172–192. N. Major, Modelling teaching strategies, Journal of Artificial Intelligence in Education 6 Ž2–3. Ž1995. 117–152. ID2 Research Group 1998, M.D. Merrill, ID expert-intelligent computer-based multimedia interactive instructional development and delivery system, Instructional Science 26 Ž3–4. Ž1998. 243–262. T. Murray, Authoring instructional expertise in knowledge based tutors, Instructional Science 26 Ž3–4. Ž1998. 263–280. S. Naidu, M. Oliver, Critical incident-based computer supported collaborative learning, Instructional Science 27 Ž5. Ž1999. 329–354. A.C. Orstein, Strategies For Effective Teaching, Dubuque, Brown and Benchmark, 1995. C.M. Reigeluth, Instructional design: what is it and why is it? in: C.M. Reigeluth ŽEd.., Instructional Design Theories And
A.K.Y. Tong, M.C. Angelidesr Decision Support Systems 29 (2000) 31–45
w21x w22x
w23x
w24x
w25x
w26x
w27x
w28x w29x
w30x
w31x
w32x
Models: An Overview of Their Current Status, Erlbaum, New Jersey, 1983, pp. 3–36. E. Sadler-Smith, R. Riding, Cognitive style and instructional preferences, Instructional Science 27 Ž5. Ž1999. 355–371. T.J. Shuell, Designing instructional computing systems for meaningful learning, in: M. Jones, P.H. Winne ŽEds.., Adaptive Learning Environment: Foundations and Frontier, Springer-Verlag, Berlin, 1992, pp. 19–54. V.J. Shute, J.W. Regian, Principles for evaluating intelligent tutoring systems, Journal of Artificial Intelligence and Education 4 Ž2–3. Ž1993. 245–271. H.S. Shyu, S.W. Brown, Learner control: the effects on learning a procedural task during computer-based videodisc instruction, International Journal of Instructional Media 22 Ž3. Ž1995. 217–231. J. Siemer, INTUITION — applying intelligent tutoring to gaming simulation, Journal of Computing and Information Technology 3 Ž1. Ž1995. 35–43. J. Siemer, M.C. Angelides, A comprehensive method for the evaluation of complete intelligent tutoring systems, Decision Support Systems 22 Ž1. Ž1998. 85–102. A. Slaats, H.G. Lodewijks, J.M. van der Sanden, Learning styles in secondary vocational education disciplinary differences, Learning and Instruction 9 Ž5. Ž1999. 475–492. J.M. Spector, The role of epistemology in instructional design, Instructional Science 26 Ž3–4. Ž1998. 193–203. A.K.Y. Tong, M.C. Angelides, Formalising tutoring strategy selection in multimedia tutoring systems, Information and Software Technology 41 Ž2. Ž1999. 67–90. K. Van Marcke, GTE: an epistemological approach to instructional modelling, Instructional Science 26 Ž3–4. Ž1998. 147– 191. B. Wasson, Facilitating dynamic pedagogical decision making: PEPE and GTE, Instructional Science 26 Ž3–4. Ž1998. 299–316. B.P. Woolf, W. Hall, Multimedia pedagogues — interactive
45
systems for teaching and learning, IEEE Computing Ž1995. 74–80, ŽMay.. Marios C. Angelides is a Professor of Computing in the Department of Information Systems and Computing at Brunel University. He holds a BSc in Computing and a PhD in Information Systems, both from the London School of Economics where he was a lecturer in information systems for several years. He has over 10 years of research experience in Multimedia Information Systems and Superhighways where he has published extensively in journal and book format. His most recent books are Multimedia Information Systems ŽKluwer Academic Publishers, 1997. and Multimedia Information Superhighways Žforthcoming.. He is a member of the management committee of the UK Multimedia Special Interest Group, the British Computer Society, the IEEE Computer Society, the ACM, the Information Resources Management Association, the UK Academy for Information Systems and the Engineering Professors’ Council. Amelia K.Y. Tong is a Lecturer in Computing in the Department of Information Systems and Computing at Brunel University. She holds a BSc in Computing, an MSc and a PhD in Information Systems, all from the London School of Economics. Her research interests are in the areas of Multimedia Information Systems and Intelligent Tutoring Systems. She is a member of the IEEE Computer Society, the ACM, and the British Computer Society.