- Email: [email protected]
A meta-cognitive tool for courseware development, maintenance, and reuse
Computers & Education 48 (2007) 548–566 www.elsevier.com/locate/compedu
A meta-cognitive tool for courseware development, maintenance, and reuse John W. CoVey
¤
The Department of Computer Science and The Institute for Human and Machine Cognition, The University of West Florida, 40 S. Alcaniz Street, Pensacola, FL 32514, USA Received 12 April 2003; accepted 29 March 2005
Abstract Novak and Iuli [Novak, J. D. & Iuli, R. J. (1991). The use of meta-cognitive tools to facilitate knowledge production. In A paper presented at the fourth Florida AI research symposium (FLAIRS ’91), Pensacola Beach, FL, May, 1991.] discuss the use of Concept Maps as meta-cognitive tools that help people to think about thinking. This work describes a network-enabled meta-cognitive tool based upon extensions to Concept Maps that can be used to help course designers visualize and plan course organizations. This tool permits the user to create a novel type of course description based on the idea of an advance organizer. Course arrangements created by this method do not have the arbitrary linear sequences of topics typically found in traditional courses at the college level. The tool is part of an environment that is designed to foster meaningful learning and reuse of course design and instructional content. This paper presents a description of this software tool, an approach to the creation of a course depiction from a Concept Map, an example of a course that was developed iteratively using the tool, and a discussion of the ways that the tool fosters course and content reuse. © 2005 Elsevier Ltd. All rights reserved. Keywords: Authoring tools and methods; Interactive learning environments; Multimedia/hypermedia systems; Metacognitive tools
¤
Tel.: +1 850 202 4414; fax: +1 850 202 4440. E-mail address: [email protected].
0360-1315/$ - see front matter © 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.compedu.2005.03.008
J.W. CoVey / Computers & Education 48 (2007) 548–566
549
1. Introduction The Concept Map (Novak & Gowin, 1984) is a well-known tool of educational psychology that aVords an explicit representation of what a person knows about a knowledge domain. Concept Maps are comprised of concepts that are, according to Novak and Gowin, an individual’s perceived regularities in events or objects. Novak and Gowin describe all knowledge as being comprised of concepts and propositions, the basic components of Concept Maps. Concept Maps oVer a Xexible framework for eliciting, representing, and communicating an individual’s personally constructed understandings of a knowledge domain. Although Concept Maps were not originally intended to represent procedural, step-by-step knowledge or items that are arranged in a sequence, the links in Concept Maps are directed arcs that could designate precedence relationships among the concepts. If the concepts represented topics in a course, then the Concept Map would be an excellent starting representation for the conceptualization of course content and prerequisite arrangements among topics. Such a graph-like representation of topics in a course would not have an arbitrary linearization of the presentation that is typical of traditional and on-line courses. A course presented in this fashion would increase the learner’s choice regarding what topic to pursue next. The purpose of this paper is to describe a software tool that provides capabilities to create non-linear course arrangements based upon Concept Maps. The remaining sections of this paper contain a description of LEO, a Learning Environment Organizer (CoVey, 2000; CoVey & Cañas, 2003) that utilizes a Concept Map-like rendering of concepts that correspond to topics, and links that indicate both conceptual and prerequisite relationships among the topics. The course designer uses this tool to create alternative conceptualizations of the organization of a course. As such, the tool serves as a meta-cognitive device that helps the course designer to think about the course design. The tool aVords the designer many additional capabilities including the ability to create links from the topics to instructional content, to designate assignments, criteria for completion of a topic, etc. This approach leads to the creation of a Concept Map-like representation of the course description, with some concepts corresponding to the topics in the course, and others providing elaboration of the relationships among topics. This paper contains a description of the software tool that supports this method of course development, an example of the utilization of this tool to convert a Concept Map into a course description, and an elaboration of how the tool fosters course component and instructional content reuse.
2. Literature pertaining to meta-cognitive tools In the broadest sense, meta-cognition is thinking about one’s thought processes. The literature suggests that the traditional application of meta-cognitive strategies has been in eVorts to help students gain awareness of how they approach reading and writing. Most of the literature addresses general-purpose tools for students and others. However, a well-conceived meta-cognitive tool could play a valuable role in helping instructors to decide what is important about the content of a course, and any necessary sequencing issues relative to mastery of the content. The remainder of this section describes methods and tools that might potentially be of use to instructors who are developing courses.
550
J.W. CoVey / Computers & Education 48 (2007) 548–566
Several graphical representations have been created that are meant to foster reXection and meta-cognition. These include Concept Maps (Novak & Gowin, 1984), Semantic Networks (Fisher, 1990), Mind Maps (Buzan & Buzan, 1996), and Knowledge Maps (Lambiotte, Skaggs, & Dansereau, 1993) or Node-Link Maps (Blankenship & Dansereau, 2000). These representations and their accompanying tools help users to externalize and understand what they know about a knowledge domain. Although all these are graphical representations comprised of conceptual knowledge on nodes and links, they diVer in a variety of ways. For instance, in some cases, the linking lines drawn between concepts are labeled, in others they are not. If linking phrases are labeled, the set of labels may be pre-speciWed or unconstrained. Concepts may be comprised of single words, short phrases, sentences, or paragraphs. Some of the tools available for these representations allow association of other resources with the concepts that are represented. SemNet (Fisher, 2000) permits creation of Fisher’s Semantic Networks. Several tools such as Mind Manager (2005) and Visual Mind (2005) facilitate creation of Mind Maps. However, none of these tools is explicitly geared to the creation of course organizers. Novak and Gowin (1984) describe Concept Maps and Vee diagrams as meta-cognitive tools. The Concept Map is the least constrained of the various meta-cognitive tools enumerated above, and provides excellent capabilities for both students and teachers to make explicit what they know about a knowledge domain. Novak and Iuli (1991) discuss the use of Concept Maps as meta-cognitive tools that help people to think about thinking. They suggest that Concept Maps may be used to facilitate learning, to organize instruction, and to help users organize their knowledge in ways that might make that knowledge more generalizable. CmapTools (Cañas et al., 1998, 2004) provide substantial support for the creation and representation of knowledge regarding a knowledge domain. CMapTools are based upon Concept Maps. These tools allow for the development and perusal of hierarchically organized Concept Maps. Additionally, accompanying explanatory resources in any electronic medium may be attached to concepts and accessed through the Concept Maps. In the broader context, Kasowitz (2000) describes a wide range of tools that she states facilitate instructional design, including advisory systems, information management systems, electronic performance support systems, and authoring tools for computer-based instruction. Her article is a general categorization that illustrates the range of tools that support development of instructional materials. Nkambou, Frasson, Gauthier, and Rouane (2001) describe their Curriculum Representation and Acquisition Model (CREAM) which supports creation and organization of the curriculum by domain, pedagogical approach, and didactic aspects of the teaching. Ritter and Blessing (1998) describe their Visual Translator, a tool designed to support development of large-scale educational systems. Other types of meta-cognitive tools have been described in the literature as well. Hedberg (1997) discusses meta-cognitive tools that he describes as being embedded in an “information landscape”. These tools are designed to assist students in their investigation of scientiWc topics and with the writing process. In his book, Hyerle (2000) examines the use of visual tools such as task-speciWc organizers, and thinking-process maps that are similar to Concept Maps and Mind Maps. He describes ways that various categories of users use these tools and the beneWcial eVects such tools have on reading and writing. While not explicitly meant as meta-cognitive tools, hypermedia authoring suites provide support for courseware design. Web development tools permit cataloguing of graphics, hierarchical directory structures to organize pages and the inclusion of links to topical materials. They also provide capabilities
J.W. CoVey / Computers & Education 48 (2007) 548–566
551
to create links to databases, to accumulate site usage statistics, etc. All of these tools provide some type of organizational factor and support for the tracking of materials that are used in courses. Programs such as Macromedia Authorware™ (Macromedia, 2003) are typical of this class of tool and are quite mature. However, authoring tools generally lack an explicit pedagogical underpinning. The idea of instructors creating representations in a meta-cognitive tool that fosters reXection on the topics that they teach and the organization of their course, and then having the representation created in the tool support the deployed course, is compelling. LEO, a Learning Environment Organizer (CoVey, 2000; CoVey & Cañas, 2003), is a component of CmapTools. LEO was designed to serve as a meta-cognitive tool for courseware designers and to utilize the designer’s representations as the interface to the deployed course for the learners. LEO is discussed in the following section.
3. LEO: a meta-cognitive tool for courseware design The ideas behind LEO, a Learning Environment Organizer, build on Novak’s advocacy of the Concept Map as a meta-cognitive tool, by adapting it to a representation that can be used to organize instruction. LEO extends the capabilities of Concept Maps for use as the organizing factor in a course. In the following sections, we describe the basic features of LEO and a strategy for the creation of a course Organizer using this tool. 3.1. A description of LEO An Organizer has two diVerent types of nodes: instructional topic nodes and explanation nodes that explain about the topics. The topic nodes have a variety of adornments that distinguish them from explanatory nodes. Fig. 1 presents a view of LEO’s editor. In Fig. 1, topic nodes are depicted as those surrounded with shadowed boxes, and populated with a variety of icons. For example, “First Pass Radionuclide Ventriculogram”, “Ejection Fraction”, “Segmental Abnormalities”, “Normal Wall Motion”, etc., are topic nodes. The topic nodes are linked together by double lines that convey prerequisite relationships. For instance, the introductory topic “First Pass Radionuclide Ventriculogram” is a prerequisite for “Ejection Fraction”, “Segmental Abnormalities”, etc. In turn, “Segmental Abnormalities” is a prerequisite for “Non-speciWc WM Abnormalities”, “Mitral Valve Prolapse”, etc. The icons beneath the topics indicate links to the instructional content that can be used to study the topic under consideration, and to the tasks or activities associated with the topic. When the user clicks the icon, a pull-down menu is displayed to indicate the links to electronic resources that are available for that topic. Separate icons exist for the various electronic media types such as text, graphics, Concept Maps, Web pages and application programs. The icon on the left side of a topic node indicates the completion requirement for the topic. Explanation nodes, which are also visible in Fig. 1, elaborate the relationships among the topic nodes, and have no adornments. 3.2. Creating a basic course organizer LEO’s editor is a modeless drawing tool. It is not necessary to switch from one mode to another in order to create nodes, linking phrases, etc. The user typically starts by placing basic concepts
552
J.W. CoVey / Computers & Education 48 (2007) 548–566
Fig. 1. The Organizer editor, showing topic and explanation nodes, icons indicating links to content, and the completion criteria dialog box.
that relate to the course on the drawing area by double clicking and typing the concept label to add a concept. Concepts can be selected and linked together to form directed acyclic graphs (DAGs). The labels on the linking phrases are added in the same fashion as the concept labels. Concepts can be selected and designated as topics or explanatory nodes. Linking lines can be selected and designated either to indicate prerequisites between topics or to indicate explanatory information on the conceptual relationships between the topics, without indicating a prerequisite relationship. The course designer may add links to online instructional content that is pertinent to the topic by dragging Wles and dropping them on the appropriate topic node. As the course is being designed, the instructor can use these icons to access the tasks, assignments, activities, etc., that are associated with the topics. Students see and use the same icons to access resources once the course is deployed. These resources may take the form of text, digital video, graphics, Web pages, or in any other electronic resource the instructor wishes to utilize.
J.W. CoVey / Computers & Education 48 (2007) 548–566
553
The instructor can specify the criteria for completion of a topic. The dialog box in the lower right portion of Fig. 1 presents the available alternatives. To set this attribute, the course designer selects one or more topics, opens the “Set Completion Criteria” dialog box, and selects the desired completion requirement alternative. Several possible alternatives have been identiWed and implemented. The instructor may require a submission of a deliverable that must be evaluated before the topic is considered completed. The student may be required to download a test (true/false or multiple choice) that is taken and graded on the spot by an automated process, with the Organizer updated immediately, or graded in an asynchronous process (CoVey & Webb, 2002). The topic could be essentially optional or suggested, in which case the student decides when to mark it completed and to continue. It is possible to allow the student to work freely through multiple topics before requiring an evaluation of work. Tests are established for a topic by creating the test in accompanying software named xTest, and dragging it onto the topic. Developers can collaborate on the asynchronous development of a course at a distance. This form of collaboration is supported by placing Organizers, Concept Maps, and other resources on CmapTools servers where the developers can access them from wherever they are. A developer can initiate the development of a course Organizer from a clean slate, or from a Concept Map created with CmapTools. The following section describes a process that enables the user to transform a Concept Map into a course Organizer.
4. Transforming Concept Maps into course Organizers In this section, we describe a basic method by which an existing Concept Map can be transformed into an Organizer for a course. This approach has two basic parts: editing the existing map to create a comprehensive map at the proper level of granularity, and transforming the map into a course description. 4.1. The basic process Novak and Gowin (1984) advocate the speciWcation of a “focus question” to keep the development of the Concept Map directed toward a goal. Even with a good focus question, Concept Maps can have undesirable attributes that should be corrected before attempting to create an Organizer. Because Concept Mapping is a minimally constrained process, it is easy to create maps that: • have detailed concepts intermingled with general concepts; • include concepts which might reasonably be judged to be irrelevant; • do not include relevant concepts. Ausubel’s (1968) notions of subsumption and progressive diVerentiation suggest that Concept Maps will contain both more general and more detailed concepts, so a judgment of the salience of the included concepts must be made. However, highly detailed or irrelevant concepts are considered to be “noise” that detracts from the truly salient elements in the map. Developers must review
554
J.W. CoVey / Computers & Education 48 (2007) 548–566
Concept Maps for such elements, which should be identiWed and removed. A judgment of degree of permissible diVerence in generality can be informed by the intuitive notion of trying to create a map with a consistent “granularity”. As an illustration, if a course contains 15 topics of the same general importance, then a concept map might have t1–2 other concepts pertaining to each of the topics, yielding a 30–45 node map. Such a map would present a rich overview of the knowledge domain or course. If some topics were more important than others, the number of corollary nodes might be adjusted accordingly. The number of accompanying nodes might also be adjusted if there were relatively more or fewer topics. Additionally, some fundamental concepts might be overlooked or omitted. Novak (1998) suggests that when concepts have excessive out-degree (for instance, more than 5 outgoing linking lines from a concept), it is probable that two or more intermediate-level concepts might be found between the superordinate and subordinate concepts. This heuristic can be used to identify missing concepts. Often, missing ideas are only detected through this heuristic, upon collaborative review of a recently created map, or upon subsequent assessment after the initial mapping session. If basic ideas have been omitted from a Concept Map, they need to be added. These three concerns with regard to the concepts in a map (irrelevancies, inappropriate level of detail, and omissions) are the basic ones that must be addressed in preparing the Concept Map for transformation into an Organizer. They may also be viewed as general guidelines for the construction of a good Concept Map. Once a Concept Map has been created that contains all the major ideas and from which irrelevancies have been removed, the transformation process commences. The Wrst step is to identify the items that are to be topics, as opposed to those that are corollary items that elaborate relationships among the topics. The second step is to determine which prerequisite relationships (if any) exist among topics. The visual representation presented by LEO assists the designer in identifying those items that should be topics as opposed to those that should be explanatory items. It also fosters the identiWcation of the necessary prerequisite relationships among topics. The topic nodes are readily identiWed, designated, and connected using LEO’s editor. Often, as a result of connecting concepts, the map becomes tangled. It is necessary to de-convolute the map to minimize the number of lines that cross, and to make the layout easy to follow and visually pleasing. Often several reviews and subsequent iterations of this process are necessary before a rendering of the topics, explanatory nodes, prerequisite relationships among topics, etc., is deemed satisfactory. Table 1 presents a summary of the entire method. When a satisfactory Organizer has been created, the attachment of resources, and the designation of completion criteria commence. Two points may be made regarding this method of converting a Concept Map into an Organizer. It is quite possible that more than one acceptable organization of the course might exist. In the example that follows, it is evident that the course may be taught with either of two diVerent sets of topics. Each of these two course organizations has diVerent prerequisite relationships and diVerent positive aspects. The second point is that this method of representation is novel in that it maps only those topic dependencies that are deemed to be necessary by the creator of the course, not an arbitrary linearization of the topics such as that contained in a typical course syllabus. The following section presents an example of the use of this method.
J.W. CoVey / Computers & Education 48 (2007) 548–566
555
Table 1 The process of converting a Concept Map into a course Organizer Analysis of the Concept Map 1. Identify possible topic nodes; identify and delete noise nodes, redundancies, and overly detailed nodes 2. Consider if any signiWcant topics are absent and add to the Concept Map 3. Consider if any potential topic nodes should be combined together or decomposed into multiple nodes 4. Reconstruct the Concept Map Transformation of the map into a course description 5. Select and mark topic nodes 6. Map dependency relationships among topic nodes, alternative organizations of the dependencies 7. Rearrange maps to accommodate dependency links 8. Review the result with the creator of the map if the creator is a diVerent person who has not been collaborating in an ongoing fashion
4.2. An example: NUCES Prior work at the Institute for Human and Machine Cognition, The University of West Florida, has led to the creation of a Concept Map-based knowledge model that pertains to radionuclide imaging of the left ventricle of the heart. It is entitled “NUCES, Nuclear Cardiology Expert System” (Ford, CoVey, Cañas, Turner, & Andrews, 1996). The purpose of this knowledge model is to represent knowledge regarding how an expert nuclear cardiologist interprets radionuclide images of the left ventricle of the heart, in order to render a diagnosis of the patient’s cardiac condition. These images, along with other diagnostic indicators, are used to arrive at a diagnosis regarding the patient’s cardiac function and the presence or absence of disease. The knowledge model is comprised of 10 Concept Maps in a three-level hierarchy, and more than 100 example images, numerous text, audio, and video resources that describe the methodology. This knowledge model was elicited from an expert in Nuclear Cardiology as the basis for the creation of an expert system on the diagnostic process. The expert had the recurring task of teaching other physicians his method of diagnosing heart disease, based upon the methodology represented in the knowledge model. A decision was made to utilize the knowledge model as instructional content for a course on the subject. However, the model itself was very large, gave no explicit indication regarding where to start or what to do to learn about the domain, and generally seemed intimidating to a number of novices who viewed it. A top level Concept Map that provides an overview of the model was utilized as a starting point for creation of a course on the topic. Fig. 2 presents this Concept Map, which was imported into LEO. The map contains descriptions of the various diagnoses that are made, as well as numerous basic concepts that are pertinent to the diagnostic process, including characteristic patterns that might be seen in the radionuclide images. The Concept Map aVorded a useful conceptual overview of the domain that proved helpful in the process of creating a course on the topic. The map contained a reference to “Ejection Fraction”, a number representing the percentage of blood volume in the left ventricle of the heart that is ejected when the ventricle contracts. The role of this measure is controversial in diagnostic nuclear cardiology, and its use when interpreting the radionuclide images is an important point of discussion. The top-level map also contained mention of diastolic increase and systolic decrease
556
J.W. CoVey / Computers & Education 48 (2007) 548–566
Fig. 2. The top level Concept Map from NUCES.
rate images, images taken from the two phases of the cardiac cycle. Both diastolic and systolic images are included in a study, and they present diVerent evidences in the process of reaching a diagnosis. Another fundamentally important distinction that this Concept Map made explicit is among normal wall motion, generalized abnormalities, and segmental abnormalities in heart wall motion. This distinction is central in the diagnostic process and in the Concept Map. The map also elaborated the various disease states that constitute a diagnosis. 4.3. The revised Concept Map Although the original map represented a good starting place that contributed substantially to the creation of an Organizer, it needed to be improved in several ways. The Wrst feature of note was found in the top right portion of the map, a discussion of the counts of gamma particles in a bolus solution. This is a low level description of how the images are made and not really salient at a global, general level of description. This discussion is absolutely necessary at some level, but its presence in a top-level map or as a basic topic in a course regarding how to interpret the images, contributes little to a broad understanding of important characteristic patterns in the images, or the formation of a diagnosis. Such information was deemed to be noise-a distraction from the more basic concepts to which attention should be drawn Wrst. This part of the Concept Map was
J.W. CoVey / Computers & Education 48 (2007) 548–566
557
removed from the Organizer, with the intention of presenting the information in the introductory materials associated with the Wrst topic in the course. A second interesting feature of the original map was that some fundamental topics were omitted. The map included most of the major disease diagnoses that the Nuclear Cardiologist makes: “Normal Wall Motion”, “Ischemic Heart Disease”, “Cardiomyopathy”, “Mitral Valve Prolapse”, “Valve Disease”, and “Severe Left Ventricular Dysfunction”. However, the map contained no mention of a state diagnosed t20% of the time, “Non-speciWc Wall Motion Abnormalities”. The map included the concepts of diastolic and systolic rate images, but it did not contain the Mean Transit Time images (MTT) which, while somewhat more diYcult to interpret than the rate images, are highly diagnostic, and often critical to the formulation of an accurate diagnosis on borderline cases. A third interesting feature was the presence of relevant but rarely used information at the top level. A characteristic pattern in the functional images called “Ice Cream Cone”, which is seen in perhaps 1 in 200 patients, was accorded the same status as the idea of “Blue Fingers”, a pattern that is seen in more than half the patients. Generally speaking, the frequency of occurrence would not be deemed an adequate measure of importance. However, in this case, a judgment was made that the concept might be addressed elsewhere and omitted from this very general view of the knowledge domain. The fundamental distinction between segmental and generalized abnormalities was preserved and highlighted (these two concepts became topic nodes later) but their locations in the map were rearranged to facilitate adding the concept “Non-speciWc Wall Motion Abnormalities”, and linking it into the concepts “Normal Wall Motion”, “Segmental Abnormalities”, and “Blue Fingers”. The concepts “MTT images” and “EDP-ESP” images were added as well. These additions completed the set of diagnoses and images that are used in the process of arriving at them. The omitted concept “Ice Cream Cone” was already prominently featured in a Concept Map in the model that is devoted to the topic of “Valve Disease”. The original map was modiWed to reXect these changes, and the map in Fig. 3 resulted. Along with the additions, the deletions essentially “leveled” the concepts in the map so that it contained all the fundamental concepts at approximately the same level of generality. 4.4. Alternative course organizations An analysis of the revised Concept Map led to the conclusion that at least two course organizations were possible. Both required laying a basic groundwork to the knowledge domain by discussing the sorts of images used in the test, and the broad categories of diagnoses that are made. In both of the Organizers, the introduction is indicated by the root node, which contains basic information on the radionuclide test itself. The information about counts in a bolus solution that had been removed from the original Concept Map was added here. Following this introduction, the distinction is made among normal wall motion, segmental and generalized abnormalities, and the role of ejection fraction in making the diagnosis. At this point, the two Organizers diverge. The layout of the Organizer presented in Fig. 4 assumes a strategy of elaborating the various types of images that are viewed in the course of the study. In the context of the distinctions among normal wall motion, segmental and generalized abnormalities, the student learns about the types of images that are perused, and gains an understanding of the characteristic patterns that might be
558
J.W. CoVey / Computers & Education 48 (2007) 548–566
Fig. 3. The revised top level Concept Map from NUCES.
seen in the various images. From within this framework, the student gains an understanding of the disease states that the patterns in these images indicate. This approach works from a study of image types and patterns that are consistently seen in images to details regarding how these patterns relate to the disease states. The Organizer presented in Fig. 5 reXects a strategy of elaborating the various disease states that might be diagnosed rather than the various categories of images that are viewed in the diagnostic process. In the context of the distinctions among normal wall motion, segmental and generalized abnormalities, each of the diagnostic states is considered. The relationships among the various disease states that might be diagnosed and the characteristic patterns that might be seen in the various images are made explicit in the Organizer, and elaborated in the accompanying materials. This approach works from diVerences in the disease states back to the patterns associated with the various disease states that are seen in the images. The two Organizers described here provide two diVerent approaches to learning the same material. Either of the two Organizers provides an explicit rendering of the major concepts in the course of study and how they interrelate, a good starting point for meaningful learning of global, structural knowledge of the course. Fig. 6 presents the student view of the Organizer from Figs. 1 and 5. The student’s view provides status bars on the left sides of the topic boxes to indicate completion status of the topic, which is one of {completed, current, ready, not ready}. These are displayed in the student view each time the student logs onto an Organizer, and they are updated as the student works
J.W. CoVey / Computers & Education 48 (2007) 548–566
559
Fig. 4. The Wrst Organizer from NUCES.
through the course and completes topics (CoVey, 2000; CoVey & Cañas, 2003). The palette at the top labeled “Display Status” enables the student to show or hide subsets of the Organizer. SpeciWcally, the user can show or hide the completed, current, ready, or not ready status topics, or the explanation nodes and linking phrases. The motivating idea behind this capability is to provide the learner with a simple means of customizing the information display.
5. Fostering reuse of course content LEO supports the reuse of both course descriptions and course content (CoVey & Cañas, 2001). The concise, explicit representation facilitates identiWcation of useful groups of topics that might be reused. These can be selected in the editor, copied, and pasted into a new course Organizer, with all existing links to course resources preserved. Additional topics may then be pre-pended, appended or interspersed with the existing materials. Additionally, LEO can be used for semiautomated generation of metadata records for course materials. The following sections illustrate
560
J.W. CoVey / Computers & Education 48 (2007) 548–566
Fig. 5. The second Organizer from NUCES.
these capabilities with material from a course in Managerial Decision Making, and the NUCES course described earlier. 5.1. Reusing course description components Fig. 7 contains an example of a course on regression models for decision-making. This course starts with an introductory topic (on the left-hand side of Fig. 7) that is followed by the topic “Business and Economic data”. In turn, that topic is followed by several topics on modeling, two topics pertaining to datasets, and descriptive statistics. Following these introductory topics, the course progresses through a series of topics on various regression models, leading to a discussion of how to make decisions based on the forecasts that these models provide. The links to content are indicated by the icons beneath the topics. Fig. 8 presents an organizer for a course in time series models. This course utilizes some of the same introductory material as the course presented in Fig. 7, in particular, topics covering modeling, types of data, and descriptive statistics. The starting point for development of the course presented in Fig. 8 was the introductory material from the course depicted in Fig. 7. The topics and explana-
J.W. CoVey / Computers & Education 48 (2007) 548–566
561
Fig. 6. The Learner’s view of the Organizer in Fig. 1.
tions on time series models that are speciWc to this course were added to the general introductory material. The links to content for the topics that were reused were already established and retained. Instructional content could be added to or deleted from any of the reused topics by selecting the topic and choosing to edit the resources associated with the topic. This form of organizer facilitates recombining previously created content to create new courses. If a variety of media exist on a topic, these media can be included in varying combinations, depending on the audience. If a course on a diVerent subset of a domain is desired, explanatory items can be made into topics and topics can be made into part of the explanatory component. 5.2. Cataloguing instructional resources for reuse LEO provides capabilities to create metadata records for the entire course and to scan through the list of course topics and resources, creating records for them as well. Fig. 9(a) presents a part of the metadata record for the Wrst topic in the NUCES course. This metadata record is structured
562
J.W. CoVey / Computers & Education 48 (2007) 548–566
Fig. 7. An organizer for a course on Regression Analysis.
Fig. 8. An organizer for a Time Series course that reuses content from the Regression Analysis course.
according to Dublin Core (DCMI, 2005) standards and the format described in the ADL/IMS speciWcation, as presented in the Sharable Content Object Reference Model (SCORM, 2005). When the metadata records for individual topics are created, the links to course content are read, and metadata records for the content are created as well. The course metadata record is placed in a subdirectory that contains all the metadata for the course. A subdirectory within that directory is created for each of the topics in the course, and a topic metadata record is placed in the subdirectory. The metadata records for the individual course content items that are linked to the topic are placed in the subdirectory for that topic. Fig. 9(b) illustrates the directory structure that is created for the metadata records associated with the NUCES course.
J.W. CoVey / Computers & Education 48 (2007) 548–566
(a) A metadata record.
563
(b) The metadata folders.
Fig. 9. A typical template metadata record and a typical folder structure for a course.
The metadata records contain a variety of information that can be automatically determined from the system. The names of the course, topics, and individual media are taken from the Wlename of the course Organizer, the topic labels in the organizer, and the names of the media that are used in the link to the content. The course instructor is the default value for creator. If an individual resource is imported into CMapTools, the person importing it has the option to Wll in a description Weld that can be used for the description of the item in the metadata record. The metadata generator software reads the directory entry for the Wle to determine the creation date of the Wle. The software provides editable default values for copyright status and placeholders for keywords that might be associated with the media, topic, or course. These values may also be edited.
6. Summary and discussion This paper contains a description of a meta-cognitive tool named LEO that allows the instructional designer to create course organizations for face-to-face, hybrid, or online courses. LEO is part of a Concept Map-based knowledge modeling toolkit named CmapTools that is in on-going development at the Institute for Human and Machine Cognition (IHMC), The University of West Florida. LEO aVords the course designer a graphical depiction of both topics in a course, and additional explanatory elements that elaborate relationships among the topics. The editor enables the course designer to add, edit, or remove concepts that are topics or explanatory nodes, to add, view and delete accompanying learning resources, and to set completion criteria for a topic.
564
J.W. CoVey / Computers & Education 48 (2007) 548–566
A drag-and-drop capability simpliWes adding learning resources to the topics. The Organizer’s editor presents capabilities to switch elements from explanatory items to topics and back, all in a graphical rendering of the course organization. The editor allows the designer to rearrange the elements in a course by simple operations such as selecting and dragging. We have presented a method that enables a courseware designer to start with a Concept Map of a course and to convert the map into an Organizer. The process involves two major activities. The Wrst activity is meant to ensure that all the salient concepts are in the map, and that overly detailed or irrelevant concepts are removed. The second part of the process involves identiWcation of the concepts that are to be the designated topics, and the sequencing of those topics that have prerequisite relationships with others. Course organizations are represented in a fashion that depicts those dependency relationships that the designer deems to be necessary, rather than an arbitrary sequence of topics typical of current courses, whether Web-based or not. One of the strengths of the editor is the visual depiction of the course that it aVords. This representation has proven useful to course designers in the process of determining how the course should be assembled – what the topics should be, how the topics should be sequenced, etc. The visual representation enables the designer to view a concise, minimal depiction of materials that might be included and to make decisions regarding which to include and which to omit. The tool puts all the information in front of the designer in a visual representation that fosters reXection on what the important concepts are in the course versus those that are secondary. The depiction also helps the instructor to include and assess the available instructional materials, the appropriate places to test or to require deliverables, etc. This approach to course organization holds the promise to facilitate the reuse of electronic instructional content by enabling the course designer to recombine course content to suit the needs of various audiences. The editor enables the instructor to copy and paste pieces of courses to tailor topics and content to diVerent audiences, and to try “what if” scenarios for course organizations. DiVerent versions of courses can be created, stored or archived, and shared over the Internet with colleagues and students. The ease with which course designers can identify, access, evaluate materials, and associate them with topics in courses also fosters reXection on basic course content. The software also provides semi-automated generation of Dublin Core Compliant XML metadata representations of the resources. These records can be used in html that wraps the resources themselves or in a metadata repository. Several Organizers have been produced and used by faculty at the University of West Florida, in knowledge domains as diverse as Managerial Decision Making (a graduate level statistics course), an upper level undergraduate English course in Shakespeare, an Introductory Psychology course, an Introduction to Computer Science, Data Structures, and NUCES. A fairly wide range in terms of the number of topics was noted. However, all of these courses exhibit signiWcant non-linearities in the arrangement of topics, indicating quite clearly that professors have fundamentally diVerent conceptualizations of courses than the linear ones typically presented in the syllabi of their courses.
Acknowledgements I thank Alberto Cañas for his help and guidance in the formulation of many of the ideas behind this work. I thank the CmapTools development team and Chad CarV, Michael Webb, John Wernicke,
J.W. CoVey / Computers & Education 48 (2007) 548–566
565
and Michael Wooten for their work on the implementation of LEO. This work was supported in part by a grant from the Chief of Naval Education and Training, Pensacola, FL. References Ausubel, D. P. (1968). Educational psychology: A cognitive view. New York: Rinehart and Winston. Blankenship, J., & Dansereau, D. F. (2000). The eVect of animated node-link displays on information recall and session evaluation. Journal of Experimental Education, 68(4), 293–308. Buzan, T., & Buzan, B. (1996). The mind map book: How to use radiant thinking to maximize your brain’s untapped potential. New York: Plume. Cañas, A. J., CoVey, J. W., Reichherzer, T., Hill, G., Suri, N., & CarV, R., et al. (1998). El-Tech: a performance support system with embedded training for electronics technicians. In Proceedings of the 11th Florida AI research symposium (FLAIRS ’98) (pp. 79-83), Sanibel Island, FL. Cañas, A. J., Hill, G., CarV, R., Suri, N., Lott, J., & Eskridge, T., et al. (2004). CmapTools: a knowledge modeling and sharing environment. In Proceedings of CMC 2004, the Wrst international conference on concept mapping, Pamplona, Spain, September 14–17, 2004. CoVey, J. W. (2000). LEO: a learning environment organizer to accompany constructivist knowledge models. Unpublished doctoral dissertation, The University of West Florida, Pensacola, FL. CoVey, J. W., & Cañas, A. J. (2001). Tools to foster course and content reuse in online instructional systems. In Proceedings of WebNet 2001: World conference on the www and internet, Orlando, FL, October 23–27, 2001. CoVey, J. W., & Cañas, A. J. (2003). LEO: A Learning Environment Organizer to support computer mediated instruction. Journal of Educational Technology Systems, 31(3), 275–290. CoVey, J. W., & Webb, M. (2002). AMEE: annotated, multi-modal electronic evaluation for dynamic student modeling. In Proceedings of e-learn 2002, world conference on e-learning in corporate, government, healthcare, and higher education, Montreal, Canada, October 19–23, 2002. DCMI. (2005). The Dublin Core Metadata Initiative: making it easier to Wnd information. Available from http:// www.dublincore.org/. Fisher, K. M. (1990). Semantic networking: The new kid on the block. Journal of Research in Science Teaching, 27(10), 1001–1018. Fisher, K. M. (2000). SemNet software as an assessment tool. In J. J. Mintzes, J. H. Wandersee, & J. D. Novak (Eds.), Assessing science understanding. San Diego: Academic Press. Ford, K. M., CoVey, J. W., Cañas, A. J., Turner, C. W., & Andrews, E. J. (1996). Diagnosis and explanation by a Nuclear Cardiology Expert System. International Journal of Expert Systems, 9(4), 499–506. Hedberg, J. (1997). Employing cognitive tools within interactive multimedia applications. In Proceedings of selected research and development presentations at the 1997 national convention of the association for educational communications and technology, 19th, Albuquerque, NM, February 14–18, 1997. Hyerle, D. (2000). A Weld guide to using visual tools. FOUND IN ERIC microWche ED443784. Kasowitz, A. (2000). Tools for automating instructional design. Educational Media and Technology Yearbook, 25, 49–52. Lambiotte, J., Skaggs, L., & Dansereau, D. (1993). Learning from lectures: EVects of knowledge maps and cooperative review strategies. Applied Cognitive Psychology, 7, 483–497. Macromedia. (2003). Macromedia Authorware 7. Available from http://www.macromedia.com/software/authorware/. Mind Manager. (2005). Mind Manager: The visual tool for brainstorming and planning. Available from http://www.mindjet.com/us/. Nkambou, R., Frasson, C., Gauthier, G., & Rouane, K. (2001). An Authoring Model and Tools for Knowledge Engineering in Intelligent Tutoring. Journal of Interactive Learning Research, 12(4), 323–357. Novak, J. D. (1998). Learning, creating, and using knowledge: Concept maps as facilitative tools in schools and corporations. Mahweh, NJ: Lawrence Erlbaum Associates. Novak, J. D., & Gowin, D. B. (1984). Learning how to learn. Cambridge, UK: Cambridge University Press. Novak, J. D. & Iuli, R. J. (1991). The use of meta-cognitive tools to facilitate knowledge production. In A paper presented at the fourth Florida AI research symposium (FLAIRS ’91), Pensacola Beach, FL, May, 1991.
566
J.W. CoVey / Computers & Education 48 (2007) 548–566
Ritter, S., & Blessing, S. B. (1998). Authoring Tools for Component-Based Learning Environments. Journal of the Learning Sciences, 7(1), 107–132. SCORM. (2005). Advanced distributed learning. Available from http://www.adlnet.org/index.cfm?fuseaction D scormabt. Visual Mind. (2005). Visual Mind: the software that helps you think. Available from http://www.visual-mind.com/.
A meta-cognitive tool for courseware development, maintenance, and reuse John W. CoVey
¤
The Department of Computer Science and The Institute for Human and Machine Cognition, The University of West Florida, 40 S. Alcaniz Street, Pensacola, FL 32514, USA Received 12 April 2003; accepted 29 March 2005
Abstract Novak and Iuli [Novak, J. D. & Iuli, R. J. (1991). The use of meta-cognitive tools to facilitate knowledge production. In A paper presented at the fourth Florida AI research symposium (FLAIRS ’91), Pensacola Beach, FL, May, 1991.] discuss the use of Concept Maps as meta-cognitive tools that help people to think about thinking. This work describes a network-enabled meta-cognitive tool based upon extensions to Concept Maps that can be used to help course designers visualize and plan course organizations. This tool permits the user to create a novel type of course description based on the idea of an advance organizer. Course arrangements created by this method do not have the arbitrary linear sequences of topics typically found in traditional courses at the college level. The tool is part of an environment that is designed to foster meaningful learning and reuse of course design and instructional content. This paper presents a description of this software tool, an approach to the creation of a course depiction from a Concept Map, an example of a course that was developed iteratively using the tool, and a discussion of the ways that the tool fosters course and content reuse. © 2005 Elsevier Ltd. All rights reserved. Keywords: Authoring tools and methods; Interactive learning environments; Multimedia/hypermedia systems; Metacognitive tools
¤
Tel.: +1 850 202 4414; fax: +1 850 202 4440. E-mail address: [email protected].
0360-1315/$ - see front matter © 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.compedu.2005.03.008
J.W. CoVey / Computers & Education 48 (2007) 548–566
549
1. Introduction The Concept Map (Novak & Gowin, 1984) is a well-known tool of educational psychology that aVords an explicit representation of what a person knows about a knowledge domain. Concept Maps are comprised of concepts that are, according to Novak and Gowin, an individual’s perceived regularities in events or objects. Novak and Gowin describe all knowledge as being comprised of concepts and propositions, the basic components of Concept Maps. Concept Maps oVer a Xexible framework for eliciting, representing, and communicating an individual’s personally constructed understandings of a knowledge domain. Although Concept Maps were not originally intended to represent procedural, step-by-step knowledge or items that are arranged in a sequence, the links in Concept Maps are directed arcs that could designate precedence relationships among the concepts. If the concepts represented topics in a course, then the Concept Map would be an excellent starting representation for the conceptualization of course content and prerequisite arrangements among topics. Such a graph-like representation of topics in a course would not have an arbitrary linearization of the presentation that is typical of traditional and on-line courses. A course presented in this fashion would increase the learner’s choice regarding what topic to pursue next. The purpose of this paper is to describe a software tool that provides capabilities to create non-linear course arrangements based upon Concept Maps. The remaining sections of this paper contain a description of LEO, a Learning Environment Organizer (CoVey, 2000; CoVey & Cañas, 2003) that utilizes a Concept Map-like rendering of concepts that correspond to topics, and links that indicate both conceptual and prerequisite relationships among the topics. The course designer uses this tool to create alternative conceptualizations of the organization of a course. As such, the tool serves as a meta-cognitive device that helps the course designer to think about the course design. The tool aVords the designer many additional capabilities including the ability to create links from the topics to instructional content, to designate assignments, criteria for completion of a topic, etc. This approach leads to the creation of a Concept Map-like representation of the course description, with some concepts corresponding to the topics in the course, and others providing elaboration of the relationships among topics. This paper contains a description of the software tool that supports this method of course development, an example of the utilization of this tool to convert a Concept Map into a course description, and an elaboration of how the tool fosters course component and instructional content reuse.
2. Literature pertaining to meta-cognitive tools In the broadest sense, meta-cognition is thinking about one’s thought processes. The literature suggests that the traditional application of meta-cognitive strategies has been in eVorts to help students gain awareness of how they approach reading and writing. Most of the literature addresses general-purpose tools for students and others. However, a well-conceived meta-cognitive tool could play a valuable role in helping instructors to decide what is important about the content of a course, and any necessary sequencing issues relative to mastery of the content. The remainder of this section describes methods and tools that might potentially be of use to instructors who are developing courses.
550
J.W. CoVey / Computers & Education 48 (2007) 548–566
Several graphical representations have been created that are meant to foster reXection and meta-cognition. These include Concept Maps (Novak & Gowin, 1984), Semantic Networks (Fisher, 1990), Mind Maps (Buzan & Buzan, 1996), and Knowledge Maps (Lambiotte, Skaggs, & Dansereau, 1993) or Node-Link Maps (Blankenship & Dansereau, 2000). These representations and their accompanying tools help users to externalize and understand what they know about a knowledge domain. Although all these are graphical representations comprised of conceptual knowledge on nodes and links, they diVer in a variety of ways. For instance, in some cases, the linking lines drawn between concepts are labeled, in others they are not. If linking phrases are labeled, the set of labels may be pre-speciWed or unconstrained. Concepts may be comprised of single words, short phrases, sentences, or paragraphs. Some of the tools available for these representations allow association of other resources with the concepts that are represented. SemNet (Fisher, 2000) permits creation of Fisher’s Semantic Networks. Several tools such as Mind Manager (2005) and Visual Mind (2005) facilitate creation of Mind Maps. However, none of these tools is explicitly geared to the creation of course organizers. Novak and Gowin (1984) describe Concept Maps and Vee diagrams as meta-cognitive tools. The Concept Map is the least constrained of the various meta-cognitive tools enumerated above, and provides excellent capabilities for both students and teachers to make explicit what they know about a knowledge domain. Novak and Iuli (1991) discuss the use of Concept Maps as meta-cognitive tools that help people to think about thinking. They suggest that Concept Maps may be used to facilitate learning, to organize instruction, and to help users organize their knowledge in ways that might make that knowledge more generalizable. CmapTools (Cañas et al., 1998, 2004) provide substantial support for the creation and representation of knowledge regarding a knowledge domain. CMapTools are based upon Concept Maps. These tools allow for the development and perusal of hierarchically organized Concept Maps. Additionally, accompanying explanatory resources in any electronic medium may be attached to concepts and accessed through the Concept Maps. In the broader context, Kasowitz (2000) describes a wide range of tools that she states facilitate instructional design, including advisory systems, information management systems, electronic performance support systems, and authoring tools for computer-based instruction. Her article is a general categorization that illustrates the range of tools that support development of instructional materials. Nkambou, Frasson, Gauthier, and Rouane (2001) describe their Curriculum Representation and Acquisition Model (CREAM) which supports creation and organization of the curriculum by domain, pedagogical approach, and didactic aspects of the teaching. Ritter and Blessing (1998) describe their Visual Translator, a tool designed to support development of large-scale educational systems. Other types of meta-cognitive tools have been described in the literature as well. Hedberg (1997) discusses meta-cognitive tools that he describes as being embedded in an “information landscape”. These tools are designed to assist students in their investigation of scientiWc topics and with the writing process. In his book, Hyerle (2000) examines the use of visual tools such as task-speciWc organizers, and thinking-process maps that are similar to Concept Maps and Mind Maps. He describes ways that various categories of users use these tools and the beneWcial eVects such tools have on reading and writing. While not explicitly meant as meta-cognitive tools, hypermedia authoring suites provide support for courseware design. Web development tools permit cataloguing of graphics, hierarchical directory structures to organize pages and the inclusion of links to topical materials. They also provide capabilities
J.W. CoVey / Computers & Education 48 (2007) 548–566
551
to create links to databases, to accumulate site usage statistics, etc. All of these tools provide some type of organizational factor and support for the tracking of materials that are used in courses. Programs such as Macromedia Authorware™ (Macromedia, 2003) are typical of this class of tool and are quite mature. However, authoring tools generally lack an explicit pedagogical underpinning. The idea of instructors creating representations in a meta-cognitive tool that fosters reXection on the topics that they teach and the organization of their course, and then having the representation created in the tool support the deployed course, is compelling. LEO, a Learning Environment Organizer (CoVey, 2000; CoVey & Cañas, 2003), is a component of CmapTools. LEO was designed to serve as a meta-cognitive tool for courseware designers and to utilize the designer’s representations as the interface to the deployed course for the learners. LEO is discussed in the following section.
3. LEO: a meta-cognitive tool for courseware design The ideas behind LEO, a Learning Environment Organizer, build on Novak’s advocacy of the Concept Map as a meta-cognitive tool, by adapting it to a representation that can be used to organize instruction. LEO extends the capabilities of Concept Maps for use as the organizing factor in a course. In the following sections, we describe the basic features of LEO and a strategy for the creation of a course Organizer using this tool. 3.1. A description of LEO An Organizer has two diVerent types of nodes: instructional topic nodes and explanation nodes that explain about the topics. The topic nodes have a variety of adornments that distinguish them from explanatory nodes. Fig. 1 presents a view of LEO’s editor. In Fig. 1, topic nodes are depicted as those surrounded with shadowed boxes, and populated with a variety of icons. For example, “First Pass Radionuclide Ventriculogram”, “Ejection Fraction”, “Segmental Abnormalities”, “Normal Wall Motion”, etc., are topic nodes. The topic nodes are linked together by double lines that convey prerequisite relationships. For instance, the introductory topic “First Pass Radionuclide Ventriculogram” is a prerequisite for “Ejection Fraction”, “Segmental Abnormalities”, etc. In turn, “Segmental Abnormalities” is a prerequisite for “Non-speciWc WM Abnormalities”, “Mitral Valve Prolapse”, etc. The icons beneath the topics indicate links to the instructional content that can be used to study the topic under consideration, and to the tasks or activities associated with the topic. When the user clicks the icon, a pull-down menu is displayed to indicate the links to electronic resources that are available for that topic. Separate icons exist for the various electronic media types such as text, graphics, Concept Maps, Web pages and application programs. The icon on the left side of a topic node indicates the completion requirement for the topic. Explanation nodes, which are also visible in Fig. 1, elaborate the relationships among the topic nodes, and have no adornments. 3.2. Creating a basic course organizer LEO’s editor is a modeless drawing tool. It is not necessary to switch from one mode to another in order to create nodes, linking phrases, etc. The user typically starts by placing basic concepts
552
J.W. CoVey / Computers & Education 48 (2007) 548–566
Fig. 1. The Organizer editor, showing topic and explanation nodes, icons indicating links to content, and the completion criteria dialog box.
that relate to the course on the drawing area by double clicking and typing the concept label to add a concept. Concepts can be selected and linked together to form directed acyclic graphs (DAGs). The labels on the linking phrases are added in the same fashion as the concept labels. Concepts can be selected and designated as topics or explanatory nodes. Linking lines can be selected and designated either to indicate prerequisites between topics or to indicate explanatory information on the conceptual relationships between the topics, without indicating a prerequisite relationship. The course designer may add links to online instructional content that is pertinent to the topic by dragging Wles and dropping them on the appropriate topic node. As the course is being designed, the instructor can use these icons to access the tasks, assignments, activities, etc., that are associated with the topics. Students see and use the same icons to access resources once the course is deployed. These resources may take the form of text, digital video, graphics, Web pages, or in any other electronic resource the instructor wishes to utilize.
J.W. CoVey / Computers & Education 48 (2007) 548–566
553
The instructor can specify the criteria for completion of a topic. The dialog box in the lower right portion of Fig. 1 presents the available alternatives. To set this attribute, the course designer selects one or more topics, opens the “Set Completion Criteria” dialog box, and selects the desired completion requirement alternative. Several possible alternatives have been identiWed and implemented. The instructor may require a submission of a deliverable that must be evaluated before the topic is considered completed. The student may be required to download a test (true/false or multiple choice) that is taken and graded on the spot by an automated process, with the Organizer updated immediately, or graded in an asynchronous process (CoVey & Webb, 2002). The topic could be essentially optional or suggested, in which case the student decides when to mark it completed and to continue. It is possible to allow the student to work freely through multiple topics before requiring an evaluation of work. Tests are established for a topic by creating the test in accompanying software named xTest, and dragging it onto the topic. Developers can collaborate on the asynchronous development of a course at a distance. This form of collaboration is supported by placing Organizers, Concept Maps, and other resources on CmapTools servers where the developers can access them from wherever they are. A developer can initiate the development of a course Organizer from a clean slate, or from a Concept Map created with CmapTools. The following section describes a process that enables the user to transform a Concept Map into a course Organizer.
4. Transforming Concept Maps into course Organizers In this section, we describe a basic method by which an existing Concept Map can be transformed into an Organizer for a course. This approach has two basic parts: editing the existing map to create a comprehensive map at the proper level of granularity, and transforming the map into a course description. 4.1. The basic process Novak and Gowin (1984) advocate the speciWcation of a “focus question” to keep the development of the Concept Map directed toward a goal. Even with a good focus question, Concept Maps can have undesirable attributes that should be corrected before attempting to create an Organizer. Because Concept Mapping is a minimally constrained process, it is easy to create maps that: • have detailed concepts intermingled with general concepts; • include concepts which might reasonably be judged to be irrelevant; • do not include relevant concepts. Ausubel’s (1968) notions of subsumption and progressive diVerentiation suggest that Concept Maps will contain both more general and more detailed concepts, so a judgment of the salience of the included concepts must be made. However, highly detailed or irrelevant concepts are considered to be “noise” that detracts from the truly salient elements in the map. Developers must review
554
J.W. CoVey / Computers & Education 48 (2007) 548–566
Concept Maps for such elements, which should be identiWed and removed. A judgment of degree of permissible diVerence in generality can be informed by the intuitive notion of trying to create a map with a consistent “granularity”. As an illustration, if a course contains 15 topics of the same general importance, then a concept map might have t1–2 other concepts pertaining to each of the topics, yielding a 30–45 node map. Such a map would present a rich overview of the knowledge domain or course. If some topics were more important than others, the number of corollary nodes might be adjusted accordingly. The number of accompanying nodes might also be adjusted if there were relatively more or fewer topics. Additionally, some fundamental concepts might be overlooked or omitted. Novak (1998) suggests that when concepts have excessive out-degree (for instance, more than 5 outgoing linking lines from a concept), it is probable that two or more intermediate-level concepts might be found between the superordinate and subordinate concepts. This heuristic can be used to identify missing concepts. Often, missing ideas are only detected through this heuristic, upon collaborative review of a recently created map, or upon subsequent assessment after the initial mapping session. If basic ideas have been omitted from a Concept Map, they need to be added. These three concerns with regard to the concepts in a map (irrelevancies, inappropriate level of detail, and omissions) are the basic ones that must be addressed in preparing the Concept Map for transformation into an Organizer. They may also be viewed as general guidelines for the construction of a good Concept Map. Once a Concept Map has been created that contains all the major ideas and from which irrelevancies have been removed, the transformation process commences. The Wrst step is to identify the items that are to be topics, as opposed to those that are corollary items that elaborate relationships among the topics. The second step is to determine which prerequisite relationships (if any) exist among topics. The visual representation presented by LEO assists the designer in identifying those items that should be topics as opposed to those that should be explanatory items. It also fosters the identiWcation of the necessary prerequisite relationships among topics. The topic nodes are readily identiWed, designated, and connected using LEO’s editor. Often, as a result of connecting concepts, the map becomes tangled. It is necessary to de-convolute the map to minimize the number of lines that cross, and to make the layout easy to follow and visually pleasing. Often several reviews and subsequent iterations of this process are necessary before a rendering of the topics, explanatory nodes, prerequisite relationships among topics, etc., is deemed satisfactory. Table 1 presents a summary of the entire method. When a satisfactory Organizer has been created, the attachment of resources, and the designation of completion criteria commence. Two points may be made regarding this method of converting a Concept Map into an Organizer. It is quite possible that more than one acceptable organization of the course might exist. In the example that follows, it is evident that the course may be taught with either of two diVerent sets of topics. Each of these two course organizations has diVerent prerequisite relationships and diVerent positive aspects. The second point is that this method of representation is novel in that it maps only those topic dependencies that are deemed to be necessary by the creator of the course, not an arbitrary linearization of the topics such as that contained in a typical course syllabus. The following section presents an example of the use of this method.
J.W. CoVey / Computers & Education 48 (2007) 548–566
555
Table 1 The process of converting a Concept Map into a course Organizer Analysis of the Concept Map 1. Identify possible topic nodes; identify and delete noise nodes, redundancies, and overly detailed nodes 2. Consider if any signiWcant topics are absent and add to the Concept Map 3. Consider if any potential topic nodes should be combined together or decomposed into multiple nodes 4. Reconstruct the Concept Map Transformation of the map into a course description 5. Select and mark topic nodes 6. Map dependency relationships among topic nodes, alternative organizations of the dependencies 7. Rearrange maps to accommodate dependency links 8. Review the result with the creator of the map if the creator is a diVerent person who has not been collaborating in an ongoing fashion
4.2. An example: NUCES Prior work at the Institute for Human and Machine Cognition, The University of West Florida, has led to the creation of a Concept Map-based knowledge model that pertains to radionuclide imaging of the left ventricle of the heart. It is entitled “NUCES, Nuclear Cardiology Expert System” (Ford, CoVey, Cañas, Turner, & Andrews, 1996). The purpose of this knowledge model is to represent knowledge regarding how an expert nuclear cardiologist interprets radionuclide images of the left ventricle of the heart, in order to render a diagnosis of the patient’s cardiac condition. These images, along with other diagnostic indicators, are used to arrive at a diagnosis regarding the patient’s cardiac function and the presence or absence of disease. The knowledge model is comprised of 10 Concept Maps in a three-level hierarchy, and more than 100 example images, numerous text, audio, and video resources that describe the methodology. This knowledge model was elicited from an expert in Nuclear Cardiology as the basis for the creation of an expert system on the diagnostic process. The expert had the recurring task of teaching other physicians his method of diagnosing heart disease, based upon the methodology represented in the knowledge model. A decision was made to utilize the knowledge model as instructional content for a course on the subject. However, the model itself was very large, gave no explicit indication regarding where to start or what to do to learn about the domain, and generally seemed intimidating to a number of novices who viewed it. A top level Concept Map that provides an overview of the model was utilized as a starting point for creation of a course on the topic. Fig. 2 presents this Concept Map, which was imported into LEO. The map contains descriptions of the various diagnoses that are made, as well as numerous basic concepts that are pertinent to the diagnostic process, including characteristic patterns that might be seen in the radionuclide images. The Concept Map aVorded a useful conceptual overview of the domain that proved helpful in the process of creating a course on the topic. The map contained a reference to “Ejection Fraction”, a number representing the percentage of blood volume in the left ventricle of the heart that is ejected when the ventricle contracts. The role of this measure is controversial in diagnostic nuclear cardiology, and its use when interpreting the radionuclide images is an important point of discussion. The top-level map also contained mention of diastolic increase and systolic decrease
556
J.W. CoVey / Computers & Education 48 (2007) 548–566
Fig. 2. The top level Concept Map from NUCES.
rate images, images taken from the two phases of the cardiac cycle. Both diastolic and systolic images are included in a study, and they present diVerent evidences in the process of reaching a diagnosis. Another fundamentally important distinction that this Concept Map made explicit is among normal wall motion, generalized abnormalities, and segmental abnormalities in heart wall motion. This distinction is central in the diagnostic process and in the Concept Map. The map also elaborated the various disease states that constitute a diagnosis. 4.3. The revised Concept Map Although the original map represented a good starting place that contributed substantially to the creation of an Organizer, it needed to be improved in several ways. The Wrst feature of note was found in the top right portion of the map, a discussion of the counts of gamma particles in a bolus solution. This is a low level description of how the images are made and not really salient at a global, general level of description. This discussion is absolutely necessary at some level, but its presence in a top-level map or as a basic topic in a course regarding how to interpret the images, contributes little to a broad understanding of important characteristic patterns in the images, or the formation of a diagnosis. Such information was deemed to be noise-a distraction from the more basic concepts to which attention should be drawn Wrst. This part of the Concept Map was
J.W. CoVey / Computers & Education 48 (2007) 548–566
557
removed from the Organizer, with the intention of presenting the information in the introductory materials associated with the Wrst topic in the course. A second interesting feature of the original map was that some fundamental topics were omitted. The map included most of the major disease diagnoses that the Nuclear Cardiologist makes: “Normal Wall Motion”, “Ischemic Heart Disease”, “Cardiomyopathy”, “Mitral Valve Prolapse”, “Valve Disease”, and “Severe Left Ventricular Dysfunction”. However, the map contained no mention of a state diagnosed t20% of the time, “Non-speciWc Wall Motion Abnormalities”. The map included the concepts of diastolic and systolic rate images, but it did not contain the Mean Transit Time images (MTT) which, while somewhat more diYcult to interpret than the rate images, are highly diagnostic, and often critical to the formulation of an accurate diagnosis on borderline cases. A third interesting feature was the presence of relevant but rarely used information at the top level. A characteristic pattern in the functional images called “Ice Cream Cone”, which is seen in perhaps 1 in 200 patients, was accorded the same status as the idea of “Blue Fingers”, a pattern that is seen in more than half the patients. Generally speaking, the frequency of occurrence would not be deemed an adequate measure of importance. However, in this case, a judgment was made that the concept might be addressed elsewhere and omitted from this very general view of the knowledge domain. The fundamental distinction between segmental and generalized abnormalities was preserved and highlighted (these two concepts became topic nodes later) but their locations in the map were rearranged to facilitate adding the concept “Non-speciWc Wall Motion Abnormalities”, and linking it into the concepts “Normal Wall Motion”, “Segmental Abnormalities”, and “Blue Fingers”. The concepts “MTT images” and “EDP-ESP” images were added as well. These additions completed the set of diagnoses and images that are used in the process of arriving at them. The omitted concept “Ice Cream Cone” was already prominently featured in a Concept Map in the model that is devoted to the topic of “Valve Disease”. The original map was modiWed to reXect these changes, and the map in Fig. 3 resulted. Along with the additions, the deletions essentially “leveled” the concepts in the map so that it contained all the fundamental concepts at approximately the same level of generality. 4.4. Alternative course organizations An analysis of the revised Concept Map led to the conclusion that at least two course organizations were possible. Both required laying a basic groundwork to the knowledge domain by discussing the sorts of images used in the test, and the broad categories of diagnoses that are made. In both of the Organizers, the introduction is indicated by the root node, which contains basic information on the radionuclide test itself. The information about counts in a bolus solution that had been removed from the original Concept Map was added here. Following this introduction, the distinction is made among normal wall motion, segmental and generalized abnormalities, and the role of ejection fraction in making the diagnosis. At this point, the two Organizers diverge. The layout of the Organizer presented in Fig. 4 assumes a strategy of elaborating the various types of images that are viewed in the course of the study. In the context of the distinctions among normal wall motion, segmental and generalized abnormalities, the student learns about the types of images that are perused, and gains an understanding of the characteristic patterns that might be
558
J.W. CoVey / Computers & Education 48 (2007) 548–566
Fig. 3. The revised top level Concept Map from NUCES.
seen in the various images. From within this framework, the student gains an understanding of the disease states that the patterns in these images indicate. This approach works from a study of image types and patterns that are consistently seen in images to details regarding how these patterns relate to the disease states. The Organizer presented in Fig. 5 reXects a strategy of elaborating the various disease states that might be diagnosed rather than the various categories of images that are viewed in the diagnostic process. In the context of the distinctions among normal wall motion, segmental and generalized abnormalities, each of the diagnostic states is considered. The relationships among the various disease states that might be diagnosed and the characteristic patterns that might be seen in the various images are made explicit in the Organizer, and elaborated in the accompanying materials. This approach works from diVerences in the disease states back to the patterns associated with the various disease states that are seen in the images. The two Organizers described here provide two diVerent approaches to learning the same material. Either of the two Organizers provides an explicit rendering of the major concepts in the course of study and how they interrelate, a good starting point for meaningful learning of global, structural knowledge of the course. Fig. 6 presents the student view of the Organizer from Figs. 1 and 5. The student’s view provides status bars on the left sides of the topic boxes to indicate completion status of the topic, which is one of {completed, current, ready, not ready}. These are displayed in the student view each time the student logs onto an Organizer, and they are updated as the student works
J.W. CoVey / Computers & Education 48 (2007) 548–566
559
Fig. 4. The Wrst Organizer from NUCES.
through the course and completes topics (CoVey, 2000; CoVey & Cañas, 2003). The palette at the top labeled “Display Status” enables the student to show or hide subsets of the Organizer. SpeciWcally, the user can show or hide the completed, current, ready, or not ready status topics, or the explanation nodes and linking phrases. The motivating idea behind this capability is to provide the learner with a simple means of customizing the information display.
5. Fostering reuse of course content LEO supports the reuse of both course descriptions and course content (CoVey & Cañas, 2001). The concise, explicit representation facilitates identiWcation of useful groups of topics that might be reused. These can be selected in the editor, copied, and pasted into a new course Organizer, with all existing links to course resources preserved. Additional topics may then be pre-pended, appended or interspersed with the existing materials. Additionally, LEO can be used for semiautomated generation of metadata records for course materials. The following sections illustrate
560
J.W. CoVey / Computers & Education 48 (2007) 548–566
Fig. 5. The second Organizer from NUCES.
these capabilities with material from a course in Managerial Decision Making, and the NUCES course described earlier. 5.1. Reusing course description components Fig. 7 contains an example of a course on regression models for decision-making. This course starts with an introductory topic (on the left-hand side of Fig. 7) that is followed by the topic “Business and Economic data”. In turn, that topic is followed by several topics on modeling, two topics pertaining to datasets, and descriptive statistics. Following these introductory topics, the course progresses through a series of topics on various regression models, leading to a discussion of how to make decisions based on the forecasts that these models provide. The links to content are indicated by the icons beneath the topics. Fig. 8 presents an organizer for a course in time series models. This course utilizes some of the same introductory material as the course presented in Fig. 7, in particular, topics covering modeling, types of data, and descriptive statistics. The starting point for development of the course presented in Fig. 8 was the introductory material from the course depicted in Fig. 7. The topics and explana-
J.W. CoVey / Computers & Education 48 (2007) 548–566
561
Fig. 6. The Learner’s view of the Organizer in Fig. 1.
tions on time series models that are speciWc to this course were added to the general introductory material. The links to content for the topics that were reused were already established and retained. Instructional content could be added to or deleted from any of the reused topics by selecting the topic and choosing to edit the resources associated with the topic. This form of organizer facilitates recombining previously created content to create new courses. If a variety of media exist on a topic, these media can be included in varying combinations, depending on the audience. If a course on a diVerent subset of a domain is desired, explanatory items can be made into topics and topics can be made into part of the explanatory component. 5.2. Cataloguing instructional resources for reuse LEO provides capabilities to create metadata records for the entire course and to scan through the list of course topics and resources, creating records for them as well. Fig. 9(a) presents a part of the metadata record for the Wrst topic in the NUCES course. This metadata record is structured
562
J.W. CoVey / Computers & Education 48 (2007) 548–566
Fig. 7. An organizer for a course on Regression Analysis.
Fig. 8. An organizer for a Time Series course that reuses content from the Regression Analysis course.
according to Dublin Core (DCMI, 2005) standards and the format described in the ADL/IMS speciWcation, as presented in the Sharable Content Object Reference Model (SCORM, 2005). When the metadata records for individual topics are created, the links to course content are read, and metadata records for the content are created as well. The course metadata record is placed in a subdirectory that contains all the metadata for the course. A subdirectory within that directory is created for each of the topics in the course, and a topic metadata record is placed in the subdirectory. The metadata records for the individual course content items that are linked to the topic are placed in the subdirectory for that topic. Fig. 9(b) illustrates the directory structure that is created for the metadata records associated with the NUCES course.
J.W. CoVey / Computers & Education 48 (2007) 548–566
(a) A metadata record.
563
(b) The metadata folders.
Fig. 9. A typical template metadata record and a typical folder structure for a course.
The metadata records contain a variety of information that can be automatically determined from the system. The names of the course, topics, and individual media are taken from the Wlename of the course Organizer, the topic labels in the organizer, and the names of the media that are used in the link to the content. The course instructor is the default value for creator. If an individual resource is imported into CMapTools, the person importing it has the option to Wll in a description Weld that can be used for the description of the item in the metadata record. The metadata generator software reads the directory entry for the Wle to determine the creation date of the Wle. The software provides editable default values for copyright status and placeholders for keywords that might be associated with the media, topic, or course. These values may also be edited.
6. Summary and discussion This paper contains a description of a meta-cognitive tool named LEO that allows the instructional designer to create course organizations for face-to-face, hybrid, or online courses. LEO is part of a Concept Map-based knowledge modeling toolkit named CmapTools that is in on-going development at the Institute for Human and Machine Cognition (IHMC), The University of West Florida. LEO aVords the course designer a graphical depiction of both topics in a course, and additional explanatory elements that elaborate relationships among the topics. The editor enables the course designer to add, edit, or remove concepts that are topics or explanatory nodes, to add, view and delete accompanying learning resources, and to set completion criteria for a topic.
564
J.W. CoVey / Computers & Education 48 (2007) 548–566
A drag-and-drop capability simpliWes adding learning resources to the topics. The Organizer’s editor presents capabilities to switch elements from explanatory items to topics and back, all in a graphical rendering of the course organization. The editor allows the designer to rearrange the elements in a course by simple operations such as selecting and dragging. We have presented a method that enables a courseware designer to start with a Concept Map of a course and to convert the map into an Organizer. The process involves two major activities. The Wrst activity is meant to ensure that all the salient concepts are in the map, and that overly detailed or irrelevant concepts are removed. The second part of the process involves identiWcation of the concepts that are to be the designated topics, and the sequencing of those topics that have prerequisite relationships with others. Course organizations are represented in a fashion that depicts those dependency relationships that the designer deems to be necessary, rather than an arbitrary sequence of topics typical of current courses, whether Web-based or not. One of the strengths of the editor is the visual depiction of the course that it aVords. This representation has proven useful to course designers in the process of determining how the course should be assembled – what the topics should be, how the topics should be sequenced, etc. The visual representation enables the designer to view a concise, minimal depiction of materials that might be included and to make decisions regarding which to include and which to omit. The tool puts all the information in front of the designer in a visual representation that fosters reXection on what the important concepts are in the course versus those that are secondary. The depiction also helps the instructor to include and assess the available instructional materials, the appropriate places to test or to require deliverables, etc. This approach to course organization holds the promise to facilitate the reuse of electronic instructional content by enabling the course designer to recombine course content to suit the needs of various audiences. The editor enables the instructor to copy and paste pieces of courses to tailor topics and content to diVerent audiences, and to try “what if” scenarios for course organizations. DiVerent versions of courses can be created, stored or archived, and shared over the Internet with colleagues and students. The ease with which course designers can identify, access, evaluate materials, and associate them with topics in courses also fosters reXection on basic course content. The software also provides semi-automated generation of Dublin Core Compliant XML metadata representations of the resources. These records can be used in html that wraps the resources themselves or in a metadata repository. Several Organizers have been produced and used by faculty at the University of West Florida, in knowledge domains as diverse as Managerial Decision Making (a graduate level statistics course), an upper level undergraduate English course in Shakespeare, an Introductory Psychology course, an Introduction to Computer Science, Data Structures, and NUCES. A fairly wide range in terms of the number of topics was noted. However, all of these courses exhibit signiWcant non-linearities in the arrangement of topics, indicating quite clearly that professors have fundamentally diVerent conceptualizations of courses than the linear ones typically presented in the syllabi of their courses.
Acknowledgements I thank Alberto Cañas for his help and guidance in the formulation of many of the ideas behind this work. I thank the CmapTools development team and Chad CarV, Michael Webb, John Wernicke,
J.W. CoVey / Computers & Education 48 (2007) 548–566
565
and Michael Wooten for their work on the implementation of LEO. This work was supported in part by a grant from the Chief of Naval Education and Training, Pensacola, FL. References Ausubel, D. P. (1968). Educational psychology: A cognitive view. New York: Rinehart and Winston. Blankenship, J., & Dansereau, D. F. (2000). The eVect of animated node-link displays on information recall and session evaluation. Journal of Experimental Education, 68(4), 293–308. Buzan, T., & Buzan, B. (1996). The mind map book: How to use radiant thinking to maximize your brain’s untapped potential. New York: Plume. Cañas, A. J., CoVey, J. W., Reichherzer, T., Hill, G., Suri, N., & CarV, R., et al. (1998). El-Tech: a performance support system with embedded training for electronics technicians. In Proceedings of the 11th Florida AI research symposium (FLAIRS ’98) (pp. 79-83), Sanibel Island, FL. Cañas, A. J., Hill, G., CarV, R., Suri, N., Lott, J., & Eskridge, T., et al. (2004). CmapTools: a knowledge modeling and sharing environment. In Proceedings of CMC 2004, the Wrst international conference on concept mapping, Pamplona, Spain, September 14–17, 2004. CoVey, J. W. (2000). LEO: a learning environment organizer to accompany constructivist knowledge models. Unpublished doctoral dissertation, The University of West Florida, Pensacola, FL. CoVey, J. W., & Cañas, A. J. (2001). Tools to foster course and content reuse in online instructional systems. In Proceedings of WebNet 2001: World conference on the www and internet, Orlando, FL, October 23–27, 2001. CoVey, J. W., & Cañas, A. J. (2003). LEO: A Learning Environment Organizer to support computer mediated instruction. Journal of Educational Technology Systems, 31(3), 275–290. CoVey, J. W., & Webb, M. (2002). AMEE: annotated, multi-modal electronic evaluation for dynamic student modeling. In Proceedings of e-learn 2002, world conference on e-learning in corporate, government, healthcare, and higher education, Montreal, Canada, October 19–23, 2002. DCMI. (2005). The Dublin Core Metadata Initiative: making it easier to Wnd information. Available from http:// www.dublincore.org/. Fisher, K. M. (1990). Semantic networking: The new kid on the block. Journal of Research in Science Teaching, 27(10), 1001–1018. Fisher, K. M. (2000). SemNet software as an assessment tool. In J. J. Mintzes, J. H. Wandersee, & J. D. Novak (Eds.), Assessing science understanding. San Diego: Academic Press. Ford, K. M., CoVey, J. W., Cañas, A. J., Turner, C. W., & Andrews, E. J. (1996). Diagnosis and explanation by a Nuclear Cardiology Expert System. International Journal of Expert Systems, 9(4), 499–506. Hedberg, J. (1997). Employing cognitive tools within interactive multimedia applications. In Proceedings of selected research and development presentations at the 1997 national convention of the association for educational communications and technology, 19th, Albuquerque, NM, February 14–18, 1997. Hyerle, D. (2000). A Weld guide to using visual tools. FOUND IN ERIC microWche ED443784. Kasowitz, A. (2000). Tools for automating instructional design. Educational Media and Technology Yearbook, 25, 49–52. Lambiotte, J., Skaggs, L., & Dansereau, D. (1993). Learning from lectures: EVects of knowledge maps and cooperative review strategies. Applied Cognitive Psychology, 7, 483–497. Macromedia. (2003). Macromedia Authorware 7. Available from http://www.macromedia.com/software/authorware/. Mind Manager. (2005). Mind Manager: The visual tool for brainstorming and planning. Available from http://www.mindjet.com/us/. Nkambou, R., Frasson, C., Gauthier, G., & Rouane, K. (2001). An Authoring Model and Tools for Knowledge Engineering in Intelligent Tutoring. Journal of Interactive Learning Research, 12(4), 323–357. Novak, J. D. (1998). Learning, creating, and using knowledge: Concept maps as facilitative tools in schools and corporations. Mahweh, NJ: Lawrence Erlbaum Associates. Novak, J. D., & Gowin, D. B. (1984). Learning how to learn. Cambridge, UK: Cambridge University Press. Novak, J. D. & Iuli, R. J. (1991). The use of meta-cognitive tools to facilitate knowledge production. In A paper presented at the fourth Florida AI research symposium (FLAIRS ’91), Pensacola Beach, FL, May, 1991.
566
J.W. CoVey / Computers & Education 48 (2007) 548–566
Ritter, S., & Blessing, S. B. (1998). Authoring Tools for Component-Based Learning Environments. Journal of the Learning Sciences, 7(1), 107–132. SCORM. (2005). Advanced distributed learning. Available from http://www.adlnet.org/index.cfm?fuseaction D scormabt. Visual Mind. (2005). Visual Mind: the software that helps you think. Available from http://www.visual-mind.com/.