- Email: [email protected]
Hypertext authoring and visualization
Int. J. Human-Computer Studies (2000) 53, 809}825 doi:10.1006/ijhc.2000.0414 Available online at http://www.idealibrary.com on
Hypertext authoring and visualization MARGIT POHL AND PETER PURGATHOFER Department for Design and Technology Assessment, University of Technology Vienna, Favoritenstrasse 9, A-1040 Wien, Austria. email: +margit,purg,@igw.tuwien.ac.at. (Received 31 January 2000, and accepted in revised form 31 May 2000) Visualizing information structure plays an increasing role in hypertext systems. Empirical evidence shows that such visualizations support the users or readers of hypertext systems. The following study investigates whether such information visualizations in the form of two-dimensional graphical overview maps also support the writers of hypertext documents. It also seems plausible to assume that the process of writing text changes when graphical overview maps integrated into hypertext systems are used. We also analysed the relationship between the authoring process and the character of the "nished documents. In general, it can be said that visualizing the information structure inherent in a document plays an important role in hypertext authoring. It was the only innovative feature of the hypertext authoring system we developed which was used intensively by almost all the authors. In contrast to that, the "nished documents were rather traditional. Authors did not exploit the novel features of hypertext/hypermedia systems like, e.g. introducing pictures, movies or sounds. On the basis of the analysis of the hypertext authoring process guidelines can be formulated to support writers of hypertext documents. 2000 Academic Press
1. Introduction A growing number of people write their own hypertext documents. Especially in university education hypertext plays an increasing role. It is an open question whether the process of writing changes with the use of hypertext authoring tools. It might be argued that especially the introduction of graphical overview maps or graphical browsers integrated into hypertext systems changes the authoring process. In the following study, we investigated in what kind of activities students engage when they create their own hypertext documents and whether they use a graphical overview map in this process. In addition, we tried to evaluate what the "nished documents looked like and whether there was a relationship between the authoring process and the character of the "nished documents. The graphical overview map was not only a tool for editing the document but also formed a part of the "nished hypertext. Therefore, the question whether visualization of information structures played an important role in hypertext authoring was central for our study.
1071-5819/00/110809#17 $35.00/0
2000 Academic Press
810
M. POHL AND P. PURGATHOFER
1.1. WRITING HYPERTEXT
Writing hypertext di!ers from writing traditional linear text in several ways. Text in hypertext documents is, e.g. more fragmented than text in linear documents. Therefore, graphical overview maps or graphical browsers representing the overall structure of the document play an important role in authoring (and also navigating) hypertext. It is plausible to assume that the use of graphical overview maps during the authoring process in#uences the three key elements found for traditional writing processes: planning, translating and reviewing. Even in traditional linear writing, planning, translating and reviewing are highly integrated processes (Hayes & Flowers, 1980, 1986). Still, it is possible to distinguish between them. Planning encompasses those thinking activities that are required before words are written on paper. The term &&translating'' is used to describe the act of putting ideas into written words. Reviewing involves evaluating and changing the text (Hartley, 1992). In the hypertext authoring processes, it is rather di$cult to distinguish between these key elements. Planning increasingly becomes part of the translation process. Concept maps or graphical overview maps are often used as planning tools even for linear texts on paper. Such methods help authors to create, clarify and order their ideas and concepts (Kellog, 1992). The integration of such planning tools into hypertext authoring systems makes it di$cult to distinguish between planning, translating and reviewing. The graphical overview map is an integral part of the "nal document whereas notes and concept maps produced during the planning stage of traditional writing are usually discarded after the document is "nished. Graphical overview maps or at least clickable tables of contents are an indispensable part of most of today's hypertext documents. Such elements of visualization of information structure seem to be the most fundamental di!erence compared to linear text because the distributed organization of text in hypertext can only be sensibly represented in a visual form. An indication of the importance of graphical forms of representing knowledge might be the intensity with which an overview map or a similar tool is used by authors. Our investigation shows that authors use the overview map to a high degree although they produce documents which are rather traditional in other respects. This is an indication that visualization of knowledge structures is one of the most attractive and usable features of hypertext systems. Hayes and Flowers (1980, 1986) used protocol analysis to develop their model of writing. This method is very time consuming and often depends on the writers' re#ections about their own cognitive processes. In addition, it disturbs the writing process fundamentally. On the other hand, it yields very rich and interesting data. With modern computer technology it is possible to monitor writing processes without the (obvious) intrusion of a human observer. In the investigation described below, we only used data produced by computer monitoring to analyse the authoring process. 1.2. CONCEPT MAPS AND LEARNING
Dillon and Maynard (1995) posit that hypertext or hypermedia systems are less useful than usually assumed, especially when real-world production of text is considered. Nevertheless, there is some empirical evidence that hypertext learning systems can be used e!ectively in certain areas. It can be argued that the use of hypertext instead of
HYPERTEXT AUTHORING
811
traditional linear text supports speci"c intellectual abilities. The success of hypertext in academic education is partly due to the fact that it uses a spatial metaphor. Jonassen (1996) describes hypertext/hypermedia and semantic networking tools as mindtools, that is, as computer-based tools functioning as &&intellectual partners'' with the learner. Mindtools are supposed to engage learners in critical thinking and higher-order learning. Hypertext is especially motivating when students construct their own hypertext documents re#ecting their perspective or understanding of the learning material. Semantic networking or concept maping helps students to analyse their own knowledge structures to integrate new information with what they already know. As mentioned above, in many hypertext authoring systems, especially in those used in an academic context, hypertext and semantic networking or concept mapping are integrated features. There is some evidence that the use of concept maps in learning environments can have bene"cial e!ects. Kozma (1992), Kommers and de Vries (1992), Fisher (1992) and McAleese (1998), who used computer-based concept mapping tools report positive results. Reader and Hammond (1994) argue that concept mapping is a more time-consuming method than traditional forms of learning. Thus, students are motivated to spend more time going through the material and analysing it. This leads to a better performance of those students developing concept maps. Concept mapping and similar methodologies can be supported easily by computer technology but they were originally developed for pencil and paper (see e.g. Novak & Gowin 1984; Novak, 1998). These methodologies are based on the theory that meaningful learning can only occur when new and old knowledge are integrated in a systematic way. The development of concept maps by students is a valuable learning aid for relating new and old knowledge which leads to the construction of structural knowledge (Jonassen, Beissner & Yacci, 1993). Jonassen and Reeves (1996) describe several experiments which support their hypothesis that hypertext/hypermedia systems are a valuable tool for developing higher-order thinking skills. Dee-Lucas (1996) investigated whether the type of content overview or the amount of segmentation of hypertext in#uenced study strategies of students. She compared linear text with a hierarchically structured hypertext and an unstructured hypertext with a menu-like list of unit titles. In general, she concludes that highly structured hypertext including spatial representations of the information organization has advantages and disadvantages for learning. On the one hand, study strategies become more e!ective, on the other hand, students explore less and tend to miss related content material. She suggests that it depends on the goal of study whether the design of an educational hypertext system should emphasize structure or not. From the results reported in the literature cited above one might conclude that hypertext can be used e!ectively in university education in some areas. The goal of the learning process should be that students acquire structural or overview knowledge. In this context, the visual representation of knowledge structures plays a decisive role.
1.3. VISUALIZATION AND HYPERTEXT
Visualization of information structure has been a popular topic in the hypertext literature. As described above, the combination of hypertext with overview maps or graphical browsers is a natural one. The distributed, fragmented nature of hypertext emphasizes
812
M. POHL AND P. PURGATHOFER
the importance of structure and relationships between the units of information contained in nodes. This structure is best represented in a topological manner. Chen (1999) and Chen and Czerwinski (1997) argue that spatial representation is one of the most important ways of information visualization in hypertext development. Marshall, Shipman and Coombs (1994) point out that a visual/spatial metaphor for hypertext enables people to express nuances of structure more easily than in other hypertext systems. They posit that using spatial metaphors in hypertext systems takes advantage of the ability of humans to navigate in physical space. The practice of developing a shared understanding of multiple information sources becomes increasingly important (Marshall & Shipman, 1997). Modern information systems provide an abundance of documents so that people lack the time to read all these documents thoroughly. Marshall and Shipman assume that hypertext systems using a spatial metaphor help people to organize and structure this information. Their empirical investigation supports this assumption. Navigating hypertextual space is often compared to navigating physical space (Dillon, McKnight & Richardson, 1993). Humans develop cognitive maps to "nd their ways in their physical environments (Golledge, 1999). Knowing the environment is a dynamic process. In western cultures physical space is usually represented as a Euclidean metric. This makes it easy to use physical space as a metaphor for semantic space. The concepts of landmarks, routes and surveys which were developed for way"nding behaviour are transferred to hypertextual space. Dillon et al. (1997) point out, however, that the relationship between physical and semantic space is more complex than usually assumed. Physical distance can be measured in a rather straightforward manner. In contrast to that, semantic distance is rather di$cult to assess. &&The extent to which two ideas are related may seem intuitively easy to assess but is unlikely to have a broadly agreed quanti"able metric.'' (Dillon et al. 1997, p. 186.) Furthermore, semantic space is an abstract linguistic concept. In contrast to physical space, it is n-dimensional. It is, therefore, impossible to &&navigate'' semantic space in the same manner as physical space. Dillon et al. conclude that the limits of the spatial metaphor for hypertext have to be considered when investigating navigational behaviour. Kaplan and Moulthrop (1994) also argue that there is a considerable di!erence between physical, &&architectonic'' space and semantic space. Other metaphors might be used in addition to &&navigation'' to get a more complete picture of hypertext usage. Dillon and Vaughan (1997a) point out that it is not obvious what constitutes a landmark in semantic space. In addition, it is not clear whether improved navigational knowledge leads to better performance. They state that a major disadvantage of the navigational metaphor is that the process of meaningful assimilation of knowledge is not considered. Therefore, they suggest the concept of &&shape'' as an alternative to the navigation metaphor. One of the main advantages of this concept is that shape integrates physical form and information content. Realworld tasks usually not only consist in locating information but also some form of meaningful processing. The concept of &&shape'' takes these aspects into consideration. Dillon et al. (1997) admit that the concept of shape is still &&somewhat elusive'' (p. 104). Nevertheless, they think it might prove interesting as it includes both the aspect of physical representation and the aspect of discourse organization. In this, it resembles the concept of schematic superstructures developed by van Dijk and Kintsch (1983). To illustrate the concept of shape they quote an experiment conducted by Dillon and
HYPERTEXT AUTHORING
813
Vaughan (1997b, quoted in Dillon & Vaughan, 1997a) where &&experts were using more complex rationales for their classi"cation than simple perceptual cues. Information content, wording, relationships to a process were all seen as indicators of location.'' (p. 100). It can be concluded that spatial metaphors play an important role in hypertext research. Spatial organization of information becomes especially important for the so-called &&new information workers'', people who work with modern, networked information systems.
2. Description of the investigation Our research in the "eld of hypertext authoring is part of a long-term study. The goal of this study is to gain insights into the application of hypertext-based information systems in teaching and learning. In the course of this project we asked our students to write their seminar papers in hypertextual form instead of traditional, linear paper form. So far, 168 students created hypertext documents using the authoring tool developed at our department. The students study computer science in their third or fourth year. The seminars cover a wide range of topics: technological and technology assessment issues as well as social sciences, psychology, information theory and design. The task of students in our seminars is to read a couple of scienti"c publications and present the underlying issues to the other participants. In addition, they have to prepare a written assignment about these publications, traditionally about 20}30 pages of text. The students were encouraged to utilize novel features of hypertext, as for example the integration of novel features of hypertext. We did not use an experimental approach in a strict sense in order to increase the ecological validity and the practical applicability of our results. The "rst steps were taken using HyperCard on the Macintosh. We found out quite fast that students (who, as computer science students, love to program, but hate to write) get entangled in the programming necessary to generate the structure so much that it actually makes them change the content in favour of an easier way to implement structure. The logical conclusion was to o!er students an authoring tool speci"cally tailored to their needs, the needs of (untrained) hypertext authors.
2.1. THE AUTHORING TOOL
Since no then-existing hypertext environment ful"lled these requirements, we decided to develop our own authoring tools. On top of Apple's &&HyperCard'' for the Macintosh, we implemented a hypertext authoring tool, embedded into a simultaneously developed multi-user networking environment. With these tools, hypertext authoring was raised from the complex task, closely related to programming, to a high-level, interactive task embedded into a context of semantically appropriate tools and actions. Many contraints were introduced in order to liberate the authors of design decisions and to constitute a common look and feel within the "nished hypertext documents. Basically, the tools consisted of two parts which are described in the following; a node editor to write text, create link anchors and embed media into the node, and an overview
814
M. POHL AND P. PURGATHOFER
FIGURE 1. Example of a node in the node editor view.
editor where nodes and links could be created and arranged freely. Figure 1 shows a typical node. The node editor o!ers a "xed, simple layout consisting of a node header (top left), the document header (top right), a main text area (right) and a sidebar for subheadings, comments and link placement. In the main text area, the authors had no choice of text fonts or styles other than plain or bold. While this decision might look like a harsh reduction of the authors' possibilities, we found that in fact it was expanding their decision space: released from the need to choose a font and a style, authors were able to dedicate their time and attention to other decisions. The toolbar at the bottom of the screen o!ers functionality to create and delete links via simple drag-and-drop operations. On the very right, some basic navigation (title node, overview map, help and last visited node) is provided. Finally, there is a mode switch between author and reader view in the top right of the window. In reader view, all but the necessary tools are removed. In addition to these visible functions, a number of administrative functions were available via the usual set of menus. Among others, new nodes could be created via the menus. The second major module of the authoring tools is the map editor or overview map. The basic function of the overview map was to give authors an instrument to create links and nodes in a structural way, and let them arrange nodes freely within the resulting map. No automatic or semi-automatic arrangement was proposed, the layout of nodes, and the resulting network of links is completely under the control of the author. Figure 2 shows a typical overview map. Again, the tools provide a simple and meaningful way to create links and nodes. Nodes can be created via drag-and-drop from the node tool, the link tool allows one to de"ne connections between nodes with the usual rubber-band functionality. Nodes are deleted by dragging them into a trash bin, and basic document information is available via an
815
HYPERTEXT AUTHORING
FIGURE 2. The overview map as authors see it.
&&i''-button. While there are no restrictions for the author in arranging nodes on the map, the arrangement is also without any consequence for the individual nodes or the content therein. To enable authors to test their work without constantly switching between author and reader modes, a modi"er key was dedicated to the emulation of reader mode in author's mode. Whenever an author held down the &&option''-key, he/she was in a &&reader-withinauthor''-mode that let him/her test the layout. 2.2. EVENT RECORDING FUNCTIONALITY
In order to enable us to evaluate the authoring process, all the actions the authors took were recorded using a simple event recording mechanism. The syntax used for event recording allowed for basic information to be provided with each event. An example of a resulting log is shown in Figure 3. While the authoring tools were in use, we were able to record 143 protocols, 95 of which turned out to be correct and usable, containing more than 48.000 single events, or a statistical mean of 408 events per protocol. From these dynamic data we can retrieve some simple statistical information like the average node and link &&age'' (relative to the number of recorded events) in a document, the mode of node creation (from the node view vs. in the overview map), etc. In addition, we retrieved some static information from the documents, like the number of nodes and links. The real challenge, however, is to "nd evaluation methods for the dynamic data that allow deeper insight into the authoring process. In the following, we will describe three methods we used to evaluate these protocols. 2.3. EVALUATION METHODS
2.3.1. Moving window analysis. The "rst method, described in the following, tries to form the basis for a quantitative analysis of the data. The problem with analysing statistically
816
M. POHL AND P. PURGATHOFER
FIGURE 3. Event recording result, showing a sequence of basic actions (change name, edit text, move node in overview map).
the dynamic data we have is that they are impossible to normalize, reduce or categorize without losing some or most of the information within: the dynamic change. With the &&moving window'' method we try to generate a set of data from the protocols that makes it easier to apply statistical methods. The focus of this method is to visualize and understand the basic strategies of the author. To get an analysis of granularity g (the number of overlapping sample intervals) of a protocol with N entries, we de"ne a &&window of observation'' with a length of l"2N/(g#1). We start by counting all di!erent classes of actions within the "rst l actions of the protocol, resulting in an accumulation vector v . Next, we &&move'' the window by an o!set of D"N/(g#1) * half the window size * and repeat the counting, getting v . This procedure is reiterated until the g accumulation vectors are acquired. The result of this process, the accumulation vectors v !v or more conveniently the E accumulation matrix <, can be visualized using a stacked bar graph. In this graph, each bar shows the relative importance of di!erent classes of actions within an interval of 2(g#1) actions, overlapping each of the neighbouring intervals by 50%. In this graph, we can then see the relative importance each class of actions had for the author throughout the creation of the document. The accumulation matrix < can be seen as some kind of redux of the authoring process. As we will show later, di!erent authoring strategies can be seen quite convincingly. Of course, < is only a coarse reduction of the rich information within the original data, leaving behind much detail eventually worth taking into account. However, < can easily be normalized, enabling one to apply more sophisticated statistical methods like cluster analysis. For the investigation described later in this text, we generated moving window analysis with granularity 19, counting "ve classes of actions: text editing (writing and deleting text), make (creating nodes and links), moving (positioning nodes in the overview map), delete (deleting nodes and links) and other actions (e.g. renaming nodes). 2.3.2. Descriptive analysis. In addition to the quantitative analysis based on the data produced by computer monitoring we also conducted a more descriptive analysis (Pohl, 1998) based on the "nished documents. Several features of these documents were evaluated by students of our department. We used two di!erent sets of categories. One set was mainly concerned with the structure of the whole document as it appeared on the overview map. The other set dealt
HYPERTEXT AUTHORING
817
with layout and screen design. The most important question the "rst set of categories tried to answer was whether students adopted traditional hierarchical forms of structuring information or not. Link types are treated insofar as they can clarify this issue. The second set of categories examined the question whether students used layout, screen design or graphical elements to convey a sense of structure. The typographic and layout features which play an important role in this context are highlighting (Mayhew, 1992), grouping (Tullis, 1991; Mayhew, 1992), and the distribution of white and black space on the screen (Watzmann, 1993). Highlighting is a possibility to point out important concepts in the text to the reader. Grouping is a powerful tool to show which elements belong together and which do not. The distribution of white and black space on the screen, which is to a certain extent related to grouping, provides a clear path through the material and gives a conceptual framework of the text (Watzmann, 1993).
3. Results In the following section two sets of results will be presented * descriptive data and dynamic data. The descriptive data contain variables like the &&structure of the overview map'', layout and screen design and link types. These variables describe the "nished documents. The basis for these data was 143 documents. These documents were analysed manually by three of our students. The automatic monitoring tool was the source of the dynamic data which describe the process of hypertext creation. The basis for these data was 95 documents. The reason for the discrepancy between these two numbers is that only 95 protocols resulting from automatic monitoring were usable. 3.1. RESULTS OF THE ANALYSIS OF THE DESCRIPTIVE DATA
Students predominantly used a hierarchical form of structuring information. Hierarchical links are by far the most frequent form of link type (85.2%). There are only three documents with no hierarchical links at all. In contrast to that, there are 83 documents with no non-hierarchical links at all. There are several documents with a considerable amount of non-hierarchical links. This indicates that the majority of students feel more comfortable with the hierarchical organization of text. Others * a minority still * use non-hierarchical links quite extensively. The term link type in this study is perhaps misleading as it is used in the literature (see e.g. deRose, 1989) to characterize link types belonging to a complete system of link classi"cation. In contrast to that, the term link type here refers to a few characteristic linking methods which students used to express their ideas about structure. This concept of link types was not developed theoretically but rather deducted from our practical experience with students' hypertext documents. These categories are supposed to re#ect whether students are able to create meaningful relationships between nodes. 3.1.1. Link between the same term on two nodes. Students very often use this method of linking two nodes. They happen to notice that one and the same term occurs on two nodes in their document. Therefore, they create a link between those two nodes. This process sometimes seems automatic and the resulting links are sometimes rather super#uous: 3804 (76.4%) out of 4977 links are links between the same term on two nodes.
818
M. POHL AND P. PURGATHOFER
3.1.2. Example links. Students fairly often use links to refer to examples which illustrate an idea brought up in the text. 1324 (26.6%) out of 4977 links are of that type. Only 26 out of 143 documents do not contain any examples. Students di!er considerably in the amount of examples they include in their documents. One document even contained 46 examples. 3.1.3. Explanation links. Students also fairly often use links to refer to explanations of terms used in their document. 1890 (37.9%) out of 4977 links are of that type. Only 9 out of 143 documents do not contain any explanatory links. The maximum of explanatory links in a document is 71. The term &&grid'' refers to the categories introduced by Watzmann (1993). She tries to formulate criteria for the quality of screen design. Good screen design, according to her, is highly structured and uses a range of di!erent topological or graphical cues. The term &&gray page'' describes a screen which resembles a traditional book with no organization of the information at all. Chunking means that information is divided into small, meaningful pieces of text. Queuing leads to an even more structured screen. Information is arranged topologically, so that the position of pieces of text conveys meaning. Mixing modes refers to a screen design which includes di!erent modes of expression (text, graphics, pictures, formulas, etc.). Chunking and queuing seem to be very popular with our students. This shows that they have realized that a computer screen o!ers possibilites which do not exist in a book. Nevertheless, the students we observed in our study did not use pictures, graphics or other modes very often. Only 41 (29%) documents out of 143 use graphics or pictures.
3.2. RESULTS OF THE ANALYSIS OF THE DYNAMIC DATA
We could observe several interesting and unexpected results when we analysed the activities students were engaged in when they wrote their hypertext documents. Ninety"ve documents were available for analysis. The categories of analysis were edit text, make, move, delete and other activities. &&Edit text'' refers to the traditional process of writing and editing text in a text "eld on a node, &&make'' refers to making links or nodes, &&move'' describes those activities which are related to moving nodes around on the overview editor, &&delete'' refers to revision of the document (especially deleting nodes and links) and &&other actions'' are all actions not covered in the other categories (e.g. renaming nodes, etc.). Figure 4 shows the average values of all analysed documents for 19 consecutive phases of hypertext creation. This "gure is rather misleading as it conveys the impression that all activities are distributed evenly across all phases (from 1 to 19). The "gures for the single documents are much more diverse. Students very often concentrate on one activity in one period or in a succession of several periods which makes the diagrams look more chaotic (see below). In addition, there are fundamental individual di!erences between students. Some students mainly write text during most periods, others prefer moving their nodes in the overview editor. Nevertheless, it is obvious that revising is an activity which is not very popular among students in general. It must be mentioned, however, that our concept of revision is more restricted than that of revision of linear text. This is due to technical reasons.
819
HYPERTEXT AUTHORING
FIGURE 4. Average values of all analysed documents/relative proportion of activities: , move; , delete; , other actions.
FIGURE 5. Document &&Das Leib}Seele Problem'': relative proportion of activities: , move; , delete; , other actions.
, edit text;
, make;
, edit text;
, make;
Figure 5 shows, on the other hand, that the visual appearance of their overview maps is very important for the students. A relevant part of their activities is made up of moving nodes around on the overview map (the links automatically move with the nodes). These actions can be interpreted in two ways. On the one hand, space and spatial relationships on the overview map are something meaningful for the students. Nodes which &&belong to each other'' are positioned near each other. Partly, students even produce meaningful sequences of nodes. Figure 6 shows a slightly chaotic overview map. Nevertheless, the authors of this overview map tried to indicate the sequence of the chapters by positioning them beside each other. The nodes &&Kapitel 1'' through to &&Kapitel 6'' refer to Chapters 1}6. On the other hand, students sometimes simply try to make the overview map look better or more orderly. The dynamic data we got by monitoring the students' activities show that students sometimes simply try to avoid the lines representing links from crossing each other. Chen and Czerwinski (1998) also point out that minimizing the number of link crossings and overlapping is one of the criteria for aesthetic layout. It is obvious from our data that visualization of information is very important for students
820
M. POHL AND P. PURGATHOFER
FIGURE 6. Document &&Das Leib}Seele Problem'': overview map.
developing hypertext documents. This is also supported by other data from our analysis. The monitoring tool we developed can distinguish between activites which took place &&on the overview map'' and activities which took place in the document itself. It was possible to make nodes &&on the overview map'' by drag and drop or by clicking on a &&node symbol''. On the other hand, it was also possible to create nodes by using the menu which is the usual Hypercard method. In 87.85% of all cases students created nodes on the overview map. Again, this is an indication that the visualization of the information is important for authors of hypertext documents, at least for the students of the sample we investigated. It is obvious that students did not &&navigate'' their document when they created it. They do not try to locate any knowledge but rather to create knowledge. They already possess survey knowledge without going through the stages of landmark and route knowledge. The concept of shape as developed by Dillon et al. (1997) seems to describe students' behaviour better than the navigational metaphor. In the following, we will discuss several cases which deviate considerably from the average. After a careful inspection of the data derived from the dynamic analysis we found that there are probably several di!erent styles of writing hypertext. Large groups either concentrate on creating links and nodes or on designing their overview maps. Smaller groups prefer to write text or revise their document. So far, this is circumstantial evidence. We intend to analyse this phenomenon statistically in the future. The following examples are typical cases for the above-mentioned groups. We want to describe the activities in which students engaged and how these activities related to the document which was produced. We will try to show that there is a relationship between the way in which documents are created and how the "nished documents are structured. As there is little experience in this area, we want to use a case-based approach to get a "rst impression of the problems which might be encountered in this area. The "rst example is a document called &&Das Leib}Seele Problem'' (the mind-body problem * Figures 5 and 6). It is outstanding insofar as the authors wrote much text but did not care very much what their overview map looked like. They moved nodes on the overview map very rarely. As a consequence, their overview map is very chaotic. In the
821
HYPERTEXT AUTHORING
FIGURE 7. Document &&Interaction Styles'': relative proportion of activities: , delete; , other actions.
, edit text;
, make;
, move;
beginning, they tried to impose some order. This is obvious from the example already described above. They tried to arrange all the chapter beginnings in one linear sequence (Kapitels 1}6). Then they apparently realized that they did not have enough space and gave up completely to edit the overview map. Especially the positions of the nodes representing the last chapters are very confusing for the readers. This is also obvious from Figure 5 which shows that they almost stopped completely moving nodes around in the last two phases. They apparently also stopped revising their document very early (from phase 7 onward). On the other hand, the text of their document is very comprehensive and interesting although the document in general is slightly confusing. The chaotic nature of the overview map adds to this confusion. This document suggests that hypertext authoring not only consists of creating nodes and writing text on these nodes but rather producing a well designed overview map is a crucial part of hypertext authoring. The next document is completely di!erent (see Figures 7 and 8). The author made many nodes and links but also deleted many of them later. He revised much more than the average student. Nevertheless, the document is slightly confusing. This document is an example for the fact that revising as such does not lead to good products. It also depends very much on the quality of the revision process. The overview map shows that the document is completely non-hierarchical. The reason for this is that the author made links from every node to almost all other nodes. There is no clear reason for this. The links seem to have been produced more or less at random. The author wrote fairly little text * text which is also confusing. One advantage of the document is that the author produced a fairly clear overview map. He moved the nodes around rather often, especially in the "nal stages of the authoring process. Therefore, the overview map is rather attractive and clear. The author of the last example apparently took great care to make his overview map look attractive. The relative proportion of move operations in Figure 9 is high above the average. Especially in the "nal stages of document creation there are phases (12}14) which almost entirely consist of moving nodes around on the screen. Consequently, the overview map is very clear and understandable (Figure 10). It is entirely hierarchical. In addition, the author also revised his document quite heavily in the last two phases. The document is clear and well-structured. There is not much text, but the text is to the point
822
M. POHL AND P. PURGATHOFER
FIGURE 8. Document &&Interaction Styles'': overview map.
FIGURE 9. Document &&User-Centred-Design Principles'': relative proportion of activities: , make; , move; , delete; , other actions.
, edit text;
FIGURE 10. Document &&User-Centred-Design Principles'': overview map.
and understandable. The links are well-designed and sensible. This case also shows the importance of including methods of visualizing knowledge structure into hypertext authoring systems. In addition, the e!ects of the revision process apparently were positive for the document.
HYPERTEXT AUTHORING
823
4. Conclusion If we compare the results from the descriptive and from the dynamic analysis one conclusion can be drawn. There was one innovative feature of our hypertext system used extensively by students when authoring hypertext documents. This feature was the overview map. Students create most of the nodes and links on the overview map and spend quite a lot of time editing the appearance of this map. There are other possibilities made available by the system which were used by a few students quite heavily like creating non-hierarchical links or introducing pictures, other graphics, movies or sounds. These novel features were apparently less attractive than the overview map. The documents which were created were often rather traditional. These results indicate that visualizing information structure is one of the most important new features of hypertext systems. Another conclusion which might be drawn from the dynamic data is that there are several di!erent styles of creating hypertext documents. The diagrams showing the relative proportions of activities during the process of writing hypertext convey a slightly contradictory picture. From the average values one might conclude that there are no signi"cant di!erences between various stages of hypertext creation. This is not the case. It must be mentioned, however, that there are no uniform patterns across students at which point in time certain activities are carried out. In addition, most of the students seem to have preferences for one activity. So far, this is only circumstantial evidence but we intend to investigate the question of hypertext creation styles in more detail in the future. The relationship between style of hypertext creation and design of the "nal documents is probably rather complex. Our experience so far suggests that students should engage in all the di!erent activities to a certain extent and not concentrate on only one to produce high-quality documents. This probably indicates that highly individualistic styles of hypertext creation might be a disadvantage for the quality of the document. More research in this area seems to be necessary. We want to thank Michaela Schuster, Alexander Sogl and Alexander Stadler who are students at our department and helped us with the analysis of the descriptive data.
References CHEN, C. (1999). Visualizing semantic spaces and author co-citation networks in digital libraries. Information Processing and Management, 35, 401}420. CHEN, C. & CZERWINSKI, M. (1997). Spatial ability and visual navigation: an empirical study. ¹he New Review of Hypermedia and Multimedia, 3, 67}89. CHEN, C. & CZERWINSKI, M. (1998). From latent semantics to spatial hypertext * an integrated approach. Proceedings of the Hypertext +98, pp. 77}86, Pittsburgh, USA. DEE-LUCAS, D. (1996). E!ects of overview structure on study strategies and text representation for instructional hypertext. In J.-F. ROUET, J. J. LEVONEN, A. DILLON & R. J. SPIRO, Eds. Hypertext and Cognition, pp. 73}107. Mahwah, NJ: Lawrence Erlbaum. VAN DIJK, T. & KINTSCH, W. (1983). Strategies of Discours Comprehension. New York: Academic Press. DILLON, A. & MAYNARD, S. (1995). &Don't forget to put the cat out' * or why collaborative authoring software and everyday writing pass one another by. ¹he New Review of Hypermedia and Multimedia, 1, 135}153. DILLON, A., MCKNIGHT, C. & RICHARDSON (1997). Space * the "nal chapter or why physical representations are not semantic intentions. In C. MCKNIGHT, A. DILLON &
824
M. POHL AND P. PURGATHOFER
J. RICHARDSON, Eds. Hypertext a Psychological Perspective, pp. 169}191. Chichester, UK: Ellis Horwood. DILLON, A. & VAUGHAN, M. (1997a). &It's the journey and the destination': shape and the emergent property of genre in evaluating digital documents. ¹he New Review of Hypermedia and Multimedia, 3, 91}106. DILLON, A. & VAUGHAN, M. (1997b). Mental representations of complex information spaces. Poster Sessions: Abridged Proceedings of the HCI International +97, 7th International Conference on Human}Computer Interaction jointly with 13th Symposium on Human Interface (Japan), p. 96. West Lafayette, IN: School of Industrial Engineering, Purdue University. FISHER, K. (1992). SemNet: a tool for personal knowledge construction. In P. A. KOMMERS, D. H. JONASSEN & J. T. MAYES, Eds. Cognitive ¹ools for ¸earning, pp. 63}75. Berlin, Heidelberg, New York: Springer. GALBRAITH, D. (1992). Conditions for discovery through writing. In M. SHARPLES, Ed. Computers and =riting. Issues and Implementations, pp. 45}71. Dordrecht, Boston, London: Kluwer. GOLLEDGE, R. G. Ed. (1999). =ay,nding Behavior. Baltimore, London: John Hopkins University Press. HARTLEY, J. (1992). Writing: a review of the research. In J. HARTLEY, Ed. ¹echnology and =riting. Readings in the Psychology of =ritten Communication, pp. 18}36. London, Philadelphia: Jessica Kingsley Publishers. HAYES, J. R. & FLOWERS, L. S. (1980). Identifying the organisation of writing processes. In L. W. GREGG & E. R. STEINBERG, Eds. Cognitive Processes in =riting, pp. 3}30. Hillsdale, NJ: Lawrence Erlbaum. HAYES, J. R. & FLOWERS, L. S. (1986). Writing research and the writer. American Psychologist, 41, 1106}1113. JONASSEN, D. H. (1996). Computers in the Classroom. Mindtools for Critical ¹hinking. Englewood Cli!s, NJ, Columbus, OH: Merill. JONASSEN, D. H., BEISSNER, K. & YACCI, M. (1993). Structural Knowledge. ¹echniques for Representing, Conveying, and Acquiring Structural Knowledge. Hillsdale, NJ, Hove and London: Lawrence Erlbaum. JONASSEN, D. H. & REEVES, T. C. (1996). Learning with technology: using computers as cognitive tools. In D. H. JONASSEN, Ed. Handbook of Research for Educational Communication and ¹echnology, pp. 693}719. New York: Simon & Schuster}Macmillan. KAPLAN, N. & MOULTHROP, S. (1994). Where no mind has gone before. Ontological design for virtual spaces. In ECHT+94 Proceedings, ACM European Conference on Hypermedia ¹echnology, pp. 206}216. KELLOG, R. T. (1992). Designing Idea Processors for Document Composition. In J. HARTLEY, Ed. ¹echnology and =riting. Readings in the Psychology of =ritten Communication, pp. 181}199, London, Philadelphia: Jessica Kingsley Publishers. KOMMERS, P. A. M. & DE VRIES, S. A. (1992). TextVision and the visualization of knowledge: school-based evaluation of its acceptance at two levels of schooling. In P. A. KOMMERS, D. H. JONASSEN & J. T. MAYES, Eds. Cognitive ¹ools for ¸earning, pp. 33}62. Berlin, Heidelberg, New York: Springer. KOZMA, R. B. (1992). Constructing knowledge with learning tool. In P. A. KOMMERS, D. H. JONASSEN & J. T. MAYES, Eds. Cognitive ¹ools for ¸earning, pp. 23}32, Berlin, Heidelberg, New York: Springer. MCALEESE, R. (1998). The knowledge arena as an extension to the concept map: re#ection in action. Interactive ¸earning Environments, 6, 1}22. MARCHIONINI, G., NEUMAN, D. & MORRELL, K. (1994). Directed and undirected tasks in hypermedia. Is variety the spice of learning? In T. OTTMANN & I. TOMEK, Eds. Proceedings of ED-MEDIA +94, pp. 373}378. Charlottesville, USA: AACE. MARSHALL, C. C. & SHIPMAN, F. M. (1997). Spatial hypertext and the practice of information triage. Proceedings of the Hypertext +97. pp. 124}133, Southhampton, UK. MARSHALL, C. C, SHIPMAN, F. M. & COOMBS, J. H. (1994). VIKI: spatial hypertext supporting emergent structure. Proceedings of the European Conference on Hypermedia ¹echnologies. pp. 13}23, Edinburgh, Scotland.
HYPERTEXT AUTHORING
825
MAYHEW, D. (1992). Principles and Guidelines in Software ;ser Interface Design. Englewood Cli!s, NJ: Prentice-Hall. NOVAK, J. D. (1998). ¸earning, Creating, and ;sing Knowledge. Mahwah, NJ, London: Lawrence Erlbaum. NOVAK, J. D. & GOWIN, D. B. (1984). ¸earning how to ¸earn. Cambrigde: Cambridge University Press. POHL, M. (1998). Hypermedia as a cognitive tool. In T. OTTMANN & I. TOMEK, Eds. Proceedings of the ED-MEDIA +98, pp. 1089}1094, Charlottesville, USA: AACE. POHL, M., PRENNER, P. & PURGATHOFER, P. (1995). Hypermedia in education * monitoring the development of hypermedia documents. Psychology ¹eaching Review, 4, 142}152. READER, W. & HAMMOND, N. (1994). Computer-based tools to support learning from hypertext: concept mapping tools and beyond. Computers in Education, 22, 99}106. DEROSE, S. (1989). Expanding the notion of links. In Proceedings of the Hypertext+89 Conference, pp. 249}258. TULLIS, T. S. (1991). Screen design. In M. HELANDER, Ed. Handbook of Human}Computer Interaction (2nd edn). pp. 377}411. Amsterdam: North-Holland. WATZMANN, S. (1993). Presenting information visually. Hypertext 93, Tutorial 22.
Hypertext authoring and visualization MARGIT POHL AND PETER PURGATHOFER Department for Design and Technology Assessment, University of Technology Vienna, Favoritenstrasse 9, A-1040 Wien, Austria. email: +margit,purg,@igw.tuwien.ac.at. (Received 31 January 2000, and accepted in revised form 31 May 2000) Visualizing information structure plays an increasing role in hypertext systems. Empirical evidence shows that such visualizations support the users or readers of hypertext systems. The following study investigates whether such information visualizations in the form of two-dimensional graphical overview maps also support the writers of hypertext documents. It also seems plausible to assume that the process of writing text changes when graphical overview maps integrated into hypertext systems are used. We also analysed the relationship between the authoring process and the character of the "nished documents. In general, it can be said that visualizing the information structure inherent in a document plays an important role in hypertext authoring. It was the only innovative feature of the hypertext authoring system we developed which was used intensively by almost all the authors. In contrast to that, the "nished documents were rather traditional. Authors did not exploit the novel features of hypertext/hypermedia systems like, e.g. introducing pictures, movies or sounds. On the basis of the analysis of the hypertext authoring process guidelines can be formulated to support writers of hypertext documents. 2000 Academic Press
1. Introduction A growing number of people write their own hypertext documents. Especially in university education hypertext plays an increasing role. It is an open question whether the process of writing changes with the use of hypertext authoring tools. It might be argued that especially the introduction of graphical overview maps or graphical browsers integrated into hypertext systems changes the authoring process. In the following study, we investigated in what kind of activities students engage when they create their own hypertext documents and whether they use a graphical overview map in this process. In addition, we tried to evaluate what the "nished documents looked like and whether there was a relationship between the authoring process and the character of the "nished documents. The graphical overview map was not only a tool for editing the document but also formed a part of the "nished hypertext. Therefore, the question whether visualization of information structures played an important role in hypertext authoring was central for our study.
1071-5819/00/110809#17 $35.00/0
2000 Academic Press
810
M. POHL AND P. PURGATHOFER
1.1. WRITING HYPERTEXT
Writing hypertext di!ers from writing traditional linear text in several ways. Text in hypertext documents is, e.g. more fragmented than text in linear documents. Therefore, graphical overview maps or graphical browsers representing the overall structure of the document play an important role in authoring (and also navigating) hypertext. It is plausible to assume that the use of graphical overview maps during the authoring process in#uences the three key elements found for traditional writing processes: planning, translating and reviewing. Even in traditional linear writing, planning, translating and reviewing are highly integrated processes (Hayes & Flowers, 1980, 1986). Still, it is possible to distinguish between them. Planning encompasses those thinking activities that are required before words are written on paper. The term &&translating'' is used to describe the act of putting ideas into written words. Reviewing involves evaluating and changing the text (Hartley, 1992). In the hypertext authoring processes, it is rather di$cult to distinguish between these key elements. Planning increasingly becomes part of the translation process. Concept maps or graphical overview maps are often used as planning tools even for linear texts on paper. Such methods help authors to create, clarify and order their ideas and concepts (Kellog, 1992). The integration of such planning tools into hypertext authoring systems makes it di$cult to distinguish between planning, translating and reviewing. The graphical overview map is an integral part of the "nal document whereas notes and concept maps produced during the planning stage of traditional writing are usually discarded after the document is "nished. Graphical overview maps or at least clickable tables of contents are an indispensable part of most of today's hypertext documents. Such elements of visualization of information structure seem to be the most fundamental di!erence compared to linear text because the distributed organization of text in hypertext can only be sensibly represented in a visual form. An indication of the importance of graphical forms of representing knowledge might be the intensity with which an overview map or a similar tool is used by authors. Our investigation shows that authors use the overview map to a high degree although they produce documents which are rather traditional in other respects. This is an indication that visualization of knowledge structures is one of the most attractive and usable features of hypertext systems. Hayes and Flowers (1980, 1986) used protocol analysis to develop their model of writing. This method is very time consuming and often depends on the writers' re#ections about their own cognitive processes. In addition, it disturbs the writing process fundamentally. On the other hand, it yields very rich and interesting data. With modern computer technology it is possible to monitor writing processes without the (obvious) intrusion of a human observer. In the investigation described below, we only used data produced by computer monitoring to analyse the authoring process. 1.2. CONCEPT MAPS AND LEARNING
Dillon and Maynard (1995) posit that hypertext or hypermedia systems are less useful than usually assumed, especially when real-world production of text is considered. Nevertheless, there is some empirical evidence that hypertext learning systems can be used e!ectively in certain areas. It can be argued that the use of hypertext instead of
HYPERTEXT AUTHORING
811
traditional linear text supports speci"c intellectual abilities. The success of hypertext in academic education is partly due to the fact that it uses a spatial metaphor. Jonassen (1996) describes hypertext/hypermedia and semantic networking tools as mindtools, that is, as computer-based tools functioning as &&intellectual partners'' with the learner. Mindtools are supposed to engage learners in critical thinking and higher-order learning. Hypertext is especially motivating when students construct their own hypertext documents re#ecting their perspective or understanding of the learning material. Semantic networking or concept maping helps students to analyse their own knowledge structures to integrate new information with what they already know. As mentioned above, in many hypertext authoring systems, especially in those used in an academic context, hypertext and semantic networking or concept mapping are integrated features. There is some evidence that the use of concept maps in learning environments can have bene"cial e!ects. Kozma (1992), Kommers and de Vries (1992), Fisher (1992) and McAleese (1998), who used computer-based concept mapping tools report positive results. Reader and Hammond (1994) argue that concept mapping is a more time-consuming method than traditional forms of learning. Thus, students are motivated to spend more time going through the material and analysing it. This leads to a better performance of those students developing concept maps. Concept mapping and similar methodologies can be supported easily by computer technology but they were originally developed for pencil and paper (see e.g. Novak & Gowin 1984; Novak, 1998). These methodologies are based on the theory that meaningful learning can only occur when new and old knowledge are integrated in a systematic way. The development of concept maps by students is a valuable learning aid for relating new and old knowledge which leads to the construction of structural knowledge (Jonassen, Beissner & Yacci, 1993). Jonassen and Reeves (1996) describe several experiments which support their hypothesis that hypertext/hypermedia systems are a valuable tool for developing higher-order thinking skills. Dee-Lucas (1996) investigated whether the type of content overview or the amount of segmentation of hypertext in#uenced study strategies of students. She compared linear text with a hierarchically structured hypertext and an unstructured hypertext with a menu-like list of unit titles. In general, she concludes that highly structured hypertext including spatial representations of the information organization has advantages and disadvantages for learning. On the one hand, study strategies become more e!ective, on the other hand, students explore less and tend to miss related content material. She suggests that it depends on the goal of study whether the design of an educational hypertext system should emphasize structure or not. From the results reported in the literature cited above one might conclude that hypertext can be used e!ectively in university education in some areas. The goal of the learning process should be that students acquire structural or overview knowledge. In this context, the visual representation of knowledge structures plays a decisive role.
1.3. VISUALIZATION AND HYPERTEXT
Visualization of information structure has been a popular topic in the hypertext literature. As described above, the combination of hypertext with overview maps or graphical browsers is a natural one. The distributed, fragmented nature of hypertext emphasizes
812
M. POHL AND P. PURGATHOFER
the importance of structure and relationships between the units of information contained in nodes. This structure is best represented in a topological manner. Chen (1999) and Chen and Czerwinski (1997) argue that spatial representation is one of the most important ways of information visualization in hypertext development. Marshall, Shipman and Coombs (1994) point out that a visual/spatial metaphor for hypertext enables people to express nuances of structure more easily than in other hypertext systems. They posit that using spatial metaphors in hypertext systems takes advantage of the ability of humans to navigate in physical space. The practice of developing a shared understanding of multiple information sources becomes increasingly important (Marshall & Shipman, 1997). Modern information systems provide an abundance of documents so that people lack the time to read all these documents thoroughly. Marshall and Shipman assume that hypertext systems using a spatial metaphor help people to organize and structure this information. Their empirical investigation supports this assumption. Navigating hypertextual space is often compared to navigating physical space (Dillon, McKnight & Richardson, 1993). Humans develop cognitive maps to "nd their ways in their physical environments (Golledge, 1999). Knowing the environment is a dynamic process. In western cultures physical space is usually represented as a Euclidean metric. This makes it easy to use physical space as a metaphor for semantic space. The concepts of landmarks, routes and surveys which were developed for way"nding behaviour are transferred to hypertextual space. Dillon et al. (1997) point out, however, that the relationship between physical and semantic space is more complex than usually assumed. Physical distance can be measured in a rather straightforward manner. In contrast to that, semantic distance is rather di$cult to assess. &&The extent to which two ideas are related may seem intuitively easy to assess but is unlikely to have a broadly agreed quanti"able metric.'' (Dillon et al. 1997, p. 186.) Furthermore, semantic space is an abstract linguistic concept. In contrast to physical space, it is n-dimensional. It is, therefore, impossible to &&navigate'' semantic space in the same manner as physical space. Dillon et al. conclude that the limits of the spatial metaphor for hypertext have to be considered when investigating navigational behaviour. Kaplan and Moulthrop (1994) also argue that there is a considerable di!erence between physical, &&architectonic'' space and semantic space. Other metaphors might be used in addition to &&navigation'' to get a more complete picture of hypertext usage. Dillon and Vaughan (1997a) point out that it is not obvious what constitutes a landmark in semantic space. In addition, it is not clear whether improved navigational knowledge leads to better performance. They state that a major disadvantage of the navigational metaphor is that the process of meaningful assimilation of knowledge is not considered. Therefore, they suggest the concept of &&shape'' as an alternative to the navigation metaphor. One of the main advantages of this concept is that shape integrates physical form and information content. Realworld tasks usually not only consist in locating information but also some form of meaningful processing. The concept of &&shape'' takes these aspects into consideration. Dillon et al. (1997) admit that the concept of shape is still &&somewhat elusive'' (p. 104). Nevertheless, they think it might prove interesting as it includes both the aspect of physical representation and the aspect of discourse organization. In this, it resembles the concept of schematic superstructures developed by van Dijk and Kintsch (1983). To illustrate the concept of shape they quote an experiment conducted by Dillon and
HYPERTEXT AUTHORING
813
Vaughan (1997b, quoted in Dillon & Vaughan, 1997a) where &&experts were using more complex rationales for their classi"cation than simple perceptual cues. Information content, wording, relationships to a process were all seen as indicators of location.'' (p. 100). It can be concluded that spatial metaphors play an important role in hypertext research. Spatial organization of information becomes especially important for the so-called &&new information workers'', people who work with modern, networked information systems.
2. Description of the investigation Our research in the "eld of hypertext authoring is part of a long-term study. The goal of this study is to gain insights into the application of hypertext-based information systems in teaching and learning. In the course of this project we asked our students to write their seminar papers in hypertextual form instead of traditional, linear paper form. So far, 168 students created hypertext documents using the authoring tool developed at our department. The students study computer science in their third or fourth year. The seminars cover a wide range of topics: technological and technology assessment issues as well as social sciences, psychology, information theory and design. The task of students in our seminars is to read a couple of scienti"c publications and present the underlying issues to the other participants. In addition, they have to prepare a written assignment about these publications, traditionally about 20}30 pages of text. The students were encouraged to utilize novel features of hypertext, as for example the integration of novel features of hypertext. We did not use an experimental approach in a strict sense in order to increase the ecological validity and the practical applicability of our results. The "rst steps were taken using HyperCard on the Macintosh. We found out quite fast that students (who, as computer science students, love to program, but hate to write) get entangled in the programming necessary to generate the structure so much that it actually makes them change the content in favour of an easier way to implement structure. The logical conclusion was to o!er students an authoring tool speci"cally tailored to their needs, the needs of (untrained) hypertext authors.
2.1. THE AUTHORING TOOL
Since no then-existing hypertext environment ful"lled these requirements, we decided to develop our own authoring tools. On top of Apple's &&HyperCard'' for the Macintosh, we implemented a hypertext authoring tool, embedded into a simultaneously developed multi-user networking environment. With these tools, hypertext authoring was raised from the complex task, closely related to programming, to a high-level, interactive task embedded into a context of semantically appropriate tools and actions. Many contraints were introduced in order to liberate the authors of design decisions and to constitute a common look and feel within the "nished hypertext documents. Basically, the tools consisted of two parts which are described in the following; a node editor to write text, create link anchors and embed media into the node, and an overview
814
M. POHL AND P. PURGATHOFER
FIGURE 1. Example of a node in the node editor view.
editor where nodes and links could be created and arranged freely. Figure 1 shows a typical node. The node editor o!ers a "xed, simple layout consisting of a node header (top left), the document header (top right), a main text area (right) and a sidebar for subheadings, comments and link placement. In the main text area, the authors had no choice of text fonts or styles other than plain or bold. While this decision might look like a harsh reduction of the authors' possibilities, we found that in fact it was expanding their decision space: released from the need to choose a font and a style, authors were able to dedicate their time and attention to other decisions. The toolbar at the bottom of the screen o!ers functionality to create and delete links via simple drag-and-drop operations. On the very right, some basic navigation (title node, overview map, help and last visited node) is provided. Finally, there is a mode switch between author and reader view in the top right of the window. In reader view, all but the necessary tools are removed. In addition to these visible functions, a number of administrative functions were available via the usual set of menus. Among others, new nodes could be created via the menus. The second major module of the authoring tools is the map editor or overview map. The basic function of the overview map was to give authors an instrument to create links and nodes in a structural way, and let them arrange nodes freely within the resulting map. No automatic or semi-automatic arrangement was proposed, the layout of nodes, and the resulting network of links is completely under the control of the author. Figure 2 shows a typical overview map. Again, the tools provide a simple and meaningful way to create links and nodes. Nodes can be created via drag-and-drop from the node tool, the link tool allows one to de"ne connections between nodes with the usual rubber-band functionality. Nodes are deleted by dragging them into a trash bin, and basic document information is available via an
815
HYPERTEXT AUTHORING
FIGURE 2. The overview map as authors see it.
&&i''-button. While there are no restrictions for the author in arranging nodes on the map, the arrangement is also without any consequence for the individual nodes or the content therein. To enable authors to test their work without constantly switching between author and reader modes, a modi"er key was dedicated to the emulation of reader mode in author's mode. Whenever an author held down the &&option''-key, he/she was in a &&reader-withinauthor''-mode that let him/her test the layout. 2.2. EVENT RECORDING FUNCTIONALITY
In order to enable us to evaluate the authoring process, all the actions the authors took were recorded using a simple event recording mechanism. The syntax used for event recording allowed for basic information to be provided with each event. An example of a resulting log is shown in Figure 3. While the authoring tools were in use, we were able to record 143 protocols, 95 of which turned out to be correct and usable, containing more than 48.000 single events, or a statistical mean of 408 events per protocol. From these dynamic data we can retrieve some simple statistical information like the average node and link &&age'' (relative to the number of recorded events) in a document, the mode of node creation (from the node view vs. in the overview map), etc. In addition, we retrieved some static information from the documents, like the number of nodes and links. The real challenge, however, is to "nd evaluation methods for the dynamic data that allow deeper insight into the authoring process. In the following, we will describe three methods we used to evaluate these protocols. 2.3. EVALUATION METHODS
2.3.1. Moving window analysis. The "rst method, described in the following, tries to form the basis for a quantitative analysis of the data. The problem with analysing statistically
816
M. POHL AND P. PURGATHOFER
FIGURE 3. Event recording result, showing a sequence of basic actions (change name, edit text, move node in overview map).
the dynamic data we have is that they are impossible to normalize, reduce or categorize without losing some or most of the information within: the dynamic change. With the &&moving window'' method we try to generate a set of data from the protocols that makes it easier to apply statistical methods. The focus of this method is to visualize and understand the basic strategies of the author. To get an analysis of granularity g (the number of overlapping sample intervals) of a protocol with N entries, we de"ne a &&window of observation'' with a length of l"2N/(g#1). We start by counting all di!erent classes of actions within the "rst l actions of the protocol, resulting in an accumulation vector v . Next, we &&move'' the window by an o!set of D"N/(g#1) * half the window size * and repeat the counting, getting v . This procedure is reiterated until the g accumulation vectors are acquired. The result of this process, the accumulation vectors v !v or more conveniently the E accumulation matrix <, can be visualized using a stacked bar graph. In this graph, each bar shows the relative importance of di!erent classes of actions within an interval of 2(g#1) actions, overlapping each of the neighbouring intervals by 50%. In this graph, we can then see the relative importance each class of actions had for the author throughout the creation of the document. The accumulation matrix < can be seen as some kind of redux of the authoring process. As we will show later, di!erent authoring strategies can be seen quite convincingly. Of course, < is only a coarse reduction of the rich information within the original data, leaving behind much detail eventually worth taking into account. However, < can easily be normalized, enabling one to apply more sophisticated statistical methods like cluster analysis. For the investigation described later in this text, we generated moving window analysis with granularity 19, counting "ve classes of actions: text editing (writing and deleting text), make (creating nodes and links), moving (positioning nodes in the overview map), delete (deleting nodes and links) and other actions (e.g. renaming nodes). 2.3.2. Descriptive analysis. In addition to the quantitative analysis based on the data produced by computer monitoring we also conducted a more descriptive analysis (Pohl, 1998) based on the "nished documents. Several features of these documents were evaluated by students of our department. We used two di!erent sets of categories. One set was mainly concerned with the structure of the whole document as it appeared on the overview map. The other set dealt
HYPERTEXT AUTHORING
817
with layout and screen design. The most important question the "rst set of categories tried to answer was whether students adopted traditional hierarchical forms of structuring information or not. Link types are treated insofar as they can clarify this issue. The second set of categories examined the question whether students used layout, screen design or graphical elements to convey a sense of structure. The typographic and layout features which play an important role in this context are highlighting (Mayhew, 1992), grouping (Tullis, 1991; Mayhew, 1992), and the distribution of white and black space on the screen (Watzmann, 1993). Highlighting is a possibility to point out important concepts in the text to the reader. Grouping is a powerful tool to show which elements belong together and which do not. The distribution of white and black space on the screen, which is to a certain extent related to grouping, provides a clear path through the material and gives a conceptual framework of the text (Watzmann, 1993).
3. Results In the following section two sets of results will be presented * descriptive data and dynamic data. The descriptive data contain variables like the &&structure of the overview map'', layout and screen design and link types. These variables describe the "nished documents. The basis for these data was 143 documents. These documents were analysed manually by three of our students. The automatic monitoring tool was the source of the dynamic data which describe the process of hypertext creation. The basis for these data was 95 documents. The reason for the discrepancy between these two numbers is that only 95 protocols resulting from automatic monitoring were usable. 3.1. RESULTS OF THE ANALYSIS OF THE DESCRIPTIVE DATA
Students predominantly used a hierarchical form of structuring information. Hierarchical links are by far the most frequent form of link type (85.2%). There are only three documents with no hierarchical links at all. In contrast to that, there are 83 documents with no non-hierarchical links at all. There are several documents with a considerable amount of non-hierarchical links. This indicates that the majority of students feel more comfortable with the hierarchical organization of text. Others * a minority still * use non-hierarchical links quite extensively. The term link type in this study is perhaps misleading as it is used in the literature (see e.g. deRose, 1989) to characterize link types belonging to a complete system of link classi"cation. In contrast to that, the term link type here refers to a few characteristic linking methods which students used to express their ideas about structure. This concept of link types was not developed theoretically but rather deducted from our practical experience with students' hypertext documents. These categories are supposed to re#ect whether students are able to create meaningful relationships between nodes. 3.1.1. Link between the same term on two nodes. Students very often use this method of linking two nodes. They happen to notice that one and the same term occurs on two nodes in their document. Therefore, they create a link between those two nodes. This process sometimes seems automatic and the resulting links are sometimes rather super#uous: 3804 (76.4%) out of 4977 links are links between the same term on two nodes.
818
M. POHL AND P. PURGATHOFER
3.1.2. Example links. Students fairly often use links to refer to examples which illustrate an idea brought up in the text. 1324 (26.6%) out of 4977 links are of that type. Only 26 out of 143 documents do not contain any examples. Students di!er considerably in the amount of examples they include in their documents. One document even contained 46 examples. 3.1.3. Explanation links. Students also fairly often use links to refer to explanations of terms used in their document. 1890 (37.9%) out of 4977 links are of that type. Only 9 out of 143 documents do not contain any explanatory links. The maximum of explanatory links in a document is 71. The term &&grid'' refers to the categories introduced by Watzmann (1993). She tries to formulate criteria for the quality of screen design. Good screen design, according to her, is highly structured and uses a range of di!erent topological or graphical cues. The term &&gray page'' describes a screen which resembles a traditional book with no organization of the information at all. Chunking means that information is divided into small, meaningful pieces of text. Queuing leads to an even more structured screen. Information is arranged topologically, so that the position of pieces of text conveys meaning. Mixing modes refers to a screen design which includes di!erent modes of expression (text, graphics, pictures, formulas, etc.). Chunking and queuing seem to be very popular with our students. This shows that they have realized that a computer screen o!ers possibilites which do not exist in a book. Nevertheless, the students we observed in our study did not use pictures, graphics or other modes very often. Only 41 (29%) documents out of 143 use graphics or pictures.
3.2. RESULTS OF THE ANALYSIS OF THE DYNAMIC DATA
We could observe several interesting and unexpected results when we analysed the activities students were engaged in when they wrote their hypertext documents. Ninety"ve documents were available for analysis. The categories of analysis were edit text, make, move, delete and other activities. &&Edit text'' refers to the traditional process of writing and editing text in a text "eld on a node, &&make'' refers to making links or nodes, &&move'' describes those activities which are related to moving nodes around on the overview editor, &&delete'' refers to revision of the document (especially deleting nodes and links) and &&other actions'' are all actions not covered in the other categories (e.g. renaming nodes, etc.). Figure 4 shows the average values of all analysed documents for 19 consecutive phases of hypertext creation. This "gure is rather misleading as it conveys the impression that all activities are distributed evenly across all phases (from 1 to 19). The "gures for the single documents are much more diverse. Students very often concentrate on one activity in one period or in a succession of several periods which makes the diagrams look more chaotic (see below). In addition, there are fundamental individual di!erences between students. Some students mainly write text during most periods, others prefer moving their nodes in the overview editor. Nevertheless, it is obvious that revising is an activity which is not very popular among students in general. It must be mentioned, however, that our concept of revision is more restricted than that of revision of linear text. This is due to technical reasons.
819
HYPERTEXT AUTHORING
FIGURE 4. Average values of all analysed documents/relative proportion of activities: , move; , delete; , other actions.
FIGURE 5. Document &&Das Leib}Seele Problem'': relative proportion of activities: , move; , delete; , other actions.
, edit text;
, make;
, edit text;
, make;
Figure 5 shows, on the other hand, that the visual appearance of their overview maps is very important for the students. A relevant part of their activities is made up of moving nodes around on the overview map (the links automatically move with the nodes). These actions can be interpreted in two ways. On the one hand, space and spatial relationships on the overview map are something meaningful for the students. Nodes which &&belong to each other'' are positioned near each other. Partly, students even produce meaningful sequences of nodes. Figure 6 shows a slightly chaotic overview map. Nevertheless, the authors of this overview map tried to indicate the sequence of the chapters by positioning them beside each other. The nodes &&Kapitel 1'' through to &&Kapitel 6'' refer to Chapters 1}6. On the other hand, students sometimes simply try to make the overview map look better or more orderly. The dynamic data we got by monitoring the students' activities show that students sometimes simply try to avoid the lines representing links from crossing each other. Chen and Czerwinski (1998) also point out that minimizing the number of link crossings and overlapping is one of the criteria for aesthetic layout. It is obvious from our data that visualization of information is very important for students
820
M. POHL AND P. PURGATHOFER
FIGURE 6. Document &&Das Leib}Seele Problem'': overview map.
developing hypertext documents. This is also supported by other data from our analysis. The monitoring tool we developed can distinguish between activites which took place &&on the overview map'' and activities which took place in the document itself. It was possible to make nodes &&on the overview map'' by drag and drop or by clicking on a &&node symbol''. On the other hand, it was also possible to create nodes by using the menu which is the usual Hypercard method. In 87.85% of all cases students created nodes on the overview map. Again, this is an indication that the visualization of the information is important for authors of hypertext documents, at least for the students of the sample we investigated. It is obvious that students did not &&navigate'' their document when they created it. They do not try to locate any knowledge but rather to create knowledge. They already possess survey knowledge without going through the stages of landmark and route knowledge. The concept of shape as developed by Dillon et al. (1997) seems to describe students' behaviour better than the navigational metaphor. In the following, we will discuss several cases which deviate considerably from the average. After a careful inspection of the data derived from the dynamic analysis we found that there are probably several di!erent styles of writing hypertext. Large groups either concentrate on creating links and nodes or on designing their overview maps. Smaller groups prefer to write text or revise their document. So far, this is circumstantial evidence. We intend to analyse this phenomenon statistically in the future. The following examples are typical cases for the above-mentioned groups. We want to describe the activities in which students engaged and how these activities related to the document which was produced. We will try to show that there is a relationship between the way in which documents are created and how the "nished documents are structured. As there is little experience in this area, we want to use a case-based approach to get a "rst impression of the problems which might be encountered in this area. The "rst example is a document called &&Das Leib}Seele Problem'' (the mind-body problem * Figures 5 and 6). It is outstanding insofar as the authors wrote much text but did not care very much what their overview map looked like. They moved nodes on the overview map very rarely. As a consequence, their overview map is very chaotic. In the
821
HYPERTEXT AUTHORING
FIGURE 7. Document &&Interaction Styles'': relative proportion of activities: , delete; , other actions.
, edit text;
, make;
, move;
beginning, they tried to impose some order. This is obvious from the example already described above. They tried to arrange all the chapter beginnings in one linear sequence (Kapitels 1}6). Then they apparently realized that they did not have enough space and gave up completely to edit the overview map. Especially the positions of the nodes representing the last chapters are very confusing for the readers. This is also obvious from Figure 5 which shows that they almost stopped completely moving nodes around in the last two phases. They apparently also stopped revising their document very early (from phase 7 onward). On the other hand, the text of their document is very comprehensive and interesting although the document in general is slightly confusing. The chaotic nature of the overview map adds to this confusion. This document suggests that hypertext authoring not only consists of creating nodes and writing text on these nodes but rather producing a well designed overview map is a crucial part of hypertext authoring. The next document is completely di!erent (see Figures 7 and 8). The author made many nodes and links but also deleted many of them later. He revised much more than the average student. Nevertheless, the document is slightly confusing. This document is an example for the fact that revising as such does not lead to good products. It also depends very much on the quality of the revision process. The overview map shows that the document is completely non-hierarchical. The reason for this is that the author made links from every node to almost all other nodes. There is no clear reason for this. The links seem to have been produced more or less at random. The author wrote fairly little text * text which is also confusing. One advantage of the document is that the author produced a fairly clear overview map. He moved the nodes around rather often, especially in the "nal stages of the authoring process. Therefore, the overview map is rather attractive and clear. The author of the last example apparently took great care to make his overview map look attractive. The relative proportion of move operations in Figure 9 is high above the average. Especially in the "nal stages of document creation there are phases (12}14) which almost entirely consist of moving nodes around on the screen. Consequently, the overview map is very clear and understandable (Figure 10). It is entirely hierarchical. In addition, the author also revised his document quite heavily in the last two phases. The document is clear and well-structured. There is not much text, but the text is to the point
822
M. POHL AND P. PURGATHOFER
FIGURE 8. Document &&Interaction Styles'': overview map.
FIGURE 9. Document &&User-Centred-Design Principles'': relative proportion of activities: , make; , move; , delete; , other actions.
, edit text;
FIGURE 10. Document &&User-Centred-Design Principles'': overview map.
and understandable. The links are well-designed and sensible. This case also shows the importance of including methods of visualizing knowledge structure into hypertext authoring systems. In addition, the e!ects of the revision process apparently were positive for the document.
HYPERTEXT AUTHORING
823
4. Conclusion If we compare the results from the descriptive and from the dynamic analysis one conclusion can be drawn. There was one innovative feature of our hypertext system used extensively by students when authoring hypertext documents. This feature was the overview map. Students create most of the nodes and links on the overview map and spend quite a lot of time editing the appearance of this map. There are other possibilities made available by the system which were used by a few students quite heavily like creating non-hierarchical links or introducing pictures, other graphics, movies or sounds. These novel features were apparently less attractive than the overview map. The documents which were created were often rather traditional. These results indicate that visualizing information structure is one of the most important new features of hypertext systems. Another conclusion which might be drawn from the dynamic data is that there are several di!erent styles of creating hypertext documents. The diagrams showing the relative proportions of activities during the process of writing hypertext convey a slightly contradictory picture. From the average values one might conclude that there are no signi"cant di!erences between various stages of hypertext creation. This is not the case. It must be mentioned, however, that there are no uniform patterns across students at which point in time certain activities are carried out. In addition, most of the students seem to have preferences for one activity. So far, this is only circumstantial evidence but we intend to investigate the question of hypertext creation styles in more detail in the future. The relationship between style of hypertext creation and design of the "nal documents is probably rather complex. Our experience so far suggests that students should engage in all the di!erent activities to a certain extent and not concentrate on only one to produce high-quality documents. This probably indicates that highly individualistic styles of hypertext creation might be a disadvantage for the quality of the document. More research in this area seems to be necessary. We want to thank Michaela Schuster, Alexander Sogl and Alexander Stadler who are students at our department and helped us with the analysis of the descriptive data.
References CHEN, C. (1999). Visualizing semantic spaces and author co-citation networks in digital libraries. Information Processing and Management, 35, 401}420. CHEN, C. & CZERWINSKI, M. (1997). Spatial ability and visual navigation: an empirical study. ¹he New Review of Hypermedia and Multimedia, 3, 67}89. CHEN, C. & CZERWINSKI, M. (1998). From latent semantics to spatial hypertext * an integrated approach. Proceedings of the Hypertext +98, pp. 77}86, Pittsburgh, USA. DEE-LUCAS, D. (1996). E!ects of overview structure on study strategies and text representation for instructional hypertext. In J.-F. ROUET, J. J. LEVONEN, A. DILLON & R. J. SPIRO, Eds. Hypertext and Cognition, pp. 73}107. Mahwah, NJ: Lawrence Erlbaum. VAN DIJK, T. & KINTSCH, W. (1983). Strategies of Discours Comprehension. New York: Academic Press. DILLON, A. & MAYNARD, S. (1995). &Don't forget to put the cat out' * or why collaborative authoring software and everyday writing pass one another by. ¹he New Review of Hypermedia and Multimedia, 1, 135}153. DILLON, A., MCKNIGHT, C. & RICHARDSON (1997). Space * the "nal chapter or why physical representations are not semantic intentions. In C. MCKNIGHT, A. DILLON &
824
M. POHL AND P. PURGATHOFER
J. RICHARDSON, Eds. Hypertext a Psychological Perspective, pp. 169}191. Chichester, UK: Ellis Horwood. DILLON, A. & VAUGHAN, M. (1997a). &It's the journey and the destination': shape and the emergent property of genre in evaluating digital documents. ¹he New Review of Hypermedia and Multimedia, 3, 91}106. DILLON, A. & VAUGHAN, M. (1997b). Mental representations of complex information spaces. Poster Sessions: Abridged Proceedings of the HCI International +97, 7th International Conference on Human}Computer Interaction jointly with 13th Symposium on Human Interface (Japan), p. 96. West Lafayette, IN: School of Industrial Engineering, Purdue University. FISHER, K. (1992). SemNet: a tool for personal knowledge construction. In P. A. KOMMERS, D. H. JONASSEN & J. T. MAYES, Eds. Cognitive ¹ools for ¸earning, pp. 63}75. Berlin, Heidelberg, New York: Springer. GALBRAITH, D. (1992). Conditions for discovery through writing. In M. SHARPLES, Ed. Computers and =riting. Issues and Implementations, pp. 45}71. Dordrecht, Boston, London: Kluwer. GOLLEDGE, R. G. Ed. (1999). =ay,nding Behavior. Baltimore, London: John Hopkins University Press. HARTLEY, J. (1992). Writing: a review of the research. In J. HARTLEY, Ed. ¹echnology and =riting. Readings in the Psychology of =ritten Communication, pp. 18}36. London, Philadelphia: Jessica Kingsley Publishers. HAYES, J. R. & FLOWERS, L. S. (1980). Identifying the organisation of writing processes. In L. W. GREGG & E. R. STEINBERG, Eds. Cognitive Processes in =riting, pp. 3}30. Hillsdale, NJ: Lawrence Erlbaum. HAYES, J. R. & FLOWERS, L. S. (1986). Writing research and the writer. American Psychologist, 41, 1106}1113. JONASSEN, D. H. (1996). Computers in the Classroom. Mindtools for Critical ¹hinking. Englewood Cli!s, NJ, Columbus, OH: Merill. JONASSEN, D. H., BEISSNER, K. & YACCI, M. (1993). Structural Knowledge. ¹echniques for Representing, Conveying, and Acquiring Structural Knowledge. Hillsdale, NJ, Hove and London: Lawrence Erlbaum. JONASSEN, D. H. & REEVES, T. C. (1996). Learning with technology: using computers as cognitive tools. In D. H. JONASSEN, Ed. Handbook of Research for Educational Communication and ¹echnology, pp. 693}719. New York: Simon & Schuster}Macmillan. KAPLAN, N. & MOULTHROP, S. (1994). Where no mind has gone before. Ontological design for virtual spaces. In ECHT+94 Proceedings, ACM European Conference on Hypermedia ¹echnology, pp. 206}216. KELLOG, R. T. (1992). Designing Idea Processors for Document Composition. In J. HARTLEY, Ed. ¹echnology and =riting. Readings in the Psychology of =ritten Communication, pp. 181}199, London, Philadelphia: Jessica Kingsley Publishers. KOMMERS, P. A. M. & DE VRIES, S. A. (1992). TextVision and the visualization of knowledge: school-based evaluation of its acceptance at two levels of schooling. In P. A. KOMMERS, D. H. JONASSEN & J. T. MAYES, Eds. Cognitive ¹ools for ¸earning, pp. 33}62. Berlin, Heidelberg, New York: Springer. KOZMA, R. B. (1992). Constructing knowledge with learning tool. In P. A. KOMMERS, D. H. JONASSEN & J. T. MAYES, Eds. Cognitive ¹ools for ¸earning, pp. 23}32, Berlin, Heidelberg, New York: Springer. MCALEESE, R. (1998). The knowledge arena as an extension to the concept map: re#ection in action. Interactive ¸earning Environments, 6, 1}22. MARCHIONINI, G., NEUMAN, D. & MORRELL, K. (1994). Directed and undirected tasks in hypermedia. Is variety the spice of learning? In T. OTTMANN & I. TOMEK, Eds. Proceedings of ED-MEDIA +94, pp. 373}378. Charlottesville, USA: AACE. MARSHALL, C. C. & SHIPMAN, F. M. (1997). Spatial hypertext and the practice of information triage. Proceedings of the Hypertext +97. pp. 124}133, Southhampton, UK. MARSHALL, C. C, SHIPMAN, F. M. & COOMBS, J. H. (1994). VIKI: spatial hypertext supporting emergent structure. Proceedings of the European Conference on Hypermedia ¹echnologies. pp. 13}23, Edinburgh, Scotland.
HYPERTEXT AUTHORING
825
MAYHEW, D. (1992). Principles and Guidelines in Software ;ser Interface Design. Englewood Cli!s, NJ: Prentice-Hall. NOVAK, J. D. (1998). ¸earning, Creating, and ;sing Knowledge. Mahwah, NJ, London: Lawrence Erlbaum. NOVAK, J. D. & GOWIN, D. B. (1984). ¸earning how to ¸earn. Cambrigde: Cambridge University Press. POHL, M. (1998). Hypermedia as a cognitive tool. In T. OTTMANN & I. TOMEK, Eds. Proceedings of the ED-MEDIA +98, pp. 1089}1094, Charlottesville, USA: AACE. POHL, M., PRENNER, P. & PURGATHOFER, P. (1995). Hypermedia in education * monitoring the development of hypermedia documents. Psychology ¹eaching Review, 4, 142}152. READER, W. & HAMMOND, N. (1994). Computer-based tools to support learning from hypertext: concept mapping tools and beyond. Computers in Education, 22, 99}106. DEROSE, S. (1989). Expanding the notion of links. In Proceedings of the Hypertext+89 Conference, pp. 249}258. TULLIS, T. S. (1991). Screen design. In M. HELANDER, Ed. Handbook of Human}Computer Interaction (2nd edn). pp. 377}411. Amsterdam: North-Holland. WATZMANN, S. (1993). Presenting information visually. Hypertext 93, Tutorial 22.