- Email: [email protected]
A visualisation tool as a demonstration aid
Computers & Education 41 (2003) 133–148 www.elsevier.com/locate/compedu
A visualisation tool as a demonstration aid Matti Lattua, Veijo Meisalob, Jorma Tarhioc,* a
Department of Teacher Education, PO Box 8 FIN-00014 University of Helsinki, Finland Department of Teacher Education, PO Box 9, FIN-00014 University of Helsinki, Finland c Helsinki University of Technology, Department of Computer Science and Engineering, PO Box 5400, FIN-020l5 HUT, Finland b
Received 18 September 2002; accepted 18 February 2003
Abstract As algorithms have been seen to be hard to teach and learn, teachers have tried to look for help in algorithm animation. While the effect of algorithm animation on learning has been studied, but not reliably evidenced, this study tries to approach the problem from a different perspective. Sixty hours of assignment sessions in an introductory programming course were observed to determine the kind of demonstration and explaining strategies teachers and students tend to use. The results show that although the variation of different visualisation types is large, there are certain common properties describing the explanation of the programs. Guidelines for demonstration tools are presented based on the results. # 2003 Elsevier Ltd. All rights reserved. Keywords: Programming; Interactive teaming environments; Improving classroom teaching
1. Introduction Traditionally, algorithms have been abstract and hard for students to understand. Therefore, interest in a deeper understanding of them has been weak even among computer science (CS) students. As understanding and developing algorithms is essential in computer science, developing novel ways for improving learning and understanding algorithms should be one of the most important areas in rethinking computer science education. To make the abstract functions of computers more concrete, CS teachers tend to use different kinds of illustrations (static drawings) or computer-based visualisation tools (moving illustrations). Recently, there has been new emphasis placed on using visualisation tools to support oral * Corresponding author. Fax: +358-9-451-3293. E-mail address: jorma.tarhio@hut.fi (J. Tarhio). 0360-1315/03/$ - see front matter # 2003 Elsevier Ltd. All rights reserved. doi:10.1016/S0360-1315(03)00032-0
134
M. Lattu et al. / Computers & Education 41 (2003) 133–148
presentation, and not only to be used independently by students (e.g. Hansen & Raizer, 2002). However, too much research on software visualisation (SV) has concentrated on finding the measurable effects of SV on learning outcomes. Thus, this paper focuses on examining the possibility of developing new teaching practices involved with program visualisation by analysing a case of one teacher and three student groups. The focus is on their illustration and visualisation methods. The discourse during their presentation is also studied. In this manner, desirable features for a visualisation tool as a demonstration aid in the classrooms have been analysed. This study is part of a larger evaluation project that examined the suitability of Jeliot, a webbased visualisation tool (Sutinen, Tarhio, & Tera¨svirta, 2003) in introductory teaching and learning. A researcher independent of the development group carried out the evaluation study. 1.1. The Research on Software Visualisation Software visualisation is a wide concept, used to cover almost anything from the layout of the written program (‘‘pretty-printers’’) to the visual languages. Petre, Blackwell, and Green (1998, 454) give four examples of different kinds of software visualisation: 1. Using the SV to present large data sets. Helping the interpretation by capitalising on perceptual effects. 2. Demonstrating the virtual machine to make the behaviour of the program visible, usually for pedagogical purposes. 3. Changing the perspective, for example, showing a large program in a single view. 4. Display-based reasoning creates an alternative symbolic re-presentation of the program to ease the programmer’s reasoning. The general assertions of the algorithm animation enthusiasts is that SV could be one way to help students or even laymen understand the logic of algorithms. Visualisation could also encourage students to approach debugging in a more analytical way, as it makes the programs and their execution more transparent than the traditional programming environments. Using a visualisation tool in teamwork situations has been shown to be beneficial in some case studies (e.g. Meisalo, Sutinen, & Tarhio, 1997). Most studies dealing with SV focus on the efficiency of the visualised program vs. traditional methods in teaching. So far, the results of evaluation studies on the advantages of using graphical elements in computer-based instruction and computer-aided learning have been contradictory (Mulholland, 1998; Whitley, 1997). Stasko and Lawrence (1998) argue, for example, that the key of the research should be: ‘‘Do algorithm animations truly assist teaching and learning, and if so, how can they be best utilized?’’ (see also Byrne, Catrambone, & Stasko, 1999). The question seems fair. It is essential in science that new ideas are first tested and the assessment is based on the results. However, there is a difference between the question of whether visualisation helps in conceptualisation and memorising, and that of whether it helps learning. The former question could be asked by a psychologist, the latter by a pedagogue, as their concepts of learning differ. While the psychologist understands learning as changes in the learner’s cognition, the pedagogue’s wider focus covers planning, organisation, evaluation and social background, to mention just a few examples. The research questions of the latter could be how students learn to use a certain programming tool (Lavonen, Meisalo, Lattu, & Sutinen, 2003), the
M. Lattu et al. / Computers & Education 41 (2003) 133–148
135
nature of problem-solving when using a certain programming tool (Ja¨rvinen, 1998), or does programming aid in learning a mathematical concept (Wilensky, 1995). It is evident that both kinds of research are needed. A firm psychological basic theory about the relations between memory, the development of concepts, and perception (in the case of SV, visualisation and animation) would have an enormous impact on teaching and learning. While waiting for such a theory, one can still study the various ways SV can be applied to support the existing teaching and learning methods, as well as develop totally new practices. In these studies, the questions should be broad, and it should always be possible to focus on emerging interesting phenomena. Without proper research, the development of pedagogy around the use of SV remains at the level of an unjustified list of ‘‘best practices’’, which is hardly the way that these issues should be dealt with in the scientific community. 1.2. Jeliot Jeliot (Sutinen et al., 2003) is a tool for the rapid generation of simple animations. It is a result of long development work. Originally, it was aimed at helping students and researchers learn and understand algorithms and data structures, but it can also be applied as a demonstration tool. Jeliot works on the Web, and so it can be used anywhere with a browser that supports Java. Jeliot allows the user to animate his or her own algorithms. The user writes and submits the algorithm as a plain Java program, and gets back its animation. This animation is generated automatically from the source code, and is displayed on the user’s screen. Animation is based on self-animation of data types: the user selects the variables of the program, which he or she wants to visualise, and Jeliot shows them on the screen. During animation, the user may adjust the appearance (shape, colour, location) of the selected variables or keep the default appearance. It is possible to store these selections for later use. Although Jeliot was designed for algorithm animation, the framework could also be used in other subjects. Many dynamic phenomena can be described as algorithms, and, therefore, visualised by Jeliot. For example, it would be possible to visualise the Mendelian rules of inheritance for a biology class (Meisalo et al., 1998). Another animation system called Jeliot 2000 (Levy, Ben-Ari, & Uronen, 2002) was developed based on the ideas of Jeliot. Jeliot 2000 is a standalone Windows application, which inherited selfanimation of some data types from the original Jeliot, but which allows no adjustments to the animation. This paper only considers the original Jeliot.
2. Empirical study and methodology This study can be best defined as an evaluational case study, where the analysis was conducted using qualitative methods supported by a small quantitative part. From the quantitative framework, the setting does not fulfil the requirements of experimental design and its low validity limits the possibility of generalisations (Campbell & Stanley, 1966, 6–12). However, from the qualitative viewpoint the decision to be able to transfer the results to other environments should be left to the receiving end, while the report has to provide significant background information to enable transferability (Lincoln & Guba, 1985, 296–298).
136
M. Lattu et al. / Computers & Education 41 (2003) 133–148
The empirical study was carried out in the Department of Computer Science at the University of Helsinki in the autumn term of 1998. The initial goal of the research was to use Jeliot, a certain www-based visualisation tool, in an introductory programming course, where Java was utilised as a programming language. The course consisted of 52 h of lectures and 24 h of assignment sessions. The lecturer, who also designed the assignments, gave the lectures to the students as a one large group. Typically, the assignments were programming tasks so the solutions were Java code. In the assignment sessions, the groups discussed students’ solutions to the assignments given in the previous week’s lectures. The assignment groups were held in smaller groups, typically 15–25 students. There were 17 assignment groups in the course, which were taught by five teachers. The students could choose which group they would attend during the course. This study focused on three assignment groups and the students were informed about this when selecting their groups. The teacher of the experiment groups was an MSc with several years’ experience in teaching CS. In the beginning, the goal of the study was to encourage the students to use Jeliot’s visualisation and follow its use. While the lecturer had clearly stated that he would not make any adjustments to the lectures because of the ongoing research in the three groups, it was meaningful to focus the resources on the three assignment groups. Because the study had an evaluational nature, a large variety of data were collected: a group of voluntary students was pre- and post-interviewed, students were asked to keep a diary of their work and the assignment sessions were observed. If Jeliot had been widely used, all these could have contained relevant data. However, a few drawbacks of Jeliot, some technical problems and the lecturer and the teacher’s lack of interest in the study lead to the use of Jeliot being minimal. The observation was non-structured. The researcher sat in the classroom and took notes on the activities, conversations and the use of different methods of illustration. The researcher observed 60 h of assignment sessions, evenly between three groups. Notes were made, which were then transcribed and imported to the qualitative data analysis program Aquad 5 (Huber, 1999). Drawings were stored on paper as they appeared in the researcher’s notebook. The coding procedure was started by following the principles of grounded theory where the concepts are built along the analysis (Strauss & Corbin 1998, 12). The researcher both conducted the data collection and made the analysis. In the first phase, the material was read and some explorative coding was made, as suggested by Strauss and Corbin (1998, 57–71). The final interest of the paper was selected at this point, as making sense of the teacher and the students’ presentations was seen as useful when designing tomorrow’s visualisation tools. The low-level codes were further processed, but as the nature of the analysis changed from qualitative analysis to a mixed-method research, the grounded theory procedures could not be followed as such. The quantitative data were compiled by going through the described presentations in the data. A presentation was defined as a continuous phase where the main presenter did not change. Presentations were described by using attributes created in the qualitative analysis: explaining strategy (see Section 3.1), number of the observation (equals roughly to the advancement of the course), presenter (teacher/student), and media (blackboard, overhead, data projector). As all these attributes were not always present in the data, there were some missing values. The main characteristics of the variables are presented in the Appendix. To find possible interplay, the cross-tabulated variables were studied statistically, using the asymmetric lambda (L), which can be used with nominal data (Siegell & Castellan, 1988, 298–303). The formula of the asymmetric lambda produces a percentage that the antecedent (independent
M. Lattu et al. / Computers & Education 41 (2003) 133–148
137
variable) can explain from the consequent (dependent variable). The significance level of 10% was chosen because of the explorative nature of the study.
3. Results As the goal of this article is to develop ideas for the further development of SV tools (c.f. Pane, Ratanamahatana, & Myers, 2001), by studying existing teaching practices, the focus was on certain elements in the data. These were the ways the teacher and students explained programs and the ways they illustrated their programs. Explaining strategies and illustration methods are significant issues for the developers of SV tools. At the very least,the new tools that aim to enhance group-work in education and industry should fit existing communication patterns. Of course, the SV tools could offer totally new ways of understanding programs and create new ways of sharing understanding of computer programs. 3.1. Explaining strategies The method used to present the program in the assignment sessions is called an explaining strategy. The grounded theory analysis found two attributes defining a strategy, namely the order of explanation (execution order or order of the written code) and the size of the chunks used. The five subcategories for this attribute are: A single statement was the smallest unit in explaining programs. This was typically used when the programs were short. A group of statements is a strategy similar to the single statement method, but here the presenter combined a few statements into one chunk. Usually the group was formed around one idea or action, for example: ‘‘In these lines, the program checks whether the age of the person is within a reasonable range’’. A single method (or a procedure) was a much used chunk size later on in the course, when the programs were large and some parts of the code were trivial, for example: ‘‘In this method, the program sets the initial values for these variables’’. Cursory explaining strategies were also encountered, when no detailed explanation was given to support the visual presentation—usually just the written code. A dynamic strategy was a mixture of the other methods. Here the presenter varied the orientation, presumably depending on the complexity of the code. To study the concept in the data, quantitative data were compiled using these attributes (see Section 2). The two orders (execution and written) were almost evenly used in the presentations, while the most used chunk size was one statement (about 50% of the findings). Table 1 shows the frequencies of the categories together with observation examples. For the statistical analysis the number of the observation (advancement of the course) was divided into categories. While there was no theory to suggest number of the categories, three different groupings were tried so that the observations were divided into three to five groups. The statistical analysis showed that the independent (background) variables (number of the
138
M. Lattu et al. / Computers & Education 41 (2003) 133–148
observation, subject or media) were not related to the explaining strategy (explaining order, chunk size). However, there were two exceptions to the general pattern. When the observations were divided into three groups, the advancement of the course and the chunk size produced lambda coefficient L=0.172 (P<0.05). Another significant result was acquired when the explaining order was explained with media type. In this case L=0.188 (P<0.10). Although both values are statistically significant, they are very modest. The summary of the analysis is presented in the Appendix. 3.2. Illustration and visualisation Explaining programs was normally done by the chalk-and-talk method, although the video projector was intensively used instead of the blackboard and the OH-projector. However, the projector was mainly used to show the program code. Because of the visual nature of Jeliot, the research team wanted to collect information on the existing visualisation methods. These notes were divided into three categories: illustrating the variables, illustrating the structure of the program and the simultaneous use of two windows in the data projector. Although in a qualitative study, and especially in a case study, the counts do not necessarily relate to the importance, this fact may be worthwhile presented along with other defining data. 3.2.1. Illustrating the variables Findings: 31. The variables were usually illustrated using the blackboard, while the program code was shown using the OH projector or the video projector. At the the beginning of the course, there was a need to understand the chain-like nature of the program execution. An attempt was made to achieve this by following the execution of the program from the code and illustrating the variables manipulated by the program. In this early stage, the variables were listed in either one row or one column (see Fig. 1). The values were written beside or under the name of the variable. The last value corresponded to the current one, but the older values were still available if needed. Later on, the use of basic variables became trivial and the illustration focused on more complex variable structures. The next stage was array. Arrays were clearly an essential factor in illustrating the programs, as most algorithms in this course manipulated data stored in arrays. Therefore, after being introduced, arrays were a popular object of visualisation throughout the course.
Fig 1. Illustrating the variables in the beginning of the course.
M. Lattu et al. / Computers & Education 41 (2003) 133–148
139
Arrays were usually illustrated by a row (or column) of boxes, including the values (see Fig. 2). Sometimes the index values were shown to underline that the index number of the first element is 0, and, therefore, the index of the last element is one less than the total number of the elements. Some students did not draw the boxes and just used a row of numbers (or characters) as an array. Arrays included a number of different kinds of data: numeric values, characters and strings, other arrays and references to objects. Slightly different from arrays, there was also a need to visualise a single string object that was not transformed into an array of characters. Complex algorithms manipulating arrays were not dealt in the assignment sessions. The basic sorting algorithms were briefly presented in the first third of the course as and later on just referred to. 3.2.2. Illustrating program structures Findings: 39. Program structures were mostly illustrated using the blackboard and transparencies. The highlighting of the program code was used especially when the code was on a transparency. The teacher underlined statements or sidelines and parentheses to focus on statements or structures. While presenting the code using the data projector, the selection feature of the mouse was often used in a similar way. The teacher presented an interesting way of giving a graphical outlook to the visibility of variables. He drew a box to represent each of the methods. The ‘‘global’’ variables of the class were outside these boxes (see Fig. 3). The visual representation also reflected a way to illustrate the order of execution of the program. The teacher drew arrows from the main method, calling it a sub-method. Another arrow was drawn to show how the control returned from the sub-method to the main method. Obviously, a great number of calls between the methods would result in a chaos of crossing arrows.
Fig 2. Illustrating the arrays on the course.
Fig 3. Illustrating global and local variables.
140
M. Lattu et al. / Computers & Education 41 (2003) 133–148
Some of the most used objects of visualisation were the objects of Java. The objects are independent subprograms with their own code and data structure. Each object can have a number of instances, much like there can be a number of different cars of the same make and model. The objects were illustrated as a circle or ring. The circle included the methods and the variables of the object. The visibility of methods and variables could be shown by closing them in a box (private) or writing them on the borderline of the circle (see Fig. 4). The objects were connected to the other data structure by a line. The connecting line could point, for example, to the arrays or to the program code where the objects were created. A similar method has been used by Arnold and Gossling (1996). Inheritance was presented at the end of the course. The teacher illustrated the relations between the classes by drawing a tree of inheritance. 3.2.3. Using two windows Findings: 11. When the teacher and the students were presenting the programs, using the data projector, they sometimes used several windows simultaneously. The number of windows was usually two. The presenters tended to show the program code in the first window while running the program in the other. Another way to utilise the second window was to present the program code of a different class, while the first window showed the code of the calling class. A few times the teacher showed a pseudo code of the program in the second window. No debugging tools were utilised in the assignment sessions. More often, when the presenters needed another view of the program, they used the blackboard or transparencies to sketch pictorial models or write some statements.
4. Discussion 4.1. General considerations This study analysed the visualisation methods used in an introductory programming course in order to support the development of the program visualisation. The results showed that the explaining strategies and methods of visualisation had a large variance. The most important conclusion is that visualisation tools should be as flexible as possible to support different strate-
Fig 4. Illustrating the objects.
M. Lattu et al. / Computers & Education 41 (2003) 133–148
141
gies. One could easily expect that the teacher and students have differences in the explaining strategy or the strategies used at the beginning of the course systematically differ from the approaches used in the end, where the students have a wider understanding of programming. However, the analysis showed that in most cases there is no such association. One obvious reason of the observed similarity of the teachers and students’ explaining strategies could be the students’ level of programming skills. In the programming course, they do not just learn how to write programs, but also how to piece together programs and explain them to themselves and others. Although no computer-based visualisation tool was ultimately used because of the technical difficulties of Jeliot, the visualisation of the programs was still an important part of the discourse. Visualisation was used more than four times in each session on average, whereas the typical number of assignments for each session varied from four to six. The visualisation was offered and used as an important way to understand programs and their operations at least in the groups studied. According to the analysis, the media used in the presentations was in association with the explaining order. There are at least two possible reasons for the relation between the media and the explaining order. At the beginning of the course, where the OH projector was mostly used, the programs were short, and, therefore, the order of execution was a natural choice. The second possible reason was that the teacher wanted to support the concept forming of the sequential nature of the program by using the execution order at the beginning of the course. The students might have followed his example, as they did not have other models of explaining. However, neither of these theories is supported by the analysis. In the course, the individuals visualised program structure, variables, and somewhat combining the previous two, the objects. When presenting the code, the presenters tended to highlight meaningful chunks of code to divide the whole into meaningful parts. Visualising variables was not needed right at the beginning of the course. However, later on complex algorithms manipulating the arrays seemed to be hard to deal with within the assignment sessions. For example, getting deeply into a sorting algorithm would take a lot of time and, therefore, they were not properly dealt with when faced. The importance of visualising the variables, either with novice or experienced programmers, has been pointed out by Gurka and Citrin (1996). Similar conclusions have been made in the present study. The variables were visualised both at the very beginning of the course and in the context of complex algorithms. Object-oriented programming brings another level of abstraction to programming. In teaching traditional procedural programming, the focus is on syntactic issues. Each element of the program (a subprogram or a variable) is tied to a specific part of the code. In teaching object-oriented programming, an additional focus is on the relations of objects and on the concepts connected with them. Thus, object-oriented programming is evidently more abstract, and the need for visualisation of object-oriented features was also clearly present on this course. 4.2. Desired features of a visualisation system Below, an outline will be presented that could be used to develop existing or new tools to be used to support an oral presentation of a program. At present, the availability and flexibility of the blackboard and OH projector are superior in showing spontaneous drawings and writing. However, if typical patterns of illustrations supporting presentations could be found, they could
142
M. Lattu et al. / Computers & Education 41 (2003) 133–148
be made by computer. Offering a pattern of useful thinking models in a program could support certain expert-like learning strategies. The data collected in the assignment sessions suggest that the presentations have common properties, which should be taken into account while implementing a visualisation tool. As the course was based on Java, the ideas presented below are influenced by the objectoriented programming. It must also be remembered that the organisation of the course strongly influenced the data collected. Basically, the students gathered to go through programming assignments given the previous week. Therefore, the discussion during the assignment sessions was focused mainly on understanding an existing piece of code. The data could have been totally different if we had followed the lectures, where the issues more generally dealt with programming-related ideas and concepts instead of program codes. Unfortunately, the lectures were not documented because of the initial goal of the research. There is a distinction between the basic explorative use (one does not know what to look for) and the demonstration use (focus on some important component) of SV tools. In the case of Table 1 The explaining strategies in assignment sessionsa Order of execution
Order of the code
Single statement on a hand-written transparency.
‘‘The student presents the program Explained the execution one statement at a time in the order of execution.’’ Findings: 18
Group of statements
‘‘— begins to explain the code in the order of execution. Selects the code using the mouse to support the presentation. Explains one group of statements at a time.’’ Findings: 2 ‘‘— presents other possible solutions by using the data projector and the less-program. Selects statements using the mouse while talking. Goes through the main features of the program, one method at a time in the execution order.’’ Findings: I ‘‘Explains the main functions of the program in the execution order.’’ Findings: 4
‘‘The student loads the program to the emacs editor and explains it using a pointer, beginning with the (sub) methods. The main program is at the bottom.’’ Findings: 9 Findings: 0
Single method
Cursory
Dynamic
a
‘‘— presents the solution in the order of execution.— Goes through the program statement by statement —. Does not enter into the single methods but explains their main function.’’ Findings: 2
Quotes are examples from the observation data.
‘‘Goes through the code in the written order by presenting the function of each method.’’ Findings: 1
‘‘Goes through the code in the order of writing. Does not go deeply into the syntax but presents the main characteristics of the implementation.’’ Findings: 9 ‘‘Presents the code in the written order. Selects a text using the mouse and presents the principle. Sometimes highlights single statements and explains the syntax.’’ Findings: 4
M. Lattu et al. / Computers & Education 41 (2003) 133–148
143
Jeliot, the explorative use visualises a program per se by showing the changes of those variables, which the user wants to look at. In this way, it works as an adjustable visual interpreter of the input program. When Jeliot is used as a demonstration tool, the main purpose is not necessarily to visualise the input program, but rather to create and show an animation. In the explorative use, the input program does not support visualisation in any way. In the demonstration use, the input program may contain data structures and statements whose only purpose is to produce additional visual effects. While this paper concentrates on visualisation to understand existing programs, the visualisation can also be used to create programs, or as in the case of Hansen & Ratzer (2002), to code the visualising object structure of a program. The distinction of these different purposes of the uses of SV tools should be kept in mind when designing or evaluating SV tools. The suggestions made below are summarised in Table 2. 4.2.1. Flexibility The individuals had three different media to use in the assignment sessions. While the long programs were usually presented using a data projector to avoid time-consuming handwriting, the blackboard was intensively used to present on-the-fly illustrations to support the code. The valuable properties of the blackboard are availability, ease of use and flexibility. The interview data outside of this paper suggest that the technical incompatibilities caused much trouble for students with limited computer skills. In an ideal situation, a true platform-independent visualisation system like Jeliot could be a common ground for students working in different environments. From the perspective of novice users, the generation of visualisation should be as automatic as possible making it also less timeconsuming. The teachers should also value the speed of creating the visualisation, just as in the survey reported by Naps et al. (2002) the four most cited reasons not to use visualisations in the teaching were related to the consumption of time and lack of effective SV tools. Rewriting the code, even small parts of it, can dramatically reduce the user’s interest in using the tool. In Jeliot, the user must write the code slightly differently than when using a standard Java compiler. The user writes applets instead of applications. The methods for text output also differ from standard Java. These differences were visible to the user, who was expected to understand them. However, the need to rewrite the programs prohibited the use of Jeliot in presentation situations and made Jeliot non-user-friendly from the novices’ perspective.
Table 2 Summary of the guidelines for demonstration tools Flexibility
Platform-independence, no need to rewrite programs for visualisation Make different explaining strategies possible Let the user quickly select a part of the code to visualise
Program Structure Data Structures Objects
Visualise program structure and help to browse around Follow the execution of the program Visualise the data structures and the manipulation of data Visualise objects and the passing of parameters
144
M. Lattu et al. / Computers & Education 41 (2003) 133–148
Last but not least, the results of the study point out that although there seems to be a great variance of different explanation strategies, they can be described using a set of properties. When visualising the execution of the program, the user should be able to quickly select the desired properties for the visualisation. 4.2.2. Visualising the program structure According to the findings of this study, needs exist to visualise both the program structure and data structure. The programs were presented, without exception, by showing the code using different explaining strategies. The presenters often needed support for the code-oriented explanations. The supporting views, often graphical, were presented by using the blackboard and the OH projector, which made it possible to draw illustrative pictures or write some statements quickly. The data projector made it possible to show the execution of the program. A pure program code was often used to explain the programs. The single-statement explanation strategy was employed especially when the programs were short. The code was browsed back and forth to follow the execution in complex programs. While developing sophisticated animation tools, the good old ways of illustrating the program should not be forgotten. Traditionally, the program structure has been visualised by using indentation, but coloured texts, underlining and sidelining could also be used. It should be possible to write and draw remarks on the code area to support the presentation. Illustrations are not the only way to make programs more understandable. A pseudo code was used a few times in the course to support presentations. The possibility of generating a flowchartlike graphical outline of the program should be studied. 4.2.3. Visualising data structures Most existing visualisation systems (e.g. Polka and XTango, Stasko & Lawrence, 1998, Byrne et al., 1999; AlgAE, Weiss, 1998) focus on visualising data structures. Normally, these are visualised by presenting the variables and the manipulation of them in a graphical form. However, the results of this paper show that there are several ways to present variables. Providing as many alternatives as possible allows the user to select that which best suits his or her personal taste or the existing conventions of the course. As there is more than one way to visualise a variable, a certain illustration can be seen in various ways (Petre, 1995). It must also be remembered that the discussion about chunk sizes applies here as well. 4.2.4. Visualising objects The user should be able to browse the code, not only up and down, but also to the subclasses and back to the main class. Optionally, when the presenter applies the order-of-execution explanation strategy, the program window should follow the execution of the program. The classes and their instances, the objects, play an important role in object-oriented programming. When taking the course, it was obvious that some students had great difficulty understanding the concept of object and the relationship between a class and an object. In the course, studied objects were usually illustrated using the blackboard. The visualisation tool could present the objects of the program in either textual or graphical form. The text mode could simply list the properties of the objects in use, while this data could also be presented in pictorial form. These features could be useful in exploring properties of
M. Lattu et al. / Computers & Education 41 (2003) 133–148
145
unfamiliar class libraries. Furthermore, the program flow could be visualised in this graphical object view, highlighting the active class and method, thus presenting the operation of the program and the parameter transmission between the objects. The idea of visualising objects is not revolutionary. A similar idea has been presented by Hosking (1996), for example. There seems to be a real need for this feature. A recent case study (Bo¨stler, Johansson, and Nordstrom, 2002) indicates that the BlueJ visualisation system helped students learn object-oriented concepts better and faster. Petre, Blackwell and Green (1998, 459–460) have claimed that there is an inherent contradiction in SV. It makes programming more concrete, but, on the other hand, using symbols has a habit of making things even more abstract. The writers have seemingly forgotten that all languages are based on symbols, and without any evidence presented one has no reason to believe that written symbols are less abstract than their pictorial equivalents.
5. Conclusions In spite of the interesting projects of SV tools, their use among everyday teaching or practising programmers appears to be limited. One possible reason is that the design of existing SV tools has not been based on manual demonstration practices. Ordinary practitioners are less sympathetic to the shortcomings of the SV tools than the enthusiasts developing the tools. Therefore, the pilot projects have failed to meet the needs of real-life. Of course, other reasons like the possible technical instability of the experimental tools must also have contributed to the outcome. Naps et al. (2002) have approached the issue by surveying the teachers’ practices using a questionnaire, which is the correct way to get a general picture of the habits of use. However, the results of this paper suggest that the developers of SV tools should also address the real-life needs and not just the technical issues. This paper shows an example of studying existing visualisation. One can believe that knowledge of existing practices results in better SV tools. When evaluating the existing visualisation systems, it appears that most of them are designed for demonstration use rather than for use in an explorative manner. However, this study suggests that flexibility is also a key element for successful visualisation at least when the learning situation is bi-directional, as in the assignment sessions (c.f. mass lectures). The variation of explanation strategies and visualisation methods is large and also the focus of visualisation has variance. Unfortunately, the requirement of all-around flexibility makes the development of an adequate SV tool a massive project. However, the knowledge gained from existing pilot projects and research on them provides a far clearer path for today’s developers than for the pioneers working in the 1980s and 1990s. Therefore, it looks like the research on SV has contributed a great deal although it has not been able to prove the miraculous effect of visualisation on learning outcomes.
Acknowledgements This work has been supported by the Ministry of Education, Finland, and the Academy of Finland. The authors thank the referees for constructive remarks which helped to improve the article.
146
M. Lattu et al. / Computers & Education 41 (2003) 133–148
Appendix. The visualisation methods used in the assignment sessions. The cross-tabulation between dependent and independent variables. Table A: Main characteristics and distributions of the variables
Explaining order Code Execution Missing
34 32 36 (35%)
Explaining chunk single statement group of statements single method cursory dynamic Total Missing
35 12 3 32 11 93 9 (9%)
Subject Teacher Student Total Missing
31 70 101 1 (1%)
Media Overhead data projector blackboard Total Missing
30 69 3 102 0 (0%)
Table B: Summary of the conducted analysis
order chunk
observ3 0.125 0.172**
observ4 0.125 0.138
observ5 0.188 0.138
subj (a) 0.086
media 0.188* 0.034
Significance level: *P<0.10, **P<0.05, other values are not statistically significant. observ3. . .5: Observations were divided to 3. . .5 successive groups by the number of the observation. (a) In this case, we were unable to count asymmetric lambda. However, the chi-square and Pearson’s correlation coefficient suggested that there is no interplay between the variables.
M. Lattu et al. / Computers & Education 41 (2003) 133–148
147
References Arnold, K., & Gosling, J. (1996). The Java programming language. Reading (MA): Addison-Wesley. Byrne, M. D., Catrambone, R., & Stasko, J. T. (1999). Evaluating animations as student aids in learning computer algorithms. Computers & Education, 33(4), 253–278. Bostler, J., Johansson, T., & Nordstro¨m, M. (2002) Teaching OO concepts—a case study using CRC-cards and BlueJ. In Proceedings of the 32nd ASEE/IEEE Frontiers in Education Conference, Paper T2G (1–6). Campbell, D. T., & Stanley, J. C. (1966). Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin. Gurka, J.S. & Citnin, W. (1996). Testing effectiveness of algorithm animation. Proceedings of 1996 IEEE Symposium on Visual Languages (182–189). Los Alamitos, USA: IEEE Computer Society Press. Hansen, K. M., & Ratzer, A. V. (2002). Tool support for collaborative teaching and learning of object-oriented modeling. SIGCSE Bulletin, 34(3), 146–150. Hosking, J.G. (1996). Visualisation of object-oriented program execution. In Proceedings of 1996 IEEE Symposium on Visual Languages (190–191). Los Alamitos, USA: IEEE Computer Society Press. Huber, G. L. (1999). Aquad 5 Home Page. Program for Qualitative Data Analysis. Retrieved February 3, 2003, from http://www.aquad.de/eng/e-index.htm. Ja¨rvinen, E.-M. (1998). The Lego/Logo learning environment in technology education: an experiment in a Finnish context. Journal of Technology Education, 9(2), 47–59. Lavonen, J. M., Meisalo, V. P., Lattu, M., & Sutinen, E. (2003). Concretising the programming task:case study in secondary school. Computers & Education, 40(2), 115–135. Levy, R. B-B., Ben-An, M., & Uronen, P. A. (2003). The Jeliot 2000 program animation system. Computers & Education, 40(1), 1–15. Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills: Sage. Meisalo, V., Sutinen, E., & Tarhio, J. (1997). CLAP: Teaching data structures in a creative way. In G. Davies (Ed.) Proc. ITiCSE 97. Integrating Technology into Computer Science Education. Uppsala University, 117–119. Meisalo, V., Sutinen, E., Tarhio, J., Terasvirta, T. (1998). Combining algorithmic and creative problem solving on the Web. In G. Davies (Ed.), Proc. Teleteaching ‘98/IFIP World Computer Congress 1998 (715–724). Austrian Computer Society. Mulholland, P. (1998). A principled approach to the evaluation of SV: a case study in Prolog. In J. Stasko, J. Domingue, M. H. Brown, & A. P. Blame (Eds.), Software visualisation. Programming as a multimedia experience (pp. 439–451). Cambridge, Massachusetts: The MIT Press. Naps, T. L., Roßling, G., Almstrum, V., Dann, W., Fleischer, R., Hundhausen, C., Korhonen, A., Malmi, L., McNally, M., Rodger, S., & Vela´zguez-Iturbide, J. A. (2002). Exploring the role of visualization and engagement in computer science education. Report of the Working Group on ‘‘Improving the Educational Impact of Algorithm Visualization’’, ITiCSE, Aarhus June, 24–26. Pane, J. F., Ratanamahatana, C. A., & Myers, B. A. (2001). Studying the language and structure in non-programmers’ solutions to programming problems. International Journal of Human-Computer Studies, 54(2), 237–264. Petre, M. (1995). Why looking isn’t always seeing: readership skills and graphical programming. Communications of the ACM, 38(6), 33–44. Petre, M., Blackwell, A., & Green, T. (1998). Cognitive questions in software visualisation. In J. Stasko, J. Domingue, M. H. Brown, & A. P. Blame (Eds.), Software visualisation. Programming as a multimedia experience (pp. 453–480). Cambridge, Massachusetts: The MIT Press. Siegel, S., & Castellan, N. J. Jr. (1988). Nonparametric statistics for the behavioral sciences. The 2nd edition. Singapore: McGraw-Hill Book Co. Stasko, J., & Lawrence, A. (1998). Empirically assessing algorithm animations as learning aids. In J. Stasko, J. Domingue, M. H. Brown, & A. P. Blame (Eds.), Software visualisation. Programming as a multimedia experience (pp. 419–438). Cambridge, Massachusetts: The MIT Press. Strauss, A., & Corbin, J. (1998). Basics of qualitative research. Techniques and procedures for developing grounded theory. The 2nd edition. Thousand Oaks: Sage. Sutinen, E., Tarhio, J., & Tera¨svirta, T. (2003). Easy algorithm animation on the Web. Multimedia Tools and Applications, 19, 179–194.
148
M. Lattu et al. / Computers & Education 41 (2003) 133–148
Weiss, M. A. (1998). Data structures and problem solving using Java. Reading, Massachusetts: Addison-Wesley. Wilensky, U. (1995). Paradox, programming, and learning probability: a case study in a connected mathematics framework. Journal of Mathematical Behavior, 14(2), 253–280. Whitley, K. N. (1997). Visual programming languages and the empirical evidence for and against. Journal of Visual Learning and Computing, 8, 109–142.
A visualisation tool as a demonstration aid Matti Lattua, Veijo Meisalob, Jorma Tarhioc,* a
Department of Teacher Education, PO Box 8 FIN-00014 University of Helsinki, Finland Department of Teacher Education, PO Box 9, FIN-00014 University of Helsinki, Finland c Helsinki University of Technology, Department of Computer Science and Engineering, PO Box 5400, FIN-020l5 HUT, Finland b
Received 18 September 2002; accepted 18 February 2003
Abstract As algorithms have been seen to be hard to teach and learn, teachers have tried to look for help in algorithm animation. While the effect of algorithm animation on learning has been studied, but not reliably evidenced, this study tries to approach the problem from a different perspective. Sixty hours of assignment sessions in an introductory programming course were observed to determine the kind of demonstration and explaining strategies teachers and students tend to use. The results show that although the variation of different visualisation types is large, there are certain common properties describing the explanation of the programs. Guidelines for demonstration tools are presented based on the results. # 2003 Elsevier Ltd. All rights reserved. Keywords: Programming; Interactive teaming environments; Improving classroom teaching
1. Introduction Traditionally, algorithms have been abstract and hard for students to understand. Therefore, interest in a deeper understanding of them has been weak even among computer science (CS) students. As understanding and developing algorithms is essential in computer science, developing novel ways for improving learning and understanding algorithms should be one of the most important areas in rethinking computer science education. To make the abstract functions of computers more concrete, CS teachers tend to use different kinds of illustrations (static drawings) or computer-based visualisation tools (moving illustrations). Recently, there has been new emphasis placed on using visualisation tools to support oral * Corresponding author. Fax: +358-9-451-3293. E-mail address: jorma.tarhio@hut.fi (J. Tarhio). 0360-1315/03/$ - see front matter # 2003 Elsevier Ltd. All rights reserved. doi:10.1016/S0360-1315(03)00032-0
134
M. Lattu et al. / Computers & Education 41 (2003) 133–148
presentation, and not only to be used independently by students (e.g. Hansen & Raizer, 2002). However, too much research on software visualisation (SV) has concentrated on finding the measurable effects of SV on learning outcomes. Thus, this paper focuses on examining the possibility of developing new teaching practices involved with program visualisation by analysing a case of one teacher and three student groups. The focus is on their illustration and visualisation methods. The discourse during their presentation is also studied. In this manner, desirable features for a visualisation tool as a demonstration aid in the classrooms have been analysed. This study is part of a larger evaluation project that examined the suitability of Jeliot, a webbased visualisation tool (Sutinen, Tarhio, & Tera¨svirta, 2003) in introductory teaching and learning. A researcher independent of the development group carried out the evaluation study. 1.1. The Research on Software Visualisation Software visualisation is a wide concept, used to cover almost anything from the layout of the written program (‘‘pretty-printers’’) to the visual languages. Petre, Blackwell, and Green (1998, 454) give four examples of different kinds of software visualisation: 1. Using the SV to present large data sets. Helping the interpretation by capitalising on perceptual effects. 2. Demonstrating the virtual machine to make the behaviour of the program visible, usually for pedagogical purposes. 3. Changing the perspective, for example, showing a large program in a single view. 4. Display-based reasoning creates an alternative symbolic re-presentation of the program to ease the programmer’s reasoning. The general assertions of the algorithm animation enthusiasts is that SV could be one way to help students or even laymen understand the logic of algorithms. Visualisation could also encourage students to approach debugging in a more analytical way, as it makes the programs and their execution more transparent than the traditional programming environments. Using a visualisation tool in teamwork situations has been shown to be beneficial in some case studies (e.g. Meisalo, Sutinen, & Tarhio, 1997). Most studies dealing with SV focus on the efficiency of the visualised program vs. traditional methods in teaching. So far, the results of evaluation studies on the advantages of using graphical elements in computer-based instruction and computer-aided learning have been contradictory (Mulholland, 1998; Whitley, 1997). Stasko and Lawrence (1998) argue, for example, that the key of the research should be: ‘‘Do algorithm animations truly assist teaching and learning, and if so, how can they be best utilized?’’ (see also Byrne, Catrambone, & Stasko, 1999). The question seems fair. It is essential in science that new ideas are first tested and the assessment is based on the results. However, there is a difference between the question of whether visualisation helps in conceptualisation and memorising, and that of whether it helps learning. The former question could be asked by a psychologist, the latter by a pedagogue, as their concepts of learning differ. While the psychologist understands learning as changes in the learner’s cognition, the pedagogue’s wider focus covers planning, organisation, evaluation and social background, to mention just a few examples. The research questions of the latter could be how students learn to use a certain programming tool (Lavonen, Meisalo, Lattu, & Sutinen, 2003), the
M. Lattu et al. / Computers & Education 41 (2003) 133–148
135
nature of problem-solving when using a certain programming tool (Ja¨rvinen, 1998), or does programming aid in learning a mathematical concept (Wilensky, 1995). It is evident that both kinds of research are needed. A firm psychological basic theory about the relations between memory, the development of concepts, and perception (in the case of SV, visualisation and animation) would have an enormous impact on teaching and learning. While waiting for such a theory, one can still study the various ways SV can be applied to support the existing teaching and learning methods, as well as develop totally new practices. In these studies, the questions should be broad, and it should always be possible to focus on emerging interesting phenomena. Without proper research, the development of pedagogy around the use of SV remains at the level of an unjustified list of ‘‘best practices’’, which is hardly the way that these issues should be dealt with in the scientific community. 1.2. Jeliot Jeliot (Sutinen et al., 2003) is a tool for the rapid generation of simple animations. It is a result of long development work. Originally, it was aimed at helping students and researchers learn and understand algorithms and data structures, but it can also be applied as a demonstration tool. Jeliot works on the Web, and so it can be used anywhere with a browser that supports Java. Jeliot allows the user to animate his or her own algorithms. The user writes and submits the algorithm as a plain Java program, and gets back its animation. This animation is generated automatically from the source code, and is displayed on the user’s screen. Animation is based on self-animation of data types: the user selects the variables of the program, which he or she wants to visualise, and Jeliot shows them on the screen. During animation, the user may adjust the appearance (shape, colour, location) of the selected variables or keep the default appearance. It is possible to store these selections for later use. Although Jeliot was designed for algorithm animation, the framework could also be used in other subjects. Many dynamic phenomena can be described as algorithms, and, therefore, visualised by Jeliot. For example, it would be possible to visualise the Mendelian rules of inheritance for a biology class (Meisalo et al., 1998). Another animation system called Jeliot 2000 (Levy, Ben-Ari, & Uronen, 2002) was developed based on the ideas of Jeliot. Jeliot 2000 is a standalone Windows application, which inherited selfanimation of some data types from the original Jeliot, but which allows no adjustments to the animation. This paper only considers the original Jeliot.
2. Empirical study and methodology This study can be best defined as an evaluational case study, where the analysis was conducted using qualitative methods supported by a small quantitative part. From the quantitative framework, the setting does not fulfil the requirements of experimental design and its low validity limits the possibility of generalisations (Campbell & Stanley, 1966, 6–12). However, from the qualitative viewpoint the decision to be able to transfer the results to other environments should be left to the receiving end, while the report has to provide significant background information to enable transferability (Lincoln & Guba, 1985, 296–298).
136
M. Lattu et al. / Computers & Education 41 (2003) 133–148
The empirical study was carried out in the Department of Computer Science at the University of Helsinki in the autumn term of 1998. The initial goal of the research was to use Jeliot, a certain www-based visualisation tool, in an introductory programming course, where Java was utilised as a programming language. The course consisted of 52 h of lectures and 24 h of assignment sessions. The lecturer, who also designed the assignments, gave the lectures to the students as a one large group. Typically, the assignments were programming tasks so the solutions were Java code. In the assignment sessions, the groups discussed students’ solutions to the assignments given in the previous week’s lectures. The assignment groups were held in smaller groups, typically 15–25 students. There were 17 assignment groups in the course, which were taught by five teachers. The students could choose which group they would attend during the course. This study focused on three assignment groups and the students were informed about this when selecting their groups. The teacher of the experiment groups was an MSc with several years’ experience in teaching CS. In the beginning, the goal of the study was to encourage the students to use Jeliot’s visualisation and follow its use. While the lecturer had clearly stated that he would not make any adjustments to the lectures because of the ongoing research in the three groups, it was meaningful to focus the resources on the three assignment groups. Because the study had an evaluational nature, a large variety of data were collected: a group of voluntary students was pre- and post-interviewed, students were asked to keep a diary of their work and the assignment sessions were observed. If Jeliot had been widely used, all these could have contained relevant data. However, a few drawbacks of Jeliot, some technical problems and the lecturer and the teacher’s lack of interest in the study lead to the use of Jeliot being minimal. The observation was non-structured. The researcher sat in the classroom and took notes on the activities, conversations and the use of different methods of illustration. The researcher observed 60 h of assignment sessions, evenly between three groups. Notes were made, which were then transcribed and imported to the qualitative data analysis program Aquad 5 (Huber, 1999). Drawings were stored on paper as they appeared in the researcher’s notebook. The coding procedure was started by following the principles of grounded theory where the concepts are built along the analysis (Strauss & Corbin 1998, 12). The researcher both conducted the data collection and made the analysis. In the first phase, the material was read and some explorative coding was made, as suggested by Strauss and Corbin (1998, 57–71). The final interest of the paper was selected at this point, as making sense of the teacher and the students’ presentations was seen as useful when designing tomorrow’s visualisation tools. The low-level codes were further processed, but as the nature of the analysis changed from qualitative analysis to a mixed-method research, the grounded theory procedures could not be followed as such. The quantitative data were compiled by going through the described presentations in the data. A presentation was defined as a continuous phase where the main presenter did not change. Presentations were described by using attributes created in the qualitative analysis: explaining strategy (see Section 3.1), number of the observation (equals roughly to the advancement of the course), presenter (teacher/student), and media (blackboard, overhead, data projector). As all these attributes were not always present in the data, there were some missing values. The main characteristics of the variables are presented in the Appendix. To find possible interplay, the cross-tabulated variables were studied statistically, using the asymmetric lambda (L), which can be used with nominal data (Siegell & Castellan, 1988, 298–303). The formula of the asymmetric lambda produces a percentage that the antecedent (independent
M. Lattu et al. / Computers & Education 41 (2003) 133–148
137
variable) can explain from the consequent (dependent variable). The significance level of 10% was chosen because of the explorative nature of the study.
3. Results As the goal of this article is to develop ideas for the further development of SV tools (c.f. Pane, Ratanamahatana, & Myers, 2001), by studying existing teaching practices, the focus was on certain elements in the data. These were the ways the teacher and students explained programs and the ways they illustrated their programs. Explaining strategies and illustration methods are significant issues for the developers of SV tools. At the very least,the new tools that aim to enhance group-work in education and industry should fit existing communication patterns. Of course, the SV tools could offer totally new ways of understanding programs and create new ways of sharing understanding of computer programs. 3.1. Explaining strategies The method used to present the program in the assignment sessions is called an explaining strategy. The grounded theory analysis found two attributes defining a strategy, namely the order of explanation (execution order or order of the written code) and the size of the chunks used. The five subcategories for this attribute are: A single statement was the smallest unit in explaining programs. This was typically used when the programs were short. A group of statements is a strategy similar to the single statement method, but here the presenter combined a few statements into one chunk. Usually the group was formed around one idea or action, for example: ‘‘In these lines, the program checks whether the age of the person is within a reasonable range’’. A single method (or a procedure) was a much used chunk size later on in the course, when the programs were large and some parts of the code were trivial, for example: ‘‘In this method, the program sets the initial values for these variables’’. Cursory explaining strategies were also encountered, when no detailed explanation was given to support the visual presentation—usually just the written code. A dynamic strategy was a mixture of the other methods. Here the presenter varied the orientation, presumably depending on the complexity of the code. To study the concept in the data, quantitative data were compiled using these attributes (see Section 2). The two orders (execution and written) were almost evenly used in the presentations, while the most used chunk size was one statement (about 50% of the findings). Table 1 shows the frequencies of the categories together with observation examples. For the statistical analysis the number of the observation (advancement of the course) was divided into categories. While there was no theory to suggest number of the categories, three different groupings were tried so that the observations were divided into three to five groups. The statistical analysis showed that the independent (background) variables (number of the
138
M. Lattu et al. / Computers & Education 41 (2003) 133–148
observation, subject or media) were not related to the explaining strategy (explaining order, chunk size). However, there were two exceptions to the general pattern. When the observations were divided into three groups, the advancement of the course and the chunk size produced lambda coefficient L=0.172 (P<0.05). Another significant result was acquired when the explaining order was explained with media type. In this case L=0.188 (P<0.10). Although both values are statistically significant, they are very modest. The summary of the analysis is presented in the Appendix. 3.2. Illustration and visualisation Explaining programs was normally done by the chalk-and-talk method, although the video projector was intensively used instead of the blackboard and the OH-projector. However, the projector was mainly used to show the program code. Because of the visual nature of Jeliot, the research team wanted to collect information on the existing visualisation methods. These notes were divided into three categories: illustrating the variables, illustrating the structure of the program and the simultaneous use of two windows in the data projector. Although in a qualitative study, and especially in a case study, the counts do not necessarily relate to the importance, this fact may be worthwhile presented along with other defining data. 3.2.1. Illustrating the variables Findings: 31. The variables were usually illustrated using the blackboard, while the program code was shown using the OH projector or the video projector. At the the beginning of the course, there was a need to understand the chain-like nature of the program execution. An attempt was made to achieve this by following the execution of the program from the code and illustrating the variables manipulated by the program. In this early stage, the variables were listed in either one row or one column (see Fig. 1). The values were written beside or under the name of the variable. The last value corresponded to the current one, but the older values were still available if needed. Later on, the use of basic variables became trivial and the illustration focused on more complex variable structures. The next stage was array. Arrays were clearly an essential factor in illustrating the programs, as most algorithms in this course manipulated data stored in arrays. Therefore, after being introduced, arrays were a popular object of visualisation throughout the course.
Fig 1. Illustrating the variables in the beginning of the course.
M. Lattu et al. / Computers & Education 41 (2003) 133–148
139
Arrays were usually illustrated by a row (or column) of boxes, including the values (see Fig. 2). Sometimes the index values were shown to underline that the index number of the first element is 0, and, therefore, the index of the last element is one less than the total number of the elements. Some students did not draw the boxes and just used a row of numbers (or characters) as an array. Arrays included a number of different kinds of data: numeric values, characters and strings, other arrays and references to objects. Slightly different from arrays, there was also a need to visualise a single string object that was not transformed into an array of characters. Complex algorithms manipulating arrays were not dealt in the assignment sessions. The basic sorting algorithms were briefly presented in the first third of the course as and later on just referred to. 3.2.2. Illustrating program structures Findings: 39. Program structures were mostly illustrated using the blackboard and transparencies. The highlighting of the program code was used especially when the code was on a transparency. The teacher underlined statements or sidelines and parentheses to focus on statements or structures. While presenting the code using the data projector, the selection feature of the mouse was often used in a similar way. The teacher presented an interesting way of giving a graphical outlook to the visibility of variables. He drew a box to represent each of the methods. The ‘‘global’’ variables of the class were outside these boxes (see Fig. 3). The visual representation also reflected a way to illustrate the order of execution of the program. The teacher drew arrows from the main method, calling it a sub-method. Another arrow was drawn to show how the control returned from the sub-method to the main method. Obviously, a great number of calls between the methods would result in a chaos of crossing arrows.
Fig 2. Illustrating the arrays on the course.
Fig 3. Illustrating global and local variables.
140
M. Lattu et al. / Computers & Education 41 (2003) 133–148
Some of the most used objects of visualisation were the objects of Java. The objects are independent subprograms with their own code and data structure. Each object can have a number of instances, much like there can be a number of different cars of the same make and model. The objects were illustrated as a circle or ring. The circle included the methods and the variables of the object. The visibility of methods and variables could be shown by closing them in a box (private) or writing them on the borderline of the circle (see Fig. 4). The objects were connected to the other data structure by a line. The connecting line could point, for example, to the arrays or to the program code where the objects were created. A similar method has been used by Arnold and Gossling (1996). Inheritance was presented at the end of the course. The teacher illustrated the relations between the classes by drawing a tree of inheritance. 3.2.3. Using two windows Findings: 11. When the teacher and the students were presenting the programs, using the data projector, they sometimes used several windows simultaneously. The number of windows was usually two. The presenters tended to show the program code in the first window while running the program in the other. Another way to utilise the second window was to present the program code of a different class, while the first window showed the code of the calling class. A few times the teacher showed a pseudo code of the program in the second window. No debugging tools were utilised in the assignment sessions. More often, when the presenters needed another view of the program, they used the blackboard or transparencies to sketch pictorial models or write some statements.
4. Discussion 4.1. General considerations This study analysed the visualisation methods used in an introductory programming course in order to support the development of the program visualisation. The results showed that the explaining strategies and methods of visualisation had a large variance. The most important conclusion is that visualisation tools should be as flexible as possible to support different strate-
Fig 4. Illustrating the objects.
M. Lattu et al. / Computers & Education 41 (2003) 133–148
141
gies. One could easily expect that the teacher and students have differences in the explaining strategy or the strategies used at the beginning of the course systematically differ from the approaches used in the end, where the students have a wider understanding of programming. However, the analysis showed that in most cases there is no such association. One obvious reason of the observed similarity of the teachers and students’ explaining strategies could be the students’ level of programming skills. In the programming course, they do not just learn how to write programs, but also how to piece together programs and explain them to themselves and others. Although no computer-based visualisation tool was ultimately used because of the technical difficulties of Jeliot, the visualisation of the programs was still an important part of the discourse. Visualisation was used more than four times in each session on average, whereas the typical number of assignments for each session varied from four to six. The visualisation was offered and used as an important way to understand programs and their operations at least in the groups studied. According to the analysis, the media used in the presentations was in association with the explaining order. There are at least two possible reasons for the relation between the media and the explaining order. At the beginning of the course, where the OH projector was mostly used, the programs were short, and, therefore, the order of execution was a natural choice. The second possible reason was that the teacher wanted to support the concept forming of the sequential nature of the program by using the execution order at the beginning of the course. The students might have followed his example, as they did not have other models of explaining. However, neither of these theories is supported by the analysis. In the course, the individuals visualised program structure, variables, and somewhat combining the previous two, the objects. When presenting the code, the presenters tended to highlight meaningful chunks of code to divide the whole into meaningful parts. Visualising variables was not needed right at the beginning of the course. However, later on complex algorithms manipulating the arrays seemed to be hard to deal with within the assignment sessions. For example, getting deeply into a sorting algorithm would take a lot of time and, therefore, they were not properly dealt with when faced. The importance of visualising the variables, either with novice or experienced programmers, has been pointed out by Gurka and Citrin (1996). Similar conclusions have been made in the present study. The variables were visualised both at the very beginning of the course and in the context of complex algorithms. Object-oriented programming brings another level of abstraction to programming. In teaching traditional procedural programming, the focus is on syntactic issues. Each element of the program (a subprogram or a variable) is tied to a specific part of the code. In teaching object-oriented programming, an additional focus is on the relations of objects and on the concepts connected with them. Thus, object-oriented programming is evidently more abstract, and the need for visualisation of object-oriented features was also clearly present on this course. 4.2. Desired features of a visualisation system Below, an outline will be presented that could be used to develop existing or new tools to be used to support an oral presentation of a program. At present, the availability and flexibility of the blackboard and OH projector are superior in showing spontaneous drawings and writing. However, if typical patterns of illustrations supporting presentations could be found, they could
142
M. Lattu et al. / Computers & Education 41 (2003) 133–148
be made by computer. Offering a pattern of useful thinking models in a program could support certain expert-like learning strategies. The data collected in the assignment sessions suggest that the presentations have common properties, which should be taken into account while implementing a visualisation tool. As the course was based on Java, the ideas presented below are influenced by the objectoriented programming. It must also be remembered that the organisation of the course strongly influenced the data collected. Basically, the students gathered to go through programming assignments given the previous week. Therefore, the discussion during the assignment sessions was focused mainly on understanding an existing piece of code. The data could have been totally different if we had followed the lectures, where the issues more generally dealt with programming-related ideas and concepts instead of program codes. Unfortunately, the lectures were not documented because of the initial goal of the research. There is a distinction between the basic explorative use (one does not know what to look for) and the demonstration use (focus on some important component) of SV tools. In the case of Table 1 The explaining strategies in assignment sessionsa Order of execution
Order of the code
Single statement on a hand-written transparency.
‘‘The student presents the program Explained the execution one statement at a time in the order of execution.’’ Findings: 18
Group of statements
‘‘— begins to explain the code in the order of execution. Selects the code using the mouse to support the presentation. Explains one group of statements at a time.’’ Findings: 2 ‘‘— presents other possible solutions by using the data projector and the less-program. Selects statements using the mouse while talking. Goes through the main features of the program, one method at a time in the execution order.’’ Findings: I ‘‘Explains the main functions of the program in the execution order.’’ Findings: 4
‘‘The student loads the program to the emacs editor and explains it using a pointer, beginning with the (sub) methods. The main program is at the bottom.’’ Findings: 9 Findings: 0
Single method
Cursory
Dynamic
a
‘‘— presents the solution in the order of execution.— Goes through the program statement by statement —. Does not enter into the single methods but explains their main function.’’ Findings: 2
Quotes are examples from the observation data.
‘‘Goes through the code in the written order by presenting the function of each method.’’ Findings: 1
‘‘Goes through the code in the order of writing. Does not go deeply into the syntax but presents the main characteristics of the implementation.’’ Findings: 9 ‘‘Presents the code in the written order. Selects a text using the mouse and presents the principle. Sometimes highlights single statements and explains the syntax.’’ Findings: 4
M. Lattu et al. / Computers & Education 41 (2003) 133–148
143
Jeliot, the explorative use visualises a program per se by showing the changes of those variables, which the user wants to look at. In this way, it works as an adjustable visual interpreter of the input program. When Jeliot is used as a demonstration tool, the main purpose is not necessarily to visualise the input program, but rather to create and show an animation. In the explorative use, the input program does not support visualisation in any way. In the demonstration use, the input program may contain data structures and statements whose only purpose is to produce additional visual effects. While this paper concentrates on visualisation to understand existing programs, the visualisation can also be used to create programs, or as in the case of Hansen & Ratzer (2002), to code the visualising object structure of a program. The distinction of these different purposes of the uses of SV tools should be kept in mind when designing or evaluating SV tools. The suggestions made below are summarised in Table 2. 4.2.1. Flexibility The individuals had three different media to use in the assignment sessions. While the long programs were usually presented using a data projector to avoid time-consuming handwriting, the blackboard was intensively used to present on-the-fly illustrations to support the code. The valuable properties of the blackboard are availability, ease of use and flexibility. The interview data outside of this paper suggest that the technical incompatibilities caused much trouble for students with limited computer skills. In an ideal situation, a true platform-independent visualisation system like Jeliot could be a common ground for students working in different environments. From the perspective of novice users, the generation of visualisation should be as automatic as possible making it also less timeconsuming. The teachers should also value the speed of creating the visualisation, just as in the survey reported by Naps et al. (2002) the four most cited reasons not to use visualisations in the teaching were related to the consumption of time and lack of effective SV tools. Rewriting the code, even small parts of it, can dramatically reduce the user’s interest in using the tool. In Jeliot, the user must write the code slightly differently than when using a standard Java compiler. The user writes applets instead of applications. The methods for text output also differ from standard Java. These differences were visible to the user, who was expected to understand them. However, the need to rewrite the programs prohibited the use of Jeliot in presentation situations and made Jeliot non-user-friendly from the novices’ perspective.
Table 2 Summary of the guidelines for demonstration tools Flexibility
Platform-independence, no need to rewrite programs for visualisation Make different explaining strategies possible Let the user quickly select a part of the code to visualise
Program Structure Data Structures Objects
Visualise program structure and help to browse around Follow the execution of the program Visualise the data structures and the manipulation of data Visualise objects and the passing of parameters
144
M. Lattu et al. / Computers & Education 41 (2003) 133–148
Last but not least, the results of the study point out that although there seems to be a great variance of different explanation strategies, they can be described using a set of properties. When visualising the execution of the program, the user should be able to quickly select the desired properties for the visualisation. 4.2.2. Visualising the program structure According to the findings of this study, needs exist to visualise both the program structure and data structure. The programs were presented, without exception, by showing the code using different explaining strategies. The presenters often needed support for the code-oriented explanations. The supporting views, often graphical, were presented by using the blackboard and the OH projector, which made it possible to draw illustrative pictures or write some statements quickly. The data projector made it possible to show the execution of the program. A pure program code was often used to explain the programs. The single-statement explanation strategy was employed especially when the programs were short. The code was browsed back and forth to follow the execution in complex programs. While developing sophisticated animation tools, the good old ways of illustrating the program should not be forgotten. Traditionally, the program structure has been visualised by using indentation, but coloured texts, underlining and sidelining could also be used. It should be possible to write and draw remarks on the code area to support the presentation. Illustrations are not the only way to make programs more understandable. A pseudo code was used a few times in the course to support presentations. The possibility of generating a flowchartlike graphical outline of the program should be studied. 4.2.3. Visualising data structures Most existing visualisation systems (e.g. Polka and XTango, Stasko & Lawrence, 1998, Byrne et al., 1999; AlgAE, Weiss, 1998) focus on visualising data structures. Normally, these are visualised by presenting the variables and the manipulation of them in a graphical form. However, the results of this paper show that there are several ways to present variables. Providing as many alternatives as possible allows the user to select that which best suits his or her personal taste or the existing conventions of the course. As there is more than one way to visualise a variable, a certain illustration can be seen in various ways (Petre, 1995). It must also be remembered that the discussion about chunk sizes applies here as well. 4.2.4. Visualising objects The user should be able to browse the code, not only up and down, but also to the subclasses and back to the main class. Optionally, when the presenter applies the order-of-execution explanation strategy, the program window should follow the execution of the program. The classes and their instances, the objects, play an important role in object-oriented programming. When taking the course, it was obvious that some students had great difficulty understanding the concept of object and the relationship between a class and an object. In the course, studied objects were usually illustrated using the blackboard. The visualisation tool could present the objects of the program in either textual or graphical form. The text mode could simply list the properties of the objects in use, while this data could also be presented in pictorial form. These features could be useful in exploring properties of
M. Lattu et al. / Computers & Education 41 (2003) 133–148
145
unfamiliar class libraries. Furthermore, the program flow could be visualised in this graphical object view, highlighting the active class and method, thus presenting the operation of the program and the parameter transmission between the objects. The idea of visualising objects is not revolutionary. A similar idea has been presented by Hosking (1996), for example. There seems to be a real need for this feature. A recent case study (Bo¨stler, Johansson, and Nordstrom, 2002) indicates that the BlueJ visualisation system helped students learn object-oriented concepts better and faster. Petre, Blackwell and Green (1998, 459–460) have claimed that there is an inherent contradiction in SV. It makes programming more concrete, but, on the other hand, using symbols has a habit of making things even more abstract. The writers have seemingly forgotten that all languages are based on symbols, and without any evidence presented one has no reason to believe that written symbols are less abstract than their pictorial equivalents.
5. Conclusions In spite of the interesting projects of SV tools, their use among everyday teaching or practising programmers appears to be limited. One possible reason is that the design of existing SV tools has not been based on manual demonstration practices. Ordinary practitioners are less sympathetic to the shortcomings of the SV tools than the enthusiasts developing the tools. Therefore, the pilot projects have failed to meet the needs of real-life. Of course, other reasons like the possible technical instability of the experimental tools must also have contributed to the outcome. Naps et al. (2002) have approached the issue by surveying the teachers’ practices using a questionnaire, which is the correct way to get a general picture of the habits of use. However, the results of this paper suggest that the developers of SV tools should also address the real-life needs and not just the technical issues. This paper shows an example of studying existing visualisation. One can believe that knowledge of existing practices results in better SV tools. When evaluating the existing visualisation systems, it appears that most of them are designed for demonstration use rather than for use in an explorative manner. However, this study suggests that flexibility is also a key element for successful visualisation at least when the learning situation is bi-directional, as in the assignment sessions (c.f. mass lectures). The variation of explanation strategies and visualisation methods is large and also the focus of visualisation has variance. Unfortunately, the requirement of all-around flexibility makes the development of an adequate SV tool a massive project. However, the knowledge gained from existing pilot projects and research on them provides a far clearer path for today’s developers than for the pioneers working in the 1980s and 1990s. Therefore, it looks like the research on SV has contributed a great deal although it has not been able to prove the miraculous effect of visualisation on learning outcomes.
Acknowledgements This work has been supported by the Ministry of Education, Finland, and the Academy of Finland. The authors thank the referees for constructive remarks which helped to improve the article.
146
M. Lattu et al. / Computers & Education 41 (2003) 133–148
Appendix. The visualisation methods used in the assignment sessions. The cross-tabulation between dependent and independent variables. Table A: Main characteristics and distributions of the variables
Explaining order Code Execution Missing
34 32 36 (35%)
Explaining chunk single statement group of statements single method cursory dynamic Total Missing
35 12 3 32 11 93 9 (9%)
Subject Teacher Student Total Missing
31 70 101 1 (1%)
Media Overhead data projector blackboard Total Missing
30 69 3 102 0 (0%)
Table B: Summary of the conducted analysis
order chunk
observ3 0.125 0.172**
observ4 0.125 0.138
observ5 0.188 0.138
subj (a) 0.086
media 0.188* 0.034
Significance level: *P<0.10, **P<0.05, other values are not statistically significant. observ3. . .5: Observations were divided to 3. . .5 successive groups by the number of the observation. (a) In this case, we were unable to count asymmetric lambda. However, the chi-square and Pearson’s correlation coefficient suggested that there is no interplay between the variables.
M. Lattu et al. / Computers & Education 41 (2003) 133–148
147
References Arnold, K., & Gosling, J. (1996). The Java programming language. Reading (MA): Addison-Wesley. Byrne, M. D., Catrambone, R., & Stasko, J. T. (1999). Evaluating animations as student aids in learning computer algorithms. Computers & Education, 33(4), 253–278. Bostler, J., Johansson, T., & Nordstro¨m, M. (2002) Teaching OO concepts—a case study using CRC-cards and BlueJ. In Proceedings of the 32nd ASEE/IEEE Frontiers in Education Conference, Paper T2G (1–6). Campbell, D. T., & Stanley, J. C. (1966). Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin. Gurka, J.S. & Citnin, W. (1996). Testing effectiveness of algorithm animation. Proceedings of 1996 IEEE Symposium on Visual Languages (182–189). Los Alamitos, USA: IEEE Computer Society Press. Hansen, K. M., & Ratzer, A. V. (2002). Tool support for collaborative teaching and learning of object-oriented modeling. SIGCSE Bulletin, 34(3), 146–150. Hosking, J.G. (1996). Visualisation of object-oriented program execution. In Proceedings of 1996 IEEE Symposium on Visual Languages (190–191). Los Alamitos, USA: IEEE Computer Society Press. Huber, G. L. (1999). Aquad 5 Home Page. Program for Qualitative Data Analysis. Retrieved February 3, 2003, from http://www.aquad.de/eng/e-index.htm. Ja¨rvinen, E.-M. (1998). The Lego/Logo learning environment in technology education: an experiment in a Finnish context. Journal of Technology Education, 9(2), 47–59. Lavonen, J. M., Meisalo, V. P., Lattu, M., & Sutinen, E. (2003). Concretising the programming task:case study in secondary school. Computers & Education, 40(2), 115–135. Levy, R. B-B., Ben-An, M., & Uronen, P. A. (2003). The Jeliot 2000 program animation system. Computers & Education, 40(1), 1–15. Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills: Sage. Meisalo, V., Sutinen, E., & Tarhio, J. (1997). CLAP: Teaching data structures in a creative way. In G. Davies (Ed.) Proc. ITiCSE 97. Integrating Technology into Computer Science Education. Uppsala University, 117–119. Meisalo, V., Sutinen, E., Tarhio, J., Terasvirta, T. (1998). Combining algorithmic and creative problem solving on the Web. In G. Davies (Ed.), Proc. Teleteaching ‘98/IFIP World Computer Congress 1998 (715–724). Austrian Computer Society. Mulholland, P. (1998). A principled approach to the evaluation of SV: a case study in Prolog. In J. Stasko, J. Domingue, M. H. Brown, & A. P. Blame (Eds.), Software visualisation. Programming as a multimedia experience (pp. 439–451). Cambridge, Massachusetts: The MIT Press. Naps, T. L., Roßling, G., Almstrum, V., Dann, W., Fleischer, R., Hundhausen, C., Korhonen, A., Malmi, L., McNally, M., Rodger, S., & Vela´zguez-Iturbide, J. A. (2002). Exploring the role of visualization and engagement in computer science education. Report of the Working Group on ‘‘Improving the Educational Impact of Algorithm Visualization’’, ITiCSE, Aarhus June, 24–26. Pane, J. F., Ratanamahatana, C. A., & Myers, B. A. (2001). Studying the language and structure in non-programmers’ solutions to programming problems. International Journal of Human-Computer Studies, 54(2), 237–264. Petre, M. (1995). Why looking isn’t always seeing: readership skills and graphical programming. Communications of the ACM, 38(6), 33–44. Petre, M., Blackwell, A., & Green, T. (1998). Cognitive questions in software visualisation. In J. Stasko, J. Domingue, M. H. Brown, & A. P. Blame (Eds.), Software visualisation. Programming as a multimedia experience (pp. 453–480). Cambridge, Massachusetts: The MIT Press. Siegel, S., & Castellan, N. J. Jr. (1988). Nonparametric statistics for the behavioral sciences. The 2nd edition. Singapore: McGraw-Hill Book Co. Stasko, J., & Lawrence, A. (1998). Empirically assessing algorithm animations as learning aids. In J. Stasko, J. Domingue, M. H. Brown, & A. P. Blame (Eds.), Software visualisation. Programming as a multimedia experience (pp. 419–438). Cambridge, Massachusetts: The MIT Press. Strauss, A., & Corbin, J. (1998). Basics of qualitative research. Techniques and procedures for developing grounded theory. The 2nd edition. Thousand Oaks: Sage. Sutinen, E., Tarhio, J., & Tera¨svirta, T. (2003). Easy algorithm animation on the Web. Multimedia Tools and Applications, 19, 179–194.
148
M. Lattu et al. / Computers & Education 41 (2003) 133–148
Weiss, M. A. (1998). Data structures and problem solving using Java. Reading, Massachusetts: Addison-Wesley. Wilensky, U. (1995). Paradox, programming, and learning probability: a case study in a connected mathematics framework. Journal of Mathematical Behavior, 14(2), 253–280. Whitley, K. N. (1997). Visual programming languages and the empirical evidence for and against. Journal of Visual Learning and Computing, 8, 109–142.