- Email: [email protected]
17 A descriptive framework for designing interaction for visual abstractions
Studies in Multidisciplinarity, Volume 2 Editor: G. Malcolm 9 2004 Elsevier B.V. All rights reserved.
17
A descriptive framework for designing interaction for visual abstractions K. Sedig a and J. Morey b
aCognitive Engineering Laboratory, Department of Computer Science and Faculty of Information and Media Studies, The University of Western Ontario, London, Ont., Canada bCognitive Engineering Laboratory, Department of Computer Science, The University of Western Ontario, London, Ont., Canada
This chapter propses a descriptive framework for categorisation and characterisation of the different forms of interaction with visual abstractions (VAs). Abstract visual representations play an important role in assisting human reasoning, thinking, and understanding processes. There are different forms of designing interaction with these representations. The goal of this chapter is to provide a descriptive framework to guide the designers and evaluators of cognitive tools to determine the appropriate forms of interaction that can facilitate the understanding of abstract concepts, patterns, structures and processes. The framework is described and substantiated using a number of VAs that represent and communicate mathematical ideas. 1.
INTRODUCTION AND BACKGROUND
Many concepts, patterns, structures, and processes are too complex to understand without the aid of external cognitive aids (Norman, 1993). Visual abstract representations can assist human reasoning and learning l This research is funded by the Natural Sciences and Engineering Research Council of Canada.
239
240
K. Sedig and J. Morey
(Jonassen et al., 1993; Glasgow et al., 1995; Peterson, 1996). The human visual system has limited channel capacity. Visuals provide high-bandwidth interaction with the mind. VAs can be defined as a set of interconnected symbols that can embody causal, functional, structural, and semantic relations and properties. Examples of VAs include visual mathematical representations, diagrams, maps, graphs, networks, and so on. Explicit, external VAs extend human memory by acting as "knowledge in the world", can stimulate cognitive activity, amplify human cognition, and assist perceptual interpretation (Nardi and Zarmer, 1993; Zhang and Norman, 1994). VAs may be primary (derived from real-world objects) or secondary (derived from representations such as patterns in raw data, textual information, or scientific and mathematical concepts). Much of the knowledge embodied in secondary VAs may not be at the surface level and readily available for reasoning or perceptible to the human mind. Allowing users to interact with VAs as cognitive tools 2 can enhance this process of reasoning, interpretation, and sense making. However, the form and style of interaction plays a crucial role in how well and how much knowledge learners can construct (de Souza and Sedig, 2001; Sedig et al., 2001). Most cognitive tools borrow interaction techniques devised for and used in productivity tools (Sedig et al., 2001). The appropriateness of some interaction techniques for problem solving and learning activities has been questioned (Golightly, 1996; Holst, 1996; Sedig et al., 2001). However, there is no clear understanding of what form of interaction cognitive tools should incorporate, de Souza and Sedig (2001) have suggested that when designing concept-centred interfaces, the availability of a general framework to guide choices among the visual representations is lacking. Additionally, there is a need for a framework to guide choices among forms of interaction with these visuals. Although Shneiderman (1991) has proposed a general taxonomy of interaction styles, this taxonomy is too broad and does not seem suitable for cognitive tools. This chapter is a step in creating a framework to categorise and characterise different forms of interaction with VAs. Existence of such a framework can provide designers of interactive cognitive tools with options as how to systematically think about design of interaction for VAs. In the following sections, we use several systems to develop our framework. All these systems have been developed by our research group and use mathematical concepts as a test-bed to assist us in thinking about the proposed framework. These systems include: Super Tangrams (Sedig and Klawe, 1996), a tool to help children learn 2D geometrical transformations (i.e. translation, rotation, and reflection); Archimedean 2Cognitive tools refer to computational tools intended to support and extend human mental activities while engaged in perceptual, reasoning, and problem solving processes (Lajoie, 2000).
A framework for designing interaction for visual abstractions
241
Kaleidoscope (Morey et al., 2001), a tool to help users visualise and explore polyhedral 3 solids; K-Lattice Machine (Sedig et al., 2002), a tool to help users explore sub-patterns in 2D regular lattice structures; Archimedean Confection, a tool to explore relationships among polyhedral solids; Lattice Space, a tool to visualise and explore 3D lattice structures; and Polyvise, an interactive tool to visualise and explore 4D Archimedean polytope structures.
2.
INTERACTION FACTORS
This chapter proposes that the form of interaction with VAs is determined by a set of factors. In this section, 10 factors are discussed: mode, flow, focus, filtering, scoping, recording, scaffolding, content, chunking, and configuration.
2.1.
Mode
The mode of interaction refers to the metaphoric bodily organ by which the user interacts with a VA. There are three basic bodily metaphors by which humans interact with entities in their surroundings: hands (handling entities), feet (walking on or through entities), and mouth (conversing with entities). Therefore, there are three modes in which a user can interact with a VA: manipulation, navigation, and conversation. Instances of these modes can be illustrated through an example. Figure 1 shows a VA representing a 3D lattice structure. As manipulation, the user can rotate it and view the whole structure from different angles; as navigation, the user can walk through or on it; and as conversation, the user can type a command to query the lattice about one of its properties or to transform it in some way. This can be done using natural language queries, speech, menus, form-fill-ins, or any type of linguistic command.
2.2.
Flow
The flow of interaction refers to the effect of the interaction on how the user perceives the relationship between cause and effect in the time-space continuum. Flow of interaction can be continuous or discrete. In continuous interaction, the user observes cause and effect simultaneously. When there is continuous flow to the interaction, a VA fluidly responds to the user's 3A polyhedron is a geometric solid bounded by polygons.
K. Sedig and J. Morey
242
Fig. 1. 3D lattice in Lattice Space. interaction with it. For instance, fig. 2 shows a VA representing the mathematical concept of 2D translation as an interactive vector. The user can click on one of the tips of the vector to change its size and direction. The user' s interaction with this VA is continuous because the movement of the mouse cursor is fluidly translated into a change in the size and direction of the vector. In discrete interaction, cause and effect are separated in time. That is, the interaction takes place in a modal fashion. For instance, fig. 3a shows a VA representing a state-transition diagram. In order to cause a transition from one state to another, the user clicks on the end point of one of the transitional links, and the state transition takes place. Although the user may see the effect of the click without any time delay, nonetheless this is a discrete interaction since the user's interaction with the VA takes place in temporal snapshots.
2.3.
Focus
The focus of interaction refers to the centre of attention of the user while interacting with an environment. There are two fundamental ways of interacting with VAs: direct and indirect.
A framework for designing interaction for visual abstractions
.
:
.
-
!
.
; .
! .
243
f .
i
i
L .....
~ .....
~ 7
~" ' ; . . . . . . .
i N-
-
..
.
.
.
.
.
.
i
.
.
,
. . . .
<
,
-:
:
-,
;
i Fig. 2.
Translation vector in Super Tangrams.
One of the popular interaction styles is direct manipulation (Hutchins et al., 1986). Direct manipulation refers to interfaces, which allow users to see graphical representations of objects and directly manipulate them on the computer screen with some kind of pointing device. In this type of interaction, the user is focused on the object of interaction. This style is widely used in productivity as well as cognitive tools. It has been suggested that direct manipulation is potentially inappropriate for activities that require
mental effort and understanding (such as problem solving), and that indirect manipulation is more appropriate (Svendsen, 1991; Golightly, 1996; Hoist, 1996). The "directness" of these interfaces has been cited as the main source
Fig. 3.
State-transition diagram (L) in K-Lattice Machine.
244
K. Sedig and J. Morey
of the problem since it makes them easy to use, not requiting much cognitive effort on the part of the user. Sedig et al. (2001) have argued that the problem with direct manipulation lies in "what is manipulated" rather than the "manipulation" itself. This has been demonstrated through empirical studies with children learning 2D transformational geometry concepts. It has been proposed that the focus of attention and interaction should be the structures and processes being investigated (Hoist, 1996; Sedig et al., 2001). Thus both direct and indirect interactions can serve different purposes. In direct interaction, the user is directly focused on a VA and interacts with it without any other intermediary representations. The VA can represent a concept, a structure, a map, or some other abstract entity. For instance, if the goal of the user is to explore the concept of translation, interaction should be directed towards its visual representation (see fig. 1; also see Sedig et al., 2001). The same technique can hold if the user wants to discover the structure of polyhedra solids (Morey et al., 2001). The VA on the fight-hand side of fig. 4 shows a polyhedron solid. The user can directly interact with it by manipulating a control point on the structure and observing how it metamorphoses from one geometric solid to another. Depending on the type of VA, this type of interaction can be called direct concept interaction (de Souza and Sedig, 2001; Sedig et al., 2001), direct structure interaction, direct entity interaction, and so on. In indirect interaction, there are at least two VAs involved. The user interacts with one VA through another VA (or other VAs). This may be the case when interaction with one VA makes it easier to understand and decode the features of another VA. For instance, lattices have local structures that infinitely repeat themselves. Morey et al. (2001) have suggested that it is
Fig. 4. Polyhedralsolids.
A framework for designing interaction for visual abstractions
245
easier to understand and reason about some of the mathematical properties of 2D regular lattice structures if they are represented using finite state machine (FSM) diagrams. The FSM diagram captures the local geometry of the lattice in a concise visual representation. Focusing the interaction on the FSM diagram helps the user discover the repetitive patterns inherent in the lattice structure. Figure 3 shows a screen capture of K-Lattice Machine where the user interacts with an FSM diagram (fig. 3a) to construct a K lattice (fig. 3b). In this example, the user interacts directly with the FSM diagram and indirectly with the lattice abstraction. As such, indirect interaction necessitates direct interaction with one VA.
2.4.
Filtering
VAs may be aggregates of several features, components, or layers of detail. This can render a VA noisy and difficult to understand. In such cases, the user may want to remove some of the features or components of the visual structure in order to make the VA more understandable. The user can choose to view a subset of the original visual structure by specifying which features and components should remain visible or hidden. This method of removing visual noise can be called filtering. When filtering is used, it can be discrete or range-based. In discrete filtering, the user can use toggles to specify which features of a VA are to be present. For instance, the VA in fig. 5a represents a 4D polytope structure derived from a hypercube, as visualised in Polyvise. This structure contains several sets of components, such as vertices, edges, polygons, and cells. The VA in fig. 5b shows the same polytope structure in which the set of polygonal faces has been removed using discrete filtering. In range-based filtering, rather than using binary toggles, the user specifies a range of values for the visual components. The visibility of components
Fig. 5. (a to b): Discrete filtering applied to a 4D polytope structure in Polyvise.
K. Sedig and J. Morey
246
depends on whether or not their values fall within or outside of this range. Finally, in both discrete and range-based filtering, logical operators can be used to combine various filters.
2.5.
Scoping
Scoping refers to the available field of view of a VA. There may be instances where a VA is too complex and the user may have difficulty understanding the components or the process by which the whole VA is constructed. In such situations, adjusting the field of view may assist the user to discover how the whole structure is made of sub-patterns or smaller building blocks. Scoping can be static or dynamic. In static scoping, the available field of view is constant, and the user cannot change what is seen on the screen. In dynamic scoping, the user can dynamically decrease or increase the available field of view. For instance, fig. 6 shows dynamic scoping applied to a visualised 4D polytope structure in Polyvise. Figure 6a shows the polytope when the field of view has been decreased, and fig. 6b shows the same polytope when the field of view is increased. In both cases, the polytope structure is the same, but the user sees either a limited or expanded view of it. Scoping differs from filtering in that scoping operates on all the constituent visual components of a VA, whereas filtering is mainly for the removal of specified visual components.
2.6.
Recording
There are times, while interacting with a VA, that a user may want to add labels to elements of a VA, record traces of his/her interaction, or leave
Fig. 6. (a to b): Dynamic scoping applied to a 4D polytope structure in Polyvise.
A framework for designing interaction for visual abstractions
247
markers on it. These labels, traces, and markers can preserve the user's experience for further reflection (Pimm, 1995), or can act as external memory aids for reasoning about paths of action, relationships among visual elements, and other cognitive processes. When recording of interaction is present in a system, it can be manual or automatic. Manual recording is when the user is responsible for and is in control of when and where to add recordings to the VA. Automatic recording is when the system keeps track of the user's interactions with the VA and adds recordings to the VA. Figure 1 shows an example of manual recording where a VA representing a 3D lattice has been augmented by the user with a number of pebble-like markers.
2.7.
Scaffolding
This factor refers to the provision of cognitive support to assist the user in the process of reasoning about and understanding a VA. When scaffolding is present, it can be permanent or progressive. Figure 7b shows a VA representing an arc of rotation used in 2D transformation geometry (Sedig et al., 2001). If the user only interacted with the VA shown in this image, scaffolding is absent. On the other hand, with scaffolding (i.e. adding the extra square image in fig. 7a), user's thinking with regard to the effect of the transformation would be supported. If this scaffold stayed constant throughout the interaction process, then it would be called permanent scaffolding. On the other hand, if, as the user interacts with the VA, the representation gradually changes from fig. 7a to 7b to 7c, then it would be called progressive scaffolding.
2.8.
Content
This factor refers to the information embedded in a VA. The content of a VA can be browsable or constructible. If the content of a VA is browsable,
Fig. 7. (a) Permanent scaffolding; (b) no scaffolding; and (a to c) progressive scaffolding.
K. Sedig and J. Morey
248
the user cannot add to the content. Such a VA is primarily intended for interpretation purposes. If the content of a VA is constructible, the user puts the visual components together to form the VA. An example of a constructible VA is shown in fig. 3. In this example, the user can use the FSM diagram (fig. 3A) to connect atomic-level Ks and construct the K-Lattice structure (fig. 3B). In this particular example, the content of the FSM is browsable, and the content of the lattice is constructible. Another example of a browsable VA is the one shown in fig. 6. This VA is browsable because the user is not constructing its content. The content is already present; the user merely applies dynamic scoping to view it at different levels of complexity.
2.9.
Chunking
There are times when understanding pattems and embedded concepts within a VA requires a grouping of some of its constituent components. People generally process information better if they can group pieces of it together. This is called chunking (Ormrod, 1995). Once several pieces of information are chunked together, they are then treated as one singular entity. When present in a system, chunking can be user-controlled or system-controlled. In user-controlled chunking, the user decides what components of a VA are to be grouped together and how to group them. For instance, in K-Lattice Machine (Sedig et al., 2002), the user can group several Ks together to form a K-Lattice Pattern, the smallest number of Ks that uniquely describes a particular set of K lattices. Figure 8 shows an example of a user-controlled chunking of Ks in the K-Lattice Machine, where the Ks have been grouped in three different ways. In system-controlled chunking, the grouping of the components of the VA is determined and controlled by the system. However, the user can still interact with the chunks.
Fig. 8. Threedifferent groupings of Ks in the K-Lattice Machine.
A framework for designing interaction for visual abstractions
2.10.
249
Configuration
There are situations in which the user needs to investigate how two VAs are related and affect one another. Such an instance occurs when one wants to understand not only the structure of polyhedral solids on their own, but also how they are related and are derived from one another. This necessitates interaction with different configurations of VAs. These configurations can be represented in terms of directed graphs, shown in fig. 9. The graphs are composed of nodes, denoting VAs, and directed edges, denoting causal relationships among the VAs. There are six different configurations of VAs" singleton, directed pair, star, sink, disconnected set, and network. For instance, figs. 1 and 2 demonstrate singleton configurations, where there is a single VA present with which the user can interact. Figure 3 shows a directed pair configuration, where one VA (the FSM diagram) affects the behaviour of the second VA (the formation of the lattice). In a star configuration, there is one VA that affects all other VAs. In a sink configuration, several VAs all affect one VA. Figure 10 shows a disconnected set configuration, where the VAs are independent of one another. Finally, fig. 11 shows a network configuration, where the user can interact with six VAs, i.e. five maps and a structure (central VA in the figure). Interacting with any one of the VAs causes simultaneous changes in all the other VAs in the figure.
3.
F O R M S OF I N T E R A C T I O N
The factors described above (namely mode, flow, focus, filtering, scoping, recording, scaffolding, content, chunking, and configuration) can combine to characterise and determine the forms of interaction with VAs at a system's interface. In this section, we use the proposed framework to analyse a few existing systems. Figure 1 shows a screen capture of Lattice Space, a tool for visualising and interacting with 3D-lattice VAs (Morey et al., 2002). It has a singleton configuration. At any given time, the user can either directly manipulate the VA in a continuous manner or directly navigate through the structure
9
-.
ri
.
.
"
"-+i'-.
- - . . .
Fig. 9. (Left to Right): singleton, directed pair, star, sink, disconnected set, and network configurations.
250
K. Sedig and J. Morey
Fig. 10.
Fig. 11.
Disconnected set of VAs.
Network configuration where all VAs affect one another in Archimedean Confection.
A framework for designing interaction for visual abstractions
251
in a discrete manner. It has manual recording allowing the user to place markers on the structure. It has static scoping. This form of interaction is similar to the one in fig. 2. However, it is important to note that in this case multiple modes and flows of interaction co-exist to complement one another. Figure 2 shows a screen capture of Super Tangrams, a tool for interacting with transformational geometry VAs (Sedig and Klawe, 1996). It has a singleton configuration. At any given time, the user can directly manipulate the VA on the screen in a continuous manner. This form of interaction can be useful when the user needs to only understand one VA in isolation. Adding progressive visual scaffolding can provide quite an effective form of interaction for the user to understand singular concepts (de Souza and Sedig, 2001; Sedig et al., 2001). The process of scaffolding can occur through a gradual change in the mode of interaction. For instance, the mode of interaction can gradually change from manipulation to conversation necessitating more cognitive effort to interact with the visual representation and thereby resulting in deeper processing of the VA. Figure 3 shows a screen capture of K-Lattice Machine, a tool for visualising and interacting with 2D K-lattice VAs (Sedig et al., 2002). It has a directed pair configuration. At any given time, the user can directly navigate the FSM diagram VA in a discrete manner, which means that the user is indirectly manipulating the lattice in a discrete manner. This can be useful when the user' s understanding of one VA depends on another VA. The system supports user-controlled chunking of lattice VAs. Its content (FSM VAs and lattice VAs) is both browsable and constructible. Figure 4 shows a screen capture of a tool for interacting with 3D polyhedral VAs. It has a network configuration. At any given time, the user can either interact with the map (left) or with the polyhedron solid (right). The user can directly navigate the map in a discrete manner, clicking on the landmark polyhedral solids, which means that the user indirectly manipulates the solid in a discrete manner, as the solid on the fight changes. Alternatively, the user can manipulate the solid in a continuous manner, which means that the user indirectly navigates the map in a discrete manner. 4 This form of interaction may be useful when one VA encodes relational knowledge spatially (e.g. the map) and the other VA encodes relational knowledge temporally (e.g. the metamorphic polyhedral solid). The interactive effect of the VAs on each other can help the user discover the temporal and spatial relationships holistically. The content of this system is browsable. Figures 5 and 6 show screen captures of Polyvise, a tool for visualising and interacting with 4D polytope VAs. The figures show a singleton 4The reason that this is discrete is that only the visual landmarks on the map get highlighted.
252
K. Sedig and J. Morey
configuration, where discrete filtering and dynamic scoping are supported. The user can directly manipulate the VA in a continuous manner. The VA's content is browsable. Figure 10 shows a screen capture of tool allowing interaction with 3D polyhedral VAs. It has a disconnected set configuration. At any given time, the user can directly manipulate any one of the VAs in a continuous manner. Interaction with one VA does not affect any other VA. This form of interaction can be useful when the user may not want interaction with one VA to affect the state of other VAs but wants the other VAs to be present to be able to compare and contrast them as a collection of small multiple visuals. 5 Figure 11 shows a screen capture of Archimedean Confection, a tool for interacting with 3D polyhedral VAs. It has a network configuration. At any given time, the user can interact with any one of the maps or with the polyhedral solid at the centre of the figure. The user can directly navigate any of the maps in a continuous manner, which means that the user indirectly manipulates the solid and indirectly navigates the other maps in a continuous manner. Additionally, the user can directly manipulate the solid in a continuous manner, which means that the user indirectly navigates all five maps in a continuous manner. This configuration, although similar to that of fig. 4, affords a more systemic interaction with the VAs. This form of interaction can be useful when the user needs to understand how all parts of a constellation of abstractions affect one another.
4.
SUMMARY AND FUTURE W O R K
This chapter has proposed a descriptive framework for categorisation and charactefisation of different forms of interaction with VAs. The goal of the framework is to provide a preliminary conceptual anchor and a systematic typology to guide the designers and evaluators of visually based tools to determine the appropriate forms of interaction that would facilitate exploration of abstract concepts, pattems, structures and processes. We have discussed 10 factors that contribute to the forms of interaction with VAs: mode, flow, focus, filtering, scoping, recording, scaffolding, content, chunking, and configuration. Each factor in tum is broken down into further categories. There are three modes of interaction (manipulation, navigation, and conversation), two types of interaction flow (discrete and continuous), two types of interaction focus (direct and indirect), two types of filtering (discrete and range-based), two types of scoping (static and dynamic), 5 See Tufte (1998) for a discussion of "Small Multiples" and their role in comparative reasoning.
A framework for designing interaction for visual abstractions
253
two types of recording (manual and automatic), two types of scaffolding ( p e r m a n e n t and p r o g r e s s i v e ) , two types of content ( b r o w s a b l e and constructible), two types of chunking (user-controlled and systemcontrolled), and six different configurations (singleton, directed pair, star, sink, disconnected set, and network). Depending on the needs of the user, multiple categories can be combined to facilitate the exploration process. The proposed framework is a step in the direction of formulating a typology of interaction with VAs. Further research is needed to investigate the effectiveness of the different forms of interaction in different contexts. Additionally, the current proposed framework is general and descriptive in nature. Much empirical research is needed for a detailed prescriptive model of design to emerge.
REFERENCES de Souza, C.S., Sedig, K., 2001. Semiotic considerations on direct concept manipulation as a distinct interface style for learnware, 2001, Proceedings of the Brazilian HumanComputer Interaction Conference (IHC2001), Florianopolis, Santa Catarina, 15-17 October, pp. 229-241. Glasgow, J., Narayanan, N.H., Chandrasekaran, A.B. (Eds.), 1995. Diagrammatic Reasoning: Cognitive and Computational Perspectives. The MIT Press, Cambridge, MA. Golightly, D., 1996. Harnessing the Interface for Domain Learning, Proceedings of CHI' 96. ACM Press, New York, pp. 37-38. Hoist, S.J., 1996. Directing learner attention with manipulation styles, Proceedings of CHI' 96. ACM Press, New York, pp. 43-44. Hutchins, E.L., Hollan, J.D., Norman, D.A., 1986. Direct manipulation interfaces. In: Norman, D.A., Draper, S.W. (Eds.), User Centered System Design: New Perspectives in Human-Computer Interaction. Lawrence Erlbaum, Hillsdale, NJ, pp. 87-124. Jonassen, D.H., Beissner, K., Yacci, M. (Eds.), 1993. Structural Knowledge: Techniques for Representing, Conveying, and Acquiring Structural Knowledge. Lawrence Erlbaum, Hillsdale, NJ. Lajoie, S. (Ed.), 2000. Computers as Cognitive Tools. Lawrence Erlbaum, Hillsdale, NJ. Morey, J., Sedig, K., Mercer, R., 2001. Interactive metamorphic visuals: exploring polyhedra relationships, The 5th International Conference on Information Visualization. IEEE Computer Society, London, UK, pp. 483-488. Morey, J., Sedig, K., Mercer, R., Wilson, W., 2001. Crystal lattice automata, Proceedings of the Sixth International Conference on Implementations and Applications of Automata (Pretoria, South Africa, July 2001), Lecture Notes in Computer Science. Springer, Berlin. Nardi, B.A., Zarmer, C.L., 1993. Beyond models and metaphors: visual formalisms in user interface design. J. Vis. Lang. Comput. 4, 5-33. Norman, D.A., 1993. Things That Make Us Smart: Defining Human Attributes in the Age of the Machine. Addison-Wesley Publishing, Reading, MA. Ormrod, J.E., 1995. Human Learning. Prentice-Hall, Englewood Cliffs, NJ. Peterson, D. (Ed.), 1996. Forms of Representation. Intellect Books, Exeter, UK. Pimm, D., 1995. Symbols and Meanings in School Mathematics. Routledge, London. Sedig (or Sedighian), K., Klawe, M.M., 1996. Super Tangrams: A child-centered approach to designing a computer supported mathematics learning environment, Proceedings of the
254
K. Sedig and J. Morey
International Conference on the Learning Sciences. Association for the Advancement of Computing in Education, Evanston, IL, pp. 490-495. Sedig, K., Klawe, M., Westrom, M., 2001. Role of interface manipulation style and scaffolding on cognition and concept learning in learnware. ACM Trans. Comput.-Hum. Interact. 1, 8, 34-59. Sedig, K., Morey, J., Mercer, R., Wilson, W., 2002. Visualizing, interacting and experimenting with lattices using a diagrammatic representation, The Current Proceedings: 2nd International Conference on Visual Representations and Interpretations, VRI'2002. Shneiderman, B., 1991. A taxonomy and rule base for the selection of interaction styles. In: Shackle, B., Richardson, S.J. (Eds.), Human Factors for Informatics Usability. Cambridge University Press, Cambridge, pp. 325-342. Svendsen, G.B., 1991. Influences of interface style on problem solving. Int. J. Man-Machine Stud. 35, 379-397. Tufte, E.R., 1998. Envisioning Information. Graphics Press, Cheshire, CT. Zhang, J., Norman, D., 1994. Representations in distributed cognitive tasks. Cognition Science. 18, 87-122.
17
A descriptive framework for designing interaction for visual abstractions K. Sedig a and J. Morey b
aCognitive Engineering Laboratory, Department of Computer Science and Faculty of Information and Media Studies, The University of Western Ontario, London, Ont., Canada bCognitive Engineering Laboratory, Department of Computer Science, The University of Western Ontario, London, Ont., Canada
This chapter propses a descriptive framework for categorisation and characterisation of the different forms of interaction with visual abstractions (VAs). Abstract visual representations play an important role in assisting human reasoning, thinking, and understanding processes. There are different forms of designing interaction with these representations. The goal of this chapter is to provide a descriptive framework to guide the designers and evaluators of cognitive tools to determine the appropriate forms of interaction that can facilitate the understanding of abstract concepts, patterns, structures and processes. The framework is described and substantiated using a number of VAs that represent and communicate mathematical ideas. 1.
INTRODUCTION AND BACKGROUND
Many concepts, patterns, structures, and processes are too complex to understand without the aid of external cognitive aids (Norman, 1993). Visual abstract representations can assist human reasoning and learning l This research is funded by the Natural Sciences and Engineering Research Council of Canada.
239
240
K. Sedig and J. Morey
(Jonassen et al., 1993; Glasgow et al., 1995; Peterson, 1996). The human visual system has limited channel capacity. Visuals provide high-bandwidth interaction with the mind. VAs can be defined as a set of interconnected symbols that can embody causal, functional, structural, and semantic relations and properties. Examples of VAs include visual mathematical representations, diagrams, maps, graphs, networks, and so on. Explicit, external VAs extend human memory by acting as "knowledge in the world", can stimulate cognitive activity, amplify human cognition, and assist perceptual interpretation (Nardi and Zarmer, 1993; Zhang and Norman, 1994). VAs may be primary (derived from real-world objects) or secondary (derived from representations such as patterns in raw data, textual information, or scientific and mathematical concepts). Much of the knowledge embodied in secondary VAs may not be at the surface level and readily available for reasoning or perceptible to the human mind. Allowing users to interact with VAs as cognitive tools 2 can enhance this process of reasoning, interpretation, and sense making. However, the form and style of interaction plays a crucial role in how well and how much knowledge learners can construct (de Souza and Sedig, 2001; Sedig et al., 2001). Most cognitive tools borrow interaction techniques devised for and used in productivity tools (Sedig et al., 2001). The appropriateness of some interaction techniques for problem solving and learning activities has been questioned (Golightly, 1996; Holst, 1996; Sedig et al., 2001). However, there is no clear understanding of what form of interaction cognitive tools should incorporate, de Souza and Sedig (2001) have suggested that when designing concept-centred interfaces, the availability of a general framework to guide choices among the visual representations is lacking. Additionally, there is a need for a framework to guide choices among forms of interaction with these visuals. Although Shneiderman (1991) has proposed a general taxonomy of interaction styles, this taxonomy is too broad and does not seem suitable for cognitive tools. This chapter is a step in creating a framework to categorise and characterise different forms of interaction with VAs. Existence of such a framework can provide designers of interactive cognitive tools with options as how to systematically think about design of interaction for VAs. In the following sections, we use several systems to develop our framework. All these systems have been developed by our research group and use mathematical concepts as a test-bed to assist us in thinking about the proposed framework. These systems include: Super Tangrams (Sedig and Klawe, 1996), a tool to help children learn 2D geometrical transformations (i.e. translation, rotation, and reflection); Archimedean 2Cognitive tools refer to computational tools intended to support and extend human mental activities while engaged in perceptual, reasoning, and problem solving processes (Lajoie, 2000).
A framework for designing interaction for visual abstractions
241
Kaleidoscope (Morey et al., 2001), a tool to help users visualise and explore polyhedral 3 solids; K-Lattice Machine (Sedig et al., 2002), a tool to help users explore sub-patterns in 2D regular lattice structures; Archimedean Confection, a tool to explore relationships among polyhedral solids; Lattice Space, a tool to visualise and explore 3D lattice structures; and Polyvise, an interactive tool to visualise and explore 4D Archimedean polytope structures.
2.
INTERACTION FACTORS
This chapter proposes that the form of interaction with VAs is determined by a set of factors. In this section, 10 factors are discussed: mode, flow, focus, filtering, scoping, recording, scaffolding, content, chunking, and configuration.
2.1.
Mode
The mode of interaction refers to the metaphoric bodily organ by which the user interacts with a VA. There are three basic bodily metaphors by which humans interact with entities in their surroundings: hands (handling entities), feet (walking on or through entities), and mouth (conversing with entities). Therefore, there are three modes in which a user can interact with a VA: manipulation, navigation, and conversation. Instances of these modes can be illustrated through an example. Figure 1 shows a VA representing a 3D lattice structure. As manipulation, the user can rotate it and view the whole structure from different angles; as navigation, the user can walk through or on it; and as conversation, the user can type a command to query the lattice about one of its properties or to transform it in some way. This can be done using natural language queries, speech, menus, form-fill-ins, or any type of linguistic command.
2.2.
Flow
The flow of interaction refers to the effect of the interaction on how the user perceives the relationship between cause and effect in the time-space continuum. Flow of interaction can be continuous or discrete. In continuous interaction, the user observes cause and effect simultaneously. When there is continuous flow to the interaction, a VA fluidly responds to the user's 3A polyhedron is a geometric solid bounded by polygons.
K. Sedig and J. Morey
242
Fig. 1. 3D lattice in Lattice Space. interaction with it. For instance, fig. 2 shows a VA representing the mathematical concept of 2D translation as an interactive vector. The user can click on one of the tips of the vector to change its size and direction. The user' s interaction with this VA is continuous because the movement of the mouse cursor is fluidly translated into a change in the size and direction of the vector. In discrete interaction, cause and effect are separated in time. That is, the interaction takes place in a modal fashion. For instance, fig. 3a shows a VA representing a state-transition diagram. In order to cause a transition from one state to another, the user clicks on the end point of one of the transitional links, and the state transition takes place. Although the user may see the effect of the click without any time delay, nonetheless this is a discrete interaction since the user's interaction with the VA takes place in temporal snapshots.
2.3.
Focus
The focus of interaction refers to the centre of attention of the user while interacting with an environment. There are two fundamental ways of interacting with VAs: direct and indirect.
A framework for designing interaction for visual abstractions
.
:
.
-
!
.
; .
! .
243
f .
i
i
L .....
~ .....
~ 7
~" ' ; . . . . . . .
i N-
-
..
.
.
.
.
.
.
i
.
.
,
. . . .
<
,
-:
:
-,
;
i Fig. 2.
Translation vector in Super Tangrams.
One of the popular interaction styles is direct manipulation (Hutchins et al., 1986). Direct manipulation refers to interfaces, which allow users to see graphical representations of objects and directly manipulate them on the computer screen with some kind of pointing device. In this type of interaction, the user is focused on the object of interaction. This style is widely used in productivity as well as cognitive tools. It has been suggested that direct manipulation is potentially inappropriate for activities that require
mental effort and understanding (such as problem solving), and that indirect manipulation is more appropriate (Svendsen, 1991; Golightly, 1996; Hoist, 1996). The "directness" of these interfaces has been cited as the main source
Fig. 3.
State-transition diagram (L) in K-Lattice Machine.
244
K. Sedig and J. Morey
of the problem since it makes them easy to use, not requiting much cognitive effort on the part of the user. Sedig et al. (2001) have argued that the problem with direct manipulation lies in "what is manipulated" rather than the "manipulation" itself. This has been demonstrated through empirical studies with children learning 2D transformational geometry concepts. It has been proposed that the focus of attention and interaction should be the structures and processes being investigated (Hoist, 1996; Sedig et al., 2001). Thus both direct and indirect interactions can serve different purposes. In direct interaction, the user is directly focused on a VA and interacts with it without any other intermediary representations. The VA can represent a concept, a structure, a map, or some other abstract entity. For instance, if the goal of the user is to explore the concept of translation, interaction should be directed towards its visual representation (see fig. 1; also see Sedig et al., 2001). The same technique can hold if the user wants to discover the structure of polyhedra solids (Morey et al., 2001). The VA on the fight-hand side of fig. 4 shows a polyhedron solid. The user can directly interact with it by manipulating a control point on the structure and observing how it metamorphoses from one geometric solid to another. Depending on the type of VA, this type of interaction can be called direct concept interaction (de Souza and Sedig, 2001; Sedig et al., 2001), direct structure interaction, direct entity interaction, and so on. In indirect interaction, there are at least two VAs involved. The user interacts with one VA through another VA (or other VAs). This may be the case when interaction with one VA makes it easier to understand and decode the features of another VA. For instance, lattices have local structures that infinitely repeat themselves. Morey et al. (2001) have suggested that it is
Fig. 4. Polyhedralsolids.
A framework for designing interaction for visual abstractions
245
easier to understand and reason about some of the mathematical properties of 2D regular lattice structures if they are represented using finite state machine (FSM) diagrams. The FSM diagram captures the local geometry of the lattice in a concise visual representation. Focusing the interaction on the FSM diagram helps the user discover the repetitive patterns inherent in the lattice structure. Figure 3 shows a screen capture of K-Lattice Machine where the user interacts with an FSM diagram (fig. 3a) to construct a K lattice (fig. 3b). In this example, the user interacts directly with the FSM diagram and indirectly with the lattice abstraction. As such, indirect interaction necessitates direct interaction with one VA.
2.4.
Filtering
VAs may be aggregates of several features, components, or layers of detail. This can render a VA noisy and difficult to understand. In such cases, the user may want to remove some of the features or components of the visual structure in order to make the VA more understandable. The user can choose to view a subset of the original visual structure by specifying which features and components should remain visible or hidden. This method of removing visual noise can be called filtering. When filtering is used, it can be discrete or range-based. In discrete filtering, the user can use toggles to specify which features of a VA are to be present. For instance, the VA in fig. 5a represents a 4D polytope structure derived from a hypercube, as visualised in Polyvise. This structure contains several sets of components, such as vertices, edges, polygons, and cells. The VA in fig. 5b shows the same polytope structure in which the set of polygonal faces has been removed using discrete filtering. In range-based filtering, rather than using binary toggles, the user specifies a range of values for the visual components. The visibility of components
Fig. 5. (a to b): Discrete filtering applied to a 4D polytope structure in Polyvise.
K. Sedig and J. Morey
246
depends on whether or not their values fall within or outside of this range. Finally, in both discrete and range-based filtering, logical operators can be used to combine various filters.
2.5.
Scoping
Scoping refers to the available field of view of a VA. There may be instances where a VA is too complex and the user may have difficulty understanding the components or the process by which the whole VA is constructed. In such situations, adjusting the field of view may assist the user to discover how the whole structure is made of sub-patterns or smaller building blocks. Scoping can be static or dynamic. In static scoping, the available field of view is constant, and the user cannot change what is seen on the screen. In dynamic scoping, the user can dynamically decrease or increase the available field of view. For instance, fig. 6 shows dynamic scoping applied to a visualised 4D polytope structure in Polyvise. Figure 6a shows the polytope when the field of view has been decreased, and fig. 6b shows the same polytope when the field of view is increased. In both cases, the polytope structure is the same, but the user sees either a limited or expanded view of it. Scoping differs from filtering in that scoping operates on all the constituent visual components of a VA, whereas filtering is mainly for the removal of specified visual components.
2.6.
Recording
There are times, while interacting with a VA, that a user may want to add labels to elements of a VA, record traces of his/her interaction, or leave
Fig. 6. (a to b): Dynamic scoping applied to a 4D polytope structure in Polyvise.
A framework for designing interaction for visual abstractions
247
markers on it. These labels, traces, and markers can preserve the user's experience for further reflection (Pimm, 1995), or can act as external memory aids for reasoning about paths of action, relationships among visual elements, and other cognitive processes. When recording of interaction is present in a system, it can be manual or automatic. Manual recording is when the user is responsible for and is in control of when and where to add recordings to the VA. Automatic recording is when the system keeps track of the user's interactions with the VA and adds recordings to the VA. Figure 1 shows an example of manual recording where a VA representing a 3D lattice has been augmented by the user with a number of pebble-like markers.
2.7.
Scaffolding
This factor refers to the provision of cognitive support to assist the user in the process of reasoning about and understanding a VA. When scaffolding is present, it can be permanent or progressive. Figure 7b shows a VA representing an arc of rotation used in 2D transformation geometry (Sedig et al., 2001). If the user only interacted with the VA shown in this image, scaffolding is absent. On the other hand, with scaffolding (i.e. adding the extra square image in fig. 7a), user's thinking with regard to the effect of the transformation would be supported. If this scaffold stayed constant throughout the interaction process, then it would be called permanent scaffolding. On the other hand, if, as the user interacts with the VA, the representation gradually changes from fig. 7a to 7b to 7c, then it would be called progressive scaffolding.
2.8.
Content
This factor refers to the information embedded in a VA. The content of a VA can be browsable or constructible. If the content of a VA is browsable,
Fig. 7. (a) Permanent scaffolding; (b) no scaffolding; and (a to c) progressive scaffolding.
K. Sedig and J. Morey
248
the user cannot add to the content. Such a VA is primarily intended for interpretation purposes. If the content of a VA is constructible, the user puts the visual components together to form the VA. An example of a constructible VA is shown in fig. 3. In this example, the user can use the FSM diagram (fig. 3A) to connect atomic-level Ks and construct the K-Lattice structure (fig. 3B). In this particular example, the content of the FSM is browsable, and the content of the lattice is constructible. Another example of a browsable VA is the one shown in fig. 6. This VA is browsable because the user is not constructing its content. The content is already present; the user merely applies dynamic scoping to view it at different levels of complexity.
2.9.
Chunking
There are times when understanding pattems and embedded concepts within a VA requires a grouping of some of its constituent components. People generally process information better if they can group pieces of it together. This is called chunking (Ormrod, 1995). Once several pieces of information are chunked together, they are then treated as one singular entity. When present in a system, chunking can be user-controlled or system-controlled. In user-controlled chunking, the user decides what components of a VA are to be grouped together and how to group them. For instance, in K-Lattice Machine (Sedig et al., 2002), the user can group several Ks together to form a K-Lattice Pattern, the smallest number of Ks that uniquely describes a particular set of K lattices. Figure 8 shows an example of a user-controlled chunking of Ks in the K-Lattice Machine, where the Ks have been grouped in three different ways. In system-controlled chunking, the grouping of the components of the VA is determined and controlled by the system. However, the user can still interact with the chunks.
Fig. 8. Threedifferent groupings of Ks in the K-Lattice Machine.
A framework for designing interaction for visual abstractions
2.10.
249
Configuration
There are situations in which the user needs to investigate how two VAs are related and affect one another. Such an instance occurs when one wants to understand not only the structure of polyhedral solids on their own, but also how they are related and are derived from one another. This necessitates interaction with different configurations of VAs. These configurations can be represented in terms of directed graphs, shown in fig. 9. The graphs are composed of nodes, denoting VAs, and directed edges, denoting causal relationships among the VAs. There are six different configurations of VAs" singleton, directed pair, star, sink, disconnected set, and network. For instance, figs. 1 and 2 demonstrate singleton configurations, where there is a single VA present with which the user can interact. Figure 3 shows a directed pair configuration, where one VA (the FSM diagram) affects the behaviour of the second VA (the formation of the lattice). In a star configuration, there is one VA that affects all other VAs. In a sink configuration, several VAs all affect one VA. Figure 10 shows a disconnected set configuration, where the VAs are independent of one another. Finally, fig. 11 shows a network configuration, where the user can interact with six VAs, i.e. five maps and a structure (central VA in the figure). Interacting with any one of the VAs causes simultaneous changes in all the other VAs in the figure.
3.
F O R M S OF I N T E R A C T I O N
The factors described above (namely mode, flow, focus, filtering, scoping, recording, scaffolding, content, chunking, and configuration) can combine to characterise and determine the forms of interaction with VAs at a system's interface. In this section, we use the proposed framework to analyse a few existing systems. Figure 1 shows a screen capture of Lattice Space, a tool for visualising and interacting with 3D-lattice VAs (Morey et al., 2002). It has a singleton configuration. At any given time, the user can either directly manipulate the VA in a continuous manner or directly navigate through the structure
9
-.
ri
.
.
"
"-+i'-.
- - . . .
Fig. 9. (Left to Right): singleton, directed pair, star, sink, disconnected set, and network configurations.
250
K. Sedig and J. Morey
Fig. 10.
Fig. 11.
Disconnected set of VAs.
Network configuration where all VAs affect one another in Archimedean Confection.
A framework for designing interaction for visual abstractions
251
in a discrete manner. It has manual recording allowing the user to place markers on the structure. It has static scoping. This form of interaction is similar to the one in fig. 2. However, it is important to note that in this case multiple modes and flows of interaction co-exist to complement one another. Figure 2 shows a screen capture of Super Tangrams, a tool for interacting with transformational geometry VAs (Sedig and Klawe, 1996). It has a singleton configuration. At any given time, the user can directly manipulate the VA on the screen in a continuous manner. This form of interaction can be useful when the user needs to only understand one VA in isolation. Adding progressive visual scaffolding can provide quite an effective form of interaction for the user to understand singular concepts (de Souza and Sedig, 2001; Sedig et al., 2001). The process of scaffolding can occur through a gradual change in the mode of interaction. For instance, the mode of interaction can gradually change from manipulation to conversation necessitating more cognitive effort to interact with the visual representation and thereby resulting in deeper processing of the VA. Figure 3 shows a screen capture of K-Lattice Machine, a tool for visualising and interacting with 2D K-lattice VAs (Sedig et al., 2002). It has a directed pair configuration. At any given time, the user can directly navigate the FSM diagram VA in a discrete manner, which means that the user is indirectly manipulating the lattice in a discrete manner. This can be useful when the user' s understanding of one VA depends on another VA. The system supports user-controlled chunking of lattice VAs. Its content (FSM VAs and lattice VAs) is both browsable and constructible. Figure 4 shows a screen capture of a tool for interacting with 3D polyhedral VAs. It has a network configuration. At any given time, the user can either interact with the map (left) or with the polyhedron solid (right). The user can directly navigate the map in a discrete manner, clicking on the landmark polyhedral solids, which means that the user indirectly manipulates the solid in a discrete manner, as the solid on the fight changes. Alternatively, the user can manipulate the solid in a continuous manner, which means that the user indirectly navigates the map in a discrete manner. 4 This form of interaction may be useful when one VA encodes relational knowledge spatially (e.g. the map) and the other VA encodes relational knowledge temporally (e.g. the metamorphic polyhedral solid). The interactive effect of the VAs on each other can help the user discover the temporal and spatial relationships holistically. The content of this system is browsable. Figures 5 and 6 show screen captures of Polyvise, a tool for visualising and interacting with 4D polytope VAs. The figures show a singleton 4The reason that this is discrete is that only the visual landmarks on the map get highlighted.
252
K. Sedig and J. Morey
configuration, where discrete filtering and dynamic scoping are supported. The user can directly manipulate the VA in a continuous manner. The VA's content is browsable. Figure 10 shows a screen capture of tool allowing interaction with 3D polyhedral VAs. It has a disconnected set configuration. At any given time, the user can directly manipulate any one of the VAs in a continuous manner. Interaction with one VA does not affect any other VA. This form of interaction can be useful when the user may not want interaction with one VA to affect the state of other VAs but wants the other VAs to be present to be able to compare and contrast them as a collection of small multiple visuals. 5 Figure 11 shows a screen capture of Archimedean Confection, a tool for interacting with 3D polyhedral VAs. It has a network configuration. At any given time, the user can interact with any one of the maps or with the polyhedral solid at the centre of the figure. The user can directly navigate any of the maps in a continuous manner, which means that the user indirectly manipulates the solid and indirectly navigates the other maps in a continuous manner. Additionally, the user can directly manipulate the solid in a continuous manner, which means that the user indirectly navigates all five maps in a continuous manner. This configuration, although similar to that of fig. 4, affords a more systemic interaction with the VAs. This form of interaction can be useful when the user needs to understand how all parts of a constellation of abstractions affect one another.
4.
SUMMARY AND FUTURE W O R K
This chapter has proposed a descriptive framework for categorisation and charactefisation of different forms of interaction with VAs. The goal of the framework is to provide a preliminary conceptual anchor and a systematic typology to guide the designers and evaluators of visually based tools to determine the appropriate forms of interaction that would facilitate exploration of abstract concepts, pattems, structures and processes. We have discussed 10 factors that contribute to the forms of interaction with VAs: mode, flow, focus, filtering, scoping, recording, scaffolding, content, chunking, and configuration. Each factor in tum is broken down into further categories. There are three modes of interaction (manipulation, navigation, and conversation), two types of interaction flow (discrete and continuous), two types of interaction focus (direct and indirect), two types of filtering (discrete and range-based), two types of scoping (static and dynamic), 5 See Tufte (1998) for a discussion of "Small Multiples" and their role in comparative reasoning.
A framework for designing interaction for visual abstractions
253
two types of recording (manual and automatic), two types of scaffolding ( p e r m a n e n t and p r o g r e s s i v e ) , two types of content ( b r o w s a b l e and constructible), two types of chunking (user-controlled and systemcontrolled), and six different configurations (singleton, directed pair, star, sink, disconnected set, and network). Depending on the needs of the user, multiple categories can be combined to facilitate the exploration process. The proposed framework is a step in the direction of formulating a typology of interaction with VAs. Further research is needed to investigate the effectiveness of the different forms of interaction in different contexts. Additionally, the current proposed framework is general and descriptive in nature. Much empirical research is needed for a detailed prescriptive model of design to emerge.
REFERENCES de Souza, C.S., Sedig, K., 2001. Semiotic considerations on direct concept manipulation as a distinct interface style for learnware, 2001, Proceedings of the Brazilian HumanComputer Interaction Conference (IHC2001), Florianopolis, Santa Catarina, 15-17 October, pp. 229-241. Glasgow, J., Narayanan, N.H., Chandrasekaran, A.B. (Eds.), 1995. Diagrammatic Reasoning: Cognitive and Computational Perspectives. The MIT Press, Cambridge, MA. Golightly, D., 1996. Harnessing the Interface for Domain Learning, Proceedings of CHI' 96. ACM Press, New York, pp. 37-38. Hoist, S.J., 1996. Directing learner attention with manipulation styles, Proceedings of CHI' 96. ACM Press, New York, pp. 43-44. Hutchins, E.L., Hollan, J.D., Norman, D.A., 1986. Direct manipulation interfaces. In: Norman, D.A., Draper, S.W. (Eds.), User Centered System Design: New Perspectives in Human-Computer Interaction. Lawrence Erlbaum, Hillsdale, NJ, pp. 87-124. Jonassen, D.H., Beissner, K., Yacci, M. (Eds.), 1993. Structural Knowledge: Techniques for Representing, Conveying, and Acquiring Structural Knowledge. Lawrence Erlbaum, Hillsdale, NJ. Lajoie, S. (Ed.), 2000. Computers as Cognitive Tools. Lawrence Erlbaum, Hillsdale, NJ. Morey, J., Sedig, K., Mercer, R., 2001. Interactive metamorphic visuals: exploring polyhedra relationships, The 5th International Conference on Information Visualization. IEEE Computer Society, London, UK, pp. 483-488. Morey, J., Sedig, K., Mercer, R., Wilson, W., 2001. Crystal lattice automata, Proceedings of the Sixth International Conference on Implementations and Applications of Automata (Pretoria, South Africa, July 2001), Lecture Notes in Computer Science. Springer, Berlin. Nardi, B.A., Zarmer, C.L., 1993. Beyond models and metaphors: visual formalisms in user interface design. J. Vis. Lang. Comput. 4, 5-33. Norman, D.A., 1993. Things That Make Us Smart: Defining Human Attributes in the Age of the Machine. Addison-Wesley Publishing, Reading, MA. Ormrod, J.E., 1995. Human Learning. Prentice-Hall, Englewood Cliffs, NJ. Peterson, D. (Ed.), 1996. Forms of Representation. Intellect Books, Exeter, UK. Pimm, D., 1995. Symbols and Meanings in School Mathematics. Routledge, London. Sedig (or Sedighian), K., Klawe, M.M., 1996. Super Tangrams: A child-centered approach to designing a computer supported mathematics learning environment, Proceedings of the
254
K. Sedig and J. Morey
International Conference on the Learning Sciences. Association for the Advancement of Computing in Education, Evanston, IL, pp. 490-495. Sedig, K., Klawe, M., Westrom, M., 2001. Role of interface manipulation style and scaffolding on cognition and concept learning in learnware. ACM Trans. Comput.-Hum. Interact. 1, 8, 34-59. Sedig, K., Morey, J., Mercer, R., Wilson, W., 2002. Visualizing, interacting and experimenting with lattices using a diagrammatic representation, The Current Proceedings: 2nd International Conference on Visual Representations and Interpretations, VRI'2002. Shneiderman, B., 1991. A taxonomy and rule base for the selection of interaction styles. In: Shackle, B., Richardson, S.J. (Eds.), Human Factors for Informatics Usability. Cambridge University Press, Cambridge, pp. 325-342. Svendsen, G.B., 1991. Influences of interface style on problem solving. Int. J. Man-Machine Stud. 35, 379-397. Tufte, E.R., 1998. Envisioning Information. Graphics Press, Cheshire, CT. Zhang, J., Norman, D., 1994. Representations in distributed cognitive tasks. Cognition Science. 18, 87-122.