- Email: [email protected]
A methodology and tools for applying context-specific usability guidelines to interface design
INTCOM 1148
Interacting with Computers 12 (2000) 225–243 www.elsevier.com/locate/intcom
A methodology and tools for applying contextspecific usability guidelines to interface design S. Henninger* Department of Computer Science and Engineering, University of Nebraska, Lincoln, NE, USA
Abstract This paper presents a methodology and an associated technology to create context-specific usability guidelines. The objective is to transform usability guidelines into a proactive resource that software developers can employ early and often in the development process. The methodology ensures conformance with established guidelines, but has the flexibility to use design experiences to adapt the guidelines to meet the emergent and diverse requirements of modern user interface design. Case-based and organizational learning technology is used to support the methodology and provide valuable resources for software developers. q 2000 Elsevier Science B.V. All rights reserved. Keywords: Guidelines; Human–computer interaction; Software development methodologies; Software engineering; Standards; Tools for working with guidelines; Usability
1. Introduction Usability guidelines have become a widely accepted method of bringing the cumulative knowledge of usability issues to bear on the software development process. While it is generally accepted that guidelines cannot replace the “golden rules” of interface design— user involvement, user feedback from early prototypes and iterative development—guidelines can play a role in improving the quality of the iterative steps, leading to an improvement in quality and reduction (but not elimination) of the number of iterations involved in the design-evaluate-redesign cycle of HCI development [1]. Guidelines usually take one of the two forms. Style guides address when and how to use interface widgets and general look-and-feel issues, often focusing on a specific platform such as Macintosh [2] or Windows [3]. Human–computer interface guidelines provide a platform-independent and comprehensive set of guidelines on various aspects of human– computer interfaces, including user-centered design techniques [4–6]. * Tel: 1 1-402-472-8394. E-mail address: [email protected] (S. Henninger) 0953-5438/00/$ - see front matter q 2000 Elsevier Science B.V. All rights reserved. PII: S0953-543 8(99)00013-2
226
S. Henninger / Interacting with Computers 12 (2000) 225–243
Fig. 1. An interface lacking in usability principles.
Many development organizations have sensed a need to extend style and general usability guidelines and have begun to create customized usability guidelines and standards [7,8]. But even with these customized approaches, there are a number of concerns about the nature of guidelines that can inhibit the impact of guidelines on the design of user interfaces [9,10]. Apart from not adequately addressing concerns facing software developers, such as which guidelines should be used under what circumstances [11], studies have shown that interface guidelines suffer from being too abstract to directly apply [11– 13]. In addition, usability guidelines have yet to become an integral part of the development process, usually relegated to a discretionary reference role or an after-the-fact human factors certification process. Approaches to improving the use of usability guidelines has mostly focused on access technology, such as hypertext [14–16] and database retrieval [17]. Web-based guidelines are becoming an increasingly popular way to disseminate usability guidelines [18–20]. But none of these technologies address the real problems of usability guidelines. Support is needed for defining when guidelines are applicable or how guidelines can be refined to meet the user task requirements for a specific set of users and a specific type of application. Little research has been done to integrate guideline use in the development process and accumulate knowledge about interface design in a form that can capture relationships between specific contexts and applicable guidelines. This paper presents an approach that addresses these issues through a combination of technology and process. The result extends the traditional notion of usability guidelines to provide detailed guidelines that are integrated into the development process in a
S. Henninger / Interacting with Computers 12 (2000) 225–243
227
conformance process. Technology is used to match design context to guidelines through a rule-based system. Guidelines are attached to cases containing information about how previous projects achieved conformance to the guidelines. This provides developers with an immediate benefit by allowing them to draw on past experiences. A deviation process is used to capture new technology and business requirements and set precedents that subsequent design efforts can follow.
2. Usability guidelines in practice While corporate usability guidelines are starting to appear [7,8,21], these guidelines are often used as part of a certification process that occurs late in the development process. Fig. 1 shows an interface developed by an IT organization with a small human factors staff and a well-documented set of usability guidelines that was available on-line to all developers. This window, which was built as an initial window for a contract support application, consists of 34 buttons with no indication of importance or ordering, groupings with little semantic significance, and button names that have marginal descriptive value. Another application from this organization currently has 120 screens and 22 subfunctions, clearly exceeding the human capacity to understand and easily learn the application. An after-the-fact human factors certification process is clearly not enough to solve the problems faced by these two examples, which are in dire need of task analyses and comprehensive re-design. Both of these applications have been shipped as-is, not because human factors personnel did not identify the problems, but because their involvement occurred too late in the development process. One solution is certainly to get the human factors people involved earlier in the design process, but this is not always feasible in the face of shortened development lifecycles and shortages in human factors specialists.
2.1. Interpreting design guidelines Placing guidelines and usability techniques early in the development process addresses the need for a proactive role for interface design [1]. Yet, it does little to resolve the problems that have been demonstrated in empirical studies of developers using guidelines, such as the difficulty of understanding and applying guidelines [12] and the difficulty of determining when a guideline has been violated [13]. These and other problems can be seen to stem from the abstract and decontextualized nature of current guideline techniques, which tend to make guidelines ambiguous or too abstract to be applied to a specific context. For example, a published guideline states that “All possible actions should be visible always” [22]. Depending on the context, this can be seen as good advice, but could also be argued that the designers of the window in Fig. 1 followed this guideline to its ultimate conclusion! More information is needed that states the circumstances under which the guideline should be followed and what to do when guidelines conflict or do not fully address the problems under consideration.
228
S. Henninger / Interacting with Computers 12 (2000) 225–243
2.2. Developing context-specific usability guidelines The decontextualized nature of current guidelines impose a significant barrier to usability guidelines becoming an integral part of the development process. For example, a guideline stating to “Always keep users informed of system states” is sound advice that has roots to usability evaluations on system modalities. However, as stated, the guideline is abstract and open to a wide variety of interpretations in different contexts. A more concrete guideline, such as “Visually identifying the need to save changes with a user action” at once conveys a specific context; that of identifying when screen information has changed and needs saving, and using visual cues to alert users to the event. Even better would be to tailor the guideline to specific design settings such as “Using color background as a cue for edit vs. read-only modes” or “Identifying changed documents in a word processor”. To the extent that the development context has been captured, the more specific the guideline, the better the support for the developer. Note also that our example defines an increasingly concrete hierarchy. If the guideline at the most concrete level does not support the design context, the developer can move up a level and see what help a more abstract guideline provides. Thus, levels of support are provided at detailed levels, but the knowledge gracefully degrades to more abstract levels as one moves up the hierarchy [23]. These contextualized guidelines are better than abstract or decontextualized ones may be clear. Less clear is how guidelines are created to meet diverse application needs, and how one applies the guidelines to a specific context. Creating and using contextualized guidelines presents a different set of problems that is exacerbated by the growing diversity of user populations and application domains to which computers are being applied. Given these kinds of diverse cross-sections, it is generally infeasible to create a “global” set of contextualized usability guidelines. Such an effort would fail for the combinatorial size of the repository and the inherent difficulty of finding the right guidelines for a given design problem and context. Techniques that provide high levels of domain-specific support [24] need to be utilized to create contextualized guidelines for issues that have strategic importance to an organization [25].
3. An organizational memory approach to developing design guidelines The discussion of guidelines has shown that it is often difficult to understand how specific guidelines apply to a designer’s current situation, or which guidelines are most appropriate. Because designers tend to “ask questions about specific problems they have with their own design rather than abstract ones” [22], there is ample evidence that designers in many domains actively seek concrete examples of existing designs to guide their work [26,27]. Thus, an effective strategy for developing a user interface is to find an existing interface that has proven effective for users with similar backgrounds and needs and adapt the ideas and methods into the new interface.
S. Henninger / Interacting with Computers 12 (2000) 225–243
229
3.1. Organizational memory and usability guidelines An organizational memory approach to design guidelines builds on this method by collecting, disseminating and improving the accumulated wisdom of developers in a development organization [25,28]. The general notion is that an organization begins with a core set of general design guidelines, combined with specific widget guidelines for the application platforms used in the organization. As interfaces are developed, developers record which guidelines were used and other criteria that significantly impact interface design. These experiences are stored in cases that are attached to the general guidelines to create a repository, capable of distributing the accumulated experiences of the organization. The cases demonstrate how different guidelines have been applied to interface designs, and can serve as examples of interface designs (both good and bad) that can be used by subsequent developers. Using the guidelines becomes a matter of describing user needs and matching them to existing cases. By tying guidelines to specific cases and providing means to locate interface problems with similar characteristics, the method provides the means to determine when the guidelines should be applied. Concrete cases help designers interpret the guidelines, making them easier to comprehend and apply to the current design problems. As the cases accumulate, the knowledge contained in the repository becomes increasingly tailored to the kinds of design problems that frequently occur in the organization. The repository therefore serves not only as a means to disseminate interface design knowledge, but also helps an organization to learn what does and does not work for their base of customers. As opposed to attempting to derive guidelines from abstract first principles, the organizational learning approach seeks to accumulate concrete information from which new principles can be derived. The method incorporates the means to reify the theoretical to the concrete [29] by providing examples that make guidelines more ecologically valid, more understandable, less open to interpretation, and more easily located [12]. But, while some of the problems of using design guidelines are solved with an organizational learning approach, other problems are exacerbated. Guidelines are faced with an inherent conflict: to be useful, guidelines need to be as specific to the design needs as possible, but this creates a proliferation of guidelines, making it even harder to find appropriate guidelines. The complexity and size of guidelines are immense to begin with (witness Smith and Mosier’s 944 general-purpose guidelines [4]), making it difficult to find guidelines that are applicable to a given kind of problem. Providing usability examples and extending guidelines to meet specific user needs provides important information, but increases the problem of locating relevant guidelines. An effective method for identifying and/or retrieving guidelines and cases is needed. Creating a classification structure involves a great deal of a priori knowledge about the domain under investigation. Our premise is that much of this knowledge does not currently exist, and must be derived by the organization as relevant issues arise in the development of interactive systems. We have therefore chosen to use case-based technology. Casebased methods do not require extensive classification to find information, and are often touted as a technique that works best in the kinds of ill-defined problem solving situations we are interested in [30,31].
230
S. Henninger / Interacting with Computers 12 (2000) 225–243
Elicit Problem Characteristics GUIDE Rules
Tailored Guidelines
Project • Guidelines • Cases
Usability Guidelines Rule Modification Review Process
Modified Guidelines
Guideline Modification
External Changes
Fig. 2. The guide development process.
3.2. A case-based approach to organizational learning Case-based reasoning is an artificial intelligence method based on cognitive models postulating that much of human problem solving involves applying past experiences to analogically related situations. While early case-based systems attempted to provide autonomous problem solving by adapting existing solutions to new situations, recent systems have emphasized providing an external memory for users through an interactive process of decision support [22,32,33]. A case-based repository for decision support can suggest how new problems can be approached, suggest the means for adapting a solution that does not quite fit, warn of possible failures, and help designers interpret and understand a situation [30]. Case-based decision aid technology is a perfect fit with an organizational memory approach to design guidelines, because we are interested in situations in which there is no formalized or algorithmic solution available, but problem solving examples exist. Human interpretation and analysis of the situation is necessary, but people need help finding relevant cases because they either forgot or did not know about existing solutions and approaches to a given problem. Case-based methods can also support the abstraction process that is so important to domain analysis by detecting patterns, such as when several cases suggest the same solution and/or are indexed with similar terms.
S. Henninger / Interacting with Computers 12 (2000) 225–243
231
Fig. 3. The guide guideline hierarchy.
4. Supporting context-specific usability guidelines with guide We have developed a prototype named guide (Guidelines for Usability through Interface Development Experiences) to demonstrate and explore a combination of technology and process to support the creation and maintenance of on-line usability guidelines. guide employs a Web-based interface using HTML, Java and Javascript with a relational database back-end for storing guidelines, cases and rules. The guide prototype can be found at http://cse-ferg40.unl.edu/guide.html. A Java-enabled Web browser supporting LiveConnect technology (Netscape Navigator 3.0 or better and Internet Explorer 4.0 or better) is required to run this Web-based program. The overall approach is depicted in Fig. 2. As development efforts or projects involving usability issues are undertaken, developers are stepped through a process that elicits problem characteristics and uses rules to match the design context to the most appropriate guidelines. The result is a set of guidelines that are tailored to the specific needs of the problem. Cases are used to document conformance with the tailored guideline set. Since this information is based on past experiences, and the current design context will probably differ from previous efforts, a review process is undertaken to assess the applicability of the guidelines assigned to the project. Together with human factors experts, developers assess which guidelines need to be followed, which guidelines should not be followed, and create any new guidelines that are necessary. In instances where deviations from the assigned guidelines are necessary, reviewers must decide whether changes to rules and/or guidelines are needed. In other words, deviations are taken as an impetus for reviewing the guide repository. For example, if
232
Will an image map be used? What graphical Navigation cues will be used?
buttons graphic icons
Yes No
Clearly delineate the clickable regions in an image map
Limit total graphics size on a page to < 30K
Provide alternate text for graphics
Is there a standard set of icons to use for navigation?
Yes No
Make click-able regions in image map look like buttons
Use a thumbnail approach to navigating between pages
Use a table of links at the top of the page
0-5 How many separate destinations are in your site?
6-15 > 15 Will typical users have access to Frames-compatible browsers?
Fig. 4. A partial decision tree.
No Yes
Use a left-frame page structure
S. Henninger / Interacting with Computers 12 (2000) 225–243
Provide alternate text links for image map destinations
S. Henninger / Interacting with Computers 12 (2000) 225–243
233
it is determined that the project should not follow an assigned guideline, the “rationale” for this action is encoded in a new rule so that subsequent projects with similar characteristics will not be assigned the guideline. In other words, a deviation from the current standard (defined by the rules and guidelines) sets a precedent that defines future actions under these circumstances. In addition, external changes to technology, usability techniques and business needs may dictate modification of guidelines and/or rules. 4.1. guide architecture As an organization evolves and creates a number of different systems, each with slightly or radically different requirements, the process described above will become too complex for any individual to fully understand. Technology is needed to keep the process simple from the developer’s perspective. The resulting repository, which contains usability guidelines, cases and rules to match design context to guidelines, should also be used to create a decision support system that allows developers to draw on the collective experience of the organization. guide uses a combination of organizational memory and learning perspectives [25,34–37], and case-based decision support technology [32] to satisfy these requirements. The general architecture of guide consists of a guideline hierarchy, cases and rules. The guideline hierarchy breaks guidelines into increasingly more detailed and context-specific information. For example, the structure in Fig. 3 shows the entire set of usability guidelines with guidelines about using graphics broken down into more specific guidelines. Up to guideline 2.4.1, the hierarchy follows the Smith and Mosier standard [4]. Deeper parts of the structure deals with context-specific guidelines that have been inserted through the process shown in Fig. 2. A “case” is assigned to a guideline for each project that is required to conform to the guideline. The cases document how individual projects achieve guideline conformance. The principle is that each time a guideline is deemed applicable to a project, a case for that guideline is created that is used to document how the project applied the guideline. For example, if a guideline states that a usability evaluation should be performed, then the project would place the results of that evaluation in the case. Cases play an important role in helping users understand how guidelines can be applied to a specific context. Rules are used to match the current design context with existing guidelines. Internally, rules are represented as question–answer pairs in a decision tree structure. The decision trees, such as the one shown in Fig. 4 determine which guidelines are assigned to a given project. Answering questions, steps the user down the decision tree until guidelines are assigned to the project and no more questions are asked. The rules are represented in a relational database. A general production system engine matches answers to question– answer pairs that define preconditions for the rules. When all the preconditions for a rule are satisfied, a set of actions are fired. 4.2. Tailoring guidelines to a project Suppose that a development team is about to undertake the design and implementation of a Web site to implement an on-line catalog for a company that has a dynamic inventory, necessitating frequent changes to keep the catalog up to date. As part of project initiation
234
S. Henninger / Interacting with Computers 12 (2000) 225–243
Fig. 5. Eliciting design context information.
and/or project refinement processes, project personnel give responses to questions in the interface shown in Fig. 5. Users are free to choose the most appropriate question from the list. Depending upon which question the user responds to and the response that is given, a subsequent set of questions is generated and guidelines are obtained that the project should conform with. The questions in this process are designed to elicit the current design context so that appropriate guidelines can be assigned to the project. For example, suppose that developers want to find some guidelines on navigation in the above Web page application. The user begins by editing project guidelines through the interface shown in Fig. 5. The figure shows that the questions “What navigation strategies will be used?” was previously answered with “graphics, hypertext”, and “What graphical navigation cues will be used?” was answered with “Graphic icons”. This causes a new set questions to appear in the “New Questions” pane. The user has selected the question “Is there a standard set of icons to use for navigation?”. The set of possible answers appear below in the “Answers” pane. Users can also view guidelines associated with the questions by clicking the “View Guidelines” button. This and other ancillary information, can provide support for choosing amongst the answers provided. If answering a question satisfies all the prerequisites for a rule, a number of different kinds of actions can be taken by guide, such as putting new questions on to the New Question stack, removing questions, adding guidelines to a project, or even sending an e-mail message to someone for notification purposes. Note that the scenario depicted in
S. Henninger / Interacting with Computers 12 (2000) 225–243
235
Fig. 6. GUIDE Project Manager.
Fig. 6. guide project manager.
Fig. 5 is following the decision tree shown in Fig. 4. If the user answers “Yes” to the current question (“Is there a standard set of icons to use for navigation?”), two guidelines will be added to the project (limiting graphics to , 30K and provide alternate text) as well as a new question (“Will an image map be used?”). The intermediate result of answering “yes” to the question is shown in Fig. 6, where the assigned guidelines have been added to the project. This interface can be treated as a checklist of tasks related to usability issues. Each item in the checklist is a case that is associated with the guideline it was created from. This can be used both by human factors specialists to track conformance with the guidelines, and by developers as a usability design resource. Because the level of detail the guidelines are capable of dealing with exceeds the ability of developers to understand all issues at the beginning of a project, guidelines are broken down iteratively. An initial pass will assign a number of guidelines to a project, such as “Language”, “Customer Service” and “Database Design” (see Fig. 6). Each of these can then be broken down further by answering questions related to that sub-topic. For example, the “Database Design” case may have some questions about how the database should be structured. Answering these questions will result in new tasks being added for the project that include usability guidelines. guide‘s architecture allows a set of questions to be associated with any guideline/case pair in the system. If a guideline is assigned to a project that needs to be broken into further detail, the initial questions will be copied to the project as part of the case. Project personnel can visit these guidelines when they have sufficient information to begin answering the questions and further refining project tasks.
236
S. Henninger / Interacting with Computers 12 (2000) 225–243
Fig. 7. Interface for modifying rules.
4.3. Modifying the guide repository to meet new requirements The previous scenario assumed that all guidelines in the guide repository were applicable and could be tailored as-is to meet the project’s requirements. Particularly in the face of diverse and changing customer needs, this will not always be the case. As stated earlier, the repository must be flexible enough to accommodate new development contexts and requirements. Requests for new guidelines and examples can come from a number of sources, including new usability technology and techniques, new interface software, issues that have arisen in the organization, as well as requirements on new projects that have not been encountered previously. From the perspective of a project encountering requirements that are not well matched to existing guidelines, a review process, as shown in Fig. 2 will be initiated. If a deviation from the guideline is needed, the review process can range from a justification for a special case to requesting the modification of guidelines or the rule structure in guide. Human factors specialists, being the curators of the repository, would review the deviation and take appropriate action(s). For example, since the results from guide are only as good as the rules, it will be important to continuously update these rules. As stated earlier, guide rules consist of preconditions and actions. The interface shown in Fig. 7 would provide those with access privileges to change rules, the ability to modify rules and test them. A rule is broken down into its preconditions and consequences (actions). In this figure, rule 108 has been chosen, showing the two preconditions that must be satisfied before the actions shown in the bottom pane can be fired. Users can edit these separately with an interface that allows the selection of questions and actions. Users can also create new questions–answer pairs and actions through these interfaces. Future efforts will explore how to make the process of creating rules easier and less error-prone. For example, a program that takes a set of rules and turns them into a graphically displayed decision tree, such as the one shown in Fig. 4, would support the process of creating and debugging rules.
S. Henninger / Interacting with Computers 12 (2000) 225–243
237
Through this deviation process, guidelines are constantly being refined to meet the context-specific needs that arise in the organization. Note that a natural feedback loop occurs, in which projects are pushed toward conformance with organizationally-defined best practices. If a project complies with the guidelines, then little else has to be done in terms of documenting their efforts. On the other hand, if they need to deviate from the guidelines, then they are considered a “trail-blazing” project for that issue that must undergo a careful documentation process to ensure that footprints are left [38] for subsequent development efforts.
5. Formative evaluation A formative evaluation has been performed with guide to assess the effectiveness of matching problem context to guidelines and defining context rules. The intention of this evaluation was to obtain early information on the feasibility of the guide approach and the interface we had developed [39]. Five subjects, all graduate students at the University of Nebraska, participated in the study. Two of the subjects had some exposure to usability issues through courses and practical experience. The others had no experience. We compared the guide approach to a hypertext-based system (the Sun Web guidelines site). Subjects were given a series of tasks asking them to find relevant guidelines for scenarios concerning the development of different kinds of Web sites. Observations were made, questionnaires were given, and subjects were interviewed about their experiences. As in other evaluations of guidelines, we found that subjects were unsure of which guidelines were appropriate. They also expressed and displayed confusion over the differences and conflicts between guidelines. When browsing on the Sun site, subjects were observed either marking all guidelines as relevant (the most prevalent approach) or choosing one or two. Some of the guidelines chosen without support from guide were contradictory or did not seem to apply to the given scenario. For example, one subject marked “Avoid using a palette of graphic navigation buttons” and “Supply alternate text for graphic navigation buttons” as relevant guidelines, clearly contradictory results. We took this to indicate that the users lacked the expertise to assess which guidelines would address their usability issues. Although our results were obtained with graduate students, our observations of software developers in industry indicate that the level of knowledge of interface issues is not very sophisticated [28]. In fact, some of the students, having taken a course in human factors, are better trained than most developers we have observed. In addition, we found that subjects found far more guidelines with the Sun site. Although this may seem to favor that approach, a closer look reveals that subjects were suffering from a form of information overload: if there is any doubt, mark it relevant. Subjects also differed widely on which guidelines they chose. While subjects found more guidelines in the browsing condition, the variance of guidelines found was ten-fold greater than in the guide condition. While this may be an obvious result stemming from the rulebased technique used in guide, the magnitude of the difference was surprising. From a design standpoint, the result of this behavior would lead to inconsistent interfaces arising from the different guidelines chosen, a problem guidelines are supposed to alleviate.
238
S. Henninger / Interacting with Computers 12 (2000) 225–243
Subject comments were generally favorable toward the guide approach, stating that answering the questions was more obvious than searching an information space whose structure seemed confusing at times (possibly due to their lack of knowledge in the area). Subjects were also able to get to relevant guidelines in a much shorter period of time and were more sure of the guidelines found. Although it is clear that the guide approach will lead to more consistent interfaces and is easier for novice users to navigate, it remains an empirical question whether the guide approach unnecessarily suppresses creative solutions or otherwise undermines usability experts. Subjects also reported that they would have liked more browsing features on the guide interface to have the freedom to explore the information space more.
6. Related work Design guidelines that attempt to distill the accumulated wisdom of the HCI field [4,5], document specific interface design methodologies [18], or enforce compliance with the look-and-feel of products [2,3], have proliferated. Hypertext systems that help designers wade through the guidelines have been designed [14,15,17], some including checklists to help users apply the reference material in a step-by-step manner [16]. While many books and manuals have been published on design guidelines, to date little work has been done on the impact of these guidelines in specific organizations. In particular, the subject of adapting guidelines to specific development organizations, classes of users (not the novice/expert distinction, but different professional fields and the like), or different kinds of tasks that users wish to accomplish, has not been addressed. There are, however a number of other research efforts that have significant impact on our approach. Design rationale is a discipline that attempts to capture and disseminate information about design issues [40]. Design rationale attempts to capture alternative design issues and the rationale for choosing amongst them [41–43]. This has led to the creation of various rationale structures, all of which have the basic structure of Rittel’s issues, answers and arguments [44]. Technologies have also been created to improve access to rationale [45], and have largely focused on creating methods to non-intrusively capture rationale [46]. These techniques suffer from both readers and writers problems. The writers problem is that the capture process adds work for developers that benefits others more than themselves, creating a serious impediment to adopting the technique [47]. The readers problem comes from the trade-off between capturing too much information and capturing the wrong information [46], making it difficult to benefit from captured rationale. Methodologies are needed that integrate rationale use and capture as a critical part of the development process that provides immediate benefit to developers and streamlines the development process instead of adding extra work for dubious benefit. In the same way that standards and guidelines suffer from abstractness, design rationale suffers from too much concreteness. Looking at the detail without any attempts to synthesize the information can confuse the designer in the myriad of detail that may or may not be relevant. Design rationale systems can also become victims of their own success. Every project has a myriad of issues needing capturing, causing an information overload problem [48,49] that demands effective information retrieval and filtering [50]. In addition to
S. Henninger / Interacting with Computers 12 (2000) 225–243
239
problems with capturing and finding relevant rationale [46], it is generally difficult to apply specific cases to new and different interfaces. There are also important questions, such as which development methods and tools should be used, that design rationale has not been concerned with. This kind of information can be a part of design rationale in principle, but none of the techniques to date have explicitly separated different kinds of concerns, choosing instead to focus on the structure of arguments on design alternatives. The focus has remained exclusively on the rationale for design decisions, leaving no room to collect crucial information on the effectiveness of development methods and tools. Perhaps, the ultimate in design guideline technology is to embody the guidelines within a development environment. Fischer and colleagues have been developing design environments that provide guidelines to support designers in creating quality designs [24]. The guidelines are embodied in knowledge-based “critics” that identify suboptimal designs based on criteria derived from domain experts [43,51]. By contextualizing the guidelines on the artifact under design, this method solves some of the problems with finding relevant guidelines. But it also relies on computationally detecting the relationship between design elements, and can only provide advice once a mis-step is taken. This means the guidelines must be formally definable, and design alternatives outside the scope of the current design cannot be supported. Our approach is to contextualize the advice given by the system by creating an evolving organizational memory of usability experiences that can support computational critics once an appropriate level of formality is achieved, but that also supplies information on less formal, but equally relevant, design guidelines. We have also been influenced by the various process improvement efforts [52]. Our approach shares the common goal of enhancing productivity and quality of software development as a continuous improvement process. These approaches, however, suffer from an over-emphasis on process aspects of development. We extend this notion to the continuous improvement of design guidelines that serve as an organizational asset that can be re-used, refined and accumulated as an organization matures in different application domains. Our approach is most closely related to some approaches in constructing organizational memory systems [53]. While the organizational learning approach outlined in this paper emphasizes the process of learning from and improving on previous efforts, these efforts focus on the first step in our domain lifecycle, collecting and disseminating design information. TeamInfo focused primarily on querying and browsing issues for an organizational memory of loosely organized e-mail messages [54] Answer Garden was built to turn knowledge into an organizational asset in a network of multiple-choice questions and answers [55,56] Their bottom-up process evolves the repository in response to user questions, and would be most useful for collecting experiences about development tools. Our framework goes further to support the process of analyzing domains and turning the individual cases into assets that can streamline the development process. Our approach is closest in scope to the Designer Assistant effort at AT&T which provides access to a repository of issues such as real-time performance constraints, local programming conventions, properties of an implementation, and others [38]. Their repository approach uses traditional knowledge-based technology, but accomplishes many of the goals we have set out to address. We are using a similar approach to collect and
240
S. Henninger / Interacting with Computers 12 (2000) 225–243
disseminate design guidelines, and are investigating design methodologies that incorporate the use and refinement of guidelines.
7. Conclusion While it is clear that “[g] uidelines should never replace thoughtful, well-executed graphical user interface design, prototyping, and usability testing with representative users” [8], making usability methods and guidelines an integral part of the development process can prevent many usability problems and improve the overall design of applications. Currently there is no lack of usability design guidelines to choose from, but there is little information available to developers that indicates how and when to apply them. guide is an approach to addressing this problem through a combination of process and technology that can capture a context-specific level of guidelines that can directly be applied to interface design problems. There is sound reasoning for making this process a proactive one, preventing potential “design disasters” before they reach a final human factors certification. In addition to the intent that guide be used throughout the entire development process to reflect evolving and context-specific project needs, a balance must be found between obligatory guidelines and providing an adequate amount of flexibility for developers. Experience has shown us that a combination of process and technology are needed to achieve the goal of incorporating a case-based approach to software development [38,57]. In order to accomplish our goals, we utilize case-based techniques in an organizational learning structure, coupled by a human factors review process for continued refinement and verification of usable guidelines. The process of determining the applicability of any guidelines, works in a fashion similar to American case law, where attorneys formally argue whether the precedent applies, demonstrating how existing cases are similar to or different from the current situation. This kind of reasoning can then be used as a form of rationale that captures the context of a decision that future efforts can benefit from. The rich hypertext cross-referencing built into guide, along with ubiquitous access and potential to support collaborative work provided by a Web-based application, provides direct fingertip access to valuable usability resources. Guidelines are only one form of resource in guide, which is supplemented by annotated examples, related guidelines, associated cases and open forum comments. Combining these tools with a process that ensures the timely incorporation of emerging information can go far to support the process of creating high-quality, consistent and usable applications. The guide prototype is a work in progress. There are a number of issues that must be resolved and features incorporated before it can be used as a central resource in organizations. We are currently refining an interface for creating and maintaining rules. While the rule-based production system architecture underlying guide makes almost any structure possible, care must be exercised so that the effort to maintain rules is not cumbersome. As the rule base becomes more complex, interface support for checking the consistency of rules [58] and actions will become necessary. Methods are also needed for “retiring” guidelines and rules that have become obsolete due to evolving organizational culture or new empirical evidence on interface design methods. A search interface, such as the one
S. Henninger / Interacting with Computers 12 (2000) 225–243
241
in Mimir [11], is also needed. Using active examples that can be executed to observe behavioral aspects of interface design is also being investigated. There are also a number of usability principles and techniques that could be incorporated into guide. For example, methods for providing end-user profiles alongside usability guidelines could be used to match guidelines to user characteristics, such as skills, knowledge, experiences and personal traits, and work environments from a variety of disciplines [8]. Integrating the results of formal usability studies would also incorporate a desired degree of scientific validity to the repository. Our hope is that through the process we have defined, guide will evolve to meet these different needs as they arise in the organization. By incrementally drawing on the collective experience and intelligence of all developers in the organization, guide provides an infrastructure to capture both usability knowledge and the relationships between development and user contexts and usability guidelines and principles. References [1] G.W. Strong, New Directions in Human–Computer Interaction Education, Research and Practice, Report sponsored by NSF and ARPA, 1994. Available at http://129.25.40.211 [2] Apple Computer Inc, Macintosh Human Interface Guidelines, Addison-Wesley, Reading, MA, 1992. [3] Microsoft Corporation, The Windows Interface: an Application Design Guide, Microsoft Press, Redmond, 1992. [4] S.L. Smith, J.N. Mosier, Guidelines for Designing User Interface Software, Technical Report, The MITRE Corporation ESD-TR-86-278, 1986. [5] C.M. Brown, Human–Computer Interface Design Guidelines, Ablex, NJ, USA, 1988. [6] P. Heckel, The Elements of Friendly Software Design, Sybex, San Francisco, 1991. [7] P.A. Billingsley, Starting from scratch: building a usability program at union Pacific Railroad, Interactions 2 (1995) 27–30. [8] E. Rosenweig, A common look and feel or a fantasy?, Interactions 3 (1996) 21–26. [9] J.D. Gould, S.J. Boies, C.H. Lewis, Making usable useful productivity-enhancing computer applications, Communications of the ACM 34 (1991) 75–85. [10] F. de Souza, N. Bevan, The use of guidelines in menu interface design: evaluation of a draft standard, Proceedings of Human–Computer Interaction—INTERACT’90, Elsevier/North-Holland, NY, USA, 1990 pp. 435–440. [11] S. Henninger, K. Haynes, M.W. Reith, A framework for developing experience-based usability guidelines, Proceeding of the Symposium on Designing Interactive Systems DIS’95, ACM Press, New York, 1995 pp. 43–53.. [12] L. Tetzlaff, D.R. Schwartz, The use of guidelines in interface design, Proceedings of the Conference on Human Factors in Computing Systems CHI’91, ACM Press, New York, 1991 pp. 329–333. [13] H. Thovtrup, J. Nielsen, Assessing the usability of a user interface standard, Proceedings of the Conference on Human Factors in Computing Systems CHI’91, ACM Press, New York, 1991 pp. 335–341. [14] R. Iannella, HyperSAM: a practical user interface guidelines management system, Proceedings of the Second Annual CHISIG (Queensland) Symposium—QCHI’94, Bond University, Australia, 1994. [15] L. Alben, J. Faris, H. Saddler, Making it Macintosh: designing the message when the message is design, Interactions 1 (1994) 10–20. [16] G. Perlman, Asynchronous design/evaluation methods for hypertext development, in: Proceedings of Hypertext ‘89, 1989, pp. 61–68. [17] J. Vanderdonckt, Accessing guidelines information with SIERRA, Proceedings Fifth IFIP International Conference on Human–Computer Interaction INTERACT’95, Chapman & Hall, London, 1995 pp. 311–316. [18] IBM, HCI Guidelines, 1997. Available at http://www.ibm.com/IBM/HCI/guidelines/guidelines.html [19] M.C. Detweiler, R.C. Omanson, Ameritech Web Page User Interface Standards and Design Guidelines,
242
[20] [21] [22] [23]
[24] [25]
[26]
[27] [28]
[29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45]
S. Henninger / Interacting with Computers 12 (2000) 225–243 Ameritech, 1996. Available at http://www.ameritech.com:1080/news/testtown/library/standard/web_guidelines/ index.html Sun Microsystems, Guide to Web Style, 1996. Available at http://www.sun.com/styleguide/ S. Weinschenk, S.C. Yeo, Guidelines for Enterprise-wide GUI Design, Wiley, New York, 1995. J. Barber, AskJef: Integration of Case-Based and Multimedia Technologies for Interface Design Support, Artificial Intelligence in Design’92, Kluwer Academic, Dordrecht, Netherlands, 1992 pp. 457–474. S.R. Henninger, A. Ignatowski, C. Rathke, D.F. Redmiles, A knowledge-based design environment for graphical network editors, IEEE Computer Society 2 (1989) 881–891 Proceedings of the 22nd Annual Hawaii Conference on System Sciences, Software Track. G. Fischer, A.C. Lemke, Construction kits and design environments: steps toward human problem-domain communication, Human–Computer Interaction 3 (1988) 179–222. S. Henninger, C. Lu, C. Faith, Using organizational learning techniques to develop context-specific usability guidelines, in: Proceedings of the Symposium on Designing Interactive Systems (DIS’97), Amsterdam, 1997, pp. 129–136. C.H. Lewis, G.M. Olson, Can the principles of cognition lower the barriers of programming?, in: G.M. Olson, E. Soloway, S. Sheppard (Eds.), Empirical Studies of Programmers, 2, Lawrence Erlbaum Associates, Hillsdale, NJ, 1987, pp. 248–263. E.A. Domeshek, J.L. Kolodner, A Case-based Design Aid for Architecture, Artificial Intelligence in Design ‘92, Kluwer Academic, Dordoecht, Netherlands, 1992 pp. 497–516. S. Henninger, K. Lappala, A. Raghavendran, An organizational learning approach to domain analysis, Proceedings of Seventeenth International Conference on Software Engineering, ACM Press, New York, 1995 pp. 95–104. V. Bellotti, Integrating Theoreticians’ and Practitioners’ perspectives with design rationale, Proceedings of InterCHI ‘93, ACM Press, New York, 1993 pp. 101–114. J.L. Kolodner, Case-Based Reasoning, Morgan-Kaufman, San Mateo, USA, 1993. S. Slade, Case-based reasoning: a research paradigm, AI Magazine 12 (1991) 42–55. J.L. Kolodner, Improving human decision making through case-based decision aiding, AI Magazine 12 (1991) 52–68. M. Pearce, A.K. Goel, J.L. Kolodner, C. Zimring, L. Sentosa, R. Billington, Case-based design support: a case study in architectural design, IEEE Expert 7 (1992) 14–20. P. Senge, The Fifth Discipline: The Art and Practice of the Learning Organization, Currency Doubleday, New York, 1990. C., Argyris, D.A. Scho¨n, Organizational Learning: a Theory of Action Perspective, Addison-Wesley, Reading, MA, 1978. E.W. Stein, V. Zwass, Actualizing organizational memory with information systems, Information Systems Research 6 (1995) 85–117. J.S. Brown, P. Duguid, Organizational learning and communities-of-practice: toward a unified view of working learning and innovation, Organization Science 2 (1991) 40–57. L.G. Terveen, P.G. Selfridge, M.D. Long, ‘Living Design Memory’—framework implementation lessons learned, Human–Computer Interaction 10 (1995) 1–37. Y. Lu, Support for the Creation of Context-Specific Usability Guidelines, Computer Science and Engineering, University of Nebraska, Lincoln, 1997. T. Moran, J. Carroll, Design Rationale: Concepts, Techniques and Use, Lawrence Erlbaum Associates, Hillsdale, NJ, 1996. J. Conklin, M.L. Begeman, gIBIS: a hypertext tool for exploratory policy discussion, Transactions on Office Information Systems 6 (1989) 303–331. A. Maclean, V. Bellotti, R. Young, T. Moran, Questions, options and criteria: elements of design space analysis, Human–Computer Interaction 6 (1991) 201–251. G. Fischer, A.C. Lemke, R. McCall, A. Morch, Making argumentation serve design, Human–Computer Interaction 6 (1991) 393–419. H.W. Rittel, W. Kunz, Issues as Elements of Information Systems, Center for Planning and Development Research, University of California Berkeley Working Paper 131, 1970. J. Lee, SIBYL: a tool for managing group design rationale, Proceedings of the Conference on ComputerSupported Cooperative Work CSCW ‘90, ACM Press, New York, 1990 pp. 79–92.
S. Henninger / Interacting with Computers 12 (2000) 225–243
243
[46] E.J. Conklin, K. Yakelovic, A process-oriented approach to design rationale, Human–Computer Interaction 6 (1991) 357–391. [47] J. Grudin, Why CSCW Applications fail: problems in the design and evaluation of organizational interfaces Proceedings of the Conference on Computer-Supported Cooperative Work CSCW ‘88, ACM Press, New York, 1988 pp. 85–93. [48] S.R. Hiltz, M. Turoff, Structuring computer-mediated communication systems to avoid information overload, Communications of the ACM 28 (1985) 680–689. [49] G. Fischer, C. Stevens, Information access in complex, poorly structured information spaces, in: Proceedings of CHI’91, New Orleans, 1991, pp. 63–70. [50] N.J. Belkin, W.B. Croft, Information filtering and information retrieval: two sides of the same coin?, Communications of the ACM 35 (1992) 29–38. [51] G. Fischer, A.C. Lemke, T. Mastaglio, A.I. Morch, Critics: an emerging approach to knowledge-based human computer interaction, International Journal of Man–Machine Studies 35 (1991) 695–721. [52] M.C. Paulk, B. Curtis, M. Chrissis, C.V. Weber, Capability Maturity Model, Version 1.1, IEEE Software 10 (1993) 18–27. [53] J.P. Walsh, G.R. Ungson, Organizational memory, Academy of Management Review 16 (1991) 57–91. [54] L.M. Berlin, R. Jeffries, V.L. O’day, A. Paepcke, C. Wharton, Where did you put it? issues in the design and use of a group memory, Proceedings InterCHI ‘93, ACM Press, New York, 1993 pp. 23–30. [55] M.S. Ackerman, Answer Garden: a Tool for Growing Organizational Memory, MIT Sloan School of Management, Boston, USA, 1994. [56] M.S. Ackerman, T.W. Malone, Answer Garden: a tool for growing organizational memory, Proceedings of the Conference on Office Information Systems, ACM Press, New York, 1990 pp. 31–39. [57] S. Henninger, Capturing and formalizing best practices in a software development organization, The Ninth International Conference on Software Engineering and Knowledge Engineering (SEKE’97), Madrid, Spain, 1997 (in press). [58] L. Terveen, D. Wroblewski, A Collaborative Interface for Browsing and Editing Large Knowledge Bases, National Conference of the American Association for AI, AAAI Press, Boston, USA, 1990 pp. 491–496.
Interacting with Computers 12 (2000) 225–243 www.elsevier.com/locate/intcom
A methodology and tools for applying contextspecific usability guidelines to interface design S. Henninger* Department of Computer Science and Engineering, University of Nebraska, Lincoln, NE, USA
Abstract This paper presents a methodology and an associated technology to create context-specific usability guidelines. The objective is to transform usability guidelines into a proactive resource that software developers can employ early and often in the development process. The methodology ensures conformance with established guidelines, but has the flexibility to use design experiences to adapt the guidelines to meet the emergent and diverse requirements of modern user interface design. Case-based and organizational learning technology is used to support the methodology and provide valuable resources for software developers. q 2000 Elsevier Science B.V. All rights reserved. Keywords: Guidelines; Human–computer interaction; Software development methodologies; Software engineering; Standards; Tools for working with guidelines; Usability
1. Introduction Usability guidelines have become a widely accepted method of bringing the cumulative knowledge of usability issues to bear on the software development process. While it is generally accepted that guidelines cannot replace the “golden rules” of interface design— user involvement, user feedback from early prototypes and iterative development—guidelines can play a role in improving the quality of the iterative steps, leading to an improvement in quality and reduction (but not elimination) of the number of iterations involved in the design-evaluate-redesign cycle of HCI development [1]. Guidelines usually take one of the two forms. Style guides address when and how to use interface widgets and general look-and-feel issues, often focusing on a specific platform such as Macintosh [2] or Windows [3]. Human–computer interface guidelines provide a platform-independent and comprehensive set of guidelines on various aspects of human– computer interfaces, including user-centered design techniques [4–6]. * Tel: 1 1-402-472-8394. E-mail address: [email protected] (S. Henninger) 0953-5438/00/$ - see front matter q 2000 Elsevier Science B.V. All rights reserved. PII: S0953-543 8(99)00013-2
226
S. Henninger / Interacting with Computers 12 (2000) 225–243
Fig. 1. An interface lacking in usability principles.
Many development organizations have sensed a need to extend style and general usability guidelines and have begun to create customized usability guidelines and standards [7,8]. But even with these customized approaches, there are a number of concerns about the nature of guidelines that can inhibit the impact of guidelines on the design of user interfaces [9,10]. Apart from not adequately addressing concerns facing software developers, such as which guidelines should be used under what circumstances [11], studies have shown that interface guidelines suffer from being too abstract to directly apply [11– 13]. In addition, usability guidelines have yet to become an integral part of the development process, usually relegated to a discretionary reference role or an after-the-fact human factors certification process. Approaches to improving the use of usability guidelines has mostly focused on access technology, such as hypertext [14–16] and database retrieval [17]. Web-based guidelines are becoming an increasingly popular way to disseminate usability guidelines [18–20]. But none of these technologies address the real problems of usability guidelines. Support is needed for defining when guidelines are applicable or how guidelines can be refined to meet the user task requirements for a specific set of users and a specific type of application. Little research has been done to integrate guideline use in the development process and accumulate knowledge about interface design in a form that can capture relationships between specific contexts and applicable guidelines. This paper presents an approach that addresses these issues through a combination of technology and process. The result extends the traditional notion of usability guidelines to provide detailed guidelines that are integrated into the development process in a
S. Henninger / Interacting with Computers 12 (2000) 225–243
227
conformance process. Technology is used to match design context to guidelines through a rule-based system. Guidelines are attached to cases containing information about how previous projects achieved conformance to the guidelines. This provides developers with an immediate benefit by allowing them to draw on past experiences. A deviation process is used to capture new technology and business requirements and set precedents that subsequent design efforts can follow.
2. Usability guidelines in practice While corporate usability guidelines are starting to appear [7,8,21], these guidelines are often used as part of a certification process that occurs late in the development process. Fig. 1 shows an interface developed by an IT organization with a small human factors staff and a well-documented set of usability guidelines that was available on-line to all developers. This window, which was built as an initial window for a contract support application, consists of 34 buttons with no indication of importance or ordering, groupings with little semantic significance, and button names that have marginal descriptive value. Another application from this organization currently has 120 screens and 22 subfunctions, clearly exceeding the human capacity to understand and easily learn the application. An after-the-fact human factors certification process is clearly not enough to solve the problems faced by these two examples, which are in dire need of task analyses and comprehensive re-design. Both of these applications have been shipped as-is, not because human factors personnel did not identify the problems, but because their involvement occurred too late in the development process. One solution is certainly to get the human factors people involved earlier in the design process, but this is not always feasible in the face of shortened development lifecycles and shortages in human factors specialists.
2.1. Interpreting design guidelines Placing guidelines and usability techniques early in the development process addresses the need for a proactive role for interface design [1]. Yet, it does little to resolve the problems that have been demonstrated in empirical studies of developers using guidelines, such as the difficulty of understanding and applying guidelines [12] and the difficulty of determining when a guideline has been violated [13]. These and other problems can be seen to stem from the abstract and decontextualized nature of current guideline techniques, which tend to make guidelines ambiguous or too abstract to be applied to a specific context. For example, a published guideline states that “All possible actions should be visible always” [22]. Depending on the context, this can be seen as good advice, but could also be argued that the designers of the window in Fig. 1 followed this guideline to its ultimate conclusion! More information is needed that states the circumstances under which the guideline should be followed and what to do when guidelines conflict or do not fully address the problems under consideration.
228
S. Henninger / Interacting with Computers 12 (2000) 225–243
2.2. Developing context-specific usability guidelines The decontextualized nature of current guidelines impose a significant barrier to usability guidelines becoming an integral part of the development process. For example, a guideline stating to “Always keep users informed of system states” is sound advice that has roots to usability evaluations on system modalities. However, as stated, the guideline is abstract and open to a wide variety of interpretations in different contexts. A more concrete guideline, such as “Visually identifying the need to save changes with a user action” at once conveys a specific context; that of identifying when screen information has changed and needs saving, and using visual cues to alert users to the event. Even better would be to tailor the guideline to specific design settings such as “Using color background as a cue for edit vs. read-only modes” or “Identifying changed documents in a word processor”. To the extent that the development context has been captured, the more specific the guideline, the better the support for the developer. Note also that our example defines an increasingly concrete hierarchy. If the guideline at the most concrete level does not support the design context, the developer can move up a level and see what help a more abstract guideline provides. Thus, levels of support are provided at detailed levels, but the knowledge gracefully degrades to more abstract levels as one moves up the hierarchy [23]. These contextualized guidelines are better than abstract or decontextualized ones may be clear. Less clear is how guidelines are created to meet diverse application needs, and how one applies the guidelines to a specific context. Creating and using contextualized guidelines presents a different set of problems that is exacerbated by the growing diversity of user populations and application domains to which computers are being applied. Given these kinds of diverse cross-sections, it is generally infeasible to create a “global” set of contextualized usability guidelines. Such an effort would fail for the combinatorial size of the repository and the inherent difficulty of finding the right guidelines for a given design problem and context. Techniques that provide high levels of domain-specific support [24] need to be utilized to create contextualized guidelines for issues that have strategic importance to an organization [25].
3. An organizational memory approach to developing design guidelines The discussion of guidelines has shown that it is often difficult to understand how specific guidelines apply to a designer’s current situation, or which guidelines are most appropriate. Because designers tend to “ask questions about specific problems they have with their own design rather than abstract ones” [22], there is ample evidence that designers in many domains actively seek concrete examples of existing designs to guide their work [26,27]. Thus, an effective strategy for developing a user interface is to find an existing interface that has proven effective for users with similar backgrounds and needs and adapt the ideas and methods into the new interface.
S. Henninger / Interacting with Computers 12 (2000) 225–243
229
3.1. Organizational memory and usability guidelines An organizational memory approach to design guidelines builds on this method by collecting, disseminating and improving the accumulated wisdom of developers in a development organization [25,28]. The general notion is that an organization begins with a core set of general design guidelines, combined with specific widget guidelines for the application platforms used in the organization. As interfaces are developed, developers record which guidelines were used and other criteria that significantly impact interface design. These experiences are stored in cases that are attached to the general guidelines to create a repository, capable of distributing the accumulated experiences of the organization. The cases demonstrate how different guidelines have been applied to interface designs, and can serve as examples of interface designs (both good and bad) that can be used by subsequent developers. Using the guidelines becomes a matter of describing user needs and matching them to existing cases. By tying guidelines to specific cases and providing means to locate interface problems with similar characteristics, the method provides the means to determine when the guidelines should be applied. Concrete cases help designers interpret the guidelines, making them easier to comprehend and apply to the current design problems. As the cases accumulate, the knowledge contained in the repository becomes increasingly tailored to the kinds of design problems that frequently occur in the organization. The repository therefore serves not only as a means to disseminate interface design knowledge, but also helps an organization to learn what does and does not work for their base of customers. As opposed to attempting to derive guidelines from abstract first principles, the organizational learning approach seeks to accumulate concrete information from which new principles can be derived. The method incorporates the means to reify the theoretical to the concrete [29] by providing examples that make guidelines more ecologically valid, more understandable, less open to interpretation, and more easily located [12]. But, while some of the problems of using design guidelines are solved with an organizational learning approach, other problems are exacerbated. Guidelines are faced with an inherent conflict: to be useful, guidelines need to be as specific to the design needs as possible, but this creates a proliferation of guidelines, making it even harder to find appropriate guidelines. The complexity and size of guidelines are immense to begin with (witness Smith and Mosier’s 944 general-purpose guidelines [4]), making it difficult to find guidelines that are applicable to a given kind of problem. Providing usability examples and extending guidelines to meet specific user needs provides important information, but increases the problem of locating relevant guidelines. An effective method for identifying and/or retrieving guidelines and cases is needed. Creating a classification structure involves a great deal of a priori knowledge about the domain under investigation. Our premise is that much of this knowledge does not currently exist, and must be derived by the organization as relevant issues arise in the development of interactive systems. We have therefore chosen to use case-based technology. Casebased methods do not require extensive classification to find information, and are often touted as a technique that works best in the kinds of ill-defined problem solving situations we are interested in [30,31].
230
S. Henninger / Interacting with Computers 12 (2000) 225–243
Elicit Problem Characteristics GUIDE Rules
Tailored Guidelines
Project • Guidelines • Cases
Usability Guidelines Rule Modification Review Process
Modified Guidelines
Guideline Modification
External Changes
Fig. 2. The guide development process.
3.2. A case-based approach to organizational learning Case-based reasoning is an artificial intelligence method based on cognitive models postulating that much of human problem solving involves applying past experiences to analogically related situations. While early case-based systems attempted to provide autonomous problem solving by adapting existing solutions to new situations, recent systems have emphasized providing an external memory for users through an interactive process of decision support [22,32,33]. A case-based repository for decision support can suggest how new problems can be approached, suggest the means for adapting a solution that does not quite fit, warn of possible failures, and help designers interpret and understand a situation [30]. Case-based decision aid technology is a perfect fit with an organizational memory approach to design guidelines, because we are interested in situations in which there is no formalized or algorithmic solution available, but problem solving examples exist. Human interpretation and analysis of the situation is necessary, but people need help finding relevant cases because they either forgot or did not know about existing solutions and approaches to a given problem. Case-based methods can also support the abstraction process that is so important to domain analysis by detecting patterns, such as when several cases suggest the same solution and/or are indexed with similar terms.
S. Henninger / Interacting with Computers 12 (2000) 225–243
231
Fig. 3. The guide guideline hierarchy.
4. Supporting context-specific usability guidelines with guide We have developed a prototype named guide (Guidelines for Usability through Interface Development Experiences) to demonstrate and explore a combination of technology and process to support the creation and maintenance of on-line usability guidelines. guide employs a Web-based interface using HTML, Java and Javascript with a relational database back-end for storing guidelines, cases and rules. The guide prototype can be found at http://cse-ferg40.unl.edu/guide.html. A Java-enabled Web browser supporting LiveConnect technology (Netscape Navigator 3.0 or better and Internet Explorer 4.0 or better) is required to run this Web-based program. The overall approach is depicted in Fig. 2. As development efforts or projects involving usability issues are undertaken, developers are stepped through a process that elicits problem characteristics and uses rules to match the design context to the most appropriate guidelines. The result is a set of guidelines that are tailored to the specific needs of the problem. Cases are used to document conformance with the tailored guideline set. Since this information is based on past experiences, and the current design context will probably differ from previous efforts, a review process is undertaken to assess the applicability of the guidelines assigned to the project. Together with human factors experts, developers assess which guidelines need to be followed, which guidelines should not be followed, and create any new guidelines that are necessary. In instances where deviations from the assigned guidelines are necessary, reviewers must decide whether changes to rules and/or guidelines are needed. In other words, deviations are taken as an impetus for reviewing the guide repository. For example, if
232
Will an image map be used? What graphical Navigation cues will be used?
buttons graphic icons
Yes No
Clearly delineate the clickable regions in an image map
Limit total graphics size on a page to < 30K
Provide alternate text for graphics
Is there a standard set of icons to use for navigation?
Yes No
Make click-able regions in image map look like buttons
Use a thumbnail approach to navigating between pages
Use a table of links at the top of the page
0-5 How many separate destinations are in your site?
6-15 > 15 Will typical users have access to Frames-compatible browsers?
Fig. 4. A partial decision tree.
No Yes
Use a left-frame page structure
S. Henninger / Interacting with Computers 12 (2000) 225–243
Provide alternate text links for image map destinations
S. Henninger / Interacting with Computers 12 (2000) 225–243
233
it is determined that the project should not follow an assigned guideline, the “rationale” for this action is encoded in a new rule so that subsequent projects with similar characteristics will not be assigned the guideline. In other words, a deviation from the current standard (defined by the rules and guidelines) sets a precedent that defines future actions under these circumstances. In addition, external changes to technology, usability techniques and business needs may dictate modification of guidelines and/or rules. 4.1. guide architecture As an organization evolves and creates a number of different systems, each with slightly or radically different requirements, the process described above will become too complex for any individual to fully understand. Technology is needed to keep the process simple from the developer’s perspective. The resulting repository, which contains usability guidelines, cases and rules to match design context to guidelines, should also be used to create a decision support system that allows developers to draw on the collective experience of the organization. guide uses a combination of organizational memory and learning perspectives [25,34–37], and case-based decision support technology [32] to satisfy these requirements. The general architecture of guide consists of a guideline hierarchy, cases and rules. The guideline hierarchy breaks guidelines into increasingly more detailed and context-specific information. For example, the structure in Fig. 3 shows the entire set of usability guidelines with guidelines about using graphics broken down into more specific guidelines. Up to guideline 2.4.1, the hierarchy follows the Smith and Mosier standard [4]. Deeper parts of the structure deals with context-specific guidelines that have been inserted through the process shown in Fig. 2. A “case” is assigned to a guideline for each project that is required to conform to the guideline. The cases document how individual projects achieve guideline conformance. The principle is that each time a guideline is deemed applicable to a project, a case for that guideline is created that is used to document how the project applied the guideline. For example, if a guideline states that a usability evaluation should be performed, then the project would place the results of that evaluation in the case. Cases play an important role in helping users understand how guidelines can be applied to a specific context. Rules are used to match the current design context with existing guidelines. Internally, rules are represented as question–answer pairs in a decision tree structure. The decision trees, such as the one shown in Fig. 4 determine which guidelines are assigned to a given project. Answering questions, steps the user down the decision tree until guidelines are assigned to the project and no more questions are asked. The rules are represented in a relational database. A general production system engine matches answers to question– answer pairs that define preconditions for the rules. When all the preconditions for a rule are satisfied, a set of actions are fired. 4.2. Tailoring guidelines to a project Suppose that a development team is about to undertake the design and implementation of a Web site to implement an on-line catalog for a company that has a dynamic inventory, necessitating frequent changes to keep the catalog up to date. As part of project initiation
234
S. Henninger / Interacting with Computers 12 (2000) 225–243
Fig. 5. Eliciting design context information.
and/or project refinement processes, project personnel give responses to questions in the interface shown in Fig. 5. Users are free to choose the most appropriate question from the list. Depending upon which question the user responds to and the response that is given, a subsequent set of questions is generated and guidelines are obtained that the project should conform with. The questions in this process are designed to elicit the current design context so that appropriate guidelines can be assigned to the project. For example, suppose that developers want to find some guidelines on navigation in the above Web page application. The user begins by editing project guidelines through the interface shown in Fig. 5. The figure shows that the questions “What navigation strategies will be used?” was previously answered with “graphics, hypertext”, and “What graphical navigation cues will be used?” was answered with “Graphic icons”. This causes a new set questions to appear in the “New Questions” pane. The user has selected the question “Is there a standard set of icons to use for navigation?”. The set of possible answers appear below in the “Answers” pane. Users can also view guidelines associated with the questions by clicking the “View Guidelines” button. This and other ancillary information, can provide support for choosing amongst the answers provided. If answering a question satisfies all the prerequisites for a rule, a number of different kinds of actions can be taken by guide, such as putting new questions on to the New Question stack, removing questions, adding guidelines to a project, or even sending an e-mail message to someone for notification purposes. Note that the scenario depicted in
S. Henninger / Interacting with Computers 12 (2000) 225–243
235
Fig. 6. GUIDE Project Manager.
Fig. 6. guide project manager.
Fig. 5 is following the decision tree shown in Fig. 4. If the user answers “Yes” to the current question (“Is there a standard set of icons to use for navigation?”), two guidelines will be added to the project (limiting graphics to , 30K and provide alternate text) as well as a new question (“Will an image map be used?”). The intermediate result of answering “yes” to the question is shown in Fig. 6, where the assigned guidelines have been added to the project. This interface can be treated as a checklist of tasks related to usability issues. Each item in the checklist is a case that is associated with the guideline it was created from. This can be used both by human factors specialists to track conformance with the guidelines, and by developers as a usability design resource. Because the level of detail the guidelines are capable of dealing with exceeds the ability of developers to understand all issues at the beginning of a project, guidelines are broken down iteratively. An initial pass will assign a number of guidelines to a project, such as “Language”, “Customer Service” and “Database Design” (see Fig. 6). Each of these can then be broken down further by answering questions related to that sub-topic. For example, the “Database Design” case may have some questions about how the database should be structured. Answering these questions will result in new tasks being added for the project that include usability guidelines. guide‘s architecture allows a set of questions to be associated with any guideline/case pair in the system. If a guideline is assigned to a project that needs to be broken into further detail, the initial questions will be copied to the project as part of the case. Project personnel can visit these guidelines when they have sufficient information to begin answering the questions and further refining project tasks.
236
S. Henninger / Interacting with Computers 12 (2000) 225–243
Fig. 7. Interface for modifying rules.
4.3. Modifying the guide repository to meet new requirements The previous scenario assumed that all guidelines in the guide repository were applicable and could be tailored as-is to meet the project’s requirements. Particularly in the face of diverse and changing customer needs, this will not always be the case. As stated earlier, the repository must be flexible enough to accommodate new development contexts and requirements. Requests for new guidelines and examples can come from a number of sources, including new usability technology and techniques, new interface software, issues that have arisen in the organization, as well as requirements on new projects that have not been encountered previously. From the perspective of a project encountering requirements that are not well matched to existing guidelines, a review process, as shown in Fig. 2 will be initiated. If a deviation from the guideline is needed, the review process can range from a justification for a special case to requesting the modification of guidelines or the rule structure in guide. Human factors specialists, being the curators of the repository, would review the deviation and take appropriate action(s). For example, since the results from guide are only as good as the rules, it will be important to continuously update these rules. As stated earlier, guide rules consist of preconditions and actions. The interface shown in Fig. 7 would provide those with access privileges to change rules, the ability to modify rules and test them. A rule is broken down into its preconditions and consequences (actions). In this figure, rule 108 has been chosen, showing the two preconditions that must be satisfied before the actions shown in the bottom pane can be fired. Users can edit these separately with an interface that allows the selection of questions and actions. Users can also create new questions–answer pairs and actions through these interfaces. Future efforts will explore how to make the process of creating rules easier and less error-prone. For example, a program that takes a set of rules and turns them into a graphically displayed decision tree, such as the one shown in Fig. 4, would support the process of creating and debugging rules.
S. Henninger / Interacting with Computers 12 (2000) 225–243
237
Through this deviation process, guidelines are constantly being refined to meet the context-specific needs that arise in the organization. Note that a natural feedback loop occurs, in which projects are pushed toward conformance with organizationally-defined best practices. If a project complies with the guidelines, then little else has to be done in terms of documenting their efforts. On the other hand, if they need to deviate from the guidelines, then they are considered a “trail-blazing” project for that issue that must undergo a careful documentation process to ensure that footprints are left [38] for subsequent development efforts.
5. Formative evaluation A formative evaluation has been performed with guide to assess the effectiveness of matching problem context to guidelines and defining context rules. The intention of this evaluation was to obtain early information on the feasibility of the guide approach and the interface we had developed [39]. Five subjects, all graduate students at the University of Nebraska, participated in the study. Two of the subjects had some exposure to usability issues through courses and practical experience. The others had no experience. We compared the guide approach to a hypertext-based system (the Sun Web guidelines site). Subjects were given a series of tasks asking them to find relevant guidelines for scenarios concerning the development of different kinds of Web sites. Observations were made, questionnaires were given, and subjects were interviewed about their experiences. As in other evaluations of guidelines, we found that subjects were unsure of which guidelines were appropriate. They also expressed and displayed confusion over the differences and conflicts between guidelines. When browsing on the Sun site, subjects were observed either marking all guidelines as relevant (the most prevalent approach) or choosing one or two. Some of the guidelines chosen without support from guide were contradictory or did not seem to apply to the given scenario. For example, one subject marked “Avoid using a palette of graphic navigation buttons” and “Supply alternate text for graphic navigation buttons” as relevant guidelines, clearly contradictory results. We took this to indicate that the users lacked the expertise to assess which guidelines would address their usability issues. Although our results were obtained with graduate students, our observations of software developers in industry indicate that the level of knowledge of interface issues is not very sophisticated [28]. In fact, some of the students, having taken a course in human factors, are better trained than most developers we have observed. In addition, we found that subjects found far more guidelines with the Sun site. Although this may seem to favor that approach, a closer look reveals that subjects were suffering from a form of information overload: if there is any doubt, mark it relevant. Subjects also differed widely on which guidelines they chose. While subjects found more guidelines in the browsing condition, the variance of guidelines found was ten-fold greater than in the guide condition. While this may be an obvious result stemming from the rulebased technique used in guide, the magnitude of the difference was surprising. From a design standpoint, the result of this behavior would lead to inconsistent interfaces arising from the different guidelines chosen, a problem guidelines are supposed to alleviate.
238
S. Henninger / Interacting with Computers 12 (2000) 225–243
Subject comments were generally favorable toward the guide approach, stating that answering the questions was more obvious than searching an information space whose structure seemed confusing at times (possibly due to their lack of knowledge in the area). Subjects were also able to get to relevant guidelines in a much shorter period of time and were more sure of the guidelines found. Although it is clear that the guide approach will lead to more consistent interfaces and is easier for novice users to navigate, it remains an empirical question whether the guide approach unnecessarily suppresses creative solutions or otherwise undermines usability experts. Subjects also reported that they would have liked more browsing features on the guide interface to have the freedom to explore the information space more.
6. Related work Design guidelines that attempt to distill the accumulated wisdom of the HCI field [4,5], document specific interface design methodologies [18], or enforce compliance with the look-and-feel of products [2,3], have proliferated. Hypertext systems that help designers wade through the guidelines have been designed [14,15,17], some including checklists to help users apply the reference material in a step-by-step manner [16]. While many books and manuals have been published on design guidelines, to date little work has been done on the impact of these guidelines in specific organizations. In particular, the subject of adapting guidelines to specific development organizations, classes of users (not the novice/expert distinction, but different professional fields and the like), or different kinds of tasks that users wish to accomplish, has not been addressed. There are, however a number of other research efforts that have significant impact on our approach. Design rationale is a discipline that attempts to capture and disseminate information about design issues [40]. Design rationale attempts to capture alternative design issues and the rationale for choosing amongst them [41–43]. This has led to the creation of various rationale structures, all of which have the basic structure of Rittel’s issues, answers and arguments [44]. Technologies have also been created to improve access to rationale [45], and have largely focused on creating methods to non-intrusively capture rationale [46]. These techniques suffer from both readers and writers problems. The writers problem is that the capture process adds work for developers that benefits others more than themselves, creating a serious impediment to adopting the technique [47]. The readers problem comes from the trade-off between capturing too much information and capturing the wrong information [46], making it difficult to benefit from captured rationale. Methodologies are needed that integrate rationale use and capture as a critical part of the development process that provides immediate benefit to developers and streamlines the development process instead of adding extra work for dubious benefit. In the same way that standards and guidelines suffer from abstractness, design rationale suffers from too much concreteness. Looking at the detail without any attempts to synthesize the information can confuse the designer in the myriad of detail that may or may not be relevant. Design rationale systems can also become victims of their own success. Every project has a myriad of issues needing capturing, causing an information overload problem [48,49] that demands effective information retrieval and filtering [50]. In addition to
S. Henninger / Interacting with Computers 12 (2000) 225–243
239
problems with capturing and finding relevant rationale [46], it is generally difficult to apply specific cases to new and different interfaces. There are also important questions, such as which development methods and tools should be used, that design rationale has not been concerned with. This kind of information can be a part of design rationale in principle, but none of the techniques to date have explicitly separated different kinds of concerns, choosing instead to focus on the structure of arguments on design alternatives. The focus has remained exclusively on the rationale for design decisions, leaving no room to collect crucial information on the effectiveness of development methods and tools. Perhaps, the ultimate in design guideline technology is to embody the guidelines within a development environment. Fischer and colleagues have been developing design environments that provide guidelines to support designers in creating quality designs [24]. The guidelines are embodied in knowledge-based “critics” that identify suboptimal designs based on criteria derived from domain experts [43,51]. By contextualizing the guidelines on the artifact under design, this method solves some of the problems with finding relevant guidelines. But it also relies on computationally detecting the relationship between design elements, and can only provide advice once a mis-step is taken. This means the guidelines must be formally definable, and design alternatives outside the scope of the current design cannot be supported. Our approach is to contextualize the advice given by the system by creating an evolving organizational memory of usability experiences that can support computational critics once an appropriate level of formality is achieved, but that also supplies information on less formal, but equally relevant, design guidelines. We have also been influenced by the various process improvement efforts [52]. Our approach shares the common goal of enhancing productivity and quality of software development as a continuous improvement process. These approaches, however, suffer from an over-emphasis on process aspects of development. We extend this notion to the continuous improvement of design guidelines that serve as an organizational asset that can be re-used, refined and accumulated as an organization matures in different application domains. Our approach is most closely related to some approaches in constructing organizational memory systems [53]. While the organizational learning approach outlined in this paper emphasizes the process of learning from and improving on previous efforts, these efforts focus on the first step in our domain lifecycle, collecting and disseminating design information. TeamInfo focused primarily on querying and browsing issues for an organizational memory of loosely organized e-mail messages [54] Answer Garden was built to turn knowledge into an organizational asset in a network of multiple-choice questions and answers [55,56] Their bottom-up process evolves the repository in response to user questions, and would be most useful for collecting experiences about development tools. Our framework goes further to support the process of analyzing domains and turning the individual cases into assets that can streamline the development process. Our approach is closest in scope to the Designer Assistant effort at AT&T which provides access to a repository of issues such as real-time performance constraints, local programming conventions, properties of an implementation, and others [38]. Their repository approach uses traditional knowledge-based technology, but accomplishes many of the goals we have set out to address. We are using a similar approach to collect and
240
S. Henninger / Interacting with Computers 12 (2000) 225–243
disseminate design guidelines, and are investigating design methodologies that incorporate the use and refinement of guidelines.
7. Conclusion While it is clear that “[g] uidelines should never replace thoughtful, well-executed graphical user interface design, prototyping, and usability testing with representative users” [8], making usability methods and guidelines an integral part of the development process can prevent many usability problems and improve the overall design of applications. Currently there is no lack of usability design guidelines to choose from, but there is little information available to developers that indicates how and when to apply them. guide is an approach to addressing this problem through a combination of process and technology that can capture a context-specific level of guidelines that can directly be applied to interface design problems. There is sound reasoning for making this process a proactive one, preventing potential “design disasters” before they reach a final human factors certification. In addition to the intent that guide be used throughout the entire development process to reflect evolving and context-specific project needs, a balance must be found between obligatory guidelines and providing an adequate amount of flexibility for developers. Experience has shown us that a combination of process and technology are needed to achieve the goal of incorporating a case-based approach to software development [38,57]. In order to accomplish our goals, we utilize case-based techniques in an organizational learning structure, coupled by a human factors review process for continued refinement and verification of usable guidelines. The process of determining the applicability of any guidelines, works in a fashion similar to American case law, where attorneys formally argue whether the precedent applies, demonstrating how existing cases are similar to or different from the current situation. This kind of reasoning can then be used as a form of rationale that captures the context of a decision that future efforts can benefit from. The rich hypertext cross-referencing built into guide, along with ubiquitous access and potential to support collaborative work provided by a Web-based application, provides direct fingertip access to valuable usability resources. Guidelines are only one form of resource in guide, which is supplemented by annotated examples, related guidelines, associated cases and open forum comments. Combining these tools with a process that ensures the timely incorporation of emerging information can go far to support the process of creating high-quality, consistent and usable applications. The guide prototype is a work in progress. There are a number of issues that must be resolved and features incorporated before it can be used as a central resource in organizations. We are currently refining an interface for creating and maintaining rules. While the rule-based production system architecture underlying guide makes almost any structure possible, care must be exercised so that the effort to maintain rules is not cumbersome. As the rule base becomes more complex, interface support for checking the consistency of rules [58] and actions will become necessary. Methods are also needed for “retiring” guidelines and rules that have become obsolete due to evolving organizational culture or new empirical evidence on interface design methods. A search interface, such as the one
S. Henninger / Interacting with Computers 12 (2000) 225–243
241
in Mimir [11], is also needed. Using active examples that can be executed to observe behavioral aspects of interface design is also being investigated. There are also a number of usability principles and techniques that could be incorporated into guide. For example, methods for providing end-user profiles alongside usability guidelines could be used to match guidelines to user characteristics, such as skills, knowledge, experiences and personal traits, and work environments from a variety of disciplines [8]. Integrating the results of formal usability studies would also incorporate a desired degree of scientific validity to the repository. Our hope is that through the process we have defined, guide will evolve to meet these different needs as they arise in the organization. By incrementally drawing on the collective experience and intelligence of all developers in the organization, guide provides an infrastructure to capture both usability knowledge and the relationships between development and user contexts and usability guidelines and principles. References [1] G.W. Strong, New Directions in Human–Computer Interaction Education, Research and Practice, Report sponsored by NSF and ARPA, 1994. Available at http://129.25.40.211 [2] Apple Computer Inc, Macintosh Human Interface Guidelines, Addison-Wesley, Reading, MA, 1992. [3] Microsoft Corporation, The Windows Interface: an Application Design Guide, Microsoft Press, Redmond, 1992. [4] S.L. Smith, J.N. Mosier, Guidelines for Designing User Interface Software, Technical Report, The MITRE Corporation ESD-TR-86-278, 1986. [5] C.M. Brown, Human–Computer Interface Design Guidelines, Ablex, NJ, USA, 1988. [6] P. Heckel, The Elements of Friendly Software Design, Sybex, San Francisco, 1991. [7] P.A. Billingsley, Starting from scratch: building a usability program at union Pacific Railroad, Interactions 2 (1995) 27–30. [8] E. Rosenweig, A common look and feel or a fantasy?, Interactions 3 (1996) 21–26. [9] J.D. Gould, S.J. Boies, C.H. Lewis, Making usable useful productivity-enhancing computer applications, Communications of the ACM 34 (1991) 75–85. [10] F. de Souza, N. Bevan, The use of guidelines in menu interface design: evaluation of a draft standard, Proceedings of Human–Computer Interaction—INTERACT’90, Elsevier/North-Holland, NY, USA, 1990 pp. 435–440. [11] S. Henninger, K. Haynes, M.W. Reith, A framework for developing experience-based usability guidelines, Proceeding of the Symposium on Designing Interactive Systems DIS’95, ACM Press, New York, 1995 pp. 43–53.. [12] L. Tetzlaff, D.R. Schwartz, The use of guidelines in interface design, Proceedings of the Conference on Human Factors in Computing Systems CHI’91, ACM Press, New York, 1991 pp. 329–333. [13] H. Thovtrup, J. Nielsen, Assessing the usability of a user interface standard, Proceedings of the Conference on Human Factors in Computing Systems CHI’91, ACM Press, New York, 1991 pp. 335–341. [14] R. Iannella, HyperSAM: a practical user interface guidelines management system, Proceedings of the Second Annual CHISIG (Queensland) Symposium—QCHI’94, Bond University, Australia, 1994. [15] L. Alben, J. Faris, H. Saddler, Making it Macintosh: designing the message when the message is design, Interactions 1 (1994) 10–20. [16] G. Perlman, Asynchronous design/evaluation methods for hypertext development, in: Proceedings of Hypertext ‘89, 1989, pp. 61–68. [17] J. Vanderdonckt, Accessing guidelines information with SIERRA, Proceedings Fifth IFIP International Conference on Human–Computer Interaction INTERACT’95, Chapman & Hall, London, 1995 pp. 311–316. [18] IBM, HCI Guidelines, 1997. Available at http://www.ibm.com/IBM/HCI/guidelines/guidelines.html [19] M.C. Detweiler, R.C. Omanson, Ameritech Web Page User Interface Standards and Design Guidelines,
242
[20] [21] [22] [23]
[24] [25]
[26]
[27] [28]
[29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45]
S. Henninger / Interacting with Computers 12 (2000) 225–243 Ameritech, 1996. Available at http://www.ameritech.com:1080/news/testtown/library/standard/web_guidelines/ index.html Sun Microsystems, Guide to Web Style, 1996. Available at http://www.sun.com/styleguide/ S. Weinschenk, S.C. Yeo, Guidelines for Enterprise-wide GUI Design, Wiley, New York, 1995. J. Barber, AskJef: Integration of Case-Based and Multimedia Technologies for Interface Design Support, Artificial Intelligence in Design’92, Kluwer Academic, Dordrecht, Netherlands, 1992 pp. 457–474. S.R. Henninger, A. Ignatowski, C. Rathke, D.F. Redmiles, A knowledge-based design environment for graphical network editors, IEEE Computer Society 2 (1989) 881–891 Proceedings of the 22nd Annual Hawaii Conference on System Sciences, Software Track. G. Fischer, A.C. Lemke, Construction kits and design environments: steps toward human problem-domain communication, Human–Computer Interaction 3 (1988) 179–222. S. Henninger, C. Lu, C. Faith, Using organizational learning techniques to develop context-specific usability guidelines, in: Proceedings of the Symposium on Designing Interactive Systems (DIS’97), Amsterdam, 1997, pp. 129–136. C.H. Lewis, G.M. Olson, Can the principles of cognition lower the barriers of programming?, in: G.M. Olson, E. Soloway, S. Sheppard (Eds.), Empirical Studies of Programmers, 2, Lawrence Erlbaum Associates, Hillsdale, NJ, 1987, pp. 248–263. E.A. Domeshek, J.L. Kolodner, A Case-based Design Aid for Architecture, Artificial Intelligence in Design ‘92, Kluwer Academic, Dordoecht, Netherlands, 1992 pp. 497–516. S. Henninger, K. Lappala, A. Raghavendran, An organizational learning approach to domain analysis, Proceedings of Seventeenth International Conference on Software Engineering, ACM Press, New York, 1995 pp. 95–104. V. Bellotti, Integrating Theoreticians’ and Practitioners’ perspectives with design rationale, Proceedings of InterCHI ‘93, ACM Press, New York, 1993 pp. 101–114. J.L. Kolodner, Case-Based Reasoning, Morgan-Kaufman, San Mateo, USA, 1993. S. Slade, Case-based reasoning: a research paradigm, AI Magazine 12 (1991) 42–55. J.L. Kolodner, Improving human decision making through case-based decision aiding, AI Magazine 12 (1991) 52–68. M. Pearce, A.K. Goel, J.L. Kolodner, C. Zimring, L. Sentosa, R. Billington, Case-based design support: a case study in architectural design, IEEE Expert 7 (1992) 14–20. P. Senge, The Fifth Discipline: The Art and Practice of the Learning Organization, Currency Doubleday, New York, 1990. C., Argyris, D.A. Scho¨n, Organizational Learning: a Theory of Action Perspective, Addison-Wesley, Reading, MA, 1978. E.W. Stein, V. Zwass, Actualizing organizational memory with information systems, Information Systems Research 6 (1995) 85–117. J.S. Brown, P. Duguid, Organizational learning and communities-of-practice: toward a unified view of working learning and innovation, Organization Science 2 (1991) 40–57. L.G. Terveen, P.G. Selfridge, M.D. Long, ‘Living Design Memory’—framework implementation lessons learned, Human–Computer Interaction 10 (1995) 1–37. Y. Lu, Support for the Creation of Context-Specific Usability Guidelines, Computer Science and Engineering, University of Nebraska, Lincoln, 1997. T. Moran, J. Carroll, Design Rationale: Concepts, Techniques and Use, Lawrence Erlbaum Associates, Hillsdale, NJ, 1996. J. Conklin, M.L. Begeman, gIBIS: a hypertext tool for exploratory policy discussion, Transactions on Office Information Systems 6 (1989) 303–331. A. Maclean, V. Bellotti, R. Young, T. Moran, Questions, options and criteria: elements of design space analysis, Human–Computer Interaction 6 (1991) 201–251. G. Fischer, A.C. Lemke, R. McCall, A. Morch, Making argumentation serve design, Human–Computer Interaction 6 (1991) 393–419. H.W. Rittel, W. Kunz, Issues as Elements of Information Systems, Center for Planning and Development Research, University of California Berkeley Working Paper 131, 1970. J. Lee, SIBYL: a tool for managing group design rationale, Proceedings of the Conference on ComputerSupported Cooperative Work CSCW ‘90, ACM Press, New York, 1990 pp. 79–92.
S. Henninger / Interacting with Computers 12 (2000) 225–243
243
[46] E.J. Conklin, K. Yakelovic, A process-oriented approach to design rationale, Human–Computer Interaction 6 (1991) 357–391. [47] J. Grudin, Why CSCW Applications fail: problems in the design and evaluation of organizational interfaces Proceedings of the Conference on Computer-Supported Cooperative Work CSCW ‘88, ACM Press, New York, 1988 pp. 85–93. [48] S.R. Hiltz, M. Turoff, Structuring computer-mediated communication systems to avoid information overload, Communications of the ACM 28 (1985) 680–689. [49] G. Fischer, C. Stevens, Information access in complex, poorly structured information spaces, in: Proceedings of CHI’91, New Orleans, 1991, pp. 63–70. [50] N.J. Belkin, W.B. Croft, Information filtering and information retrieval: two sides of the same coin?, Communications of the ACM 35 (1992) 29–38. [51] G. Fischer, A.C. Lemke, T. Mastaglio, A.I. Morch, Critics: an emerging approach to knowledge-based human computer interaction, International Journal of Man–Machine Studies 35 (1991) 695–721. [52] M.C. Paulk, B. Curtis, M. Chrissis, C.V. Weber, Capability Maturity Model, Version 1.1, IEEE Software 10 (1993) 18–27. [53] J.P. Walsh, G.R. Ungson, Organizational memory, Academy of Management Review 16 (1991) 57–91. [54] L.M. Berlin, R. Jeffries, V.L. O’day, A. Paepcke, C. Wharton, Where did you put it? issues in the design and use of a group memory, Proceedings InterCHI ‘93, ACM Press, New York, 1993 pp. 23–30. [55] M.S. Ackerman, Answer Garden: a Tool for Growing Organizational Memory, MIT Sloan School of Management, Boston, USA, 1994. [56] M.S. Ackerman, T.W. Malone, Answer Garden: a tool for growing organizational memory, Proceedings of the Conference on Office Information Systems, ACM Press, New York, 1990 pp. 31–39. [57] S. Henninger, Capturing and formalizing best practices in a software development organization, The Ninth International Conference on Software Engineering and Knowledge Engineering (SEKE’97), Madrid, Spain, 1997 (in press). [58] L. Terveen, D. Wroblewski, A Collaborative Interface for Browsing and Editing Large Knowledge Bases, National Conference of the American Association for AI, AAAI Press, Boston, USA, 1990 pp. 491–496.