Intelligent interfaces: The IMPACT survey

,

Intelligent interfaces: the I M P A C T survey* Brian Sharratt

Logica Cambridge Ltd, Betjeman House, 104 Hills Road, Cambridfe, CB2 1LQ, UK The present study was commissioned by the I M P A C T programme of the CEC (DG XIII B/2) with the aim- of providing: (i) an overview of current research and development activity on intelligent interfaces; and (ii) support for the formulation of the I M P A C T II programme in the area of intelligent interfaces. The study consisted of a short review of intelligent interface research and a survey of current R&D projects in this area. The survey was conducted by phone and email and used a short questionnaire that addressed a project's experiences of developing an intelligent interface and general issues concerning the market for intelligent interfaces. The survey covered 43 R&D projects in Europe and the United States. Key Words: intelligent interfaces, intelligent assistant/apprentice systems, power tools, intelligent help systems knowledge-based systems.

1. I N T R O D U C T I O N

-

AIMS AND O B J E C T I V E S

The present study was undertaken for the Information Market Policy Action programme (IMPACT) of the Commission of the European Community (CEC). The I M P A C T programme is involved in the promotion of projects in the European information services market and covers key information areas such as patents, standards and tourism. The CEC is divided into a number of Directorate Generals (DGs) each with their own area of responsibility and the I M P A C T programme is part of D G XIII B/2. The aims of this study were to provide an overview of current research and development activity on intelligent interfaces which will be used to help formulate the I M P A C T programme's research strategy in this area. The study was undertaken between November 1989 and March 1990 and consisted of two components - a brief review of current research on intelligent interfaces and a survey of research projects working in this area. Essentially, the survey represented a quick horizontal slice taken through the current research and development activity on intelligent interfaces. In this survey the emphasis has been more on achieving a broad coverage rather than a detailed investigation of a few market areas. The objectives of this study are to: • •

•

Provide a brief overview of current research on intelligent interfaces. Survey research and development activity on intelligent interfaces within major pan-European research programmes (e.g. ESPRIT) and R&D programmes/projects within individual EEC member states (e.g. Alvey programme In UK) as well as a brief exploration of projects in USA. Briefly evaluate and categorize research activity in terms of: (i) the types of support provided by current

* The present study was funded by the IMPACT programme of the CEC. The views expressed in this paper the author's and are not necessarily shared by the IMPACT programme. Accepted October 1990. Discussion closes June 1991

.~) 1991 Computational Mechanics Publications

intelligent interfaces; (ii) the strengths and weaknesses of particular intelligent interfaces; (iii) the successes and failures of the research in the market. Identify research projects leading to commercial systems and what commercial systems are available. Identify key features preventing commercial exploitation of current research. The present paper is divided into four sections. Section 2 contains a review of research on intelligent interfaces. This review does three things - it describes the general structure of intelligent interfaces (in terms of the types of knowledge in the interface and the range of user support covered); it provides examples of intelligent interfaces taken from the current research literature; and it raises a few general issues for the development of intelligent interfaces. Section 3 describes the methodology used in the survey of research projects. The results of this survey are then presented in Section 4.

2. REVIEW O F I N T E L L I G E N T I N T E R F A C E S

2.1. Introduction to intelligent interfaces In recent years there have been a number of developments in computer interfaces, many of which have been aimed at reducing the disparity between the user's own organisation of a task and how that task is performed at the interface. This disparity has been tackled by making changes to the physical structure and representation of the interface and by introducing some form of user and task support. A wide variety of structural changes have been made ranging from the adoption of more meaningful command names to the production of grahical interfaces employing spatial input devices, such as mice and tablets. In addition some researchers and interface developers have addressed the issue of how users can be actively supported during the execution of their tasks by some form of intelligent interface (see Chignell and Hancock 2). The key features of these intelligent interfaces are the forms of knowledge they contain and what types of

Artificial Intelligence in Engineering, 1991, Vol 6, No 1 5

Intelligent interfaces." the I M P A C T survey." B. Sharratt support can be provided based on that knowledge. Rissland 11 has discussed both knowledge and support. She has identified seven types of knowledge: •

• •

•

•

•

•

User Knowledge - which covers the user's expertise with the computer system, their preferences for certain modes/default conditions and the history of their interaction with the system. Task Knowledge which covers the generalpurpose of user tasks and specific goals for individual tasks. Domain Knowledge which covers the general context surrounding the interface, the type of job undertaken by the user (e.g. librarian, accountant, secretary, etc.), the situation in which the interface is used (e.g. office, home or laboratory) and the objects and concepts employed in that domain. Application/Tool Knowledge which covers detailed knowledge of the application being used, e.g. the organisation and data types within particular databases and parameter settings for communication packages. Representation Knowledge which covers the ways in which users' input is handled (e.g. the use of typed commands, pointing devices or speech) and how this relates to the information presented on different output devices (e.g. feedback on the selection of protected objects in a graphical interface). Interaction Knowledge - which deals with how the interface monitors the users actions, the extent to which it intervenes during these actions and when relevant information is presented to the user (e.g. the interface may make a prompt intervention for certain types of crucial user errors). Evaluation Knowledge - which is concerned with evaluation of the current support provided and modification of that support (e.g. an assessment of whether interface interventions are slowing the task down and the restriction of such interventions).

These different types of knowledge may be used to provide various forms of user support. According to Rissland 11 an intelligent interface can: •

•

•

•

• •

Take over certain tasks from the user. These tasks are typically infrequent low-level configuration tasks (such as the setting of terminal characteristics) or routine tasks (e.g. creating backups of user files). Provide assistance with more complex tasks (e.g. inform the user of command sequences to be followed) Provide easy access to applications/tools within the computer system (e.g. present a common interface to a variety of different databases) Provide status information during tasks (e.g. indicating the position of valves in a process control application). Provide on-line assistance using help procedures that take into account the user's tasks and the form of errors made. Allow multi-tasking, where the interface monitors concurrent tasks undertaken at the interface (e.g. the routing of a number of aircraft through an airspace).

These descriptions of knowledge types and forms of support provide the basis for a general definition of an intelligent interface, which is:

6

Artificial Intelligence in Engineering, 1991, Vol 6, No 1

an interface which helps a user overcome their incomplete understanding of the computer system and, in this support role, employs knowledge of the user, the user's tasks, the task domain and the style of interaction required by different users. By its very nature this definition covers a range of intelligent interfaces which vary in: • •

the degree of support provided the extent of the user/task/domain contained in the interface.

knowledge

At its simplest an intelligent interface could support a restricted range of tasks and present an easy-to-use interface where many of the complexities of accessing and using an application (or set of applications) are hidden from the user. For example, an intelligent interface for database access could allow users to build limited queries using simple interactions, such as form filling or menu selection. Such an interface would handle: (i) the details of the log-on process ifa remote database is used; and (ii) the translation of user queries into the more complex query languages used by the database applications. More complex intelligent interfaces would contain extensive user/task/domain knowledge and provide a variety of user support. For example, an intelligent interface for office tasks could provide intelligent mail filters (that sort incoming email by sender or selected keywords or phrases) and allow task-based interaction between different applications and users (e.g. when figures in a spreadsheet are updated other related documents are also updated and the changes are emailed to the document's recipients).

2.2. Examples of intelligent interfaces Rissland has developed a broad view of intelligent interfaces and described the full range of features which can be included within such an interface. Specific projects have concentrated on certain application areas and types of user support. Examples of current research on intelligent interfaces which illustrate this specialisation are IDA (Intelligent Database Assistant - Jacobsen et al.V), POISE (Croft and Lefkowitz4), COKES (Carleton Office Knowledge Engineering System Kaye and Karam 9) and ACTIVIST (Active Help System Fischer Lemke and Schwab6). The IDA system is an intelligent database assistant which combines database expertise with an intelligent interface. The system provides a number of types of user support: •

• •

•

User-orientated communication - natural language dialogue and menu selection are used rather than specialised query languages. Automatic database selection - where IDA handles the selection of databases to handle queries rather than having the user make an explicit selection. Automatic query generation - where a correct query is formulated based on the user's natural language input and transformed into a database-specific query. This transformation is handled by a query planner implemented as a rule-based expert system. Portable access to different database systems - the system converts the user's query into a database-

Intelligent interfaces: the I M P A C T survey." B. Sharratt independent virtual query which is in turn transformed into a database-specific query. This staged transformation allows access to new or different databases as and when they become available. The interesting features of IDA are the types of knowledge used in the interface, the limitations of the current system and its relationship with knowledge-based systems. The emphasis in IDA is on simply enabling better database access and it relies on application/tool knowledge in conjunction with representation knowledge and some domain knowledge (the domain knowledge is used to aid the natural language parsing the construction of virtual queries). The system does not take account of different types of users, their tasks and the errors made during database query. Another important limitation is that the system deals with just the input side and little is done in the present system to present the information from different databases in a consistent and unified format. Finally, IDA operates as both an intelligent interface and a knowledge-based system. As an intelligent interface it helps with the formulation of correct queries and as knowledge-based system it maintains knowledge about the domain, the available databases and query formulation for these databases. A more user-centered form of intelligent interface is demonstrated by the POISE system. POISE acts as an intelligent assistant to users of an office system and provides a range of assistance from the automation of simple tasks to agenda maintenance and error checking with more complex tasks. The assistance is context dependent and the system maintains a database containing descriptions of the roles of different office workers, the tasks they perform, the available tools, the office domain and the current state of the user's activities. From this information POISE can recognise the tasks being currently undertaken, plan actions to complete that task and ask the user for input where necessary. Using this approach the system is able to automate routine tasks, describe courses of action to a user when decision points are reached in specific tasks and recognise actions that are syntactically correct but inappropriate in the current context. The C O K E S system is another intelligent interface for office systems and it makes a number of important extensions to the approach taken in POISE. The main extensions in C O K E S are the handing of concurrent user tasks, the integration of a computer-based message system and the shift in focus towards knowledge-based systems and higher levels of user support. In C O K E S the task structures can run in parallel and intertask communications are provided to enable their synchronisation. This allows the system to support the resumption of previously suspended tasks as well as tasks distributed across different users. With these distributed tasks the message system is used by C O K E S to ensure user coordination and the system sends requests for information or contributions to users involved in such tasks. For example, C O K E S can support the production of technical progress reports requiring multiple inputs from different engineers by requesting specific information from these users at the appropriate time. Within C O K E S a knowledge-based system is used to handle the organisation and coordination of quite high-level tasks undertaken by one or more users. The emphasis is on supporting this coordination rather than providing an

intelligent interface for more detailed tasks using specific applications such as editors or spreadsheets. POISE and C O K E S are systems which provide the user with practical assistance during the execution of tasks. Their success in supporting office workers depends on the accuracy and integration of their user, task and domain knowledge. To recognise users' intentions they require detailed knowledge of the range of tasks in the domain, the different ways in which these tasks can be performed and the ways adopted by different users. Gaps in these forms of knowledge can, at best, leave users unsupported for certain tasks or, at worst, mean that the user is presented with very inapropriate forms of support. The ACTIVIST system contrasts with the other intelligent interfaces mentioned above in that it concentrates on one particular type of user support. ACTIVIST is concerned with active on-line help, where the system monitors user actions at the interface, detects problematic user behaviour and provides advice on correct actions. The system contains detailed task plans for the domain of operation of the application being supported. These plans are used to detect specific users errors and Sub-optimal user actions, i.e. situations where a whole sequence of commands issued by the user which could be replaced by a smaller set of more complex commands. The system evaluates these errors and sub-optimal actions and presents the user with advice on how to avoid these problems. The important features of the ACTIVIST system are the need for a variety of knowledge types to deliver a single form of user support and the development of this approach in more advanced help systems. To provide its active help the ACTIVIST requires a combination of task, user and application knowledge (to detect and record individual user problems) coupled with interaction and evaluation knowledge (to enable the presentation of suitable help at the correct point in the user's task). ACTIVIST has, in part, formed the basis for work on more general help systems (e.g. E U R O H E L P - see Stehouwer and Bruggen 12) and help systems providing a mixture of "how-to-do-it" and "how-it-works" help (SmartHelp system see Carroll and Aeronson 1). As with POISE and COKES, the ACTIVIST system's operation depends on having accurate knowledge of user, task and application which enables correct plan recognition. A more general weakness of ACTIVIST and other help systems are their lack of integration with other forms of user support.

2.3. General issues for intelligent interfaces Four general issues are raised by the research and development of intelligent interfaces. These issues are:

1. The relationship between intelligent interfaces and knowledge-based systems The brief descriptions of IDA and COKES in the previous section illustrate the close relationship between intelligent interfaces and knowledge-based systems. This relationship may be represented as a continuum. At one end of the continuum are systems where the user support is just provided by the intelligent interface and that interface contains some components of a knowledgebased system. Typically these systems have intelligent interfaces grafted onto established applications. At the other end of the continuum are full knowledge-based systems where the majority of user support comes from

Artificial Intelligence in Engineering, 1991, Vol 6, No 1 7

Intelligent interfaces: the I M P A C T survey: B. Sharratt the underlying system rather than being a function of the interface. In this case the interface may provide some extra user support such as on-line help. For the systems in the middle of this continuum, where intelligence is divided between the interface and the application, there is a division of labour problem in which the user support (and stored knowledge to provide this support) has to be split between interface and application, and a close coupling has to be achieved between the two parts of the system.

2. Fundamental problems identified in current research In their discussions of intelligent interfaces Rissland 1 and Chignell and Hancock 2 raise a number of problems facing the development of these interfaces. These problems are: •

•

•

•

8

Control - a central issue is who takes control of the interaction and when is that control is exercised. Whilst control has to be shared between user and system the balance depends on the user and the task being performed. For novice users the interface may control most of the interaction, taking charge of routine tasks and directing users in the steps required for more complex tasks. Experienced users may wish to have more control over the configuration of the interface and the handling of tasks. In this situation the interface operates as a tool enabling the user to structure the task environment. The control issue leads to a distinction between intelligent interfaces which operate as an "intelligent apprentice" and those which are more like a "power tool" (see Neches et al.l°; the tool perspective is discussed at greater length in Kammersgaard 8. The intelligent interfaces discussed in Section 3.2 are very much "intelligent apprentice" type systems for the less experienced user. Communication - one of the main aims of an intelligent interface is to improve the communication between system and user. This requires a change in the form of both intput and output. Current research has focused on making the input more user orientated by the use of natural language, menus and direct manipulation. Apart from the area of intelligent help systems, less attention has been paid to making the output understandable by the user and appropriate to the tasks being undertaken. Adaptation a feature of intelligent interfaces discussed by Rissland is the capacity to change over time as the user becomes more experienced with the system and undertakes more advanced tasks. Adaptation requires that the user's performance be monitored and a set of interface changes be associated with certain performance levels. Adaptation adds further complications to the structure and operation of intelligent interfaces. An example of adaptation is provided by POISE, where the system's task descriptions contain information on the preferred task methods for individual users and this is used to customise the system for each user (Croft4). Knowledge C a p t u r e - the operation of an intelligent interface depends on the knowledge contained in the interface. To recognise the tasks performed by users and provide the necessary support this knowledge has to be both comprehensive and accurate. Identifying this knowledge and representing it in the

Artificial Intelligence in Engineering, 1991, Vol 6, No 1

interface is an important part of intelligent interface development. This knowledge capture can be expensive in terms of time and effort required.

3. The application of intelligent interfaces The application of intelligent interfaces raises the question of when such interfaces are appropriate. Three important factors influence the use of intelligent interfaces for current applications. These factors are: 1.

2.

3.

Task complexity - the user's task should require several distinct operations. There is little point in developing an intelligent interface when the task only contains a small number of operations which can be easily learnt by users. Interface complexity - the interface is adding to the task complexity by forcing the user to engage in extra operation not directly related to the user's tasks. For example, lengthy log-on sequences to connect with remote databases is an extra burden on the user whose task may be to simply cross check two or more sets of information. Existing task support - the interface provides little task support and the user has to perform the majority of task operations.

High task and interface complexity coupled with little task support can lead to: • • •

users failing to understand the task as it is structured at the interface the need for large amounts of training significant number of user errors occurring after training (Chignell and Hancock2).

In such situations an intelligent interface can help to reduce or remove these problems. For new applications/ systems the same factors apply, in that the potential task and interface complexity are high and little task support would lead to problems with user understanding, training and errors.

4. The benefits of intelligent interfaces There are a range of practical benefits associated with intelligent interfaces. The benefits of such interfaces are that they: •

•

lead to a reduction in training time and user errors; enable a wider user population to use the system (i.e. users with different backgrounds and levels of computer experience) allow better access to the underlying functionality of the system.

When intelligent interfaces are applied to existing applications they can transform the usability of the application, enable it to be integrated with other applications under a common interface (e.g. an intelligent interface combining both database access and other applications) and introduce these applications to new users. For new systems an intelligent interface would help ensure that the system has high usability and provides good support during the user's initial learning phase.

Intelligent interfaces: the I M P A C T survey: B. Sharratt 3. SURVEY O F I N T E L L I G E N T I N T E R F A C E S R&D ACTIVITY The survey consisted of a short questionnaire presented by phone or e-mail to key individuals on research projects concerned with the development of intelligent user interfaces. These research projects were identified using published information on pan-European research projects (e.g. Esprit'89) and personal contacts in various universities and industrial research centres. The questionnaire consisted of a general definition of intelligent interfaces, a request for a brief description of the project or projects involving intelligent interfaces, followed by project-specific and market-orientated questions. From our initial discussions with researchers working in the area it was clear that there was some confusion over what an intelligent interface was. A broad definition based on Rissland 11 was presented which covered the types of knowledge which could be contained in the interface and the forms of user support which could be provided. This definition ensured that a common understanding of what constituted an intelligent interface was achieved and respondents could identify the forms of support provided by their interface (see Appendix A for the full questionnaire). A series of eight project-specific questions were used. These questions addressed the following issues: • •

•

•

•

•

•

Whether any commercial products were developed during the project. Which commercial products were used for interface development (this question dealt with the use of programming language and more specialised software development tools). Whether any problems were encountered during interface development (a problem was defined as any part of the interface that required extra effort/ resources or any feature that proved difficult to build). Whether anything was easy or straightforward to do (i.e. features that were easy to develop or were based on existing software/research). Whether any standards were used during the project (these standards covered both technical standards for software development and user interface guidelines and standards). Whether the interface has been evaluated using formal testing (i.e. planned testing with defined user groups) or informal testing during the development process, and what the general results were. Other projects or people who should be contacted as part of this survey.

Three market-orientated questions were used. These questions dealt with more general issues to do with the development of the market for intelligent interfaces and covered: • • •

The use and impact of new technologies, such as C D - R O M and new input devices, on intelligent interfaces. The market for intelligent interfaces in terms of its current size, rate of growth and future size. The general problems facing the development of intelligent interfaces and the process of bringing them to the marketplace.

The questionnaire concluded with a request for any general comments concerning this survey. A number of comments were received concerning the I M P A C T programme and the publication of the survey. These comments are discussed at the end of the next section. Whilst every attempt was made to find people who could give a good overview of their particular projects, in some cases the responses to the project-specific questions were personal observations, i.e. these views may not be shared by project partners and may not reflect their employers view on intelligent interfaces. However, the nature of the market-orientated questions meant that all these responses have to be treated as personal observations.

4. E V A L U A T I O N O F I N T E L L I G E N T INTERFACES This section describes the projects covered in the survey, presents the general findings for project-specific questions and market-orientated questions, and discusses some of the more general issues raised by the survey. In the sections dealing with project-specific and marketorientated questions the aim has been to concentrate on the general positions reflected in the responses rather than provide a very detailed breakdown of responses made.

The projects covered in the survey The survey obtained responses from 35 people (mainly researchers and system developers) working in the area of intelligent interfaces*. Some of these people were involved in more than one intelligent interface project and as a result the survey covered 43 projects (the list of people/projects in the survey are shown in Fig. 1). These projects were divided into two types: •

•

Application-focused - these projects were developing intelligent interfaces with clear applications in mind. For example, the production of an intelligent interface for particular statistics packages and programming language libraries. Research-focused - these projects were concerned with more long term research on the components used in intelligent interfaces and their operation within the interface. For example, research on the provision of intelligent help systems that can then be tailored for different applications.

A general breakdown of application- and researchfocused projects in the survey is shown in Fig. 2. In Fig. 2 the application-focused projects are categorised by application area and the research-focused projects are categorised by the interface features being researched. These categorisations were simply based on the project descriptions provided at the start of the questionnaire. In some cases the categorisations are complementary in that they deal with the same basic form of user support (e.g. advice systems-advisory situations and speech systems/speech I/O) but they differ in their focus on specific applications or more general research.

* Due to time constraints two people did not go through the full questionnaire. They just gave a brief synopsis of their intelligent interface project.

Artificial Intelligence in Engineering, 1991, Vol 6, No 1 9

Intelligent interfaces." the I M P A C T survey." B. Sharratt

ProjectName

Project Funding ,Project Type ADKMS ESPRIT 1 A - databases HUFIT ESPRIT 1 IR - speech I/O SPICOS Philips/Siemens JP IR - speech I/O Reading Board IPOIP R- speech I/O CRI ITSIE E. Holnagel ESPRIT 2 A - CBT Exp. Operators Assistant ESA IP A - process control T. Weaver Philips~ UK Multiple Agent Systems Vhilips n, R - intell,agents BEES Philips IP R - expert systems Philips IP Intelligent C D I A -CDI ESPRIT 2 FOCUS A - lib/slats, packages R. Isles NAG Philips, NL ESPRIT 2 A - lib/slats, packages M. Brouwer FOCUS EUROHELP ESPRIT 1 ICL Plc A - advice systems M. Smith PROMISE ESPRIT 2 A - process control P. Puncello Tecsiel S.A. J. Ilion SUNSTAR ESPRIT 2 R - speech I/O AEG Int. Interactive Home Damlier-Benz IP R - graphical sys. R. Oed Damlier-Benz ESPRIT 1 R - multi-media Multiworks ESPRIT1 R - graphical sys. M. Jarke Passau University DAIDA Multiquest France-Canada JP A - databases S. Soudoplatos CAP Sogeti USIS EUREKA A - design tools A - simul./modelling SERC ~tJK) D. Robertson Edinburgh Uni. ECO D. MacRandal Rutherford Labs/ Intelligent Frontend for CABD Rutherford Labs JP A- simul./modelling Strathclyde Uni. Medical DB iAIM Initative A - databases I. Dezegler BIM S.A. AIM Inilative Leeds University AIDMED A - databases M. Howes MMI2 ESPRIT 2 R - expert systems BIM S.A. D. Sedlock A - proj. management PIMS ESPRIT 2 J. Paris CAP Sogcti ESTEAM ESPRIT 1 A - advice systems T. Grossi C A P Sogeti British Telecom IP A - databases J. Noseworthy British Telecom Product Selection DB DEC IP A - databases Intelligent DB Queries P. Coutier DEC Valbonne R - embedded intell. Embedded Intelligence Projects DEC IPs J. Barrett DEC Reading BP Research Material Selection DB BP IP A - databases M. Davies BP Research Communications Network BP IP A - simulJmodelling P. Wilhelmaj R - advisory situations ESPRIT 1 EUROHELP B. Koza Axion A.S. A - process control GRADIENT ESPRIT 1 G. Weir Strathclyde Uni. A - databases GTEIP IDA/CALIDA G. Jacobsen GTE Lab% USA A - databases TomelP T o m e Associates Tome Searcher A. Vickery R - advisory situations HITS MCC IP W.Hill MCCI,USA N Y N E X IP A - speech system Voice Messaging System NYNEX, USA M. Atwood A - CBT N Y N E X IP Cobol Intell. tutoring System A - design tools B B N IP B. Papazian BBN Labs~ USA II for Interface Design A - databases Nordic Finances IANI T. Andersen Axion A.S. A - advice systems Conversational Advisory Sys. CSI IP T. Wolf CSI~ USA !A - office systems Natural Language Gen. system CSI IP A - databases INFOTAP IP Intelligent Information B. Mahon INFOTAP R - coop. systems various Coop Problem Solving G. Fischer Colorado Uni. A - application focused R - research focused JP - joint project Key: IP - internal project Person J. Samuel D. Bowhuis

Fig. 1.

Organisation Nixdorf IPO

Contacts and project list

For details of the allocation of projects to these categories see Fig. 1. Figure 2 shows that the survey has concentrated on application-focused projects. Within this group the main application area has been databases, followed by process control, advisory systems, CBT, design tools and simulation/modelling. The research-focused projects

10

Artificial Intelligence in Engineering, 1991, Vol 6, No 1

cover a broad spectrum from I/O concerns for intelligent interfaces through to study of the structure of intelligent systems.

4.2 Findings from project-specific questions The project-specific questions produced a number of interesting findings. For clarity, these findings are

Intelligent interfaces: the I M P A C T survey: B. Sharratt .

presented on a question-by-question basis (see Appendix A for the actual questions asked). The findings were:

•

Development of intelligent interface products

The projects covered in this survey were categorised into four types: 1. No product plans these projects have just concentrated on the development of prototype intelligent interfaces; 2. Possible Product - these projects would like to develop products but they do not have, or are not prepared to discuss, any product plants; 3. Restricted Product - these projects have produced a product for either a single customer for their in-house use. This category was further sub-divided into: 3a product released and being used; and 3b - release of product anticipated;

For projects in categories 3b and 4b details of the anticipated release of products were not revealed by respondents, but from the types of responses given these products may be expected in either the short term (i.e. a prototype intelligent interface has been completed and is being converted into a product) or to medium term (i.e. further development work is required before product release within the next few years). The projects in categories 2, 3 and 4, and their application areas are shown in Fig. 3 (the application areas are taken from the Project Type column in Fig. 1). Two important issues raised by the projects shown in Fig. 3 are: • •

Application-focused Projects (n=29)

Databases Process Control

11 3 CDI 1 Lib./Stats Packages 1 CBT 2 Advice Systems 3 Office Systems 1 Design Tools 2 Simulation/Modelling 3 Project Management 1 Speech Systems 1

Fig. 2.

Research-Focused Projects (n=14) Speech I/O

4

Intelligent Agents/ Embedded Intelligence Expert Systems Multi-Media Graphical Systems Advisory Situations Cooperative Systems

2 2 1 2 2 1

Breakdown of projects in survey

Product Status General Market Product (released)

General Market Product (anticipated)

Restricted Product (released)

Restricted Product (anticipated) Possible Product Fig. 3.

General Market Product - these projects have products available in the marketplace. This category was also sub-divided into: 4a - product released and being used; and 4b - release of product anticipated.

the application areas covered by released and anticipated products the nature of the intelligent interfaces provided in released products.

The main application areas for released products are databases and advisory systems. Anticipated products address a wider range of applications, such as process control, CDI and multi-media, however, the expected release dates for these products is uncertain. From the descriptions provided in the survey, the released General Market Products tend to contain fairly simple intelligent interfaces to specific application areas, i.e. the interface supports a restricted range of tasks for set applications and hides much of the complexities of accessing and using those applications from the user (see also Section 2.1). In contrast the released Restricted Products tend to demonstrate more complex intelligent

Project

Application Area

Tome Searcher Voice Messaging System Intell Tutoring System IANI Intelligent Information ADKMS HUFIT Intelligent CDI FOCUS Mulfiworks Material Selection DB Communications Network IDA/CALIDA Conver. Advisory Sys. N.l_amguage Gen. System ITSIE Exp. Operators Assistant Multiple Agent Systems BEES HITS

Databases Speech system CBT Databases Databases Databases Speech I/O CDI Lib/stats. packages Multi-media

Databases Advisory systems Databases Advisor,/systems Office systems CBT Process control Intelligent agents Expert systems Advisory situations

Product status

Artificial Intelligence in Engineering, 1991, Vol 6, No 1

11

Intelligent interfaces." the IMPACT survey: B. Sharratt interfaces offering varied forms of user support. These Restricted Products also appear to be making some progress towards the development of generic intelligent interface products (i.e. interface products which are separable from the underlying application and can be used with other applications). Of the products mentioned, the one closest to being a generic product was American, and was the CALIDA project (where G T E have used their IDA system as a kernel for the development of intelligent database interfaces for marketing). However, it should be noted that the development and use of generic intelligent interfaces is still some way off and more experience has to be gained building and using simple intelligent interfaces tied to specific applications. •

Use of commercial products

A small range of commercial products or tools have been used on the intelligent interface projects. The most common element was Prolog, which was used by over half the projects. A number of projects used more specialised development environments (such as KEE, LOOPS, Flavours, Interlisp-D) mainly those in product categories 2 (Possible Product), 3 (Restricted Product) and 4 (General Market Product). In addition there was a smaller collection of projects using object orientated tools (such as CLOS and CLUE). •

General problems encountered

Four general problems were highlighted in the survey: 1.

2.

3.

4.

Separation between intelligent interfaces and knowledge-based systems - this was the main problem encountered in the survey. Those projects using an intelligent interface in conjunction with a knowledgebased system had difficulties in achieving a suitable separation between the interface and underlying knowledge-based system in terms of the knowledge stored and its use. For example, both the interface and the knowledge-based system require knowledge concerning the user and tasks undertaken. There is often an overlap between the storage and processing of this knowledge in the interface and the underlying system. This then leads to control and communication problems when the interface is in operation. Problems with requirements stage and knowledge engineering - a couple of people emphasised the difficulty of specifying the types of knowledge required in the interface and determining (i) the form of intelligent interface and (ii) the levels of user support needed for different user populations. Difficulties in developing graphical interfaces on top of the intelligent interface software - in some cases the developers found that the interface tool kits were at too low a level and a lot of work was needed to produce a suitable interface. The projects experiencing these difficulties were those who were not using the more specialised programming environments mentioned above. Software translation problems - a few projects encountered a number of problems when they moved between different programming language, such as changing between types of Prolog. The first two _problems are more theoretical in nature

12

Artificial Intelligence in Engineering, 1991, Vol 6, No 1

and reflect the need for a clear understanding of what forms of user support the interface will deliver, the types of users the system is designed for and how these requirements are captured in a workng system. The other two problems represent more general software issues which occur in other types of software development projects. •

Easier aspects of intelligent interface development

The almost universal response was that intelligent interfaces are a difficult area to work in and that there are few things which are easy and straightforward to do. However, two related responses that were particularly interesting were: (i) once the intelligent interface has been developed, customising it for different user groups was relatively straightforward; and (ii) once an intelligent interface has been shown to users, they provide numerous suggestions for interface improvement. This suggests that the fundamental problem is the initial provision of an intelligent interface. After that has been achieved further developments or enhancements become much easier. Therefore, interface adaptation (raised in Section 2.3) may not prove such a problem once the basic forms of user support have been provided. •

Use of standards

The main finding was there was a noticeable absence of standards in the projects surveyed. In some cases standards relating to the underlying application (e.g. the use of SQL for databases) or communications (e.g. the X.400 and X.500 standards for communication protocols) were required. A few commercial organisations had their own internal HC1 standards but, from the descriptions given, these seem to apply to more general HC1 issues rather than being tailored towards intelligent interfaces. The interface style guidelines in Open Look and Motif were currently being used or evaluated by a few projects tackling advice systems, design tools and process control. •

Evaluation of intelligent interfaces

Interface evaluation was an integral component of most of the projects surveyed. The majority of evaluations (either planned or completed) were informal and whose purpose was to refine the interface during the development process. Despite the widespread occurrence of evaluation few results were reported. Whilst the potential advantages of intelligent interfaces have been emphasised in the research literature (see Section 2.1) evidence of their actual benefits is lacking.

4.3. Findings from market-orientated questions The market-orientated questions addressed to the market for intelligent interface products, the problems facing the development of these products and the impact of new technologies on intelligent interfaces (again see Appendix A for the questions asked). Generally speaking the potential market was seen as huge. One respondent even went so far as to predict that within 10 years all industrial control systems will use intelligent interfaces. However, views on the current size of the market and the rate of growth were far more uncertain. The current market (i.e. the number and range of intelligent interfaces in place) was seen as very small and specialised. The rate

Intelligent interfaces: the I M P A C T survey: B. Sharratt of growth of the market produced very divergent views respondents envisaged either slow growth or very rapid growth. The people anticipating slow growth often raised various technological barriers to growth, such as need for better speech processing and specialised hardware. In addition a few people cited an awareness problem (i.e. how intelligent interfaces can be applied to existing applications and what is currently possible) as a delaying factor for market growth. The perceived problems facing product development fell into five groups:

speech output is required. Important hardware issues were seen as the introduction of ISDN (to enable higher bandwidth communication between machines) and the need for faster/special purpose processors for intelligent interfaces.

4.4. General issues and overall impression of results A number of general issues are raised by the survey per se as well as the results obtained. When conducting the survey two features were very apparent: •

•

•

•

•

•

Interface construction problems - which covered the separation of the interface from underlying applications, the shortage of tools for developing these interfaces, the difficulty of acquiring knowledge of users/tasks and modelling this knowledge in the interface and how to ensure robust and reliable intelligent interfaces; Standards - two types of standards were mentioned, those relating to the usability of the interface (i.e. setting the levels of user performance which can be achieved when different types of support are provided by the intelligent interface) and standards for interface structure (i.e. standards for the interconnection between components of the interface and between the interface and the underlying applications). Hardware issues - these focused on the processing power required and hardware costs - a few respondents saw the need for delivery of intelligent interfaces on cheaper hardware than the high performance workstations currently being used; Demonstrable benefits - a number of people felt that the costly hardware and long development times needed justification, and that actual benefits of intelligent interfaces remain largely unproven. As one respondent put it consumers are interested in whole products (i.e. interfaces and applications) with clear benefits and not just interfaces with potential benefits. The inherent conservatism within industry - a few people pointed out that the inherent conservatism in many industrial areas will cause a slow introduction of intelligent interfaces. The two factors mentioned as possible reducing this conservatism were the demand from end-users for more suitable interfaces and the demonstration of good working interfaces in key market areas. One key market area raised was office systems.

Statements concerning the impact of new technologies tended to be very general and raised the issues of input/ output modalities, multi-media/graphical interfaces, the role of C D - R O M / C D I and hardware developments. Responses concerning modalities emphasised how speech input and output would have a strong-influence on future intelligent interfaces. This coupled with the introduction of multi-media and graphical interfaces were seen as having a strong impact as they increased interface versatility, i.e. they allowed users a broader range interaction with the system. The few comments on C D - R O M / C D I stressed two features: (i) their important potential role in the information services/database area; and (ii) the need for such devices to reduce processing and storage overheads in situations where large volume

Intelligent interfaces as a unifying concept - the term "intelligent interface" was generally not well understood. It was only after a definition was provided (see Section 3.1) that a lot of the people could see how their particular interface operated as an intelligent interface. The basic problem revolved around "intelligence" being seen as some deep and complex understanding of user and task captured in the interface. Whereas the definition presented at the start of the survey concentrated on the types of practical support provided and how these rely on some basic knowledge of user, task and domain (see Appendix A). Support for intelligent interface research - most of the people contacted as part of this survey were not aware that funding initiatives such the I M P A C T programme existed. In the case of some of the AIM (Artificial Intelligence in Medicine) projects they commented on how relevant the I M P A C T programme would be for the later stages of their project.

•

Furthermore, the nature of the survey needs to be kept in mind when interpreting the results presented. The survey is horizontal in nature, i.e. it went for a broad coverage of research and development activity on intelligent interfaces rather than take a vertical slice through specific market areas. The survey therefore provides a broad perspective on the current use of intelligent interfaces rather than take a vertical slice through specific market areas. The main impressions of the survey and important consequences arising from these impressions are: •

•

•

The current intelligent interfaces are largely specialised single systems. They are specialised in that they: (i) provide a subset of the possible range of user assistance which may be delivered by an intelligent user interface; and (ii) are tied to specific applications. At present these specialised single systems deliver quite limited forms of intelligent interface but they can form the basis for more widespread use of intelligent interfaces (i.e. they are changed and used for new applications). Attention is beginning to shift towards generic intelligent interfaces. This work is mainly driven by the extension of "one-off" interfaces into generic interfaces. This form of product development will play a key role as it wll enable the construction of a common intelligent interface for a range of related applications or an interface that bridges different underlying applications. Such products then provide an important platform for the development of generic intelligent interfaces. Standards for intelligent interfaces are lacking.

Artificial Intelligence in Engineering, 1991, Vol 6, No 1

13

Intelligent interfaces." the I M P A C T survey: B. Sharratt

•

•

•

•

•

•

If standards were present they would form a framework for intelligent interface development within which some consistency could be introduced to both the presentation styles used by intelligent interfaces (i.e. the delivery of the user support) and the connection between interfaces and the underlying applications. Generic intelligent interfaces are still some way off. However, the emergence of generic interfaces needs to be considered as a result the question is raised of whether the impetus for such interfaces will come from either research-focused projects or from market forces. The market forces could provide this impetus and generic interfaces could emerge in areas where good standards exist for underlying applications and the tasks/users are well understood. In many ways the information services market fulfils these requirements and can provide the arena for development of generic intelligent interfaces. The projects are mainly working on "intelligent assistant" type interfaces. These projects are encountering numerous problems with interface control and provision of user support (see Section 3.3). Clearly there is a need for more work on intelligent interfaces that operate as a "power tool", where users have the interface configured to the tasks they undertake and they maintain control over the interface operation. The reduction of control problems in "power tool" interfaces means that they are easier to develop as they do not require complex monitoring of user tasks and mechanisms for choosing when to offer assistance. "Over engineering" is a risk in this product area. The responses for some intelligent assistant interfaces indicated an element of "over engineering" (i.e. they store extensive knowledge about user, task and domain which has little bearing on the level of user support provided) and to redress this imbalance attention should be focussed on "power tools". Separability problems between interface and knowledge-based system. In many ways a clear 'division of labour" is needed between the intelligent interface and the knowledgebased system. The present difficulties in this area mean that short term developments in intelligent interfaces will probably occur in systems that do not have the added complication of a large underlying knowledge-based system. Good user involvement is essential in intelligent interface development. The more successful projects surveyed contained good user involvement for both good requirements capture and the delivery of suitable forms of support. Good demonstrations of the actual benefits of intelligent interfaces are lacking. The work on evaluation is largely aimed at improving particular components of the interface rather than showing the higher level of usability obtained compared with more standard interfaces. The demonstration of these benefits to potential purchasers will be a strong influence on the take up of intelligent interfaces in the marketplace.

GLOSSARY active help systems - on-line help systems that not only

14

Artificial Intelligence in Eng&eering, 1991, Vol 6, No 1

respond to users errors but also notice sub-optimal actions (i.e. those user actions which can be replaced by a shorter, more efficient sequence) and provide appropriate suggestions of more efficient command sequences (see Fischer et al.6). AIM - Artificial Intelligence in Medicine, a CECX initiative. CBT Computer-Based Training, where the delivery of training material is via a computer system which can be used to present text, graphics and audio material. CDI - Compact Disc Interactive, a standard developed by Philips, Matsushita and Sony that allows a flexible mix of audio, video, text, graphics, cartoon animation, computer data and still pictures to be stored on a disc that is the same size as a domestic audio compact disc. CD-ROM Compact Disc-Read Only Memory, is related to the domestic audio compact disc but stores information that can be read by a computer using a C D - R O M reader. concurrent user tasks these are groups of user tasks that are undertaken together and the user switches between individual tasks when necessary. They may include both computer-based tasks, which can be represented by collections of applications (such a wordprocessors, graphics packages and spreadsheets) open in different windows on the computer screen, and other noncomputer tasks (such as dealing with telephone queries and searching printed information). context dependent assistance - the assistance provided by the intelligent interface is appropriate to the tasks being undertaken by the user and any difficulties encountered. For example, an intelligent help system could recognise the error being made and tailor its help messages to that situation. domain knowledge - knowledge about the general context surrounding the interface, the type of job undertaken by the user, the work situation in which the interface is used and the range of objects and concepts employed in the domain. generic intelligent interface- an intelligent interface which is not tied to a specific application (e.g. a particular wordprocessor) and can be used in conjunction with other applications with the minimum of change. "how-it-works" help - messages provided by an on-line help system that stress objectives and goals, and avoid mentioning the names of specific commands or operations. These messages are intended to provide model-based explanations of errors. For example, for the user error of not knowing what to put for a file name a "how-it-works" message might be: "Before create file, you must name it. This is a list of files that already exist. You must choose a different name of up to eight characters. The first part of the filename has been filled in for you." (see Carroll and Aaronsonl.) "how-to-do-it" help - messages provided by an on-line help systems that are strictly procedural. For example, for the above file-name user error a "how-to-do-it" message might be: "Type a new filename (of up to eight characters) where it says Drive or file name" (again see Carroll and Aaronson 1). intelligent apprentice - see intelligent assistant. intelligent assistant - is a system that provides a user with full access to a backend system but uses knowledge-based techniques to provide them with support in formulating requests and interpreting responses from the system. intelligent help s y s t e m s - on-line help systems that provide

Intelligent interfaces: the I M P A C T survey." B. Sharratt

both active help (see above) and passive help (presentation of standard text messages usually taken from user manuals) and are sensitive to the experience level of the user. Intelligent help systems aim to expand the knowledge and skills of the user in order to p r o m o t e efficient use of the full functionality of the system (see Stehouer and Bruggen12). ISDN Integrated Systems Digital Network, used for simultaneous audio and slow m o t i o n video uses 2 digital channels and a third channel reserved for control. Motif a user interface style developed for the O p e n Software F o u n d a t i o n , which is compatible with OS/2's Presentation Manager. Open Look - similar to Motif but developed by Sun, A T & T and Xerox. It is more prescriptive than Motif in that it gives detailed interface style conventions. over-engineering - where the interface or application contains a range of facilities well b e y o n d those required by the user population. power tool - an interface to an application that provides the user with a powerful c o m m a n d set tailored to the tasks that user performs (see Neches et al.l°). Power tools are usually aimed at more experienced users who wish to perform complex tasks in an efficient manner. task knowledge knowledge a b o u t the general purpose of user tasks and the specific goals for individual tasks. user knowledge knowledge which covers the user's expertise with the c o m p u t e r system, their preferences for certain modes/default conditions and their interaction with the system. X.400/X.500 C C I T T telecom standards, with X.400 dealing with message passing and X.500 dealing with application standards and directory services.

ACKNOWLEDGEMENTS

I wish to thank Ian Clowes for his insight into this research area and his n u m e r o u s comments on early versions of this paper. I also wish to thank Terry Casilli and Geoffrey Stephenson of the I M P A C T p r o g r a m m e for their help in setting up this survey and their review of the initial findings. Finally m y thanks go to all the contributors to this survey.

9 10 11 12

APPENDIX A

2

3 4 5 6 7 8

Carroll, J. M. and Aaronsn, A. P. Learning by doing with simulated intelligent help, Communications o[' the ACM, 1988, 31, 1064 1079 Chignell, M. H. and Hancock, P. A. An introduction to intelligent interfaces. In Hancock, P. A. and Chignell, M. H. Intelligent Interjaces: Theory, Research and Design, Elsevier Science Publishers: Amsterdam, 1989, 1 26 Croft, W. B. The role of context and adaptation in user interfaces, Int. J. Man-Machine Studies, 21, 1984, 283-292 Croft, W. B. and Lefkowitz, L. S. Task support in an office system, ACM Trans. Office Information Systems, 1984, 2, 197 212 ESPRIT'89, Proceedings of the 6th Annual ESPRIT Conference, Brussels, Nov 27th-Dec ! st, 1989. Kluwer Academic Publishers: Dordrecht Fischer,G., Lemke, A. and Schwab, T. Knowledge-based help system, Proceedings CH1'85, ACM, 1985, 161 167 Jacobson, G., Laford, C., Nyberg, E. and Piatetsky-Shapiro, G. An intelligent database assistant, 1EEE Expert, Summer Issue, 1986, 65 79 Kammersgaard, J. Four different perspectives on humancomputer interaction, Int. J. Man-Machine Studies, 1988, 28, 343 362

QUESTIONNAIRE STRUCTURE

Intelligent interfaces survey The HCI group at Logica Cambridge are conducting a survey of R & D work on Intelligent User Interfaces (IUIs) fr the I M P A C T (Information Market Policy Action) p r o g r a m m e of the Commission of the E u r o p e a n C o m m u n i t y ( D G X I I I B). The objectives of the survey are to:

1.

2. 3.

Provide an overview of current research on I U I s within major p a n - E u r o p e a n research p r o g r a m m e s (such as E S P R I T ) and R & D projects within E E C member states, as well as recent developments in USA. Identify work that is leading to products (i.e. tools to build I U I s or commercial IUIs). Identify features preventing further development and exploitation of current R & D .

The time available for this survey is very restricted and it is being conducted via email and phone, and we are hoping to gather the necessary information by mid-January 1990. With this timescale a p r o m p t reply would help. Definition o f I U I We are taking a very b r o a d definition of I U I and are looking at interfaces that contain (to some degree) knowledge of: the user, the user's tasks, the task d o m a i n and the style of interaction required by different users. An I U I could function as an intelligent assistant and provide services such as:

handling of routine tasks assistance for more complex tasks allow easy access to tools provide status information to aid task progress deliver on-line assistance/help support multi-tasking

REFERENCES

1

Kaye, A. R. and Karam, G. M. Cooperating knowledge-based assistants for the office,A CM Trans. Office Information Systems, 1987 5, 297-326 Neches,R., Seely Brown, J., Malone, T., Sondheimer, N. and Williams, M. Intelligence in interfaces, Proceedings CHI'87, ACM, 1987, 267-269 Rissland,E. L. Ingredients of intelligent user interfaces, Int. J. Man-Machine Studies, 1984, 21, 377-388 Stehouwer,M. and Van Bruggen, J. Performance interpretation in an intelligent help system. In ESPRIT'89, op. cit., 248-257

In addition an I U I could deal with a range of user experience levels and support the development of user skills (i.e. the m o v e m e n t from novice to expert) as well as allowing users to customise tasks and features of the interface. The survey The survey itself consists of a series of questions. It you are able to answer all or some of the following questions we would be very grateful. The questions are of two types project specific and market orientated. To start with could you provide a brief description of the project or projects involving IUIs. The project specific questions are:

1.

Will any commercial products arise from the project (such as tools)?

Artificial Intelligence in Engineering, 1991, Vol 6, N o 1

15

Intelligent interfaces." the I M P A C T survey: B. Sharratt 2.

3.

4. 5.

6.

7.

16

Is the project using any commercial products (this question can cover programming languages and more specialised toolkits)? Have there been any problems during the IUI development, i.e. parts of the interface that have required extra effort and resources or have been difficult to construct? Are there any things which have been relatively easy or straightforward to do? Have any standards been used during the project (this covers both technical standards for software development and user interface guidelines/standards)? Has the interface been evaluated and has the evaluation been informal or formal (planned testing with a defined user group)? What are your general results? Are there any other projects or people we should contact as part of this survey?

Artificial Intelligence in Engineering, 1991, Vol 6, No 1

The market orientated questions are: 1. 2. 3.

What do you see as the use of new technology (such as CD-ROM, new user input devices, speech) in IUIs? How would you assess the market for IUIs? (we are interested in your views of its current size, rate of growth and future size). What do you see as the problems facing IUIs and the process of bringing IUI products to the marketplace.

Finally, are there any other comments you would like to make concerning this survey?

Thank you for your help. Brian Sharratt, Logica Cambrige Ltd.