A cooperative problem-solver for investment management

InternationalJournal of Information Management,Vol. 16, No. 2, pp. 133-147, 1996 Copyright © 1996 Elsevier Science Ltd Printed in Great Britain. All fights reserved 0268-4012/96 $15.00 + 0.00

Pergamon 0268-4012(95)00074-7

A Cooperative Problem-solver for Investment Management JDUGDALE

Classical expert systems which are acting as autonomous problem-solvers ere unsuRable when the problem needs to be solved cooperatively and when the users of the system are themselves experts. As a consequence a new type of system is emerging--the cooperative problem-solving system. This paper details the development of a cooperative problem-solving system for the domain of investment management. Much of investment management involves comparing alternative solutions. An assumption-based truth maintenance system (ATMS), which allows multiple hypothetical scenarios to be modelled, is the technique used to store each agents portfolio solution, constraints and underlying decisions. Analysis of transcripts obtained during knowledge acquisition revealed seven functions that a cooperative problem-solving system should seek to provide. The resulting system provides a cooperative environment in which multiple users can investigate and compare different solutions, build new approaches to obtaining a solution and introduce new concepts to engender discussion.

Julie Dugdale is currently a lecturer in Expert Systems at De Montfort University, Milton Keynes, UK, and member of the Artificial Intelligence Group. The author would like to acknowledge the support of ICL PLC in this work.

Introduction Expert systems and cooperative problem-solvers

Early expert systems such as MYCIN 1 and PROSPECTOR 2 aimed to solve a problem autonomously, using a model of the domain and a set of data about a specific problem. Although such autonomous problemISHORTLIFFE, E H (1976) Computer-Based Medical Consultations: MYCIN Elsevier, solvers perform equally well, or in some cases better, than their human New York counterparts, many fail to be used routinely. 3 It has been argued that 2DUDA, R O, HART, P E, BARRETT, P, GASthe major reason for the lack of acceptance of such systems is not due to CHNIG, J, KONOL1GE, K, REBOH, R AND SLOCtJM, j (1978) 'Development of the their problem-solving ability, but rather the way these systems interact PROSPECTOR Consultation System for with their u s e r s . 4 This is referred to by Hickman and his colleagues as Mineral Exploration' Final Report SRI Projects 5821 and 6415, Artificial Intelli- the cooperativity problem. 5 gence Centre, SRI International, Menlo Classical expert systems, being autonomous problem-solvers, have Park, CA been criticized for ignoring the problem-solving ability of the user. By 3DAVIS, R (ED) (1989) "Expert systems: how disregarding the capabilities of the user, a system loses a potentially far can they go?' A1 Magazine Summer; FENN, J A (1987) 'Implementing cooperavaluable aid to its problem-solving ability. In addition, by excluding the tive expert systems' in First International human from the problem-solving process, autonomous problem-solvers Symposium, Artificial Intelligence and Exrequire a comprehensive knowledge-base covering all aspects of the pert Systems, Volume 2, AMK Berlin, 125157 tasks being performed. This is infeasible as there will always be some 4OBERQUELLE, H, KUPKA, I AND MAASS, S relevant factors which will have been excluded. 6 A third problem with (1983) 'A view of human-machine comclassical expert systems is that they fail to accommodate the potentially munication and cooperation' International Journal of Man-Machine Studies 19 309- different problem-solving approaches of their users. A further criticism 333; WOODS, D D (1985) 'Cognitive tech- was noted by Fischer and his colleagues when they recognized that nologies: the design of joint humanmachine cognitive systems' AI Magazine 6 users, rather than expecting a system to derive a solution independently, continued on page 134 actually enjoy participating in the problem-solving process. 7 Finally, the 133

Cooperative problem-solver for investment management: J Dugdale I Identify and define the problem

I H .-,°-H'mP'-°"' H

= alternative solutions

alternative

the

solutions

decision

Evaluate the decision

I

Figure 1 Generalized model of management problem-solving

user's acceptance of the final solution proposed by the autonomous problem-solver can have strong moral and possibly legal implications. Expecting a user who has not been involved in deriving a solution to judge whether or not that solution is correct can result in what Woods calls the 'responsibility/authority double bind'. 8 This is a dilemma whereby the user either accepts without question, or consistently rejects, the system's decisions. In many real-world situations, problems are solved not in isolation but cooperatively. For example, in the business domain, managers will consult with their colleagues when deciding which investment projects to finance in the coming year. A new type of system--the cooperative problem-solving system is emerging to handle this situation. The goal of cooperative problem-solving systems is to derive a solution which is superior to that produced by either an autonomous problem-solver or a human problem-solver working separately. Expert systems deal primarily in the domain of logical or deductive reasoning on well-structured problems. Management problem-solving has long been recognized as an area in which the problems, although having a logical proclivity, require an element of creativity and intuition. 9 Cooperative problem-solving systems exploit the potentially synergistic relationship between humans and computers.

continued from page 133

IPart 4) 86-92 HICKMAN~ F R, KILLIN, J L, LAND~ L, MULHALL~ T~ PORTER~ D AND TAYLOR~ R M

(1989) Analysis for Knowledge-Based Systems--A Practical Guide to the KADS Methodology Ellis Horwood 6LEMKE, A C AND FISCHER, G (1990) ' A cooperative problem solving system for user interface design' AAAI-90 Proceedings of the Eighth National Conference in Artificial Intelligence MIT Press, 479-484; STOLZE, M (1992) 'From knowledge engineering to work-oriented development of knowledge systems' PhD Thesis, University of Ziirich

A model ofmanagementproblem-solving Managerial problem-solving is a sequential process that is often approached in a highly systematic manner, t° A general model which shows the five stages of management problem-solving is shown in Figure 1. The first step in the process is to recognize that a problem exists. The second stage of the model involves generating alternative solutions. This incorporates a process of searching for relevant information. In the next stage, the alternative solutions are compared and evaluated. The penultimate stage of the process deals with the implementation of the decision. Finally, successfully implemented decisions require follow up and control. The cooperative system developed in this work is aimed at supporting managers during the second and third stages of the model.

7FISCHER~ G~ LEMKE~ A C~ MASTAGL10~ T

AND MORCH, A I (1991) 'The role of critiquing in cooperative problem solving' A CM Transactions on Information Systems 9 (3) 123-151 SWOOPS, op City Ref 4 9KHARBANDA~ O P AND STALLWORTHY, E A

(1990) 'Managerial decision making, Part 2' Management Decision 28 (4) 29-35; MARGERISOH, C J (1974) Managerial Problem Solving McGraw-Hill 10DONNELLY~ J H JR, GIBSON, J L AND IVAN-

CEVlCH, J M (1990) Fundamentals of Management Seventh Edition, BPI Irwin 1L MOTr, G (1987) Investment Appraisal for Managers Second Edition, Gower

134

Investment management A specific example of managerial problem-solving is contained within the domain of investment management. Investment management is a complex discipline which is concerned with investing funds in some future event or product with the aim of generating a return, it In order to arrive at a solution, which is acceptable to all participants, a high degree of cooperative problem-solving is required. To state it in its simplest terms, investment management is concerned with selecting a portfolio of projects to fund in the coming financial year. A company has only limited resources and thus management must be selective in choosing which future investments to fund. The most

Cooperative problem-solver for investment management: J Dugdale

Table 1 Typical data associated with a project Cost

Evolves infrastructure Fit to strategy Key competencies M a rket size Market duration Payback period Relationship to competition Revenue/cost ratio

E250 000

Marginally Aligns Exploits La rg e 7 years 5 years

Ahead 3

common way of evaluating proposed investments is to apply financial techniques, for example calculating the revenue/cost ratio, payback period and net present value of the project. In recent years these techniques have been subject to some criticism lz and as a result the use of non-financial criteria is becoming more widespread. The trend in many businesses is to use non-financial criteria, in addition to standard financial techniques. Within the cooperative system developed in this work both the financial and non-financial aspects of an investment project are modelled. Table 1 gives an example of the type of data associated with a project. In terms of the generalized model of management problem-solving, the first task is to characterize the problem. This is a two-stage process which involves recognizing any potential constraints on the solution, and then developing a set of achievable objectives. A frequent constraint imposed on the derivation of an investment portfolio is a budget which is set by company policy. Other constraints may include the number of personnel required or the level of expertise needed. Once the constraints on the solution have been identified, alternative ways of achieving the stated objectives are devised. It is at this point that both financial techniques and the application of non-financial criteria are used to evaluate projects. Expert system principles are used to arrive at a partial portfolio solution. Cooperation between managers is then required in order to arrive at a final, mutually acceptable, solution. Once a solution has been chosen and implemented, the investment's progress is monitored to ensure that the original objectives are realized. From the preamble above, a tool which would keep track of alternative solutions each of which may exist under a different set of circumstances or assumptions would be extremely useful. Such a tool would allow the user to compare potentially different solutions and see any underlying assumptions or constraints. The tool should also ideally be able to show the consequences of any change to the underlying assumptions. A technique which is ideally suited to this task is a truth maintenance system. 12ASHFORD, R W, DYSON, R G AND HODGES, S ~ (1986) 'The capital investment appraisal of new technology: problems, misconceptions, and research directions Warwick Papers in Management 7 1-27; KAPLAN, R S (1986) 'Must CIM be justified by faith alone?' Harvard Business Review, MarchApril, 87-93; PETERS, T (1988) Thriving on Chaos Macmillan; SWANN, K AND O'KEEFE, W O (1990) 'Advanced manufacturing technology: investment decision process. Part 2' Management Decision 28 (3) 27-34

Truth maintenance The general architectural model of an overall problem-solving system has two distinct components: the Problem-solver and the Truth Maintenance System (TMS) (Figure 2). Inferences made by the Problem-solver are communicated to the TMS in the form of assumptions and associated justifications. Using this information, the TMS is able to determine the set of current beliefs and any contradictions. Notification of any contradictions together with the 135

Cooperative problem-solver for investment management: J Dugdale Justificationsand

assumptions D

Problem solver

I al

Truth

maintenance system

Beliefs and contradictions

Figure2

Architecture of an overall problem-solving system

set of current beliefs are then passed back to the problem-solver for use in further inferencing. A TMS records the dependencies between inferences and detects any inconsistencies that may arise. As a consequence of maintaining a dependency network of inferences, the TMS is able to identify the derivation of any conclusion. This furnishes the TMS with the ability to provide rational explanations. Three principle treatments to Truth Maintenance Systems have been identified: Assumption-based Truth Maintenance Systems (ATMS), Justification-based Truth Maintenance Systems (JTMS), and Logicbased Truth Maintenance Systems (LTMS).13 One of the critical issues in the choice of a TMS is its ability to support single or multiple contexts. The standard view of a TMS is that it operates with one consistent set of data, or context; these are single context systems. This is analogous to working in only one scenario or viewpoint. Multiple context systems, on the other hand, allow several sets of data to be handled concurrently whilst still maintaining the internal consistency of any one context. By allowing multiple contexts, it is possible to investigate alternative scenarios or potential solutions which may be mutually exclusive. Typically, Justification-based systems, such as those of McAllester and Doyle are single context, 14 whereas de Kleer's Assumption-based system is designed to operate in multiple contexts. 15

Analysis of the domain and human problem-solving performance This section describes the development of the cooperative problemsolving system. Knowledge acquisition for traditional expert systems is primarily concerned with understanding the concepts in the domain and how the expert performs the task. For a cooperative system, additional analysis should be carried out to identify the cooperative interaction between the problem-solvers. In this work multiple experts were used as opposed to using a single domain expert. This provided alternative views and perspectives. The presence of multiple viewpoints closely mimics the decision-making process in investment management. In this work, 13 experts were used in the initial stages of knowledge acquisition. Twelve secondary experts were chosen in addition to a 13DEKLEER,J (1986) 'An assumption-based primary expert. The roles and contributions of the experts were as TMS' Artificial Intelligence 28 127-162; follows: DOYLE, J (1979) 'A truth maintenance system' Artificial Intelligence 12 231-272; MCALLESTER, O A (1980) 'An outlook on truth maintenance' Report No AIM 551, Massachusetts Institute of Technology, AI Laboratory 14DOYLE, op cit, Ref 13; MCALLESTER,op cit, Ref 13 15DEKLEER,op cit, Ref 13

136

• Secondary experts were used to provide additional knowledge that may have been inadvertently omitted by the primary expert. • The primary expert was used to validate and arbitrate knowledge gained from the secondary experts. • Secondary experts were used as specialists on a particular area of investment management.

Cooperativeproblem-solverfor investment management: J Dugdale The duration of each knowledge acquisition session varied with the material covered, with a typical session lasting approximately one-anda-half-hours. The total time spent on knowledge acquisition using both primary and secondary experts is estimated to be between 50 and 60 hours. The knowledge acquisition sessions were conducted on a one-toone basis with the expert, although occasionally two experts were interviewed together. The exceptions to this format were the management meetings which are explained later in this section. In addition to making written notes each session was audio recorded and later transcribed verbatim. The aim of the analysis was to develop an insight in to the domain, the problem-solving process, and type of cooperation used in solving the problem. By analysing the cooperative activity the intention was to identify the cooperative functions required of a cooperative system. Each of the three stages of knowledge acquisition and analysis will now be explained.

Stage One: domain familiarization Initial familiarity with the concepts and terminology of investment management was achieved primarily by interviewing the experts. The interviews at this stage were predominantly unstructured, since the goal was simply to discover the general principles in the domain. Once these principles were determined, focused interviews were used to uncover greater detail.

Stage Two: the problem-solving process Detail of the problem-solving process was obtained by employing the protocol analysis method. Protocol analysis is classical knowledge acquisition technique which involves asking the expert to 'think-aloud' while solving a problem. From the tape recording, the expert's utterances are then transcribed into a 'protocol'. Both the primary and secondary experts were called upon to describe the processes involved in deriving an acceptable portfolio. Protocol analysis, unfortunately, has the disadvantage that the expert can introduce bias which may be misleading. To overcome any expert bias, real-life management meetings were attended. The role of the author at these meetings was as a silent observer, although each meeting was recorded. A typical management meeting operates under strict time constraints and lasts about two hours. The nature of these meetings limits their frequency to usually only two a year. Interviews with selected experts immediately after the meetings and retrospective discussions of the meeting transcripts, provided a richer insight into the problem-solving method. The analysis also uncovered the problem of hidden management agendas. For example, a manager may choose to support a project sponsored by another manager in order to gain reciprocal support. Modelling such a situation within a system would be both difficult, and of dubious value. Indeed, one of the main strengths of a computerized problem-solver is its ability to remain unbiased. The answer to these problems was to move to an analysis of the protocols obtained during simulated management meetings. Using the knowledge gained of the domain, a case-study was devised which could be presented to a team of participants having management experience. The case-study was veri137

Cooperative problem-solver for investment management: J Dugdale

Identifycandidate } elements

Identifyconstraints I

1 IdentifycriteriaSelection[

Rank criteria selection

I Reviewnew solutionfor weaknesses

I I

[

Selectelements satisfyingtop/next selectioncriteria Apply constraints (identifyviolations)

Suggestremedy to violation

I.

I

Figure 3 Task model

lied by the primary expert as being representative of the problems faced in investment management. By analysing the transcripts obtained from the simulated and real management meetings the method used by the experts to solve the problem began to emerge. This is shown in the form of a Task Model in Figure 3. The first task in the model is to identify the candidate elements, or investment projects, which form the solution space. Once these have been established, the constraints, which act upon the solution are determined (shown by the identify constraints task). The third task is to identify the criteria, either financial or non-financial, by which the elements are selected. A criterion may be a single factor, such as the decision to select only those projects which have a low payback period, a combination of several factors, or the specification of a particular project. The criteria are then ranked in order of importance. For example, fit to strategy may be considered more important than evolving the company infrastructure. If some of the selection criteria are unknown at the outset then identifying these becomes part of the iteration. With the candidate elements, constraints, and selection criteria identified, a propose-and-revise cycle begins. The first task of the cycle is to choose a set of elements which satisfies the top selection criterion. This generates a partial solution. The constraints are then applied to this solution and any violations are identified. If a violation is uncovered, a possible remedy may be suggested. The new solution set is then reviewed for any weaknesses. For instance, the partial portfolio may lead to problems with the short term cash flow. This is rectified by seeking similar projects with shorter payback periods. 138

Cooperative problem-solver for investment management: J Dugdale

To confirm that the derived Task Model was the strategy used by the experts, teachback interviews were employed. 16 This technique involves the expert describing a procedure to the interviewer, who then 'teaches' it back using the expert's terms, and to the expert's satisfaction. The primary expert first explained the approach used in the derivation of a portfolio. An old investment case was then selected for the author to work through. Each step in composing the portfolio was then taught back to the expert.

Stage Three: analysis of cooperation and the identification of cooperative functions

16jOHNSON, L AND JOHNSON, N E

(1988)

'Knowledge elicitation involving teachback interviewing' in K1DD, g (ED) Knowledge

Acquisition for Expert Systems--a practical handbook Plenum Press, 91-108 (1991) 'Task level frameworks for cooperative expert system design' A1 Communications 4 (2/3) 98-106; OOODSON, 17STOLZE, M

J L AND SCttMIDT, C F

(1990) 'The design o f

cooperative person-machine problemsolving systems: a methodology and example' in ROBERTSON, S P, ZACHARY, W AND BLACK, J B (EDS) Cognition, Computing and Cooperation Ablex, 187-223

The objective of Stage Three was to identify the functions that a cooperative system should provide. There are very few approaches to analysing cooperation for cooperative systems. Previous approaches focused on identifying and assigning individual tasks to the various agents. 17 This is inappropriate for the interpretation of cooperation used here in which all agents are able to develop a suitable portfolio. Thus, the relative merits of independently derived solutions are investigated by participants in order to arrive at a solution that is mutually acceptable. To identify the cooperative functions, a new approach to cooperative analysis was developed. Central to this approach are the transcripts obtained from the real and simulated management meetings. The first step towards identifying the functions was to decompose the transcripts to see if there was any structure to the dialogue. The transcripts are made up of individual statements made by each participant, for example 'So payback period is the only criterion we are looking at?' and 'The revenue/cost ratio is the total amount of revenue you will get back over the whole time of the project, the market duration'. From the second stage of knowledge acquisition, (which involved developing a Task Model) the statements could be grouped into tasks. For example, the statements above were concerned with the task of selecting projects by reference to their payback period. Although the identification of the tasks showed the various procedures that the system should perform to solve the problem, it gave no indication of the cooperative functions that should be provided. The question that required answering was 'What was the role or aim of cooperation in performing the task?'. More precisely, 'What were the participants trying to achieve when conversing?'. This information cannot be gained from individual statements as they needed to be taken in context. Likewise, the task information shows the objective and not the part played by cooperation. An intermediate level, called the interaction level, was identified which answered the above questions and thus identified the functions that a cooperative system should provide. The interaction level represents the interactions that are necessary in order to perform a task. The example statements given above are considered at the interaction level: Manager 1: 'Can I just clarify one point, this revenue/cost ratio applies only over the payback period, doesn't it?' Manager 2: 'The revenue/cost ratio is the total amount of revenue you will get back over the whole time of the project: the market duration.' In response to the question posed by Manager 1, Manager 2 is trying to

explain part of the concept of revenue/cost ratio. Likewise: 139

Cooperative problem-solver for investment management." J Dugdale Manager 1: 'So payback period is the only criterion we are

looking at?' Manager 2: 'Yes, at the moment.' Manager 1 is trying to understand (or model) part of the problemsolving process. In response, Manager 2 is confirming Manager l's understanding. The interaction level of the transcripts shows the particular type, or purpose of cooperation, exhibited by participants when trying to accomplish a task. For example, the interaction may aim to explain a particular concept or recognize some constraint. This work identifies seven cooperative functions in the transcripts at the interaction level that a cooperative system should aim to provide: explanation, constraint recognition, problem-solving, problemmodelling, information provision, confirmation, and user-modelling. Some of the functions, such as explanation, have been identified previously as contributing to a system's cooperative ability. 18 However, researchers have tended to look at the functions individually; the implication being that cooperation will automatically increase if one function is enhanced. However, the aim of a cooperative environment is to bring together all of the functions in order to facilitate cooperation.

An implementation of a cooperative problem-solving system The previous section established the role of the user in the problemsolving process and the underlying functions required of a cooperative system. This section shows how this information was used to implement the cooperative system. Figure 4 shows the conceptual architecture of the system.

Knowledge-base From the Task Model, three major knowledge types are indicated, two of which are stored in the Problem-solving Knowledge-base. Firstly, domain knowledge on the solution constraints and selection criteria is required together with more fundamental knowledge on the domain. The second type of knowledge indicated by the Task Model is strategic knowledge. Strategic knowledge is the knowledge used to decide what action to perform in a given situation. Lastly, information on the cooperative interaction between the system and the user is stored. This cooperative knowledge is stored in a separate knowledge-base which is accessed only by the Cooperator module. Contained within this knowledge-base is information on how to receive and present information, how to explain and how to compare portfolios. Also included is information about the users and the system and the portfolios they have created.

Portfolio generator 18CROOK, N T (1992) 'A theory of explanation applied to medical expert systems' PhD Thesis, School of Computing and Mathematical Sciences, Oxford Brookes University, Research Report No. KER 92/1

140

The problem-solving component of the system is called the Portfolio Generator. The Portfolio Generator is concerned with constructing solutions from information gained from the Cooperator. The module treats the system and the user equally by handling the opinions of each in the same manner. From the Portfolio Generator's point of view, each user (including the system) is a client. Each client may develop many

Cooperative problem-solver for investment management: J Dugdale

P

Cooperative knowledge

knowledge 1

Fa°';':°::'or::nts'

Cooperator

T Beliefs]

I

Beliefs

[ Explanations ] critiques

I

Portfolio generator

Assumptions premises

Beliefs

contradictions

derived facts

ATMS

Figure 4 system

Conceptual architecture of the cooperative problem-solving

portfolios. Constraints and decisions are passed to the Portfolio Generator by the Cooperator either from the user or the system's opinion. The solution generated by the system acts as a catalyst and base mark to which other users can refer and develop their own solutions. As a solution is constructed, the assumptions, premises, facts and associated justifications of both the system and the user are passed onto the ATMS. Any inconsistency in the information contained within the ATMS is reported back to the Portfolio Generator and, if necessary, to the user. By using an ATMS to store each client's decisions, the opinions of multiple clients may be represented.

Cooperator The Cooperator module acts as an interface between the client and the Portfolio Generator and thus mediates all problem-solving activity. The Cooperator supports a range of different types of interaction most of which are focused on the processes of constructing and evaluating a portfolio solution. The module provides a simple Windows-based user interface which functions as a work-pad for the user and a communication outlet for the system. The functions supported by the Cooperator include the following: • Formulation of constraints, facts, and decisions to be used by the Portfolio Generator. 141

Cooperative problem-solverfor investment management: J Dugdale

• • • •

Verification of information provided by the user. Presentation of the system's and users' solutions. Presentation of critiques and explanations. Presentation of relevant information.

The module allows input of the user's decisions and constraints. These are then passed to the Portfolio Generator for use in constructing the user's solution. Information on the system's and any alternative solutions is passed back from the ATMS via the Portfolio Generator to the Cooperator. Knowledge required in presenting the information is obtained from the Cooperative Knowledge-base.

ATMS The ATMS is the mechanism used to store the decisions of the user and the system. It is not concerned with the actual generation of the portfolio; instead, it holds the actions and justifications for any given portfolio. Information concerning the system's and user's decisions is received directly from the Portfolio Generator. This information is then represented in the ATMS. Current decisions are then passed back to the Portfolio Generator to be used to infer other actions. Like the Portfolio Generator, the ATMS treats the system and the user equally, regarding each as a client.

Interacting with the cooperative problem-solver A consultation starts with the system formulating a portfolio solution using knowledge contained within the problem-solving and domain knowledge-bases. The system follows the same problem-solving methodology as the user, applying prioritized selection criteria and constraints to the set of available projects. At this stage in the consultation, the system is functioning as a traditional autonomous problem-solver. The decisions and derived facts from the inference are passed from the Portfolio Generator to the ATMS. As each task is executed, the dependency network develops. With the system's proposed portfolio and full dependency network in existence, the user is free to review the system-generated solution. The user is first presented with a brief overview of the system's portfolio solution. This shows which projects are included in the portfolio, which projects are not included (still available) and which have been explicitly excluded. By clicking on a particular project, the user can request an explanation of the derivation and justification for the projects inclusion or exclusion. In addition, the user can also investigate the underlying constraints and selection criteria used by the system.

Defining constraints One of the first tasks in the construction of a portfolio is to define the constraints which operate on the solution space. A typical constraint might be to restrict the budget for the portfolio to 20 million pounds. The user has the option to justify the constraint by relating it to any underlying domain knowledge. For example, a client may wish to link the reasons for the budgetary constraint to the underlying concepts of growth targets and profit. This action adds to the dependency structure formed in the ATMS. The user can view any existing constraints or construct an entirely

142

Cooperative problem-solverfor investmentmanagement:J Dugdale new constraint. Constraints which already exist are grouped, by the system, into two sets: constraints which apply to the present portfolio, and those which operate in other portfolio contexts. Any newly defined constraint will operate only in the current portfolio. When a new constraint is defined, the ATMS checks that there is no inconsistency with any other constraints in that context. For example, a client cannot define two budgets to operate in the same portfolio. Recall that the ATMS maintains consistency within a context, but allows inconsistent assumptions over several contexts.

Defining selection criteria The selection criteria which are to be applied to the evolving portfolio fall into four categories: an individual attribute, a combination of the attributes, user-defined criteria, or the specification of a named project. For example, in the first category, a user can request to include those projects in the portfolio which satisfy one particular criterion, such as having a high strategic fit, or a long market duration. The second category allows the user to be more specific by combining the criteria, for example, specifying the inclusion of projects which put the company ahead of the competition and have a short payback period. The third category permits the user to define their own criterion. For example, because of staffing levels, a user may wish to undertake those projects which require only 20, or less, staff. As there is no data in the system on how many staff are needed for each project, the user must supply the relevant information so that it may be used in reasoning. The ability to provide information which will be used in reasoning is a principal characteristic of human-human cooperation. The last category enables the user to stipulate the inclusion of a particular project, for example, Project L must be included in the portfolio, regardless of any other criteria. The Portfolio Generator filters out all of the projects satisfying the selection criterion at the specified level. Once these projects have been identified, the Portfolio Generator checks to see that no constraints have been broken. If a constraint has been broken, the Portfolio Generator does not allow the task which resulted in the violation to be performed. The user can either redefine the selection criteria or explore an alternative portfolio with relaxed constraints.

Providing and receiving information To allow the user a more active role in problem-solving, and support mixed-initiative interaction, the user can volunteer information to the system. This ability to add information that was previously unknown to the system is one of the major features of the system. Information can be provided which relates to a particular project, or general background data which is added to the body of support knowledge contained within the system. The system recognizes the difference between information about an entity of which it is unaware, a new attribute of a known entity, and a conflicting value. An analysis of the transcripts from both the real and simulated management meetings established the types of questions that were frequently asked during problem-solving. Four generic question types, for which the system can provide answers, were identified: • What is the value of an attribute for a certain project? 143

Cooperative problem-solver for investment management: J Dugdale

Figure 5

Explaining the presence of a project in a portfolio

• Which projects have the attribute with a particular value? • In which portfolios is a project included? • How far is the current portfolio from violating a constraint?

Explanation. The explanation facility supports the three categories of explanation which were identified in the transcripts: Explanations about actions Explanations about concepts Explanations about the problem-solving method Requests for an explanation relating to the first category generally take the form of questions such as 'Why is Project X included in Portfolio Y?'. Consider Figure 5 in which the user has requested an explanation as to why Project G was included in the Jones' first portfolio. The figure shows three windows; the Portfolio Window (top left-hand corner), the Project Selection Window (top right-hand corner) and the Explanation Window. The Portfolio Window shows all the clients and their respective portfolios that are currently being considered by the system. Notice that Jones has two portfolios currently under development (Jones 1 and Jones 2). The Project Selection Window shows those projects which, by virtue of the selection criteria and constraints, are included or excluded in the current portfolio (for simplicity the projects are simply identified by letters of the alphabet). The Explanation window shows why Project G was included in Jones' portfolio. The first line shows which portfolio is being considered (Jones 1), and the constraints imposed upon that portfolio (Budget 40000). More importantly it shows the main reason for the projects inclusion; that Jones chose to include those projects which have a fit to company strategy equal to 3 (indicating a fair fit to strategy) (Strategy=3). This was her most important selection criterion (Strategy=3 1). The explanation also tells the user in which other portfolios Project G is included and under what circumstances. (System 1 Budget 30000 System 1 Policy=Windows 1) 144

Cooperative problem-solver for investment management: J Dugdale

Meaning that Project G is included in the portfolio that was derived by the system when using a budget of £30 000. It was included as the system believed that the most important criterion was to include those projects which encompassed a Windows interface. Note that it is also included in Jones' portfolio for a second reason; as the third selection criterion Jones wanted to include Windows based projects. Notice that this information was given supplementary to the the fact that it was included because it fitted with the company's strategy. This is because the system always gives the most important and relevant reason for a project's inclusion first. (Jones 1 Budget 40000 Jones 1 Policy=Windows 3)

In addition to the three categories of explanation, three levels of explanation are provided: full, standard, and simple. With each level, the explanations are first grouped into contexts. Thus, the system will explain all supporting information as to why a project is included as a result of one selection criterion before moving onto the next criterion. Similarly, if a project is included in multiple portfolios, the reason for its inclusion in one portfolio is explained before moving onto the next portfolio. With all three levels, the explanation provided is non-repetitive, so that information that was previously displayed in a context is not repeated within that context. Critiquing

19CLARKE, A A AND SMYTH, M G G

(1993)

'A cooperative computer based on the principles of human cooperation' International Journal of Man-Machine Studies 38 3-22. Z°Op cit, Ref 7

A cooperative system should prompt the user to adopt a more divergent style of thinking during problem-solving. 19 This is achieved by critiquing. Critiquing is the presentation of a reasoned opinion about a product or service. 2° In the domain of investment management the critiquing approach is widely applied. It is common for a manager to present a tentative portfolio of investment projects to his or her peer group in order to evoke comment or criticism. From the discussion that ensues, the solution is refined, or new solutions are developed. The cooperative system uses a differential approach to critiquing by comparing two alternative solutions. Thus, any portfolio can be the subject of a critique, and any other portfolio can be its critic, regardless of who actually owns the portfolio. The system can therefore critique two portfolios owned by the same agent, as well as critiquing portfolios owned by different agents. In Figure 6 the user has requested a critique of the portfolio derived by Jones using the system-generated portfolio. The critique is displayed in four windows. Starting from the top left-hand corner and working clockwise, the first window shows the overall scores for each attribute of the critic portfolio compared with those of the subject at the same state of completion. This provides a measure of how good the developing (subject's) portfolio is compared with the critic's portfolio at the same stage. In the example above it appears that the system's portfolio is fairing better on utilizing the company's key competencies than Jones Portfolio (45.0 versus 35.0 respectively). However, the scores should be viewed in respect of the cost of the included projects, which in the example, is higher in the system's portfolio. The second window depicts the averages of the attributes of both portfolios. It uses the overall (final) average for the critic portfolio and the current average for the subject portfolio. This shows where the 145

Cooperative problem-solver for investment management: J Dugdale

Figure 6 Critiquing strengths and weaknesses of the subject's portfolio lie focusing attention on areas of the portfolio that require further consideration. The two lower windows show the origin of the critic and subject portfolios in terms of the tasks undertaken and the constraints imposed. It may be thought that a comparison between portfolios which use different constraints is meaningless. This may be true when there are large differences between constraint values. However, when constraints vary only marginally, then critiques can be useful. The cooperative system adopts a passive approach to critiquing. Passive critics are explicitly invoked by users when an evaluation is desired, whereas active critics continuously monitor, and respond to, individual user actions. A passive critic is more appropriate when the critic information is used only at certain times during problem-solving, for example at the end of constructing a portfolio. Consider the position after the first task had been undertaken, an active critic would attempt to advise the user of the problems with the portfolio without taking into account the user's future actions. Unlike, the active critic, the passive approach allows for a less intrusive style of critiquing making it germane when the user is likely to be an expert, as in the case of a cooperative system.

Results and conclusion To help evaluate the cooperative problem-solving system a questionnaire was formulated. The questionnaire aimed to establish what the user thought of the system's solution and also how successful the system was at providing a cooperative environment. The evaluators of the system were those people who had participated in the simulated case-study. The major results from the evaluation were: (1) Evaluators generally felt that the system increased the overall quality of the final solution. More specifically, evaluators believed that the final solution was better than that which could have been

146

Cooperative problem-solver for investment management: J Dugdale

produced by either the system or a user alone. This suggests that the aims of a cooperative system (see earlier section) have been achieved. (2) A major benefit derived from the system was the ability to model different possible solutions. This allowed the user to compare the relative merits of the solutions and view the underlying decisions. This facility was provided via the utilization of an ATMS. An additional advantage was the ability of the ATMS to provide meaningful and relevant explanations. The relevance of the explanations (the explanation provides the most significant reason for a project's inclusion in a portfolio first) was felt by the evaluators to be very important and a particularly valuable feature of the system. (3) The solution generated by the system was useful only insofar as it encouraged the users to investigate other possible solutions. Users were able to see the weaknesses in the system's solution. The user could then modify the approach used by the system in an attempt to overcome these weaknesses. (4) There are four main limitations of the system. The first is concerned with the explanation facility. Whilst three levels of explanation were considered to be useful, the evaluators felt that the full explanations were difficult to follow. This was largely attributed to the lack of a natural language interface. The second limitation is concerned with the system's efficiency. The evaluators thought the system to be efficient in terms of speed and response time. However, testing carried out independently by the author found that efficiency would suffer under exceptional circumstances. A further limitation is the format of the constraints that the user may impose. Currently the system can only deal with simple numeric constraints. The final limitation concerns the system's solution. The opinion of the system is static and does not develop during the interaction thus the user becomes familiar with the system's reasoning. Further work would focus on addressing these four limitations. Investment management is a very eclectic environment. It is difficult to make hard and fast rules about what will lead to success and to failure. For this reason the systems currently in use focus on decision support, rather than providing a serious attempt to help managers solve the problem. This work addresses this issue by describing a system that is both acceptable and useful to managers.

147