- Email: [email protected]
Designing knowledge-based systems for incremental development
Expert Systems With Applications, Vol. 4, pp. 87-97, 1992
0957--4174/92 $5.00 + .00 © 1991 Pergamon Pressplc
Printed in the USA.
Designing Knowledge-Based Systems for Incremental Development PATRICK J. LYONS St. John's University, Jamaica, New York 11439
AbstractmMany knowledge-based systems are developed in an incremental fashion because the knowledge is not fully understood at the start. To facilitate this process, the knowledge bases shouM be designed to accommodate revisions easily. This article presents a methodology for designing the knowledge bases to achieve this.
1. INTRODUCTION
the results of some of that research will be discussed along with its implications for the proposed methodology.
a powerful technology that must be well managed if its potential is to be achieved. As Chapnick (1990) noted, rapid prototyping (and expert system technology) may help build systems more quickly, but building incorrect or incomplete systems ten times faster is no gain. The objectives of this article are twofold. The first is to review some of the management issues relating to the successful implementation of expert system technology. The second is to present a methodology for designing the knowledge bases so that these management issues can be achieved. In Section 2, several issues relating to the successful implementation of expert system technology are reviewed, including: • Designing robust information systems • Viewing information systems as change agents • Altering the traditional system development life cycle • Assuring the quality of knowledge bases Section 3 describes a methodology for designing the knowledge bases so that the management issues can be achieved. In Section 4, a case is given to illustrate the methodology. Section 5 presents some of the experience using the methodology. Finally, Section 6 discusses the benefits of the methodology to the chosen case in particular and to knowledge systems in general. EXPERT SYSTEM TECHNOLOGY is
2.1. Designing Robust Information Systems As El Sawy and Nanus (1989) noted, the concept of robustness revolves around having the capability of adapting efficiently to changing environments and continuing to function effectively under harsh dynamic conditions. An information system can be defined as robust if it yields satisficing (satisfactory and sufficing) performance under all environments that are assessed as having an appreciable probability of occurring, and if it also has an associated scanning component that alerts it to environmental changes and enables switching to another configuration quickly and inexpensively to take advantage of the environmental change. The implication of robustness for the proposed methodology is that a Risk Management Plan, such as that described in Boehm (1988), should be developed. Such a plan identifies the top risk items for the project and develops a strategy for resolving them. For projects using expert system technology, the risks usually center on developing the wrong software functions and/or developing the wrong user interface. As will be shown, by developing a Risk Management Plan early in the project and using the proposed methodology, the system will be designed to accommodate more easily environmental changes. The result, by definition, will be a more robust knowledge system.
2. MANAGEMENT ISSUES There has been extensive research on the development of information systems in general and decision support systems and expert systems in particular. In this section,
2.2. Viewing Information Systems as Change Agents The concept of change agency is important for designing information systems. Silver (1990), in his study of Decision Support Systems (DSS), identified two vari-
Requests for reprints should be sent to Patrick J. Lyons, Department of Management, College of Business Administration, St. John's University, Jamaica, New York 11439.
87
88
eties of change agencies reflecting two different attitudes on the part of the system designer. On one hand, when designers comprehend that change will occur and deliberately attempt through a DSS to force the direction of that change, we have an instance of "directed" change. On the other hand, when designers understand that change will occur but do not try to influence the direction of that change, allowing it to be determined instead by the decision maker through DSS use over time, we have a case of "nondirected" change. Silver develops four design strategies for directed change and five design strategies for nondirected change. These strategies are based on the system restrictiveness and decisional guidance. The system restrictiveness is defined as the degree to which a DSS limits its users' decision-making processes to a subset of all possible processes. Decisional guidance refers to the degree to which a DSS guides its users in constructing and executing decision-making processes. The implication of change agency for the proposed methodology is that serious thought must be given at the beginning of the project to the impact that the new system will have on the organizational unit. Then one of the strategies identified in (Silver, 1990) (or a combination of strategies, if appropriate) should be adopted. The proposed methodology can be used in conjunction with the strategy for directed or nondirected change.
P. J. Lyons
greater flexibility, it typically will require more effort to control the many processes. Boehm (1988) presented a spiral model of the general software development process. The major distinguishing feature of this model is that it is a risk-driven approach to the software process rather than a primarily document-driven or code-driven process. In particular, risk-driven specifications can have varying degrees of completeness, formality, and granularity, depending on the relative risks of doing too little or too much specification. From a general management perspective, the risk analysis of the spiral model is an example of an ABC Analysis applied to project risk. For a general discussion of an ABC Analysis, see Dilworth (1989). The U.S. Department of Defense has established a standard on software management, DoD-Std-2167, which requires that developers produce and use risk management plans (Boehm, 1988). O'Keefe (1990) presents an explicit modification of Boehm's spiral model for expert system technology. The primary emphasis of O'Keefe's model is to identify verification and validation risk factors early in the software development process. However the life cycle is modified, the implication for the proposed methodology is that it must accommodate incremental development. The example in Section 4 illustrates this.
2A. Assuring the Quality of Knowledge Bases 2.3. Altering the Traditional System Development Life Cycle Because prototypes can be developed rapidly with expert system technology, there has been some research into altering the traditional systems development life cycle to take advantage of this. For example, Cupello and Mishelevich (1988) have proposed a three-stage process. In the first stage, a demonstration prototype is built that behaves intelligently in only a very limited segment of the domain. This proves the technology. The second stage is a full prototype that incorporates both breadth and depth across the problem domain. This system can be used to verify and validate the knowledge. The third stage produces the production system. In Weitzel and Kerschberg (1989), another modification to the life cycle is examined. To avoid the rigidity of sequential phases, the life cycle considered here consists of processes. Processes are activated, deactivated, and reactivated. They can run concurrently. Initially, a process is activated after the process above it has been activated at least once. Active processes can reactivate any previously activated process. The processes are Identify Problem, Definition/Feasibility, Identify Subproblems, Identify and Define Conceptual Structure, Conceptual Design, Detail Design, Code, Test Reasoning, Test Knowledge, Validation, Maintain/Enhance. Although this life cycle offers
Quality assurance techniques have been applied to conventional information systems for years. Many of these techniques, such as walk-throughs and testing, can be adapted for knowledge systems. As noted in Carrico, Girard, & Jones (1989), walk-throughs are an effective form of testing. The proposed methodology actively encourages experts and users to become involved in walk-throughs and testing. 3. METHODOLOGY The objective of this section is to present a methodology for designing the knowledge bases so that the foregoing management issues can be achieved. The methodology consists of four basic steps: Assessment Step, Dialog Specification Step, Rule Specification Step, and Coding Step. These steps have been created to facilitate the acquisition of the control knowledge. For business applications, the knowledge is composed primarily of two types: judgmental and control. Judgmental knowledge consists of rules that can be used to make judgments about the state of the universe. An example of a judgmental rule from a real estate loan knowledge system is: If real estate developer is building his or her specialty and developer is not building to sell project and developer has favorable previous experience then developer experience is high.
Designing Knowledged-Based Systems Control knowledge consists of rules that control the use of judgmental knowledge. A control rule from the real estate loan knowledge system is: If real estate developer has favorablelending experience with the bank, then don't inquire about developer experience. Usually, much of the control knowledge is undocumented and as a result, more difficult to acquire. However, it is necessary if a user-friendly robust knowledge system is desired. The Dialog Specification defines the control knowledge in a way which is intuitive for the business professional. See Lyons (1989) for a presentation of the methodology with an in depth example.
3.1. Assessment Step The objective of the Assessment Step is to assess the suitability of the potential application for expert system technology. For an in depth discussion of application selection, see Cupello and Mishelevich (1988), Keller (1987), Waterman (1986), and Weitzel and Kerschberg (1989). The end result is a document with the following information: • Task Domain--short statement of the general task domain the knowledge system will address • Task--statement of the specific task the knowledge system will solve • Expert--identification of domain expert and degree of commitment • Description of Dialog with Knowledge System~general description of typical user dialog with the knowledge system • Sample Recommendations--general description of some sample recommendations (outputs) of the knowledge system • Rules--general description of the rules and calculations used • Knowledge Resources---description of resources to be used as knowledge sources
3.2. Dialog Specification Step The objective of the Dialog Specification Step is to specify the dialog that the knowledge system will conduct with the user. By focusing on only the inputs/ outputs, and not the rules, the business professional can concentrate on an aspect of the knowledge system with which he/she is very familiar. However, by clarifying the dialog, the control knowledge necessary to handle the different types of consultations can be determined. The end result is a document with the following information: • Task--statement (possibly revised) of the specific task the knowledge system will solve • Text of Dialog with Knowledge System--the complete text of a typical user dialog with the future
89
knowledge system including questions, answers, and recommendations • Test Cases---documentation of other typical dialogs that the knowledge system will conduct • Recommendations--a complete list of all possible recommendations • Scope--brief description of any related problems which the knowledge system will not solve • Knowledge Resources--description (possibly revised) of resources to be used as knowledge sources
33. Rule Specification Step The objective of the Rule Specification Step is to specify the rules the knowledge system will require. This is accomplished by specifying a Goal Hierarchy Outline and a set of corresponding decision tables. The partitioning of the knowledge base into modules (decision tables) helps focus attention on meaningful chunks of knowledge. Research in human problem solving indicates that a human can maintain from four to seven chunks of knowledge in his/her short-term memory at the same time (Harmon & King, 1985). It also appears, based on the author's experience developing knowledge systems, that there should be no more than four to seven conditions in a rule. Usually there are natural groupings of the conditions so that there are no more than seven conditions in a rule. The process of determining the grouping of the conditions and the Goal Hierarchy Outline is explained in the following example. The end result of the Rule Specification Step is a document with the following information: • Taskmstatement (possibly revised) of the specific task the knowledge system will solve. • Text of Dialog with the Knowledge Systemmthe complete (possibly revised) text of a typical user dialog with the future knowledge system including questions, answers, and recommendations. • Goal Hierarchy Outline--this outline summarizes the backward chaining procedure that the knowledge system will use to conduct a consultation. This outline also specifies the source of each variable. The primary sources are a response from the user, a Lotus worksheet, a dBase file or a decision table. • Decision Tables--this set of hierarchical decision tables specifies groups of rules for every variable whose source is a decision table in the Goal Hierarchy Outline. The decision tables provide a compact format to analyze the logical correctness and completeness of a set of rules.
3.4. Coding Step The objective of the Coding Step is to implement the knowledge system with the selected expert system tool. The knowledge base is coded in stages. In the first stage, the highest-level decision table is implemented. With
90
P. J. Lyons
only this set of rules, the user can execute the knowledge base. However, the user is asked directly for the values of the secondary variables. As more decision tables are added, the knowledge base becomes more robust. As the encoder experiences this development process, he/ she acquires a better understanding of expert system technology. The final result of the Coding Step is a document with the following information: • Introduction---description of the case situation in sufficient detail so that a professional working in the field of the task domain would understand the application • Typical User Dialog--possibly revised • Goal Hierarchy Outline--possibly revised • Decision Tables--possibly revised • Listing of Knowledge Base
TABLE 2 Goal Hierarchy Outline for Project Eveluetor Suitability (Table A) Mgt_need_.satisfactn (Table B) Assist_sales (User) Replace_manuals (User) Service_products (User) Train_employees (User)
Appropriateness (Table C) Task_time (User) Total_recommendations (User) Totat_rules (User) Interface_Lotus (User) Interface_dBase (User) Utility (Table D) Task_relationship (User) Prototype_use (User)
4. PROJECT EVALUATOR The objective of this section is to demonstrate the use of the methodology with a straightforward stand-alone application. The case study chosen is a knowledge system to evaluate potential knowledge system projects. This system, the Project Evaluator knowledge system, is utilized by the author in management courses. The students execute the system after the first class meeting to create a one-page evaluation of their individual cases for use as projects in the course. This output of the system then becomes part of the Assessment Step, which the students submit in the third class meeting. The system illustrates very convincingly to the students how expert system technology can be used for effective communications.
4.1. Assessment Step The objective of the Assessment Step is to assess the suitability of the potential application for expert system
TABLE 1 Test Cases Condition Assist sales Replace manuals Service products Train employees Task time Total recommendations Total rules Interface Lotus Interface dBase Task relationship Prototype use Suitability
1
2
3
None
Partial
None
Partial Complete Partial 3-60
Partial None Partial 3-60
Complete Partial Partial 3-60
3-30 5-100 No No Position Useful
3-30 5-100 No No Organization Useful
3-30 <5 No Yes None None
Excellent
Good
Good
technology. Section A. 1.4 of the Appendix at the end of the article illustrates how the dialog can be described in a compact fashion but with sufficient detail to communicate the nature of the variables involved. Section A. 1.6 also illustrates how the rules can be described in a similar meaningful compact fashion.
4.2. Dialog Specification Step The objective of the Dialog Specification Step is to specify the dialog that the knowledge system will conduct with the user. By focusing on only the inputs/ outputs, and not the rules, the business professional can concentrate on an aspect of the knowledge system, with which he/she is very familiar. However, by clarifying the dialog, the control knowledge necessary to handle the different types of consultations can be determined. Section A.2.2 in the Appendix illustrates how the complete text of a dialog can be specified in a fashion that is intuitive to the business professional. There are three key advantages of producing a dialog specification document. First, it helps manage expectations. Early in the project everyone is made aware of what problems the resulting knowledge system will and will not solve (as specified in the Scope statement of Section A.2.5). A second advantage is the reduced time spent with the expert in the knowledge acquisition process. Without a comprehensive set of test cases, much time is spent reworking rules because the expert finds it unnatural to specify the rules explicitly. A third advantage is the reduced effort to design the knowledge base. With a comprehensive set of test cases, the business professional can use the process explained in Sections 3.3 and 4.3 to determine in a straightforward fashion the rules required for the knowledge base. The effort spent in performing the dialog specification step greatly increases the probability of successfully implementing expert system technology.
Designing Know~edged-Based Systems
91 TABLE 3 Decision Table A--Suitability Rule
Value
Mgt _need_satisfaction
1
2
3
4
5
High Medium .
Medium High
--
Appropriateness Utility
High High High
High High .
Suitability
Excellent
Good
4.3. Rule Specification Step The objective of the Rule Specification Step is to specify the rules the knowledge system will require. This is accomplished by specifying a Goal Hierarchy Outline and a set of corresponding decision tables. The partitioning of the knowledge base into modules (decision tables) helps focus attention on meaningful chunks of knowledge. The essential part of this step is to document the following issues. 4.3.1. Goal Hierarchy Outfine. The Goal Hierarchy Outline summarizes the backward chaining process for the knowledge base. The Goal Hierarchy Outline is determined by analyzing the test cases given in Table 1. One way to implement the knowledge base would be to create a knowledge base with four rules, each with 11 conditions, corresponding to the one dialog and the three test cases of Table 1. Such a knowledge system would handle each test case correctly but not much else. Implementing the knowledge base with such rules is not desirable. Another, more natural, way to design the knowledge base is to impose a limit of approximately seven as to the number of conditions in a rule. The primary reason for doing this is that seven appears to be the maximum number of conditions that people feel comfortable with when specifying heuristic rules. Looking down the list of conditions in the test cases given in Table l, it appears that the four conditions of Assist_sales, Replace_manuals, Service_products and Train_ employees form a logical grouping. This grouping is given a name, Mgt_need_satisfaction. Continuing down the list of conditions in the test cases, the next five conditions of Task_time, Total_recommendatns, Total rules, Interface__Lotus and Interface_dBase, can be grouped under the name of Appropriateness because they inquire about the appropriateness of the task for expert system technology. The remaining two conditions, Task_relationship and Prototype_use, form the last group called Utility because they describe the utility of the resulting prototype. The Goal Hierarchy Outline given in Table 2 specifies these relationships. 4.3.2. Decision Tables. Once the groupings have been determined, they are used to specify four sets of col-
.
Good
--
. Good
Poor
lectively exhaustive rules. The first-level rules specify the suitability in terms of Mgt_need ~tisfaction, Appropriateness, and Utility. For example, if the Mgt_need mtisfaction is high, the Appropriateness is high, and the Utility is high, then the suitability of the potential case is Excellent. Rules 2 through 4 of Table 3 define the good cases. Rule 5 is called a dummy rulem it has no explicit conditions. For example, if none of the first four rules is true for a potential case, then the suitability will be set to low. Thus the five rules determine the three values of suitability for all possible values of Mgt_need_satisfaction, Appropriateness, and Utility. At the second level, there are three decision tables-Mgt_need_satisfaction, Appropriateness, and Utility. One table for each variable in the first level decision table. The Mgt_need_satisfaction rules can be paraphrased as: If one of the needs is satisfied completely, then mgt_need_satisfaction is high. If not, if one of the needs is satisfied partially, then mgt_need_satisfaction is medium. If not, then mgt_need__satisfactionis low. These three rules determine a value for mgt_need_satisfaction for all values of Assist_sales, Replace_manuals, Service_products and Train_ employees. Table 4 specifies these rules explicitly in terms of these four variables. In a similar fashion, Table 5 documents the rules for Appropriateness. Note that if none of the first eight rules is true, then rule nine will be executed and Appropriateness will be set to low. TABLE 4 Decision TaMe BmManagement Need Satisfaction Rule Value
1
2
Assist_sales Replace._manuals
Complete or Complete or
Partial or Partial or
Service_products Train_employee
Complete or Partial or Complete or Partial or
Mgt _need_satisfaction
High
Medium
3
Low
92
P. J. Lyons
IIL'o .-I
3
AAA
Thus this decision table determines a value for Appropriateness for all possible values of the given conditions. Table 6 also has this property for Utility. Thus, no matter what values the 11 second-level conditions are given, there is a unique value specified for each of the second-level variables, Mgt._need r,atisfaction, Appropriateness, and Utility. These three variables, in turn, specify a unique value for suitability.
E 4.4. Coding Step
E tD
Q T.-
E
tt~
d 0
QO
Q
E
1= O,
dd
,,J m
0
Q
Q O
m
•
!
O
The objective of the Coding Step is to implement the knowledge system. The knowledge base is coded in four stages. In stage 1, the highest-level decision table, Decision Table A, is implemented. With only this set of rules, the user can execute the knowledge base (see Table 7). However, the user is asked directly for the values of the secondary variables. In stage 2, Decision Table B is encoded. The temporary Ask and Choices statements for Mgt_.need._satisfactn are removed and replaced with Rules B 1, B2, and the Ask and Choices statements for Assist_sales, Replace_manuals, Service_products and Train_employees (see Table 8). In a similar fashion, stages 3 and 4 encode Decision Tables C and D. As more decision tables are added, the knowledge base becomes more robust. By experiencing this development process, the student/professional acquires a better understanding of expert system technology. 5. E X P E R I E N C E U S I N G T H E METHODOLOGY
t~ e-
N
>-
17
i
"1-
¢.. "1"
Q Q O O ¢D ¢.. Z Z
"1-
¢-
._o ID ¢.-
> .tQ.
o
Q. m
B
<
The methodology has been used successfully for several semesters in an M.B.A. (Master of Business Administration) program. The majority of the students work full time and attend school part time. As a result the students are able to use their own business experience for the case studies. Each student submits an Assessment Step document; half of the cases are selected. The students are formed into pairs. The person who submitted the case acts as the expert while his/her partner functions as the knowledge engineer. An overview of some of the projects follows: The PC Advisor Knowledge System assists internal consultants in formulating solutions to end-user PC requirements (uses dBase files). The Acquisition Evaluator Knowledge System assists in evaluating the suitability of potential acquisition candidates (uses Lotus worksheet). The Workstation Selection Knowledge System assists in determining the proper display device for the internal user's access to computing resources. The Bid/No-Bid Knowledge System helps to determine if a small to medium sized defense company should pursue a new competitive procurement, the strategy and the effort to be expended.
DesigningKnowledged-BasedSystems
93 TABLE 6 DeeL..-~n Table D--Utility
Rule Value
1
Task_relationship Prototype_use
Position Useful
Utility
High
2
3
4
5
Position Demo
Organization Useful
Organization Demo
m
Medium
Medium
Medium
The Broker Knowledge System recommends brokers for sales distribution of woodenware products for a major wood products company. The MedicaidEligibility KnowledgeSystem determines if a person is eligible for medicaid support based on certain qualifications. The PC Shipping Route Selection Knowledge System attempts to recommend vendor and service selection for items shipped through a mail and shipping department. The methodology has been used successfully by the author on consulting assignments to develop larger systems. As with the foregoing students, the practicing professionals found the methodology very straightforward to use. 6. BENEFITS OF THE METHODOLOGY In Section 2, several issues relating to the successful implementation of expert system technology are reviewed. The objective of this section is to discuss the benefits of the methodology to the chosen case in particular and to knowledge systems in general.
6.1. Designing Robust Information Systems As discussed in Section 2.1, the concept of robustness revolves around having the capability of adapting efficiently to changing environments and continuing to function effectively under harsh dynamic conditions. As an example of this, consider the following situation. Suppose that a revised Risk Management Plan has identified a new need to consider the development cost when evaluating the suitability. Because the rules have been partitioned into modules (decision tables), the new condition---development cost--can be added to Decision Table A and a new decision table, Decision Table E, can be included with the rules to specify the different values of development cost. As a result, the Project Evaluator can be adapted efficiently to a changing environment. In general, if the knowledge base is developed according to the proposed methodology, then the rules will have seven or fewer conditions. The resulting knowledge base will be modularized. Just as modular programming eases maintenance of conventional software, modularizing the rules in the knowledge base
Low
will ease maintenance and result in a more robust knowledge system.
6.2. Viewing Information Systems as Change Agents After a knowledge system has been in use, it is not unusual for the need to modify the type of change agency from directed to nondirected or vise versa. If the proposed methodology has been used to develop explicitly the control knowledge (see Section 3), then it is a straightforward process to modify the type of change agency. For example, in the Project Evaluator, the knowledge base could be changed to give the user the option to give their own assessment of the management need satisfaction directly or to use the rules in Decision Table B. In general, if the rules for the knowledge base are developed according to the proposed methodology, then it will be a straightforward process to modify the type of change agency as required.
6.3. Altering the Traditional System Development Life Cycle There is an overall trend to use prototyping (and incremental development) in developing information systems, in general, and knowledge systems, in particular. In their survey on prototyping, Necco, Tsai, and Gordon (1989), found that 38% of a broad cross section of organizations throughout the United States use prototyping. A majority of the respondents stated that results achieved with using prototyping include the fact that changes are detected sooner, systems are developed in less time, users of the systems are more satisfied, and the systems are less expensive. As we discussed in Section 4.4, the Coding Step of the proposed methodology can be used to develop a series of knowledge bases as each decision table is added. On a higher level, the Dialog Specification Step, Rule Specification Step, and the Coding Step can be iterated in a fashion similar to that described in Boehm (1988) and Weitzel and Kerschberg (1989). For. example, the change in decisional guidance discussed in Section 6.2 can be implemented by iterating those three steps. In general, an iterative system development life cycle can be used with the proposed methodology.
94
P. J. Lyons TABLE 7 Knowledge Base with Decision Table A
! !
Project Evaluator Knowledge System II Patrick Lyons
! Overview--The Project Evaluator Knowledge System II is a ! simplified version of the Project Evaluator Knowledge System. ! It is used to illustrate the backward chaining process. ACTIONS Display" Project Evaluator II Press any key to begin the consultation.-" FIND Suitability DISPLAY" The suitability is { Suitability } ."; ! Decision Table A--Suitability ! This table determines the suitability of a particular case ! based on management need satisfaction, the appropriateness of ! the case for expert system technology and the utility of the ! resulting prototype. The value for management need ! satisfaction is found in Decision Table B. The values for ! appropriateness and utility are obtained from the user. ! The three possible values for suitability are Excellent, Good ! or Poor. Rule A1 IF Mgt_need_satisfactn = High AND Appropriateness = High AND Utility -- High THEN Suitability = excellent Rule A2 IF Mgt._need_satisfactn Appropriateness Utility THEN Suitability
= High AND = High AND < > Unknown = Good;
Rule A3 IF Mgt._need_satisfactn Appropriateness Utility THEN Suitability
= High AND = Medium AND < > Unknown = Good;
Rule A4 IF Mgt__need_satisfactn Appropriateness Utility THEN Suitability ELSE Suitability
= Medium AND = High AND < > Unknown = Good = Poor;
ASK Mgt_neecL satisfactn:" How well does the case satisfy management needs?"; CHOICES Mgt_need_satisfactn: High, Medium, Low; ASK Appropriateness:" How appropriate is this case for expert system technology?"; CHOICES Appropriateness: High, Medium, Low; ASK Utility:" How useful will the resulting prototype be?"; CHOICES Utility: High, Medium, Low;
Designing Knowledged-BasedSystems
95
TABLE 8 Knowledge Base with Decision Tables A and B (Partial Usting)
ACTIONS •
.
.
The suitability is {suitability}.";
!
Decision Table AmSuitability
A'sk Appropriateness:" How appropriate is this case for expert system technology?"; Choices Appropriateness: High, Medium, Low; Ask Utility:" How useful will the resulting prototype be?"; Choices Utility: High, Medium, Low;
[ Decision Table BmMgt._need_satisfactn [ This decision table determines the extent to which the case i satisfies cetain management needs in business. The four ! management needs considered are: ! the need to assist new sales people, ! the need to replace traditional procedure manuals, ! the need to service products, and ! the need to train or retrain employees. Rule B1 If Assist_sales = Complete or Replace__manuals = Complete or Service_products = Complete or Train_employees = Complete Then Mgt_need_satisfactn = High; Rule B2 If Assist_sales = Partial or Replace_.manuals = Partial or Service_products -- Partial or Train_employees = Partial Then Mgt_.need_satisfactn = Medium Else Mgt._need_satisfactn = Low; Ask Assist_sales:" Knowledge systems can assist new salespeople t o . . . " ; Ask Replace_manuals:" Knowledge systems can be used t o . . . " ; Ask Service_products:" Products and services have become m o r e . . . " ; Ask Train_employees:". Most corporations need to t r a i n . . . " ; Choices Assist_sales, Replace_manuais, Service_products, Train_employees: Complete, Partial, None;
6.4. Assuring the Quality of Knowledge Bases
port, Quality Measures and Assurance for AI Software
As discussed in Section 2.4, walk-throughs are a very effective form of testing• With the proposed methodology, the experts and users can be involved in quality assuring any of the decision tables that they developed. For more on verification and validation, see Geissman and Schultz (1988), Kang and Bahill (1990), Liebowitz (1990a,b), and Newquist (1988). Chapnick (1990) refers to an SRI International re-
by John Rushby, which states: "The determining factor in the deployment of AI software for serious applications may not be in the informal evidence of how well it is able to perform, but the extent to which formal guarantees can be given on how badly it can perform." Referring to an expert system that determines the loading sequence for material into the cargo bays o f an airplane, Rushby writes that we must evaluate it by saying " . . . the system generally seems to get as much
96
P . £ Lyons
material o n the plane as an experienced quartermaster, b u t it is g u a r a n t e e d never to l o a d the p l a n e in such a w a y that its center o f gravity is outside the allowable range." By using the p r o p o s e d m e t h o d o l o g y , experts a n d users can d e v e l o p explicitly similar rules a n d t h e r e b y g u a r a n t e e certain levels o f p e r f o r m a n c e . 7. C O N C L U S I O N T h e ability o f expert system t e c h n o l o g y to i m p l e m e n t systems not viable with conventional technology, m u s t be t e m p e r e d with appropriate m a n a g e m e n t techniques. As stated in B o e h m (1988), software d e v e l o p m e n t m e t h o d o l o g i e s m u s t be used with software process m o d e l s to d e t e r m i n e the o r d e r o f the stages involved in software d e v e l o p m e n t a n d to establish the t r a n s i t i o n criteria for progressing from o n e stage to another. This article has p r e s e n t e d such a m e t h o d o l o g y , which addressed the following m a n a g e m e n t issues: • Designing r o b u s t i n f o r m a t i o n systems • Viewing i n f o r m a t i o n systems as change agents • Altering the traditional system d e v e l o p m e n t life cycle • A s s u r i n g the quality o f k n o w l e d g e bases. R a t h e r t h a n discuss these issues in only b r o a d terms, this article presents the m e t h o d o l o g y , along with a detailed case. T h e m a i n c o n c l u s i o n is that, b y using the p r o p o s e d m e t h o d o l o g y , several benefits relating to the a b o v e m a n a g e m e n t issues can be achieved. Acknowledgments--This research was partially supported by the BusinessResearch Institute at St. John's University. The author wishes to thank the reviewers for their comments and suggestions.
Liebowitz, J. (Ed.). (1990b). "Verification and validation of knowledge-basedsystems." Expert Systems with Applications, C. Culbert (Guest Ed.), 1(3), 199-322. Lyons, P. (1989). A structured knowledge engineering methodology for Lotus-literate business professionals. Intelligent Systems Review, 2(1), 3-20. Necco, C., Tsai, N., & Gordon, C. (1989). Prototyping: Use in the development of computer-based information systems. Journal of Computer Information Systems, 30(1 ), 62-66. Newquist, H. (1988). Struggling to maintain. AI Expert, 3(8), 6971. Silver, M.S. (1990). Decision support systems: Directed and nondirected change. Information Systems Research, 1(1), 47-70. O'Keefe, R.M. (1990). An integrative model of expert system verification and validation. Expert Systems with Applications, 1(3), 231-236. Waterman, D.A. (1986). A guide to expert systems. Reading, MA: Addison-Wesley. Weitzel, J., & Kerschberg, L. (1989). Developing knowledge-based systems: Reorganizing the system development life cycle. Communications of the ACM, 32(4), 482--488.
APPENDIX: PROJECT
EVALUATOR
A.I. Assessment Step A. 1.1. Task Domain--the task domain consists of all those tasks associated with assisting students in learning about expert system technology. A. 1.2. Task--the Project Evaluator knowledge system assists a student in evaluating the suitability of a case for use as a knowledge system project. A.I.3. Expert--Dr. Patrick Lyons will devote 100 hours to this project during a three-week period.
REFERENCES Boehm, B.W. (1988). A spiral model of software development and enhancement. Computer. 21(5), 61-72. Carrico, M.A., Girard, J., & Jones, J. (1989). Building knowledge systems. New York: Intertext Publication of McGraw-Hill. Chapnick, P. (1990). Software quality assurance. AI Expert, 5(2), 56. Cupello, J.M., & Mishelevich, D. (1988, May). Managing prototype knowledge/expert system projects. Communications oftheACM, 31(5), 534-541. DeSalvo, D.A., Liebowitz, J. (Eds.). (1990). Managing artificial intelligence and expert systems. Englewood Cliffs,NJ: Prentice Hall. Dilworth, J.B. (1989). Production and operations management: Manufacturing and nonmanufacturing. New York: Random House, Business Division. El Sawy, O.A., & Nanus, B. (1989). Toward the design of robust information systems. Journal of Management Information Systems, 5(4), 33-54. Geissman, J., & Schultz, R. (1988). Verification and validation of expert systems. AI Expert, 3(2), 26-33. Harmon, P., & King, D. (1985). Expert systems: Artificial intelligence in business (p. 24). New York: Wiley. Kan~ Y., & BahiU, A.T. (1990). A tool for detecting expert system errors. AI Expert, 5(2), 46-51. Keller, R. (1987). Expert system technology. Englewood Cliffs, NJ: Prentice Hall. Liebowitz, J. (Ed.). (1990a). Expert systems for business and management. Englewood Cliffs, N J: Prentice Hall.
A.I.4. Description o f Dialog with Knowledge System--the dialog is divided into three parts. In the first part, a series of questions is asked to determine the potential of the case for satisfying some of the management needs in business today. Such needs are assisting new sales people, replacing traditional procedures manuals, servicing complex (financial) products, and training. The second part of the dialog centers on the appropriateness of the specific task to expert system technology, in general, and the course project, in specific. How long does it take to accomplish the typical task? How many possible recommendations are there? How many rules are there? What are the type of calculations? Does it involve the examination of diagrams? The third part of the dialog focuses on the value of using a knowledge system to perform the task. Does the task relate directly to your current position? Will the resulting prototype be useful? A. 1.5. Sample Recommendations--as a result of the dialog, the suitability of the case will be evaluated as one of three possible recommendations--excellent, good, or poor. A. 1.6. General Description o f Rules--as with the dialog, the rules are divided into parts. The first set of rules evaluates
Designing Knowledged-Based Systems the potential of the case to satisfy management needs. For example, if a case satisfies one of the identified needs completely, then its need satisfaction is high. The next set of rules classifies the appropriateness of the specific task to expert system technology, in general, and the course project, in specific. For example, if it takes 15 minutes to accomplish the task, there are about 50 rules and the computations are simple, then the appropriateness is high. The third group of rules seeks the utility of using the knowledge system to perform the task. For example, if the task relates directly to your current position and the resulting prototype will be useful, then the utility is high. The last set of rules prescribes the suitability of the case based on the management need satisfaction, the appropriateness of the specific task to expert system technology and the utility of using the knowledge system to perform the task. For example, if the need satisfaction is high, the appropriateness is high and the utility is high, then the suitability is excellent. All of the rules are qualitative and require no computation. A.I.7. KnowledgeResources--for the purposes of this project, the knowledge resources are limited to the project information sheets distributed to the class and Dr. Lyons.
A.2. Dialog-SpecificationStep A.2.1. Task--the Project Evaluator knowledge system assists a student in evaluating the suitability of a case for use as a knowledge system project. A.2.2. Text of Dialog with Knowledge System--Knowledge systems have the potential for satisfying many of the management needs in business today. The first part of the dialog will identify some of those needs. This may help you in choosing and/or refining your case. Your case may satisfy more than one of these management needs. Knowledge systems can assist new salespeople to provide customers with the answers and recommendations that only the best-trained and most experienced salespeople can provide. To what extent does your case involve the need for sales assistance? None, partial, or complete. Answer: none. Knowledge systems can be used to replace and/or automate some of the traditional procedures manuals. To what extent does your case involve the need to communicate and document new procedures and policies? None, partial, or complete. Answer. complete. Products and services have become more numerous and complex. Businesses in general and financial service organizations in particular are faced with the problem of servicing
97 their products. Knowledge systems can provide consistently competent service. To what extent does your case involve the need to service or repair products? None, partial, or complete. Answer: partial. Most corporations need to train or retrain large numbers of people. Knowledge systems can perform as patient tutors. To what extent does your case involve the need to train? None, partial, or complete. Answer: partial. The second part oftbe dialog centers on the appropriateness of the specific task to expert system technology. How long does it iake to accomplish the typical task? Less than 3 minutes? Between 3 and 60 minutes? More than one hour? Answer: between 3 and 60 minutes. The total possible number of recommendations are: Less than 3? Between 3 and 30? More than 30? Answer. between 3 and 30. An estimate as to the number of rules is: Less than 5? Between 5 and 100? More than 100? Answer: between 5 and 100. Will the knowledge system interface with Lotus worksheets? Yes or no. Answer: no. Will the knowledge system interface with dBase files? Yes or no. Answer: no. The third part of the dialog focuses on the value of using a knowledge system to perform the task. Does the task that the knowledge system solves relate directly to your: Current position? Current organization, but not current position? There is no relationship with current organization. Answer: position. Will the resulting prototype: Be directly useful to your organization? Be an interesting demo of expert system technology, but not directly useful? Have no use to your organization? Answer: useful. The suitability of the project is excellent. A.2.3. Test Cases~see Table 1. A.2.4. Recommendations--the Project Evaluator knowledge system will recommend a possible student case as (1) excellent, (2) good, or (3) poor. A.2.5. Scopemthe Project Evaluator knowledge system will evaluate the suitability of the case for use in a course, not in an organization. As a result, such factors as explicit cost/ benefit results, the organization's acceptance of knowledge systems, and the employee's computer skills will not be considered. A.2.6. KnowledgeResourcesmthe knowledge sources for the Project Evaluator knowledge system are limited to the project information sheets distributed to the class and Dr. Lyons.
0957--4174/92 $5.00 + .00 © 1991 Pergamon Pressplc
Printed in the USA.
Designing Knowledge-Based Systems for Incremental Development PATRICK J. LYONS St. John's University, Jamaica, New York 11439
AbstractmMany knowledge-based systems are developed in an incremental fashion because the knowledge is not fully understood at the start. To facilitate this process, the knowledge bases shouM be designed to accommodate revisions easily. This article presents a methodology for designing the knowledge bases to achieve this.
1. INTRODUCTION
the results of some of that research will be discussed along with its implications for the proposed methodology.
a powerful technology that must be well managed if its potential is to be achieved. As Chapnick (1990) noted, rapid prototyping (and expert system technology) may help build systems more quickly, but building incorrect or incomplete systems ten times faster is no gain. The objectives of this article are twofold. The first is to review some of the management issues relating to the successful implementation of expert system technology. The second is to present a methodology for designing the knowledge bases so that these management issues can be achieved. In Section 2, several issues relating to the successful implementation of expert system technology are reviewed, including: • Designing robust information systems • Viewing information systems as change agents • Altering the traditional system development life cycle • Assuring the quality of knowledge bases Section 3 describes a methodology for designing the knowledge bases so that the management issues can be achieved. In Section 4, a case is given to illustrate the methodology. Section 5 presents some of the experience using the methodology. Finally, Section 6 discusses the benefits of the methodology to the chosen case in particular and to knowledge systems in general. EXPERT SYSTEM TECHNOLOGY is
2.1. Designing Robust Information Systems As El Sawy and Nanus (1989) noted, the concept of robustness revolves around having the capability of adapting efficiently to changing environments and continuing to function effectively under harsh dynamic conditions. An information system can be defined as robust if it yields satisficing (satisfactory and sufficing) performance under all environments that are assessed as having an appreciable probability of occurring, and if it also has an associated scanning component that alerts it to environmental changes and enables switching to another configuration quickly and inexpensively to take advantage of the environmental change. The implication of robustness for the proposed methodology is that a Risk Management Plan, such as that described in Boehm (1988), should be developed. Such a plan identifies the top risk items for the project and develops a strategy for resolving them. For projects using expert system technology, the risks usually center on developing the wrong software functions and/or developing the wrong user interface. As will be shown, by developing a Risk Management Plan early in the project and using the proposed methodology, the system will be designed to accommodate more easily environmental changes. The result, by definition, will be a more robust knowledge system.
2. MANAGEMENT ISSUES There has been extensive research on the development of information systems in general and decision support systems and expert systems in particular. In this section,
2.2. Viewing Information Systems as Change Agents The concept of change agency is important for designing information systems. Silver (1990), in his study of Decision Support Systems (DSS), identified two vari-
Requests for reprints should be sent to Patrick J. Lyons, Department of Management, College of Business Administration, St. John's University, Jamaica, New York 11439.
87
88
eties of change agencies reflecting two different attitudes on the part of the system designer. On one hand, when designers comprehend that change will occur and deliberately attempt through a DSS to force the direction of that change, we have an instance of "directed" change. On the other hand, when designers understand that change will occur but do not try to influence the direction of that change, allowing it to be determined instead by the decision maker through DSS use over time, we have a case of "nondirected" change. Silver develops four design strategies for directed change and five design strategies for nondirected change. These strategies are based on the system restrictiveness and decisional guidance. The system restrictiveness is defined as the degree to which a DSS limits its users' decision-making processes to a subset of all possible processes. Decisional guidance refers to the degree to which a DSS guides its users in constructing and executing decision-making processes. The implication of change agency for the proposed methodology is that serious thought must be given at the beginning of the project to the impact that the new system will have on the organizational unit. Then one of the strategies identified in (Silver, 1990) (or a combination of strategies, if appropriate) should be adopted. The proposed methodology can be used in conjunction with the strategy for directed or nondirected change.
P. J. Lyons
greater flexibility, it typically will require more effort to control the many processes. Boehm (1988) presented a spiral model of the general software development process. The major distinguishing feature of this model is that it is a risk-driven approach to the software process rather than a primarily document-driven or code-driven process. In particular, risk-driven specifications can have varying degrees of completeness, formality, and granularity, depending on the relative risks of doing too little or too much specification. From a general management perspective, the risk analysis of the spiral model is an example of an ABC Analysis applied to project risk. For a general discussion of an ABC Analysis, see Dilworth (1989). The U.S. Department of Defense has established a standard on software management, DoD-Std-2167, which requires that developers produce and use risk management plans (Boehm, 1988). O'Keefe (1990) presents an explicit modification of Boehm's spiral model for expert system technology. The primary emphasis of O'Keefe's model is to identify verification and validation risk factors early in the software development process. However the life cycle is modified, the implication for the proposed methodology is that it must accommodate incremental development. The example in Section 4 illustrates this.
2A. Assuring the Quality of Knowledge Bases 2.3. Altering the Traditional System Development Life Cycle Because prototypes can be developed rapidly with expert system technology, there has been some research into altering the traditional systems development life cycle to take advantage of this. For example, Cupello and Mishelevich (1988) have proposed a three-stage process. In the first stage, a demonstration prototype is built that behaves intelligently in only a very limited segment of the domain. This proves the technology. The second stage is a full prototype that incorporates both breadth and depth across the problem domain. This system can be used to verify and validate the knowledge. The third stage produces the production system. In Weitzel and Kerschberg (1989), another modification to the life cycle is examined. To avoid the rigidity of sequential phases, the life cycle considered here consists of processes. Processes are activated, deactivated, and reactivated. They can run concurrently. Initially, a process is activated after the process above it has been activated at least once. Active processes can reactivate any previously activated process. The processes are Identify Problem, Definition/Feasibility, Identify Subproblems, Identify and Define Conceptual Structure, Conceptual Design, Detail Design, Code, Test Reasoning, Test Knowledge, Validation, Maintain/Enhance. Although this life cycle offers
Quality assurance techniques have been applied to conventional information systems for years. Many of these techniques, such as walk-throughs and testing, can be adapted for knowledge systems. As noted in Carrico, Girard, & Jones (1989), walk-throughs are an effective form of testing. The proposed methodology actively encourages experts and users to become involved in walk-throughs and testing. 3. METHODOLOGY The objective of this section is to present a methodology for designing the knowledge bases so that the foregoing management issues can be achieved. The methodology consists of four basic steps: Assessment Step, Dialog Specification Step, Rule Specification Step, and Coding Step. These steps have been created to facilitate the acquisition of the control knowledge. For business applications, the knowledge is composed primarily of two types: judgmental and control. Judgmental knowledge consists of rules that can be used to make judgments about the state of the universe. An example of a judgmental rule from a real estate loan knowledge system is: If real estate developer is building his or her specialty and developer is not building to sell project and developer has favorable previous experience then developer experience is high.
Designing Knowledged-Based Systems Control knowledge consists of rules that control the use of judgmental knowledge. A control rule from the real estate loan knowledge system is: If real estate developer has favorablelending experience with the bank, then don't inquire about developer experience. Usually, much of the control knowledge is undocumented and as a result, more difficult to acquire. However, it is necessary if a user-friendly robust knowledge system is desired. The Dialog Specification defines the control knowledge in a way which is intuitive for the business professional. See Lyons (1989) for a presentation of the methodology with an in depth example.
3.1. Assessment Step The objective of the Assessment Step is to assess the suitability of the potential application for expert system technology. For an in depth discussion of application selection, see Cupello and Mishelevich (1988), Keller (1987), Waterman (1986), and Weitzel and Kerschberg (1989). The end result is a document with the following information: • Task Domain--short statement of the general task domain the knowledge system will address • Task--statement of the specific task the knowledge system will solve • Expert--identification of domain expert and degree of commitment • Description of Dialog with Knowledge System~general description of typical user dialog with the knowledge system • Sample Recommendations--general description of some sample recommendations (outputs) of the knowledge system • Rules--general description of the rules and calculations used • Knowledge Resources---description of resources to be used as knowledge sources
3.2. Dialog Specification Step The objective of the Dialog Specification Step is to specify the dialog that the knowledge system will conduct with the user. By focusing on only the inputs/ outputs, and not the rules, the business professional can concentrate on an aspect of the knowledge system with which he/she is very familiar. However, by clarifying the dialog, the control knowledge necessary to handle the different types of consultations can be determined. The end result is a document with the following information: • Task--statement (possibly revised) of the specific task the knowledge system will solve • Text of Dialog with Knowledge System--the complete text of a typical user dialog with the future
89
knowledge system including questions, answers, and recommendations • Test Cases---documentation of other typical dialogs that the knowledge system will conduct • Recommendations--a complete list of all possible recommendations • Scope--brief description of any related problems which the knowledge system will not solve • Knowledge Resources--description (possibly revised) of resources to be used as knowledge sources
33. Rule Specification Step The objective of the Rule Specification Step is to specify the rules the knowledge system will require. This is accomplished by specifying a Goal Hierarchy Outline and a set of corresponding decision tables. The partitioning of the knowledge base into modules (decision tables) helps focus attention on meaningful chunks of knowledge. Research in human problem solving indicates that a human can maintain from four to seven chunks of knowledge in his/her short-term memory at the same time (Harmon & King, 1985). It also appears, based on the author's experience developing knowledge systems, that there should be no more than four to seven conditions in a rule. Usually there are natural groupings of the conditions so that there are no more than seven conditions in a rule. The process of determining the grouping of the conditions and the Goal Hierarchy Outline is explained in the following example. The end result of the Rule Specification Step is a document with the following information: • Taskmstatement (possibly revised) of the specific task the knowledge system will solve. • Text of Dialog with the Knowledge Systemmthe complete (possibly revised) text of a typical user dialog with the future knowledge system including questions, answers, and recommendations. • Goal Hierarchy Outline--this outline summarizes the backward chaining procedure that the knowledge system will use to conduct a consultation. This outline also specifies the source of each variable. The primary sources are a response from the user, a Lotus worksheet, a dBase file or a decision table. • Decision Tables--this set of hierarchical decision tables specifies groups of rules for every variable whose source is a decision table in the Goal Hierarchy Outline. The decision tables provide a compact format to analyze the logical correctness and completeness of a set of rules.
3.4. Coding Step The objective of the Coding Step is to implement the knowledge system with the selected expert system tool. The knowledge base is coded in stages. In the first stage, the highest-level decision table is implemented. With
90
P. J. Lyons
only this set of rules, the user can execute the knowledge base. However, the user is asked directly for the values of the secondary variables. As more decision tables are added, the knowledge base becomes more robust. As the encoder experiences this development process, he/ she acquires a better understanding of expert system technology. The final result of the Coding Step is a document with the following information: • Introduction---description of the case situation in sufficient detail so that a professional working in the field of the task domain would understand the application • Typical User Dialog--possibly revised • Goal Hierarchy Outline--possibly revised • Decision Tables--possibly revised • Listing of Knowledge Base
TABLE 2 Goal Hierarchy Outline for Project Eveluetor Suitability (Table A) Mgt_need_.satisfactn (Table B) Assist_sales (User) Replace_manuals (User) Service_products (User) Train_employees (User)
Appropriateness (Table C) Task_time (User) Total_recommendations (User) Totat_rules (User) Interface_Lotus (User) Interface_dBase (User) Utility (Table D) Task_relationship (User) Prototype_use (User)
4. PROJECT EVALUATOR The objective of this section is to demonstrate the use of the methodology with a straightforward stand-alone application. The case study chosen is a knowledge system to evaluate potential knowledge system projects. This system, the Project Evaluator knowledge system, is utilized by the author in management courses. The students execute the system after the first class meeting to create a one-page evaluation of their individual cases for use as projects in the course. This output of the system then becomes part of the Assessment Step, which the students submit in the third class meeting. The system illustrates very convincingly to the students how expert system technology can be used for effective communications.
4.1. Assessment Step The objective of the Assessment Step is to assess the suitability of the potential application for expert system
TABLE 1 Test Cases Condition Assist sales Replace manuals Service products Train employees Task time Total recommendations Total rules Interface Lotus Interface dBase Task relationship Prototype use Suitability
1
2
3
None
Partial
None
Partial Complete Partial 3-60
Partial None Partial 3-60
Complete Partial Partial 3-60
3-30 5-100 No No Position Useful
3-30 5-100 No No Organization Useful
3-30 <5 No Yes None None
Excellent
Good
Good
technology. Section A. 1.4 of the Appendix at the end of the article illustrates how the dialog can be described in a compact fashion but with sufficient detail to communicate the nature of the variables involved. Section A. 1.6 also illustrates how the rules can be described in a similar meaningful compact fashion.
4.2. Dialog Specification Step The objective of the Dialog Specification Step is to specify the dialog that the knowledge system will conduct with the user. By focusing on only the inputs/ outputs, and not the rules, the business professional can concentrate on an aspect of the knowledge system, with which he/she is very familiar. However, by clarifying the dialog, the control knowledge necessary to handle the different types of consultations can be determined. Section A.2.2 in the Appendix illustrates how the complete text of a dialog can be specified in a fashion that is intuitive to the business professional. There are three key advantages of producing a dialog specification document. First, it helps manage expectations. Early in the project everyone is made aware of what problems the resulting knowledge system will and will not solve (as specified in the Scope statement of Section A.2.5). A second advantage is the reduced time spent with the expert in the knowledge acquisition process. Without a comprehensive set of test cases, much time is spent reworking rules because the expert finds it unnatural to specify the rules explicitly. A third advantage is the reduced effort to design the knowledge base. With a comprehensive set of test cases, the business professional can use the process explained in Sections 3.3 and 4.3 to determine in a straightforward fashion the rules required for the knowledge base. The effort spent in performing the dialog specification step greatly increases the probability of successfully implementing expert system technology.
Designing Know~edged-Based Systems
91 TABLE 3 Decision Table A--Suitability Rule
Value
Mgt _need_satisfaction
1
2
3
4
5
High Medium .
Medium High
--
Appropriateness Utility
High High High
High High .
Suitability
Excellent
Good
4.3. Rule Specification Step The objective of the Rule Specification Step is to specify the rules the knowledge system will require. This is accomplished by specifying a Goal Hierarchy Outline and a set of corresponding decision tables. The partitioning of the knowledge base into modules (decision tables) helps focus attention on meaningful chunks of knowledge. The essential part of this step is to document the following issues. 4.3.1. Goal Hierarchy Outfine. The Goal Hierarchy Outline summarizes the backward chaining process for the knowledge base. The Goal Hierarchy Outline is determined by analyzing the test cases given in Table 1. One way to implement the knowledge base would be to create a knowledge base with four rules, each with 11 conditions, corresponding to the one dialog and the three test cases of Table 1. Such a knowledge system would handle each test case correctly but not much else. Implementing the knowledge base with such rules is not desirable. Another, more natural, way to design the knowledge base is to impose a limit of approximately seven as to the number of conditions in a rule. The primary reason for doing this is that seven appears to be the maximum number of conditions that people feel comfortable with when specifying heuristic rules. Looking down the list of conditions in the test cases given in Table l, it appears that the four conditions of Assist_sales, Replace_manuals, Service_products and Train_ employees form a logical grouping. This grouping is given a name, Mgt_need_satisfaction. Continuing down the list of conditions in the test cases, the next five conditions of Task_time, Total_recommendatns, Total rules, Interface__Lotus and Interface_dBase, can be grouped under the name of Appropriateness because they inquire about the appropriateness of the task for expert system technology. The remaining two conditions, Task_relationship and Prototype_use, form the last group called Utility because they describe the utility of the resulting prototype. The Goal Hierarchy Outline given in Table 2 specifies these relationships. 4.3.2. Decision Tables. Once the groupings have been determined, they are used to specify four sets of col-
.
Good
--
. Good
Poor
lectively exhaustive rules. The first-level rules specify the suitability in terms of Mgt_need ~tisfaction, Appropriateness, and Utility. For example, if the Mgt_need mtisfaction is high, the Appropriateness is high, and the Utility is high, then the suitability of the potential case is Excellent. Rules 2 through 4 of Table 3 define the good cases. Rule 5 is called a dummy rulem it has no explicit conditions. For example, if none of the first four rules is true for a potential case, then the suitability will be set to low. Thus the five rules determine the three values of suitability for all possible values of Mgt_need_satisfaction, Appropriateness, and Utility. At the second level, there are three decision tables-Mgt_need_satisfaction, Appropriateness, and Utility. One table for each variable in the first level decision table. The Mgt_need_satisfaction rules can be paraphrased as: If one of the needs is satisfied completely, then mgt_need_satisfaction is high. If not, if one of the needs is satisfied partially, then mgt_need_satisfaction is medium. If not, then mgt_need__satisfactionis low. These three rules determine a value for mgt_need_satisfaction for all values of Assist_sales, Replace_manuals, Service_products and Train_ employees. Table 4 specifies these rules explicitly in terms of these four variables. In a similar fashion, Table 5 documents the rules for Appropriateness. Note that if none of the first eight rules is true, then rule nine will be executed and Appropriateness will be set to low. TABLE 4 Decision TaMe BmManagement Need Satisfaction Rule Value
1
2
Assist_sales Replace._manuals
Complete or Complete or
Partial or Partial or
Service_products Train_employee
Complete or Partial or Complete or Partial or
Mgt _need_satisfaction
High
Medium
3
Low
92
P. J. Lyons
IIL'o .-I
3
AAA
Thus this decision table determines a value for Appropriateness for all possible values of the given conditions. Table 6 also has this property for Utility. Thus, no matter what values the 11 second-level conditions are given, there is a unique value specified for each of the second-level variables, Mgt._need r,atisfaction, Appropriateness, and Utility. These three variables, in turn, specify a unique value for suitability.
E 4.4. Coding Step
E tD
Q T.-
E
tt~
d 0
QO
Q
E
1= O,
dd
,,J m
0
Q
Q O
m
•
!
O
The objective of the Coding Step is to implement the knowledge system. The knowledge base is coded in four stages. In stage 1, the highest-level decision table, Decision Table A, is implemented. With only this set of rules, the user can execute the knowledge base (see Table 7). However, the user is asked directly for the values of the secondary variables. In stage 2, Decision Table B is encoded. The temporary Ask and Choices statements for Mgt_.need._satisfactn are removed and replaced with Rules B 1, B2, and the Ask and Choices statements for Assist_sales, Replace_manuals, Service_products and Train_employees (see Table 8). In a similar fashion, stages 3 and 4 encode Decision Tables C and D. As more decision tables are added, the knowledge base becomes more robust. By experiencing this development process, the student/professional acquires a better understanding of expert system technology. 5. E X P E R I E N C E U S I N G T H E METHODOLOGY
t~ e-
N
>-
17
i
"1-
¢.. "1"
Q Q O O ¢D ¢.. Z Z
"1-
¢-
._o ID ¢.-
> .tQ.
o
Q. m
B
<
The methodology has been used successfully for several semesters in an M.B.A. (Master of Business Administration) program. The majority of the students work full time and attend school part time. As a result the students are able to use their own business experience for the case studies. Each student submits an Assessment Step document; half of the cases are selected. The students are formed into pairs. The person who submitted the case acts as the expert while his/her partner functions as the knowledge engineer. An overview of some of the projects follows: The PC Advisor Knowledge System assists internal consultants in formulating solutions to end-user PC requirements (uses dBase files). The Acquisition Evaluator Knowledge System assists in evaluating the suitability of potential acquisition candidates (uses Lotus worksheet). The Workstation Selection Knowledge System assists in determining the proper display device for the internal user's access to computing resources. The Bid/No-Bid Knowledge System helps to determine if a small to medium sized defense company should pursue a new competitive procurement, the strategy and the effort to be expended.
DesigningKnowledged-BasedSystems
93 TABLE 6 DeeL..-~n Table D--Utility
Rule Value
1
Task_relationship Prototype_use
Position Useful
Utility
High
2
3
4
5
Position Demo
Organization Useful
Organization Demo
m
Medium
Medium
Medium
The Broker Knowledge System recommends brokers for sales distribution of woodenware products for a major wood products company. The MedicaidEligibility KnowledgeSystem determines if a person is eligible for medicaid support based on certain qualifications. The PC Shipping Route Selection Knowledge System attempts to recommend vendor and service selection for items shipped through a mail and shipping department. The methodology has been used successfully by the author on consulting assignments to develop larger systems. As with the foregoing students, the practicing professionals found the methodology very straightforward to use. 6. BENEFITS OF THE METHODOLOGY In Section 2, several issues relating to the successful implementation of expert system technology are reviewed. The objective of this section is to discuss the benefits of the methodology to the chosen case in particular and to knowledge systems in general.
6.1. Designing Robust Information Systems As discussed in Section 2.1, the concept of robustness revolves around having the capability of adapting efficiently to changing environments and continuing to function effectively under harsh dynamic conditions. As an example of this, consider the following situation. Suppose that a revised Risk Management Plan has identified a new need to consider the development cost when evaluating the suitability. Because the rules have been partitioned into modules (decision tables), the new condition---development cost--can be added to Decision Table A and a new decision table, Decision Table E, can be included with the rules to specify the different values of development cost. As a result, the Project Evaluator can be adapted efficiently to a changing environment. In general, if the knowledge base is developed according to the proposed methodology, then the rules will have seven or fewer conditions. The resulting knowledge base will be modularized. Just as modular programming eases maintenance of conventional software, modularizing the rules in the knowledge base
Low
will ease maintenance and result in a more robust knowledge system.
6.2. Viewing Information Systems as Change Agents After a knowledge system has been in use, it is not unusual for the need to modify the type of change agency from directed to nondirected or vise versa. If the proposed methodology has been used to develop explicitly the control knowledge (see Section 3), then it is a straightforward process to modify the type of change agency. For example, in the Project Evaluator, the knowledge base could be changed to give the user the option to give their own assessment of the management need satisfaction directly or to use the rules in Decision Table B. In general, if the rules for the knowledge base are developed according to the proposed methodology, then it will be a straightforward process to modify the type of change agency as required.
6.3. Altering the Traditional System Development Life Cycle There is an overall trend to use prototyping (and incremental development) in developing information systems, in general, and knowledge systems, in particular. In their survey on prototyping, Necco, Tsai, and Gordon (1989), found that 38% of a broad cross section of organizations throughout the United States use prototyping. A majority of the respondents stated that results achieved with using prototyping include the fact that changes are detected sooner, systems are developed in less time, users of the systems are more satisfied, and the systems are less expensive. As we discussed in Section 4.4, the Coding Step of the proposed methodology can be used to develop a series of knowledge bases as each decision table is added. On a higher level, the Dialog Specification Step, Rule Specification Step, and the Coding Step can be iterated in a fashion similar to that described in Boehm (1988) and Weitzel and Kerschberg (1989). For. example, the change in decisional guidance discussed in Section 6.2 can be implemented by iterating those three steps. In general, an iterative system development life cycle can be used with the proposed methodology.
94
P. J. Lyons TABLE 7 Knowledge Base with Decision Table A
! !
Project Evaluator Knowledge System II Patrick Lyons
! Overview--The Project Evaluator Knowledge System II is a ! simplified version of the Project Evaluator Knowledge System. ! It is used to illustrate the backward chaining process. ACTIONS Display" Project Evaluator II Press any key to begin the consultation.-" FIND Suitability DISPLAY" The suitability is { Suitability } ."; ! Decision Table A--Suitability ! This table determines the suitability of a particular case ! based on management need satisfaction, the appropriateness of ! the case for expert system technology and the utility of the ! resulting prototype. The value for management need ! satisfaction is found in Decision Table B. The values for ! appropriateness and utility are obtained from the user. ! The three possible values for suitability are Excellent, Good ! or Poor. Rule A1 IF Mgt_need_satisfactn = High AND Appropriateness = High AND Utility -- High THEN Suitability = excellent Rule A2 IF Mgt._need_satisfactn Appropriateness Utility THEN Suitability
= High AND = High AND < > Unknown = Good;
Rule A3 IF Mgt._need_satisfactn Appropriateness Utility THEN Suitability
= High AND = Medium AND < > Unknown = Good;
Rule A4 IF Mgt__need_satisfactn Appropriateness Utility THEN Suitability ELSE Suitability
= Medium AND = High AND < > Unknown = Good = Poor;
ASK Mgt_neecL satisfactn:" How well does the case satisfy management needs?"; CHOICES Mgt_need_satisfactn: High, Medium, Low; ASK Appropriateness:" How appropriate is this case for expert system technology?"; CHOICES Appropriateness: High, Medium, Low; ASK Utility:" How useful will the resulting prototype be?"; CHOICES Utility: High, Medium, Low;
Designing Knowledged-BasedSystems
95
TABLE 8 Knowledge Base with Decision Tables A and B (Partial Usting)
ACTIONS •
.
.
The suitability is {suitability}.";
!
Decision Table AmSuitability
A'sk Appropriateness:" How appropriate is this case for expert system technology?"; Choices Appropriateness: High, Medium, Low; Ask Utility:" How useful will the resulting prototype be?"; Choices Utility: High, Medium, Low;
[ Decision Table BmMgt._need_satisfactn [ This decision table determines the extent to which the case i satisfies cetain management needs in business. The four ! management needs considered are: ! the need to assist new sales people, ! the need to replace traditional procedure manuals, ! the need to service products, and ! the need to train or retrain employees. Rule B1 If Assist_sales = Complete or Replace__manuals = Complete or Service_products = Complete or Train_employees = Complete Then Mgt_need_satisfactn = High; Rule B2 If Assist_sales = Partial or Replace_.manuals = Partial or Service_products -- Partial or Train_employees = Partial Then Mgt_.need_satisfactn = Medium Else Mgt._need_satisfactn = Low; Ask Assist_sales:" Knowledge systems can assist new salespeople t o . . . " ; Ask Replace_manuals:" Knowledge systems can be used t o . . . " ; Ask Service_products:" Products and services have become m o r e . . . " ; Ask Train_employees:". Most corporations need to t r a i n . . . " ; Choices Assist_sales, Replace_manuais, Service_products, Train_employees: Complete, Partial, None;
6.4. Assuring the Quality of Knowledge Bases
port, Quality Measures and Assurance for AI Software
As discussed in Section 2.4, walk-throughs are a very effective form of testing• With the proposed methodology, the experts and users can be involved in quality assuring any of the decision tables that they developed. For more on verification and validation, see Geissman and Schultz (1988), Kang and Bahill (1990), Liebowitz (1990a,b), and Newquist (1988). Chapnick (1990) refers to an SRI International re-
by John Rushby, which states: "The determining factor in the deployment of AI software for serious applications may not be in the informal evidence of how well it is able to perform, but the extent to which formal guarantees can be given on how badly it can perform." Referring to an expert system that determines the loading sequence for material into the cargo bays o f an airplane, Rushby writes that we must evaluate it by saying " . . . the system generally seems to get as much
96
P . £ Lyons
material o n the plane as an experienced quartermaster, b u t it is g u a r a n t e e d never to l o a d the p l a n e in such a w a y that its center o f gravity is outside the allowable range." By using the p r o p o s e d m e t h o d o l o g y , experts a n d users can d e v e l o p explicitly similar rules a n d t h e r e b y g u a r a n t e e certain levels o f p e r f o r m a n c e . 7. C O N C L U S I O N T h e ability o f expert system t e c h n o l o g y to i m p l e m e n t systems not viable with conventional technology, m u s t be t e m p e r e d with appropriate m a n a g e m e n t techniques. As stated in B o e h m (1988), software d e v e l o p m e n t m e t h o d o l o g i e s m u s t be used with software process m o d e l s to d e t e r m i n e the o r d e r o f the stages involved in software d e v e l o p m e n t a n d to establish the t r a n s i t i o n criteria for progressing from o n e stage to another. This article has p r e s e n t e d such a m e t h o d o l o g y , which addressed the following m a n a g e m e n t issues: • Designing r o b u s t i n f o r m a t i o n systems • Viewing i n f o r m a t i o n systems as change agents • Altering the traditional system d e v e l o p m e n t life cycle • A s s u r i n g the quality o f k n o w l e d g e bases. R a t h e r t h a n discuss these issues in only b r o a d terms, this article presents the m e t h o d o l o g y , along with a detailed case. T h e m a i n c o n c l u s i o n is that, b y using the p r o p o s e d m e t h o d o l o g y , several benefits relating to the a b o v e m a n a g e m e n t issues can be achieved. Acknowledgments--This research was partially supported by the BusinessResearch Institute at St. John's University. The author wishes to thank the reviewers for their comments and suggestions.
Liebowitz, J. (Ed.). (1990b). "Verification and validation of knowledge-basedsystems." Expert Systems with Applications, C. Culbert (Guest Ed.), 1(3), 199-322. Lyons, P. (1989). A structured knowledge engineering methodology for Lotus-literate business professionals. Intelligent Systems Review, 2(1), 3-20. Necco, C., Tsai, N., & Gordon, C. (1989). Prototyping: Use in the development of computer-based information systems. Journal of Computer Information Systems, 30(1 ), 62-66. Newquist, H. (1988). Struggling to maintain. AI Expert, 3(8), 6971. Silver, M.S. (1990). Decision support systems: Directed and nondirected change. Information Systems Research, 1(1), 47-70. O'Keefe, R.M. (1990). An integrative model of expert system verification and validation. Expert Systems with Applications, 1(3), 231-236. Waterman, D.A. (1986). A guide to expert systems. Reading, MA: Addison-Wesley. Weitzel, J., & Kerschberg, L. (1989). Developing knowledge-based systems: Reorganizing the system development life cycle. Communications of the ACM, 32(4), 482--488.
APPENDIX: PROJECT
EVALUATOR
A.I. Assessment Step A. 1.1. Task Domain--the task domain consists of all those tasks associated with assisting students in learning about expert system technology. A. 1.2. Task--the Project Evaluator knowledge system assists a student in evaluating the suitability of a case for use as a knowledge system project. A.I.3. Expert--Dr. Patrick Lyons will devote 100 hours to this project during a three-week period.
REFERENCES Boehm, B.W. (1988). A spiral model of software development and enhancement. Computer. 21(5), 61-72. Carrico, M.A., Girard, J., & Jones, J. (1989). Building knowledge systems. New York: Intertext Publication of McGraw-Hill. Chapnick, P. (1990). Software quality assurance. AI Expert, 5(2), 56. Cupello, J.M., & Mishelevich, D. (1988, May). Managing prototype knowledge/expert system projects. Communications oftheACM, 31(5), 534-541. DeSalvo, D.A., Liebowitz, J. (Eds.). (1990). Managing artificial intelligence and expert systems. Englewood Cliffs,NJ: Prentice Hall. Dilworth, J.B. (1989). Production and operations management: Manufacturing and nonmanufacturing. New York: Random House, Business Division. El Sawy, O.A., & Nanus, B. (1989). Toward the design of robust information systems. Journal of Management Information Systems, 5(4), 33-54. Geissman, J., & Schultz, R. (1988). Verification and validation of expert systems. AI Expert, 3(2), 26-33. Harmon, P., & King, D. (1985). Expert systems: Artificial intelligence in business (p. 24). New York: Wiley. Kan~ Y., & BahiU, A.T. (1990). A tool for detecting expert system errors. AI Expert, 5(2), 46-51. Keller, R. (1987). Expert system technology. Englewood Cliffs, NJ: Prentice Hall. Liebowitz, J. (Ed.). (1990a). Expert systems for business and management. Englewood Cliffs, N J: Prentice Hall.
A.I.4. Description o f Dialog with Knowledge System--the dialog is divided into three parts. In the first part, a series of questions is asked to determine the potential of the case for satisfying some of the management needs in business today. Such needs are assisting new sales people, replacing traditional procedures manuals, servicing complex (financial) products, and training. The second part of the dialog centers on the appropriateness of the specific task to expert system technology, in general, and the course project, in specific. How long does it take to accomplish the typical task? How many possible recommendations are there? How many rules are there? What are the type of calculations? Does it involve the examination of diagrams? The third part of the dialog focuses on the value of using a knowledge system to perform the task. Does the task relate directly to your current position? Will the resulting prototype be useful? A. 1.5. Sample Recommendations--as a result of the dialog, the suitability of the case will be evaluated as one of three possible recommendations--excellent, good, or poor. A. 1.6. General Description o f Rules--as with the dialog, the rules are divided into parts. The first set of rules evaluates
Designing Knowledged-Based Systems the potential of the case to satisfy management needs. For example, if a case satisfies one of the identified needs completely, then its need satisfaction is high. The next set of rules classifies the appropriateness of the specific task to expert system technology, in general, and the course project, in specific. For example, if it takes 15 minutes to accomplish the task, there are about 50 rules and the computations are simple, then the appropriateness is high. The third group of rules seeks the utility of using the knowledge system to perform the task. For example, if the task relates directly to your current position and the resulting prototype will be useful, then the utility is high. The last set of rules prescribes the suitability of the case based on the management need satisfaction, the appropriateness of the specific task to expert system technology and the utility of using the knowledge system to perform the task. For example, if the need satisfaction is high, the appropriateness is high and the utility is high, then the suitability is excellent. All of the rules are qualitative and require no computation. A.I.7. KnowledgeResources--for the purposes of this project, the knowledge resources are limited to the project information sheets distributed to the class and Dr. Lyons.
A.2. Dialog-SpecificationStep A.2.1. Task--the Project Evaluator knowledge system assists a student in evaluating the suitability of a case for use as a knowledge system project. A.2.2. Text of Dialog with Knowledge System--Knowledge systems have the potential for satisfying many of the management needs in business today. The first part of the dialog will identify some of those needs. This may help you in choosing and/or refining your case. Your case may satisfy more than one of these management needs. Knowledge systems can assist new salespeople to provide customers with the answers and recommendations that only the best-trained and most experienced salespeople can provide. To what extent does your case involve the need for sales assistance? None, partial, or complete. Answer: none. Knowledge systems can be used to replace and/or automate some of the traditional procedures manuals. To what extent does your case involve the need to communicate and document new procedures and policies? None, partial, or complete. Answer. complete. Products and services have become more numerous and complex. Businesses in general and financial service organizations in particular are faced with the problem of servicing
97 their products. Knowledge systems can provide consistently competent service. To what extent does your case involve the need to service or repair products? None, partial, or complete. Answer: partial. Most corporations need to train or retrain large numbers of people. Knowledge systems can perform as patient tutors. To what extent does your case involve the need to train? None, partial, or complete. Answer: partial. The second part oftbe dialog centers on the appropriateness of the specific task to expert system technology. How long does it iake to accomplish the typical task? Less than 3 minutes? Between 3 and 60 minutes? More than one hour? Answer: between 3 and 60 minutes. The total possible number of recommendations are: Less than 3? Between 3 and 30? More than 30? Answer. between 3 and 30. An estimate as to the number of rules is: Less than 5? Between 5 and 100? More than 100? Answer: between 5 and 100. Will the knowledge system interface with Lotus worksheets? Yes or no. Answer: no. Will the knowledge system interface with dBase files? Yes or no. Answer: no. The third part of the dialog focuses on the value of using a knowledge system to perform the task. Does the task that the knowledge system solves relate directly to your: Current position? Current organization, but not current position? There is no relationship with current organization. Answer: position. Will the resulting prototype: Be directly useful to your organization? Be an interesting demo of expert system technology, but not directly useful? Have no use to your organization? Answer: useful. The suitability of the project is excellent. A.2.3. Test Cases~see Table 1. A.2.4. Recommendations--the Project Evaluator knowledge system will recommend a possible student case as (1) excellent, (2) good, or (3) poor. A.2.5. Scopemthe Project Evaluator knowledge system will evaluate the suitability of the case for use in a course, not in an organization. As a result, such factors as explicit cost/ benefit results, the organization's acceptance of knowledge systems, and the employee's computer skills will not be considered. A.2.6. KnowledgeResourcesmthe knowledge sources for the Project Evaluator knowledge system are limited to the project information sheets distributed to the class and Dr. Lyons.