- Email: [email protected]
Validation of an accounting expert system for business combinations
Expert SystemsWith Applications, Vol. 7, No. 2, pp. 175-183, 1994 Copyright © 1994 ElsevierScience Lid Printed in the USA. All rights reserved 0957-4174/94 $6.00 + .00
Pergamon
Validation of an Accounting Expert System for Business Combinations R. STEVE McDUFFIE Department of Accounting, Southwest Missouri State University, Springfield, MO
L. MURPHY SMITH Department of Accounting,TexasA&MUniversity,CollegeStation,TX
STEVEN M. FLORY Department of Accounting, University of Southwestern Louisiana, Lafayette, LA
Abstract--This article presents the five stages involved in the construction of a knowledge-based system using VP-EXPERT, with emphasis on the final step, knowledge validation. The knowledgebased system, called PURPOOL, enables the user to determine the proper accounting treatment for a business combination. PURPOOL can be used effectively by educators as a pedagogical device to teach their students how to account for business combinations. PURPOOL also can be used by accounting practitioners in training inexperienced staff accountants or in determining the proper accounting treatment of actual problems encountered in practice. PURPOOL is one of thefirst GAAPbased expert systems reported in the accounting literature. A GAAP-based system relies upon the current authoritative accounting literature (Financial Accounting Standards Board Statements, Accounting Principles Board Opinions, etc.)for the knowledge base rules. PURPOOL captures the content of APB Opinion No. 16, Business Combinations in its knowledge base. The validation of PURPOOL utilized a framework consisting of nine elements. This process included interaction with a human expert, review of the accounting literature pertaining to business combinations, development of test cases, consultations with audit seniors from a regional accounting firm, and consultations with accounting information systems and advanced accountingprofessors. As a result of using theframework, a thorough validation of the system was accomplished, and that process is described in this article.
1. INTRODUCTION
in the AICPA report departs from the typical accounting expert systems developed thus far. The traditional accounting expert system chiefly depends upon a human expert in the construction of the knowledge base of the expert system. The new type of expert system presented in the special report chiefly depends upon generally accepted accounting principles (GAAP) to provide information for the development of the rules contained in the knowledge base. These GAAP-based expert systems still rely on the human expert for system validation, but primarily rely upon the current authoritative accounting literature for the information to be included in the system's knowledge base. At least one such expert system has been reported in the literature (Boer & Livnat, 1990). The researchers developed an expert system to solve lease problems
IN 1987 THE AMERICAN INSTITUTE OF CERTIFIED PUBLIC ACCOUNTANTS( A I C P A ) published An Intro-
duction to Artificial Intelligence and Expert Systems.
One section of the report presented applications suitable for future expert system development. The applications included leases, foreign currency translation, acquisitions, pensions, and income taxes. Expert systems are being used and developed for accounting, auditing, and tax practice (Brown, 1991). The type of expert system development described
Requests for reprints should be sent to R. Steve McDutfie, Department of Accounting, Southwest Missouri State University, Springfield, MO 65804-0094.
175
176
R. S. McDuffie et al.
from the standpoint of the lessee. The system was used as a teaching aid by the researchers. Students who used the expert system performed better in solving lease problems than students who did not use the expert system. This paper describes the five stages involved in the construction of the GAAP-based expert system, PURPOOL. The focus of the paper is upon the final stage of construction, knowledge validation. The system determines the proper accounting treatment for business combinations. PURPOOL indicates whether a combination should be treated as a purchase or as a pooling of interests. The five stages in constructing a GAAP-based expert system are as follows: 1. Problem selection 2. Knowledge acquisition 3. Knowledge representation 4. Knowledge programming 5. Knowledge validation 2. P R O B L E M S E L E C T I O N Anyone familiar with advanced accounting understands the difficulty of determining whether a business combination should be recorded as a purchase or as a pooling of interests. Several CPAs were queried about the potential benefits of PURPOOL. They indicated that a commercial version of PURPOOL could be beneficial to their firms. A senior manager for a national accounting firm indicated during a conversation that her firm had developed a similar type of expert system prototype. She further indicated that an expert system like PURPOOL would be of substantial value. Former Deputy Chief Accountant of the Securities and Exchange Commission (SEC), Glen L. Davison, indicated that the issue of purchase versus pooling of interests for business combinations was the most frequent problem area considered by the SEC during his 3-year tenure with the Commission. Mr. Davison further provided that while he was a manager with a national accounting firm's central office in New York, the issue of purchase versus pooling of interests for business combinations was the most common problem addressed by that office. Certain task characteristics should be present to identify a problem as suitable for the development of an expert system. The problem of determining the proper accounting treatment of a business combination appears to match these characteristics. The task characteristics, as they related to PURPOOL, are discussed below. 2.1. Well-Bounded Task
A well-bounded task is one confined to a narrow, welldefined problem domain. The determination of the
proper method of accounting for a business combination is a well-bound task. At the prototype stage, P U R P O O L evaluates the 12 criteria contained in APB Opinion No. 16. 2.2. Few Minutes to a Few Hours to Solve
The evaluation of the 12 criteria contained in Opinion No. 16 would take at least a few hours, because the opinion itself is over 40 pages long. 2.3. Task is Frequently Performed
From the point of view of a national accounting firm, the decision of how a business combination should be recorded could occur numerous times on a firm-wide basis during a year. 2.4. Test Data is Available
The criteria in Opinion No. 16 lends itself to the construction of case studies. Case studies provided appropriate test data for the purpose of validating PURPOOL. 2.5. General Consensus
After a detailed review, the criteria in Opinion No. 16 should lead accountants to a similar conclusion concerning the accounting treatment of a particular business combination. Because of the reaction of the various CPAs to the potential benefits of PURPOOL, the comments from former Deputy Chief Accountant Davison, and the task characteristics associated with determining if a business combination is a purchase or pooling of interests, the authors concluded that this was a suitable problem area for expert system development. 3. K N O W L E D G E ACQUISITION Knowledge acquisition for PURPOOL was different from the acquisition used for typical accounting expert systems. The primary source of knowledge for PURPOOL was Accounting Principles Board (APB ) Opinion No. 16, which defines a business combination as a purchase or as a pooling of interests. Hollander (1987) used a similar approach for knowledge acquisition in her Ph.D. dissertation. She extracted knowledge from accounting text books, articles, etc. to be used in an expert system prototype that determined which standard cost variances should be analyzed by a company's management. With the knowledge extracted in this fashion, she constructed a prototype that identified price and use variances for material and labor that should be reviewed by management.
Accounting Expert System A domain expert was used, to a limited extent, during the knowledge acquisition stage of the construction of PURPOOL. As a partner in charge of a national accounting firm office, the expert was experienced with the problems associated with business combinations. The domain expert was queried at length during the development of PURPOOL and reviewed the system, including the rules it contained during validation. In addition to Opinion No. 16, two national public accounting firms provided sections from their accounting manuals concerning the purchase method and the pooling of interests method of recording business combinations. Also, explanatory material concerning Opinion No. 16 were reviewed, and several advanced accounting text books were consulted. 4. KNOWLEDGE REPRESENTATION A production rule format was selected for the development of PURPOOL. The knowledge pertaining to business combinations lends itself to the use of production rules (i.e., if/then rules). Production rules are the most popular method of knowledge representation in expert system development (Liebowitz, 1988, p. 47 ). 5. KNOWLEDGE PROGRAMMING Early in the development of PURPOOL the authors decided against using a formal programming language, such as LISP or PROLOG, to encode the knowledge acquired into the knowledge base. Using such programming languages requires extensive training, time, and effort beyond the resources of most accountants. Expert system shell software has been identified as an excellent tool for use by accountants in constructing expert systems (cf., Akers, Porter, Blocker, & Mister, 1986; Dologite, 1993; Press, 1988; Sena & Smith, 1987). VP-EXPERT (Wordtech Systems, Inc., 1990) was the shell selected for the construction of PURPOOL. The selection process is described below. Several criteria should be considered in selecting an expert system shell. Some of the criteria include: documentation, explanation and interaction facilities, funds, and the nature of the problem. These criteria were evaluated as a part of the selection of VP-EXPERT. 5.1. Documentation
The documentation of VP-EXPERT was reviewed and found to be excellent. With the exception of the mathematical functions, the shell documentation was clearly explained in the user's manual. 5.2. Explanation and Interaction Facilities
VP-EXPERT provides a built-in explanation facility and an interaction facility. This allows the user to in-
177 teract with the shell and obtain an explanation of why a particular question is asked. These facilities were used during the construction of PURPOOL and were of great assistance. 5.3. Funds
VP-EXPERT is inexpensive. The shell costs less than $300. 5.4. Nature of the Problem
VP-EXPERT uses production rules (i.e., if/then rules) in structuring the knowledge base. Determining the accounting treatment for a business combination lends itself to the use of production rules. If the criteria contained in Opinion No. 16 are assigned particular values, then the business combination should be recorded as either a purchase or a pooling of interests. In 1990 when PURPOOL was constructed, VPEXPERT had been compared favorably with other expert system shells that were more costly. One researcher (Press, 1988) compared three expert system shells: Ml, Personal Consultant Plus, and VP-EXPERT. With almost every function that was compared, such as user interface, explanation of why a question is asked, debugging aids, etc., VP-EXPERT performed as well as the other two expert system shells. The researcher concluded: "For experimenting with expert systems or developing small prototypes, choose VP-EXPERT." 6. KNOWLEDGE VALIDATION Knowledge validation is the fifth major stage in the construction of an expert system (cf., Bahill, 1991; O'Leary, Goul, Moffitt, & Radwan, 1990). O'Leary (1987) defines knowledge validation as the analyzing of the knowledge base and the decision-making capacity of an expert system. This process requires four steps. They are as follows: 1. Determination of what an expert system knows, does not know, or knows incorrectly. 2. Determination of the expertise level of an expert system. 3. Ascertaining if an expert system is based on a decision-making theory in the domain. 4. Ascertaining the reliability of an expert system. Liebowitz (1988) indicates that validation is a method for determining the accuracy of an expert system, which influences whether the system performs as it is expected to perform. During the validation process of an expert system, the knowledge engineer is attempting to test the quality of the system's decisions and conclusions and the correctness of the system's reasoning. Several methods of knowledge validation exist for expert systems. One method is static analysis. Most
178 static analyses are performed by parsers and translators residing in compilers. Errors ranging from ill-formed arithmetic equations to syntax errors are detected in this fashion. Another validation approach is the use of error rates. This approach involves determining the number of correct and incorrect consultations that the expert system generates. Ifa significant number of errors are discovered, the knowledge base must be redefined in order to achieve better performance by the system. Proof of correctness is yet another validation method. Proof techniques involve the validation of the consistency of the output generated by the expert system with respect to input specifications. Another validation approach is backcasting. This requires using the expert system on a past case to see if the results from the system are the same as the historical results. Bailey, Hackenbrack, De, and Dillard (1987) propose a validation approach that uses the domain expert and then additional human experts. Initially the outcome from consultations with the expert system is compared to the domain expert's conclusions, which are assumed to be correct. If significant differences are discovered, the knowledge engineer must develop the knowledge base further. The second stage of knowledge validation integrates additional human experts into the process. The additional experts are used to determine if the system's consultation results compare favorably with the results found among other experts. With this approach, the knowledge engineer is attempting to include the best available heuristics in the knowledge base. O'Leary (1987) provides a framework for the validation of an expert system, which includes the following nine elements: I. content validity 2. criterion validity 3. construct validity 4. objectivity 5. economics 6. reliability 7. systematic variance 8. extraneous variance 9. error variance This validation framework was the method used in the validation of PURPOOL because, in the authors' view, it was very comprehensive and included aspects of other approaches described above. Each of the framework elements is described below and related to the validation of PURPOOL.
6.1. Content Validity Content validity is the first element in the framework used in the validation of PURPOOL. In the validation of an expert system, content validity refers to determining what the system knows, does not know, or
R. S. McDuffie et al. knows incorrectly. The content validity of an expert system can be determined by direct examination of the system or testing the system by human experts (as with a modified Turing test). As discussed in the knowledge acquisition section of this paper, a domain expert reviewed the rules conrained in PURPOOL's knowledge base. In addition, the criteria prescribed in Opinion No. 16 were examined by the domain expert and compared against the rules captured in the system to ensure all criteria were evaluated by appropriate rule (s). Two accounting professors at a regional state university directly examined PURPOOL with the use of VP-EXPERT. One of the accounting professors has written over 100 articles in the area of accounting information systems (AIS); the other professor has taught advanced accounting for a number of years. These two professors were selected because of their extensive background in AIS and business combinations issues. The two accounting professors independently performed a consultation with PURPOOL using VP-EXPERT. The professors answered the questions posed by PURPOOL by referring to a hypothetical business combination they developed. The consultations were conducted in the same distraction-free setting, and the two professors found no problem or difficulty with PURPOOL and the system's content validity. 6.2. Criterion Validity The second element in the validation framework is cflteflon validity. In the validation of an expert system, this framework element consists of the criteria used to validate the system, such as ascertaining the system's level of expertise. One of the best methods to determine the criterion validity of an expert system is to evaluate the system's performance on a set of test problems. The criterion validity of PURPOOL was validated in this fashion. One aspect of criterion validity concerns the consistency, accuracy, and completeness of the knowledge base. Consistency involves the information contained in the knowledge base and the ability of the inference engine to process the information consistently. Accuracy is the correctness of the knowledge stored in the knowledge base. Completeness concerns whether the system can handle all the potential problems that may arise within the problem domain (in this case, business combinations). A total of 29 cases that encompassed all criteria contained in Opinion No. 16 were used by the authors in consultations with PURPOOL. When developing the cases, the authors selected a criterion from Opinion No. 16 and developed a short case to test the criterion with a business combination structured as a purchase and as a pooling of interests. A second criterion was added to the case, as a purchase and as a pooling, and
Accounting Expert System
179
both criteria were tested. This process was continued until all 12 criteria were contained in the case, with all possible variation on assumptions applied and tested. In every consultation, PURPOOL reached the proper conclusion concerning the business combination described in the case and the method by which it would be recorded. Table 1 presents a summary of the case test results.
6 3 . Construct Validity The third element of the validation framework used to validate PURPOOL is construct validity. This pertains to the constructs or factors that the expert system is designed to discover. The point that distinguishes construct validity from other aspects of validity is the importance of theory and theoretical constructs. This type of validity suggests the importance of an existing theory upon which the system is based. The constructs upon which PURPOOL was based were the 12 criteria contained in Opinion No. 16. The Opinion defines geiaerally accepted accounting principles for business combinations. Therefore, the information contained in Opinion No. 16 was the foundation of PURPOOL's knowledge base.
TABLE 1 Business Combination Cases--Test Results
Case I Case 2 Case 3 Case 4 Case 5 Case 6 Case 7 Case 8 Case 9 Case 10 Case 11 Case 12 Case 13 Case 14 Case 15 Case 16 Case 17 Case 18 Case 19 Case 20 Case 21 Case 22 Case 23 Case 24 Case 25 Case 26 Case 27 Case 28 Case 29
Proper Accounting Treatment per Opinion No. 16
Purpool Results
Error
Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Purchase Pooling Pooling Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling
Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Purchase Pooling Pooling Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling
None None None None None None None None None None None None None None None None None None None None None None None None None None None None None
6.4. Objectivity The fourth element in the validation framework is objectivity, which includes such components as programmer validation, sponsor or end-user validation, and independent administration of validation. Programmer validation involves the knowledge engineer performing periodic, informal validation of the expert system. During the development of PURPOOL, the rules that evaluate each of the 12 criteria contained in Opinion No. 16 were tested by consultation with hypothetical situations as they were programmed into the knowledge base. As a result, when the system was completed, programmer validation was accomplished. Sponsor or end-user validation suggests that the ultimate user of the expert system should give final approval or sign-offfor the system. This assumes the enduser of the system is sufficiently knowledgeable to judge the quality of the system. No formal sponsor or end-user existed for PURPOOL. However, a commercial version of PURPOOL would be of value to auditors employed by public accounting firms. Therefore, four audit seniors from a regional accounting firm were selected to function as the end-users of PURPOOL for validation purposes. The four audit seniors were allowed to perform a consultation with PURPOOL using VP-EXPERT independently. They were provided basic instructions and asked to document their responses to the questions PURPOOL posed. Each consultation was conducted in the same distraction-free setting. At the completion of each consultation, the audit seniors were questioned concerning the user friendliness of the system and the conclusion reached by PURPOOL for the business combination information provided. The audit seniors indicated the questions possessed by the system were easily understood, and the results were correct based upon their knowledge of business combinations. Each of the audit seniors was asked if their firm provided an expert system such as PURPOOL, and, if so, if the system would be a productive audit tool. Each senior answered affirmatively. One of the seniors went so far as to inquire if the authors had developed other such systems and if they were available for use. Independent administration of the validation process for an expert system is necessary because validation of a system entirely by the knowledge engineer might lead to a subjective bias or a conflict of interest. Therefore, the two accounting professors and the four audit seniors were an important part of the validation of PURPOOL. They ensured some level of independence in the validation of the prototype.
6.5. Economics The fifth element of the framework used in the validation of PURPOOL is economics or cost-versus-ben-
180
efits. Since accounting and auditing expert systems are constructed and validated using resources, cost-versusbenefits analysis is a key factor in the validation process of this element. One factor in determining any benefits that will be derived from an expert system is how the system will be used. The expert system may have commercial use, or it may be a prototype constructed to determine the feasibility of developing a commercial system. Two factors that determine the validation cost of an expert system are the extent of validation required and the formality of the validation process considered necessa.ry. The extent of validation refers to the depth of the validation of the system, including the number of test problems required. The formality of validation refers to the breadth of the system's validation, which may include independent validation of the expert system. Economics or cost-versus-benefits was not a major issue in the validation of PURPOOL as the system was constructed and validated in an academic environment. The construction of PURPOOL indicated that the process of determining the accounting treatment for business combinations could be represented as an expert system and that perhaps such a system could be economically beneficial to an accounting firm.
6.6. Reliability The sixth element in the validation framework is reliability. The reliability of an expert system refers to the system's ability to generate identical solutions when inputs are the same. One method of determining the reliability of an expert system is to test the system with slight changes of test problem information. As rules were developed to evaluate each of the 12 criteria contained in Opinion No. 16, the "what i f " option of VP-EXPERT was used extensively. This allowed for a change in the value of any variable used in a consultation. In this way, slight changes were integrated into test consultations during the development of PURPOOL. In all instances, the prototype responded with the proper conclusions. Therefore, PURPOOL was concluded to be reliable in the context of the validation framework.
6.7. Systematic Variance The seventh element in the framework used in the validation of P U R P O O L is systematic variance, or experimental variance. The use of test problems is one method of determining the systematic variance of an expert system. The test problems must include significant variation in the values assigned to the problem variables. Also, a significant number of test problems is necessary. The use of only a few problems will not adequately test the knowledge base. A total of 29 test cases was used to determine if any systematic variance existed in PURPOOL. As discussed
R. S. McDu~e et al.
earlier in the criterion validity section of this article, the test cases included all possible assumptions concerning the 12 criteria contained in Opinion No. 16. No variation was noted in the performance of PURPOOL. In every case, a consultation with PURPOOL resulted in the correct conclusion concerning the business combination described in that case.
6.8. Extraneous Variance The eighth validation element in the framework is extraneous variance. This refers to the influences of independent variables extraneous to the purpose of a study. These should be minimized, nullified, or isolated. One extraneous variable, location of judges during testing, was considered in the validation of PURPOOL. The two accounting professors and the four audit seniors that tested PURPOOL conducted their consultations at the same location. Noise was limited, and no interruptions occurred during the consultations. In this way, the extraneous variable--location of judges during testing--was controlled and did not vary among consultations.
6.9. Error Variance The final element of the validation framework is error variance. No errors were noted during the testing of P U R P O O L Therefore, even though extensive testing was used by the authors, error variance was not a factor in the validation of PURPOOL. 7. C O N S U L T A T I O N W I T H P U R P O O L Appendix A contains a case developed and used during the construction and validation of PURPOOL. Appendix B presents a consultation with PURPOOL, using the information from the case in Appendix A. All the screens generated during the consultation are provided in Appendix B, including introductory and instructional material, specific questions asked by PURPOOL and answers provided by the user from information contained in the case, and conclusions reached by the system. PURPOOL is most effectively demonstrated using an overhead projector, a viewer projection pad, and a portable computer. As shown in Appendix A, Imperial Company is the issuer firm (acquiring company), and Cisco Company is the combiner company (acquired company). The results of the consultation with PURPOOL indicate that a pooling of interests is the correct accounting treatment of the combination. 8. F U T U R E D E V E L O P M E N T O F GAAPBASED E X P E R T S Y S T E M S P U R P O O L is an example of a GAAP-based expert system. As discussed in this paper, the system captures
Accounting Expert System
the authoritative accounting literature, A P B Opinion No. 16, that defines G A A P for business combinations. As a result, P U R P O O L is capable o f determining if a business c o m b i n a t i o n is a purchase or a pooling o f interests. In the future, it appears that additional GAAP-based expert systems m a y be developed. T h e A I C P A has suggested that similar systems be developed in the areas o f foreign currency translation, pensions, i n c o m e taxes, etc. Boer a n d Livnat ( 1 9 9 0 ) suggested several areas suitable for expert system development. Those areas included earnings per share, pension accounting, foreign currency translations, and accounting for stock and other deferred c o m p e n s a t i o n packages for employees. Expert systems developed in m a n y o f these areas will rely heavily u p o n authoritative accounting literature defining G A A P for the area. With the i n u n d a t i o n o f accounting literature f r o m various accounting organizations, accountants a n d auditors can utilize G A A P - b a s e d expert systems to improve productivity a n d enhance performance. Rather than review a lengthy accounting p r o n o u n c e m e n t , an expert system can provide the a c c o u n t a n t or auditor with information concerning the proper treatment for the accounting problem being considered. 9. C O N C L U S I O N S This paper has described the construction process o f a GAAP-based expert system for business combinations. Emphasis was placed on the knowledge validation process, the final step in construction. A nine-step validation framework was used to validate the expert system. A c c o u n t i n g expert systems can be extremely useful to both educators and practitioners. Expert systems can be used as pedagogical devices to present complex acc o u n t i n g problems to students. Expert systems can be used by practitioners in training inexperienced staff as well as in resolving actual problems encountered in practice.
REFERENCES Aker, M.D., Porter, G.L., Blocker, E.J., & Mister, W.G. (1986). Expert systems for management accountants. Management Accounting, 67, 30-34. American Institute of Certified Public Accountants. (1987). Introduction to artificial intelligence and expert systems. New York: American Institute of Certified Public Accountants. Bahill, T. ( 1991). Verifyingand validatingpersonal computer-based expert systems. Engk wood Cliffs, NJ: Prentice-Hall. Bully, A., Hackenbrack, K., De, P., & Dillard, J. (1987). Artificial intelligence, cognitiw science, and computational modeling in auditing research: A esearch approach. Journal of Information Systems, 1, 20-40. Boer, G., & Livnat, J. ( 1~90). Using expert systemsto teach complex accounting issues. Is,. ues in Accounting Education, 5, 108-119. Brown, C. E. (Ed.) ( 1991 ). Expert Systems for Accounting,Auditing, and Tax [Special lssu~ ]. Expert @stems with Applications, 3, ( 1).
181
Dologite, D.G. (1993). Developing knowledge-based systems using VP-Expert. New York: Macmillan. Hollander, A.S. (1987). A rule-based approach to variance analysis. (Doctoral dissertation, University of Tennessee). Liebowitz, J. (1988). Introduction to expert systems. Santa Cruz: Mitchell Publishing. O'Leary, D. (1987). Validation of expert systems:With applications to auditing and accounting expert systems. Decision Sciences, 18, 468-486. O'Leary, T., Goul, M., Motiitt, K., & Radwan, A. (1990). Validating expert systems. IEEE Expert, 51-58. Press, L. (1988). Expert system shells for personal computers. (Personal Computing) Abacus, 5, 64-69. Sena, J., & Smith, L.M. ( 1987). A sample expert system for financial statement analysis. Journal of Accounting and EDP, 3, 15-22. Wordtech Systems, Inc. (1990). VP-EXPERT: Rule-based expert system development tool. Orinda, CA: Wordtech Systems, Inc.
APPENDIX A: B U S I N E S S COMBINATION
CASE
Imperial Company and Cisco Company have agreed to merge in a transaction that will qualify either as a purchase or as a pooling of interests. Imperial Company is the issuer firm and will continue in existence. Cisco Company is the combiner firm and will no longer exist. The initiation date of the business combination is March 1, 1990. Neither Imperial Company nor Cisco Company have ever been a subsidiary or division of another company. On March 1, 1990 and until the business combination was consummated, Imperial Company and Cisco Company owned none of the outstanding common stock of the other firm. The business combination involving Imperial Company and Cisco Company was accomplished by a single transaction that occurred on August 1, 1990. Imperial Company issued common stock to consummate the business combination with Cisco Company. The stock issued had rights identical to other Imperial Company outstanding common stock. During 1988 and 1989, Imperial Company and Cisco Company distributed dividends to stockholders that were normal when compared to dividend distributions in years prior to 1988. In 1990 Imperial Company and Cisco Company acquired treasury stock that was normal when compared to treasury stock acquisitions in prior years. Imperial Company issued 9,250 shares of common stock to consummate the business combination on August 1, 1990. The stock was distributed based on the combiner company stockholders' ownership in Cisco Company. The voting rights in the common stock issued by Imperial Company to consummate the business combination were exercisable immediately after the consummation date. There were no contingent consideration agreements based on events occurring subsequent to the consummation of the combination involving Imperial Company and Cisco Company. Imperial Company did not agree to reacquire any of the common stock that was issued to consummate the business combination with Cisco Company.
182
R. S. McDuffie et al.
Imperial Company did not agree to provide financial assistance to any of Cisco Company's stockholders. Imperial Company does not plan to sell any of the assets of the combining firms for at least 2 years after the consummation date of the business combination. Additional facts about the business combination: • Cisco Company stock outstanding: 20,000 shares. • Cisco Company stock acquired by Imperial Company: 18,500 shares. • Imperial Company stock issued: 9,250 shares. • Imperial Company stock owned by Cisco Company: None. • Imperial Company's acquisitions of Cisco Company's net assets: None. Results of the case analysis were as follows: Business Combination Recorded As: Expected results
Pooling of interests
PURPOOL results
Pooling of interests
A P P E N D I X B: P U R P O O L C O N S U L T A T I O N SCREENS USING THE CASE IN A P P E N D I X A - -
Indicates separation of screen content when using P U R P O O L . )
This expert system uses the 12 criteria contained in APB Opinion No. 16 to determine if a business combination should be recorded as a purchase or as a pooling of interests. Press any key to continue. Answer the questions that appear on the screen. To select an answer when a choice is provided, use the arrow keys to move the highlighted area to the answer you desire, press the enter key, and then finalize your answer by pressing the end key. To answer a question when a choice is not provided, type your answer and press the enter key. Press any key to continue. Based upon your answers, the 12 criteria in APB Opinion No. 16 will be evaluated. If any of the criteria are not satisfied, the business combination should be recorded as a purchase, and this will be indicated on the screen. The consultation may be continued to evaluate the remaining criteria or discontinued. You will be given instructions that describe how to continue or discontinue the consultation. Press any key to continue.
Have the issuer company or the combiner company been a subsidiary or a division of another company during the 2year period prior to the initiation date of the business combination? YES
NO X
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. Ignoring shares acquired as a part of the consummation of the business combination, have the issuer company and the combiner company owned 10% or less of the outstanding stock of the other company involved in the combination for a period of time beginning with the initiation date of the combination until the combination was consummated? YES X
NO
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. Ignoring delay caused by deliberations of regulatory agencies and litigation to prevent the business combination, was the combination accomplished with a single transaction or by a specific plan, within 1 year after the initiation date? YES X
NO
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. Was the business combination consummated with the issuance of issuer company common stock, which had rights identical to other issuer company common stock that was outstanding? YES X
NO
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. How many shares of the combiner company's common stock were outstanding at the date the business combination was consummated? 20,000
If all the criteria in APB Opinion No. 16 are satisfied, the business combination should be recorded as a pooling of interest, and this will be indicated on the screen. The consultation is completed, and you will be given instructions that describe how to exit from the expert system. Press any key to continue. The expert system can describe to you why a question is asked. Before answering a question, type slash (/), then W, and a brief description of why the question is being asked will be provided. Press any key to begin the consultation.
How many shares of the combiner company's common stock were acquired by the issuer company in consummating the business combination? 18,500 How many shares of common stock did the issuer company issue to consummate the business combination? 9,250 How many shares of the issuer company's common stock were owned by the combiner company when the business combination was consummated?
Accounting Expert System 0 Was the business combination consummated by the issuer company acquiring all the net assets of the combiner company? YES
NO X
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. For 2 years prior to the initiation date of the business combination until the combination was consummated, did the issuer company and the combiner company distribute dividends that were less than or equal to dividends distributed prior to this period? YES X
NO
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation.
183 exercisable immediately after the combination was consummated? YES X
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. Were there any contingent consideration agreements for the stockholders' benefit that were based upon events subsequent to the consummation of the business combination? YES
YES X
NO
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. Was the issuer company's share of common stock that were issued to consummate the business combination distributed to the combiner company stockholders based upon the ratio of their ownership in combiner company stock? YES X
NO
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. Were the voting rights in the shares of stock issued by the issuer company to consummate the business combination
NO X
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. Were there any agreements to retire or reacquire any of the stock issued to consummate the business combination? YES
From the date the business combination was initiated and until the consummation of the combination, did the issuer company and the combiner company acquire shares of treasury stock that were less than or equal to the number of treasury shares acquired during the years prior to this time period and use the treasury shares for purposes other than for the consummation of the combination?
NO
NO X
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. Were there any agreements to provide financial aid to stockholders of the combiner company? YES
NO X
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. Except for disposals in the ordinary course of business and disposals to eliminate duplicate facilities or excess capacity, were there any plans to dispose of a significant part of the assets of the combining firms during the 2 years after the business combination was consummated? YES
NO X
The business combination has satisfied the criterion. All the criteria in APB Opinion No. 16 have been met, and the combination should be recorded as a pooling of interests. The consultation is concluded. Type Q, and type Q again.
Pergamon
Validation of an Accounting Expert System for Business Combinations R. STEVE McDUFFIE Department of Accounting, Southwest Missouri State University, Springfield, MO
L. MURPHY SMITH Department of Accounting,TexasA&MUniversity,CollegeStation,TX
STEVEN M. FLORY Department of Accounting, University of Southwestern Louisiana, Lafayette, LA
Abstract--This article presents the five stages involved in the construction of a knowledge-based system using VP-EXPERT, with emphasis on the final step, knowledge validation. The knowledgebased system, called PURPOOL, enables the user to determine the proper accounting treatment for a business combination. PURPOOL can be used effectively by educators as a pedagogical device to teach their students how to account for business combinations. PURPOOL also can be used by accounting practitioners in training inexperienced staff accountants or in determining the proper accounting treatment of actual problems encountered in practice. PURPOOL is one of thefirst GAAPbased expert systems reported in the accounting literature. A GAAP-based system relies upon the current authoritative accounting literature (Financial Accounting Standards Board Statements, Accounting Principles Board Opinions, etc.)for the knowledge base rules. PURPOOL captures the content of APB Opinion No. 16, Business Combinations in its knowledge base. The validation of PURPOOL utilized a framework consisting of nine elements. This process included interaction with a human expert, review of the accounting literature pertaining to business combinations, development of test cases, consultations with audit seniors from a regional accounting firm, and consultations with accounting information systems and advanced accountingprofessors. As a result of using theframework, a thorough validation of the system was accomplished, and that process is described in this article.
1. INTRODUCTION
in the AICPA report departs from the typical accounting expert systems developed thus far. The traditional accounting expert system chiefly depends upon a human expert in the construction of the knowledge base of the expert system. The new type of expert system presented in the special report chiefly depends upon generally accepted accounting principles (GAAP) to provide information for the development of the rules contained in the knowledge base. These GAAP-based expert systems still rely on the human expert for system validation, but primarily rely upon the current authoritative accounting literature for the information to be included in the system's knowledge base. At least one such expert system has been reported in the literature (Boer & Livnat, 1990). The researchers developed an expert system to solve lease problems
IN 1987 THE AMERICAN INSTITUTE OF CERTIFIED PUBLIC ACCOUNTANTS( A I C P A ) published An Intro-
duction to Artificial Intelligence and Expert Systems.
One section of the report presented applications suitable for future expert system development. The applications included leases, foreign currency translation, acquisitions, pensions, and income taxes. Expert systems are being used and developed for accounting, auditing, and tax practice (Brown, 1991). The type of expert system development described
Requests for reprints should be sent to R. Steve McDutfie, Department of Accounting, Southwest Missouri State University, Springfield, MO 65804-0094.
175
176
R. S. McDuffie et al.
from the standpoint of the lessee. The system was used as a teaching aid by the researchers. Students who used the expert system performed better in solving lease problems than students who did not use the expert system. This paper describes the five stages involved in the construction of the GAAP-based expert system, PURPOOL. The focus of the paper is upon the final stage of construction, knowledge validation. The system determines the proper accounting treatment for business combinations. PURPOOL indicates whether a combination should be treated as a purchase or as a pooling of interests. The five stages in constructing a GAAP-based expert system are as follows: 1. Problem selection 2. Knowledge acquisition 3. Knowledge representation 4. Knowledge programming 5. Knowledge validation 2. P R O B L E M S E L E C T I O N Anyone familiar with advanced accounting understands the difficulty of determining whether a business combination should be recorded as a purchase or as a pooling of interests. Several CPAs were queried about the potential benefits of PURPOOL. They indicated that a commercial version of PURPOOL could be beneficial to their firms. A senior manager for a national accounting firm indicated during a conversation that her firm had developed a similar type of expert system prototype. She further indicated that an expert system like PURPOOL would be of substantial value. Former Deputy Chief Accountant of the Securities and Exchange Commission (SEC), Glen L. Davison, indicated that the issue of purchase versus pooling of interests for business combinations was the most frequent problem area considered by the SEC during his 3-year tenure with the Commission. Mr. Davison further provided that while he was a manager with a national accounting firm's central office in New York, the issue of purchase versus pooling of interests for business combinations was the most common problem addressed by that office. Certain task characteristics should be present to identify a problem as suitable for the development of an expert system. The problem of determining the proper accounting treatment of a business combination appears to match these characteristics. The task characteristics, as they related to PURPOOL, are discussed below. 2.1. Well-Bounded Task
A well-bounded task is one confined to a narrow, welldefined problem domain. The determination of the
proper method of accounting for a business combination is a well-bound task. At the prototype stage, P U R P O O L evaluates the 12 criteria contained in APB Opinion No. 16. 2.2. Few Minutes to a Few Hours to Solve
The evaluation of the 12 criteria contained in Opinion No. 16 would take at least a few hours, because the opinion itself is over 40 pages long. 2.3. Task is Frequently Performed
From the point of view of a national accounting firm, the decision of how a business combination should be recorded could occur numerous times on a firm-wide basis during a year. 2.4. Test Data is Available
The criteria in Opinion No. 16 lends itself to the construction of case studies. Case studies provided appropriate test data for the purpose of validating PURPOOL. 2.5. General Consensus
After a detailed review, the criteria in Opinion No. 16 should lead accountants to a similar conclusion concerning the accounting treatment of a particular business combination. Because of the reaction of the various CPAs to the potential benefits of PURPOOL, the comments from former Deputy Chief Accountant Davison, and the task characteristics associated with determining if a business combination is a purchase or pooling of interests, the authors concluded that this was a suitable problem area for expert system development. 3. K N O W L E D G E ACQUISITION Knowledge acquisition for PURPOOL was different from the acquisition used for typical accounting expert systems. The primary source of knowledge for PURPOOL was Accounting Principles Board (APB ) Opinion No. 16, which defines a business combination as a purchase or as a pooling of interests. Hollander (1987) used a similar approach for knowledge acquisition in her Ph.D. dissertation. She extracted knowledge from accounting text books, articles, etc. to be used in an expert system prototype that determined which standard cost variances should be analyzed by a company's management. With the knowledge extracted in this fashion, she constructed a prototype that identified price and use variances for material and labor that should be reviewed by management.
Accounting Expert System A domain expert was used, to a limited extent, during the knowledge acquisition stage of the construction of PURPOOL. As a partner in charge of a national accounting firm office, the expert was experienced with the problems associated with business combinations. The domain expert was queried at length during the development of PURPOOL and reviewed the system, including the rules it contained during validation. In addition to Opinion No. 16, two national public accounting firms provided sections from their accounting manuals concerning the purchase method and the pooling of interests method of recording business combinations. Also, explanatory material concerning Opinion No. 16 were reviewed, and several advanced accounting text books were consulted. 4. KNOWLEDGE REPRESENTATION A production rule format was selected for the development of PURPOOL. The knowledge pertaining to business combinations lends itself to the use of production rules (i.e., if/then rules). Production rules are the most popular method of knowledge representation in expert system development (Liebowitz, 1988, p. 47 ). 5. KNOWLEDGE PROGRAMMING Early in the development of PURPOOL the authors decided against using a formal programming language, such as LISP or PROLOG, to encode the knowledge acquired into the knowledge base. Using such programming languages requires extensive training, time, and effort beyond the resources of most accountants. Expert system shell software has been identified as an excellent tool for use by accountants in constructing expert systems (cf., Akers, Porter, Blocker, & Mister, 1986; Dologite, 1993; Press, 1988; Sena & Smith, 1987). VP-EXPERT (Wordtech Systems, Inc., 1990) was the shell selected for the construction of PURPOOL. The selection process is described below. Several criteria should be considered in selecting an expert system shell. Some of the criteria include: documentation, explanation and interaction facilities, funds, and the nature of the problem. These criteria were evaluated as a part of the selection of VP-EXPERT. 5.1. Documentation
The documentation of VP-EXPERT was reviewed and found to be excellent. With the exception of the mathematical functions, the shell documentation was clearly explained in the user's manual. 5.2. Explanation and Interaction Facilities
VP-EXPERT provides a built-in explanation facility and an interaction facility. This allows the user to in-
177 teract with the shell and obtain an explanation of why a particular question is asked. These facilities were used during the construction of PURPOOL and were of great assistance. 5.3. Funds
VP-EXPERT is inexpensive. The shell costs less than $300. 5.4. Nature of the Problem
VP-EXPERT uses production rules (i.e., if/then rules) in structuring the knowledge base. Determining the accounting treatment for a business combination lends itself to the use of production rules. If the criteria contained in Opinion No. 16 are assigned particular values, then the business combination should be recorded as either a purchase or a pooling of interests. In 1990 when PURPOOL was constructed, VPEXPERT had been compared favorably with other expert system shells that were more costly. One researcher (Press, 1988) compared three expert system shells: Ml, Personal Consultant Plus, and VP-EXPERT. With almost every function that was compared, such as user interface, explanation of why a question is asked, debugging aids, etc., VP-EXPERT performed as well as the other two expert system shells. The researcher concluded: "For experimenting with expert systems or developing small prototypes, choose VP-EXPERT." 6. KNOWLEDGE VALIDATION Knowledge validation is the fifth major stage in the construction of an expert system (cf., Bahill, 1991; O'Leary, Goul, Moffitt, & Radwan, 1990). O'Leary (1987) defines knowledge validation as the analyzing of the knowledge base and the decision-making capacity of an expert system. This process requires four steps. They are as follows: 1. Determination of what an expert system knows, does not know, or knows incorrectly. 2. Determination of the expertise level of an expert system. 3. Ascertaining if an expert system is based on a decision-making theory in the domain. 4. Ascertaining the reliability of an expert system. Liebowitz (1988) indicates that validation is a method for determining the accuracy of an expert system, which influences whether the system performs as it is expected to perform. During the validation process of an expert system, the knowledge engineer is attempting to test the quality of the system's decisions and conclusions and the correctness of the system's reasoning. Several methods of knowledge validation exist for expert systems. One method is static analysis. Most
178 static analyses are performed by parsers and translators residing in compilers. Errors ranging from ill-formed arithmetic equations to syntax errors are detected in this fashion. Another validation approach is the use of error rates. This approach involves determining the number of correct and incorrect consultations that the expert system generates. Ifa significant number of errors are discovered, the knowledge base must be redefined in order to achieve better performance by the system. Proof of correctness is yet another validation method. Proof techniques involve the validation of the consistency of the output generated by the expert system with respect to input specifications. Another validation approach is backcasting. This requires using the expert system on a past case to see if the results from the system are the same as the historical results. Bailey, Hackenbrack, De, and Dillard (1987) propose a validation approach that uses the domain expert and then additional human experts. Initially the outcome from consultations with the expert system is compared to the domain expert's conclusions, which are assumed to be correct. If significant differences are discovered, the knowledge engineer must develop the knowledge base further. The second stage of knowledge validation integrates additional human experts into the process. The additional experts are used to determine if the system's consultation results compare favorably with the results found among other experts. With this approach, the knowledge engineer is attempting to include the best available heuristics in the knowledge base. O'Leary (1987) provides a framework for the validation of an expert system, which includes the following nine elements: I. content validity 2. criterion validity 3. construct validity 4. objectivity 5. economics 6. reliability 7. systematic variance 8. extraneous variance 9. error variance This validation framework was the method used in the validation of PURPOOL because, in the authors' view, it was very comprehensive and included aspects of other approaches described above. Each of the framework elements is described below and related to the validation of PURPOOL.
6.1. Content Validity Content validity is the first element in the framework used in the validation of PURPOOL. In the validation of an expert system, content validity refers to determining what the system knows, does not know, or
R. S. McDuffie et al. knows incorrectly. The content validity of an expert system can be determined by direct examination of the system or testing the system by human experts (as with a modified Turing test). As discussed in the knowledge acquisition section of this paper, a domain expert reviewed the rules conrained in PURPOOL's knowledge base. In addition, the criteria prescribed in Opinion No. 16 were examined by the domain expert and compared against the rules captured in the system to ensure all criteria were evaluated by appropriate rule (s). Two accounting professors at a regional state university directly examined PURPOOL with the use of VP-EXPERT. One of the accounting professors has written over 100 articles in the area of accounting information systems (AIS); the other professor has taught advanced accounting for a number of years. These two professors were selected because of their extensive background in AIS and business combinations issues. The two accounting professors independently performed a consultation with PURPOOL using VP-EXPERT. The professors answered the questions posed by PURPOOL by referring to a hypothetical business combination they developed. The consultations were conducted in the same distraction-free setting, and the two professors found no problem or difficulty with PURPOOL and the system's content validity. 6.2. Criterion Validity The second element in the validation framework is cflteflon validity. In the validation of an expert system, this framework element consists of the criteria used to validate the system, such as ascertaining the system's level of expertise. One of the best methods to determine the criterion validity of an expert system is to evaluate the system's performance on a set of test problems. The criterion validity of PURPOOL was validated in this fashion. One aspect of criterion validity concerns the consistency, accuracy, and completeness of the knowledge base. Consistency involves the information contained in the knowledge base and the ability of the inference engine to process the information consistently. Accuracy is the correctness of the knowledge stored in the knowledge base. Completeness concerns whether the system can handle all the potential problems that may arise within the problem domain (in this case, business combinations). A total of 29 cases that encompassed all criteria contained in Opinion No. 16 were used by the authors in consultations with PURPOOL. When developing the cases, the authors selected a criterion from Opinion No. 16 and developed a short case to test the criterion with a business combination structured as a purchase and as a pooling of interests. A second criterion was added to the case, as a purchase and as a pooling, and
Accounting Expert System
179
both criteria were tested. This process was continued until all 12 criteria were contained in the case, with all possible variation on assumptions applied and tested. In every consultation, PURPOOL reached the proper conclusion concerning the business combination described in the case and the method by which it would be recorded. Table 1 presents a summary of the case test results.
6 3 . Construct Validity The third element of the validation framework used to validate PURPOOL is construct validity. This pertains to the constructs or factors that the expert system is designed to discover. The point that distinguishes construct validity from other aspects of validity is the importance of theory and theoretical constructs. This type of validity suggests the importance of an existing theory upon which the system is based. The constructs upon which PURPOOL was based were the 12 criteria contained in Opinion No. 16. The Opinion defines geiaerally accepted accounting principles for business combinations. Therefore, the information contained in Opinion No. 16 was the foundation of PURPOOL's knowledge base.
TABLE 1 Business Combination Cases--Test Results
Case I Case 2 Case 3 Case 4 Case 5 Case 6 Case 7 Case 8 Case 9 Case 10 Case 11 Case 12 Case 13 Case 14 Case 15 Case 16 Case 17 Case 18 Case 19 Case 20 Case 21 Case 22 Case 23 Case 24 Case 25 Case 26 Case 27 Case 28 Case 29
Proper Accounting Treatment per Opinion No. 16
Purpool Results
Error
Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Purchase Pooling Pooling Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling
Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Purchase Pooling Pooling Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling Purchase Pooling
None None None None None None None None None None None None None None None None None None None None None None None None None None None None None
6.4. Objectivity The fourth element in the validation framework is objectivity, which includes such components as programmer validation, sponsor or end-user validation, and independent administration of validation. Programmer validation involves the knowledge engineer performing periodic, informal validation of the expert system. During the development of PURPOOL, the rules that evaluate each of the 12 criteria contained in Opinion No. 16 were tested by consultation with hypothetical situations as they were programmed into the knowledge base. As a result, when the system was completed, programmer validation was accomplished. Sponsor or end-user validation suggests that the ultimate user of the expert system should give final approval or sign-offfor the system. This assumes the enduser of the system is sufficiently knowledgeable to judge the quality of the system. No formal sponsor or end-user existed for PURPOOL. However, a commercial version of PURPOOL would be of value to auditors employed by public accounting firms. Therefore, four audit seniors from a regional accounting firm were selected to function as the end-users of PURPOOL for validation purposes. The four audit seniors were allowed to perform a consultation with PURPOOL using VP-EXPERT independently. They were provided basic instructions and asked to document their responses to the questions PURPOOL posed. Each consultation was conducted in the same distraction-free setting. At the completion of each consultation, the audit seniors were questioned concerning the user friendliness of the system and the conclusion reached by PURPOOL for the business combination information provided. The audit seniors indicated the questions possessed by the system were easily understood, and the results were correct based upon their knowledge of business combinations. Each of the audit seniors was asked if their firm provided an expert system such as PURPOOL, and, if so, if the system would be a productive audit tool. Each senior answered affirmatively. One of the seniors went so far as to inquire if the authors had developed other such systems and if they were available for use. Independent administration of the validation process for an expert system is necessary because validation of a system entirely by the knowledge engineer might lead to a subjective bias or a conflict of interest. Therefore, the two accounting professors and the four audit seniors were an important part of the validation of PURPOOL. They ensured some level of independence in the validation of the prototype.
6.5. Economics The fifth element of the framework used in the validation of PURPOOL is economics or cost-versus-ben-
180
efits. Since accounting and auditing expert systems are constructed and validated using resources, cost-versusbenefits analysis is a key factor in the validation process of this element. One factor in determining any benefits that will be derived from an expert system is how the system will be used. The expert system may have commercial use, or it may be a prototype constructed to determine the feasibility of developing a commercial system. Two factors that determine the validation cost of an expert system are the extent of validation required and the formality of the validation process considered necessa.ry. The extent of validation refers to the depth of the validation of the system, including the number of test problems required. The formality of validation refers to the breadth of the system's validation, which may include independent validation of the expert system. Economics or cost-versus-benefits was not a major issue in the validation of PURPOOL as the system was constructed and validated in an academic environment. The construction of PURPOOL indicated that the process of determining the accounting treatment for business combinations could be represented as an expert system and that perhaps such a system could be economically beneficial to an accounting firm.
6.6. Reliability The sixth element in the validation framework is reliability. The reliability of an expert system refers to the system's ability to generate identical solutions when inputs are the same. One method of determining the reliability of an expert system is to test the system with slight changes of test problem information. As rules were developed to evaluate each of the 12 criteria contained in Opinion No. 16, the "what i f " option of VP-EXPERT was used extensively. This allowed for a change in the value of any variable used in a consultation. In this way, slight changes were integrated into test consultations during the development of PURPOOL. In all instances, the prototype responded with the proper conclusions. Therefore, PURPOOL was concluded to be reliable in the context of the validation framework.
6.7. Systematic Variance The seventh element in the framework used in the validation of P U R P O O L is systematic variance, or experimental variance. The use of test problems is one method of determining the systematic variance of an expert system. The test problems must include significant variation in the values assigned to the problem variables. Also, a significant number of test problems is necessary. The use of only a few problems will not adequately test the knowledge base. A total of 29 test cases was used to determine if any systematic variance existed in PURPOOL. As discussed
R. S. McDu~e et al.
earlier in the criterion validity section of this article, the test cases included all possible assumptions concerning the 12 criteria contained in Opinion No. 16. No variation was noted in the performance of PURPOOL. In every case, a consultation with PURPOOL resulted in the correct conclusion concerning the business combination described in that case.
6.8. Extraneous Variance The eighth validation element in the framework is extraneous variance. This refers to the influences of independent variables extraneous to the purpose of a study. These should be minimized, nullified, or isolated. One extraneous variable, location of judges during testing, was considered in the validation of PURPOOL. The two accounting professors and the four audit seniors that tested PURPOOL conducted their consultations at the same location. Noise was limited, and no interruptions occurred during the consultations. In this way, the extraneous variable--location of judges during testing--was controlled and did not vary among consultations.
6.9. Error Variance The final element of the validation framework is error variance. No errors were noted during the testing of P U R P O O L Therefore, even though extensive testing was used by the authors, error variance was not a factor in the validation of PURPOOL. 7. C O N S U L T A T I O N W I T H P U R P O O L Appendix A contains a case developed and used during the construction and validation of PURPOOL. Appendix B presents a consultation with PURPOOL, using the information from the case in Appendix A. All the screens generated during the consultation are provided in Appendix B, including introductory and instructional material, specific questions asked by PURPOOL and answers provided by the user from information contained in the case, and conclusions reached by the system. PURPOOL is most effectively demonstrated using an overhead projector, a viewer projection pad, and a portable computer. As shown in Appendix A, Imperial Company is the issuer firm (acquiring company), and Cisco Company is the combiner company (acquired company). The results of the consultation with PURPOOL indicate that a pooling of interests is the correct accounting treatment of the combination. 8. F U T U R E D E V E L O P M E N T O F GAAPBASED E X P E R T S Y S T E M S P U R P O O L is an example of a GAAP-based expert system. As discussed in this paper, the system captures
Accounting Expert System
the authoritative accounting literature, A P B Opinion No. 16, that defines G A A P for business combinations. As a result, P U R P O O L is capable o f determining if a business c o m b i n a t i o n is a purchase or a pooling o f interests. In the future, it appears that additional GAAP-based expert systems m a y be developed. T h e A I C P A has suggested that similar systems be developed in the areas o f foreign currency translation, pensions, i n c o m e taxes, etc. Boer a n d Livnat ( 1 9 9 0 ) suggested several areas suitable for expert system development. Those areas included earnings per share, pension accounting, foreign currency translations, and accounting for stock and other deferred c o m p e n s a t i o n packages for employees. Expert systems developed in m a n y o f these areas will rely heavily u p o n authoritative accounting literature defining G A A P for the area. With the i n u n d a t i o n o f accounting literature f r o m various accounting organizations, accountants a n d auditors can utilize G A A P - b a s e d expert systems to improve productivity a n d enhance performance. Rather than review a lengthy accounting p r o n o u n c e m e n t , an expert system can provide the a c c o u n t a n t or auditor with information concerning the proper treatment for the accounting problem being considered. 9. C O N C L U S I O N S This paper has described the construction process o f a GAAP-based expert system for business combinations. Emphasis was placed on the knowledge validation process, the final step in construction. A nine-step validation framework was used to validate the expert system. A c c o u n t i n g expert systems can be extremely useful to both educators and practitioners. Expert systems can be used as pedagogical devices to present complex acc o u n t i n g problems to students. Expert systems can be used by practitioners in training inexperienced staff as well as in resolving actual problems encountered in practice.
REFERENCES Aker, M.D., Porter, G.L., Blocker, E.J., & Mister, W.G. (1986). Expert systems for management accountants. Management Accounting, 67, 30-34. American Institute of Certified Public Accountants. (1987). Introduction to artificial intelligence and expert systems. New York: American Institute of Certified Public Accountants. Bahill, T. ( 1991). Verifyingand validatingpersonal computer-based expert systems. Engk wood Cliffs, NJ: Prentice-Hall. Bully, A., Hackenbrack, K., De, P., & Dillard, J. (1987). Artificial intelligence, cognitiw science, and computational modeling in auditing research: A esearch approach. Journal of Information Systems, 1, 20-40. Boer, G., & Livnat, J. ( 1~90). Using expert systemsto teach complex accounting issues. Is,. ues in Accounting Education, 5, 108-119. Brown, C. E. (Ed.) ( 1991 ). Expert Systems for Accounting,Auditing, and Tax [Special lssu~ ]. Expert @stems with Applications, 3, ( 1).
181
Dologite, D.G. (1993). Developing knowledge-based systems using VP-Expert. New York: Macmillan. Hollander, A.S. (1987). A rule-based approach to variance analysis. (Doctoral dissertation, University of Tennessee). Liebowitz, J. (1988). Introduction to expert systems. Santa Cruz: Mitchell Publishing. O'Leary, D. (1987). Validation of expert systems:With applications to auditing and accounting expert systems. Decision Sciences, 18, 468-486. O'Leary, T., Goul, M., Motiitt, K., & Radwan, A. (1990). Validating expert systems. IEEE Expert, 51-58. Press, L. (1988). Expert system shells for personal computers. (Personal Computing) Abacus, 5, 64-69. Sena, J., & Smith, L.M. ( 1987). A sample expert system for financial statement analysis. Journal of Accounting and EDP, 3, 15-22. Wordtech Systems, Inc. (1990). VP-EXPERT: Rule-based expert system development tool. Orinda, CA: Wordtech Systems, Inc.
APPENDIX A: B U S I N E S S COMBINATION
CASE
Imperial Company and Cisco Company have agreed to merge in a transaction that will qualify either as a purchase or as a pooling of interests. Imperial Company is the issuer firm and will continue in existence. Cisco Company is the combiner firm and will no longer exist. The initiation date of the business combination is March 1, 1990. Neither Imperial Company nor Cisco Company have ever been a subsidiary or division of another company. On March 1, 1990 and until the business combination was consummated, Imperial Company and Cisco Company owned none of the outstanding common stock of the other firm. The business combination involving Imperial Company and Cisco Company was accomplished by a single transaction that occurred on August 1, 1990. Imperial Company issued common stock to consummate the business combination with Cisco Company. The stock issued had rights identical to other Imperial Company outstanding common stock. During 1988 and 1989, Imperial Company and Cisco Company distributed dividends to stockholders that were normal when compared to dividend distributions in years prior to 1988. In 1990 Imperial Company and Cisco Company acquired treasury stock that was normal when compared to treasury stock acquisitions in prior years. Imperial Company issued 9,250 shares of common stock to consummate the business combination on August 1, 1990. The stock was distributed based on the combiner company stockholders' ownership in Cisco Company. The voting rights in the common stock issued by Imperial Company to consummate the business combination were exercisable immediately after the consummation date. There were no contingent consideration agreements based on events occurring subsequent to the consummation of the combination involving Imperial Company and Cisco Company. Imperial Company did not agree to reacquire any of the common stock that was issued to consummate the business combination with Cisco Company.
182
R. S. McDuffie et al.
Imperial Company did not agree to provide financial assistance to any of Cisco Company's stockholders. Imperial Company does not plan to sell any of the assets of the combining firms for at least 2 years after the consummation date of the business combination. Additional facts about the business combination: • Cisco Company stock outstanding: 20,000 shares. • Cisco Company stock acquired by Imperial Company: 18,500 shares. • Imperial Company stock issued: 9,250 shares. • Imperial Company stock owned by Cisco Company: None. • Imperial Company's acquisitions of Cisco Company's net assets: None. Results of the case analysis were as follows: Business Combination Recorded As: Expected results
Pooling of interests
PURPOOL results
Pooling of interests
A P P E N D I X B: P U R P O O L C O N S U L T A T I O N SCREENS USING THE CASE IN A P P E N D I X A - -
Indicates separation of screen content when using P U R P O O L . )
This expert system uses the 12 criteria contained in APB Opinion No. 16 to determine if a business combination should be recorded as a purchase or as a pooling of interests. Press any key to continue. Answer the questions that appear on the screen. To select an answer when a choice is provided, use the arrow keys to move the highlighted area to the answer you desire, press the enter key, and then finalize your answer by pressing the end key. To answer a question when a choice is not provided, type your answer and press the enter key. Press any key to continue. Based upon your answers, the 12 criteria in APB Opinion No. 16 will be evaluated. If any of the criteria are not satisfied, the business combination should be recorded as a purchase, and this will be indicated on the screen. The consultation may be continued to evaluate the remaining criteria or discontinued. You will be given instructions that describe how to continue or discontinue the consultation. Press any key to continue.
Have the issuer company or the combiner company been a subsidiary or a division of another company during the 2year period prior to the initiation date of the business combination? YES
NO X
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. Ignoring shares acquired as a part of the consummation of the business combination, have the issuer company and the combiner company owned 10% or less of the outstanding stock of the other company involved in the combination for a period of time beginning with the initiation date of the combination until the combination was consummated? YES X
NO
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. Ignoring delay caused by deliberations of regulatory agencies and litigation to prevent the business combination, was the combination accomplished with a single transaction or by a specific plan, within 1 year after the initiation date? YES X
NO
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. Was the business combination consummated with the issuance of issuer company common stock, which had rights identical to other issuer company common stock that was outstanding? YES X
NO
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. How many shares of the combiner company's common stock were outstanding at the date the business combination was consummated? 20,000
If all the criteria in APB Opinion No. 16 are satisfied, the business combination should be recorded as a pooling of interest, and this will be indicated on the screen. The consultation is completed, and you will be given instructions that describe how to exit from the expert system. Press any key to continue. The expert system can describe to you why a question is asked. Before answering a question, type slash (/), then W, and a brief description of why the question is being asked will be provided. Press any key to begin the consultation.
How many shares of the combiner company's common stock were acquired by the issuer company in consummating the business combination? 18,500 How many shares of common stock did the issuer company issue to consummate the business combination? 9,250 How many shares of the issuer company's common stock were owned by the combiner company when the business combination was consummated?
Accounting Expert System 0 Was the business combination consummated by the issuer company acquiring all the net assets of the combiner company? YES
NO X
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. For 2 years prior to the initiation date of the business combination until the combination was consummated, did the issuer company and the combiner company distribute dividends that were less than or equal to dividends distributed prior to this period? YES X
NO
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation.
183 exercisable immediately after the combination was consummated? YES X
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. Were there any contingent consideration agreements for the stockholders' benefit that were based upon events subsequent to the consummation of the business combination? YES
YES X
NO
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. Was the issuer company's share of common stock that were issued to consummate the business combination distributed to the combiner company stockholders based upon the ratio of their ownership in combiner company stock? YES X
NO
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. Were the voting rights in the shares of stock issued by the issuer company to consummate the business combination
NO X
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. Were there any agreements to retire or reacquire any of the stock issued to consummate the business combination? YES
From the date the business combination was initiated and until the consummation of the combination, did the issuer company and the combiner company acquire shares of treasury stock that were less than or equal to the number of treasury shares acquired during the years prior to this time period and use the treasury shares for purposes other than for the consummation of the combination?
NO
NO X
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. Were there any agreements to provide financial aid to stockholders of the combiner company? YES
NO X
The business combination has satisfied the criterion, and further evaluation must be performed to determine if the combination should be recorded as a pooling of interests. Press any key to continue the consultation. Except for disposals in the ordinary course of business and disposals to eliminate duplicate facilities or excess capacity, were there any plans to dispose of a significant part of the assets of the combining firms during the 2 years after the business combination was consummated? YES
NO X
The business combination has satisfied the criterion. All the criteria in APB Opinion No. 16 have been met, and the combination should be recorded as a pooling of interests. The consultation is concluded. Type Q, and type Q again.