Transferring auditors’ internal control evaluation knowledge to management

Expert Systems with Applications PERGAMON

Expert Systems with Applications 20 (2001) 275±291

www.elsevier.com/locate/eswa

Transferring auditors' internal control evaluation knowledge to management q Chuleeporn Changchit a, Clyde W. Holsapple b,*, Ralph E. Viator c a

Department of Management Sciences, Henry B. Tippie College of Business, W284 John Pappajohn Business Building, University of Iowa, Iowa City, IA 52242-1000, USA b School of Management, Carol M. Gatton College of Business and Economics, University of Kentucky, Lexington, KY 40506-0034, USA c Area of Accountancy, College of Business Administration, Texas Tech University, Lubbock, TX 79409, USA

Abstract Due to both regulatory and competitive forces, attention to an organization's internal controls has increased signi®cantly in the 1990s. Although management is ultimately responsible for ensuring internal controls are adequate, managers often lack knowledge of internal control concepts. This study reports on an experiment testing an expert system developed to facilitate the transfer of internal control knowledge to management. Experimental results indicate that expert systems are viable aids for transferring internal control knowledge to managers whose work experience is outside of accounting and control systems. q 2001 Elsevier Science Ltd. All rights reserved. Keywords: Intelligent system; Expert system; Management; Internal controls; Knowledge transfer; Learning theory

1. Introduction Internal control is de®ned as a process designed to provide reasonable assurance regarding the achievement of objectives in the following categories: reliability of ®nancial reporting, effectiveness and ef®ciency of operations, and compliance with applicable laws and regulation (AICPA, 1996). Due to both regulatory and competitive forces, attention to an organization's internal controls has increased signi®cantly in the 1990s (Weber, 1999). Both accounting ®rms and researchers have devoted signi®cant effort to the development of decision aids, especially expert systems, to assist auditors in internal control evaluations (Bailey, Duke, Gerlach, Ko, Meservy & Whinston, 1985; Boritz, 1985; Gal, 1985; Cummings & Apostolou, 1987; Cummings, Lauer & Baker, 1988; O'Leary & Watkins, 1989; Graham, Damens & Ness, 1991; Vinze, Karan & Murthy, 1991). Researchers have long contended that expert systems could be used not only as decision aids but also to train non-expert users (Holsapple & Whinston, 1986; Biggs, Messier & Hansen, 1987; Borthick & West, 1987; Gal & q Data Availability: Data is available from the authors; however, requests for data must be speci®c regarding intended use. * Corresponding author. Tel.: 11-606-257-5236; fax: 11-606-257-8031. E-mail addresses: [email protected] (C. Changchit), [email protected] (C.W. Holsapple), [email protected] (R.E. Viator).

Steinbart, 1992; Odem & Dorr, 1995). Prior studies found that subjects who practiced making decisions with the aid of an expert system were better and quicker at reaching decisions than subjects who practiced without the support of the expert system (Oz, 1989; Murphy, 1990; Eining & Dorr, 1991; Fedorowicz, Oz & Berger, 1992). However, there are no previous reported studies of an expert system to facilitate the transfer of internal control evaluation knowledge to management. There are several reasons why managers are likely to perform internal control evaluations. First, establishment and supervision of internal control systems are the responsibility of management, not external auditors (COSO, 1992). Second, external auditors do not have the immediate and detailed insight into the operation of the internal control systems that management does. And third, the evaluation of internal control systems should be treated as a continuous rather than a discrete process, so that any weaknesses can be prevented or detected as soon as possible. Even though the literature has reported that an expert system can facilitate the transfer of internal control evaluation knowledge to novice subjects (Libby, 1995; Lieb & Gillease, 1996; Weber, 1999), all such studies used either auditors or accounting students as subjects. Without conducting the research, we are reluctant to assume that having managers as subjects will yield the same positive result. There are signi®cant reasons why managers who lack training in accounting systems may be less likely to

0957-4174/01/$ - see front matter q 2001 Elsevier Science Ltd. All rights reserved. PII: S 0957-417 4(00)00066-X

276

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291

acquire internal control evaluation knowledge. First, managers, including corporate internal auditors, come from diverse backgrounds, such as operations and information systems training, and do not necessarily have an accounting educational background (Viator & Curtis, 1998). Second, differences in educational background have been directly linked to differences in knowledge structure (Curtis & Viator, forthcoming). Studies have shown that a mismatch between knowledge structure and task structure can have a detrimental effect on performance (Nelson, Libby & Bonner, 1995), and differences in knowledge structure are systematically associated with the quality of performance in reviewing of internal control systems (Curtis & Viator, forthcoming). Given variations in educational background, work experience, and knowledge structures, the transfer of internal control evaluation knowledge appears to be less likely for managers, as compared to external auditors or accounting student novices. Third, managers may not be willing to learn the concept of the internal controls, or may not feel comfortable using expert systems. For example, they may provide incorrect inputs, not understand system outputs, ®nd it dif®cult to work with expert systems, or resist considering the advice it gives. This research examines whether managers' use of expert systems in the evaluation of internal controls can overcome the barriers identi®ed above, resulting in the transfer of internal control knowledge to those managers. 2. Background and hypotheses One of the most fundamental learning concepts is acquisition of knowledge (Cormier & Hagman, 1987). Information processing theory uses the computer as a model for human learning (Miller, 1956). Like the computer, the human mind takes in information, performs operations to change its form and content, stores and locates it, and generates responses to it. Thus, processing involves gathering and representing information (encoding), holding information (retention), and getting at the information when needed (retrieval). Several decision aids were developed to facilitate the transfer of the knowledge of one source to another. In this study, we are concerned with the impact of using expert systems to facilitate such learning. The following six hypotheses were designed to examine the value of the system as follows: H1: The improvement in accuracy scores of participants trained with an internal control evaluation expert system is higher that the improvement in accuracy scores of participants with no decision aid. Hypothesis H1 examines whether a participant's accuracy in detecting internal control weaknesses improves after being trained with an expert system. Accuracy of decision-making is examined as a measure of the system's effec-

tiveness (Sharda, Barr & McDonnell, 1988; Eining & Dorr, 1991). In order to examine the effectiveness of the system, we compared participants' improvement (Session I vs Session IV) in the accuracy scores between the Expert System (ES) group and No Decision Aid (NDA) group. We hypothesized that, on average, the ES group's improvement in accuracy scores will be higher than the NDA group's improvement. H2: The improvement in decision time of participants trained with an internal control evaluation expert system is higher than the improvement in decision time of participants with no decision aid. Hypothesis H2 examines whether a participant's decision time in accurately detecting internal control weaknesses is reduced after being trained with an expert system. The time used to make correct decisions (i.e., time per accuracy score) was examined as a measure of the system's ef®ciency. In order to measure the improvement in decision time, we compare the times used by participants in both groups in accurately detected internal control weaknesses. We hypothesized that, on average, the ES group's improvement in decision time will be higher than the NDA group's improvement. Hypotheses H3a, H3b, H4a, and H4b are based on the Technology Acceptance Model (TAM). The TAM provides a basis for explaining the determinants of computer acceptance and user behavior (Davis, Bagozzi & Warshaw, 1989). The model suggests that computer usage is determined by two criteria: perceived usefulness and perceived ease of use. Perceived usefulness is de®ned as the prospective users' subjective probability that using a speci®c application system will increase his or her job performance within an organizational context. Perceived ease of use is de®ned as the degree to which the prospective user expects the target system to be free of effort (Davis et al., 1989). H3a: Participants trained with an internal control evaluation expert system are more satis®ed with their accuracy than participants with no decision aid. H3b: Participants trained with an internal control evaluation expert system are more satis®ed with their speed than participants with no decision aid. Hypotheses H3a and H3b examine participants' perceived usefulness of the expert system in improving their performances in detecting internal control weaknesses. The hypotheses test whether there were differences in satisfaction with performances between participants practicing with an expert system (ES) and those with no decision aid (NDA). We hypothesized that, on average, participants in the ES group would be more satis®ed with their accuracy and speed than participants in the NDA group. H4a: Participants trained with an internal control evaluation expert system perceive that performing tasks requires

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291

less effort, compared to participants with no decision aid. H4b: Participants trained with an internal control evaluation expert system perceive that performing tasks is more interesting, compared to participants with no decision aid. Hypotheses H4a and H4b examine participants' perceived usefulness of the expert system in improving their attitudes toward the internal control evaluation tasks. The hypotheses test whether there were differences in participants' perceptions of the task between the ES group and the NDA group. We hypothesized that, on average, participants in the ES group would perceive that the evaluation of internal controls requires less effort and is more interesting than participants in the NDA group. 3. Research methodology To examine the impact of using an expert system to facilitate the transfer of the auditor's internal control evaluation knowledge to management, an experiment was conducted. The research design consisted of two treatment groups, three covariates, and six dependent variables. The two treatment groups were subjects with an expert system (ES) and those with no decision aid (NDA). Three covariates were measured, including the initial accuracy score from Session I, percentage of management duties, and years of work. The six dependent variables included subjects' accuracy, decision time, satisfaction with accuracy, satisfaction with speed, perception of task dif®culty, and perception of task attractiveness. 3.1. Subjects Because the main purpose of this study is to investigate whether an expert system can be devised to help managers more ef®ciently and/or more effectively detect potential weaknesses of internal control systems in their organizations, the subjects were persons with managerial responsibilities. These managers were drawn from client ®rms of three international accounting ®rms. Lists of potential subjects were obtained from these accounting ®rms. One hundred and forty subjects were randomly selected from those lists (one hundred for the experimental group and forty for a control group). To gain subjects' cooperation, the invitation letter explained clearly the importance of the study and assured con®dentiality. All subjects were told that they would receive a summary of this study. The experiment was conducted in an isolated, controlled environment provided by college laboratory and conference rooms. There were 50 participants in the experimental group (50% participation rate) and 15 in the control group (37.5% participation rate). Each participant served as an internal control decision maker. Even though some participants hold positions in accounting and ®nancial areas, telephone interviews were conducted with these participants to ensure that they had neither a background in internal control

277

Table 1 Distribution of subjects across industry Industries

No. of subjects

Consultant Legal Manufacturing Government Insurance Banking Construction Distribution Mining Transportation Education Finance Others Total

12 8 8 11 3 2 2 2 2 2 3 3 7 65

Table 2 Participants' demographics Education Average years of work Average percentage of management duties

55 Bachelor, 10 Master 5.46 53.7

Table 3 Participants' titles Title Finance and Accounting Manager General Manager Supervisor Director Senior Analyst Assistant Manager Vice President Tax Manager Administrative Manager Legal Manager Chief Accountant Engineer System Analyst Lawyer Marketing Manager Operating Manager Joint Venture Manager Total

No. of subjects 9 7 10 5 4 6 2 2 6 2 2 2 2 2 2 1 1 65

concepts nor prior hands-on experience with an expert system. Tables 1±3 show the participants' industries, education, average years of work, average percentage of management duties, and diversity of management roles. 3.2. Internal control case studies Four case studies (A, B, C, and D) were generated from the manipulation of several cues for detecting the potential

278

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291

weaknesses in internal control systems. These cues were obtained from a review of auditing texts, accounting texts, and input from accounting professors and experienced auditors. (See Appendix B) The scenario in each case dealt with the adequacy of internal control over a company's sales and collection cycle. Also, each case included background information about the ®ctitious company and a partial organization chart. Each case contained ten potential weaknesses in the internal control system. Three experienced auditors and three managers were asked to pilot test these cases to ensure their similarity with respect to the degree of dif®culty in detecting the potential internal control weaknesses. Revisions to the cases were made based on feedback provided (see Appendix C).

scores and to those attaining the best time per accuracy scores. Participants appeared to be genuinely interested in garnering prizes. In addition, as practicing managers who agreed to devote the many hours required to this study, their own senses of commitment and professionalism were factors in securing their efforts within the experiment. In arriving at an accuracy score, all points related to correctly identi®ed weaknesses were summed. In addition, to prevent the subjects from trying to detect weaknesses by guessing, they were informed at the beginning of the experiment that one-third of all points for inaccurately identi®ed weaknesses will be subtracted as the penalty for guessing. Non-response for a potential weakness results in neither addition nor subtraction.

3.3. Experimental task

3.4. Dependent variables

The experiment consisted of four sessions. Prior to the experiment, the four cases were randomly assigned to the sessions, yielding the result of Case A being assigned to Session IV, Case B to Session I, Case C to Session III, and Case D to Session II. In each session, participants were asked to play the role of a manager trying to detect the potential weaknesses of the internal control system described in the case. The maximum time allowed for each session was two hours. In Session I, participants in both groups performed the case without any decision aid. Because no participant had a background in internal control concepts, a booklet containing an overview of basic internal control activities (Appendix A) and a list of possible internal control weaknesses (Appendix B) were provided in this session. In Sessions II and III, participants in the NDA group performed the second and third case studies without the expert system, while participants in the ES group were allowed to use the expert system as a decision aid in detecting internal control weaknesses in the assigned case. In Session IV, the expert system was removed, participants in both groups performed the fourth case study without any decision aid. A booklet containing an overview of basic internal control activities and a list of possible internal control weaknesses were provided again in this session. For each session, the time required for decision-making was recorded and an accuracy score was calculated. In addition, questionnaires were given at the end of each session to measure participants' satisfaction with their performances as well as their perceptions of effort and interest in the task. This questionnaire was also used to gather data about their demographics. A performance-based reward system was used here to induce optimal participant behavior and reduce problems due to guessing (Smith, 1982). Participants were told that their performances across the series of cases would be used to calculate two scores: an accuracy score and a time per accuracy score. They were informed that prizes would be given to those participants attaining the highest accuracy

Two experienced auditors and two accounting professors evaluated the list of internal control weaknesses previously described. Each evaluator assigned a score to each weakness using a scale of 0±10, based on the degree of importance of each weakness (0Ðthe least important, 10Ðvery important). The average of the scores for each weakness (i.e., divided by four) was assigned as its importance. In order to examine the impact of using the expert system to facilitate the transfer of knowledge, the improvement in accuracy was compared across the two treatment groups: participants in the ES group versus participants in the NDA group. The time used to make decisions accurately (i.e., Time per Accuracy Score) was examined as a measure of ef®ciency. The time per accuracy score was computed as follows: Time Used to Perform Task Accuracy Score For example, Mr. A correctly detects 5 weaknesses and incorrectly identi®es 5 weaknesses. The time used to do the case is 40 min. Assuming that each weakness has 3 points associated with it. Mr. A's accuracy score is 10 ˆ ((5 £ 3) 2 (5 £ 3)/3) and his time per accuracy score is 40/10, or 4 min per accuracy score. Four questions (detailed in Appendix D) were asked to investigate participant satisfaction and perception of the task. These questionnaires were pretested with ten MBA students to ensure their clarity and construct validity. 1. On a scale of 1 (very unsatisfactory) to 7 (very satisfactory), how satis®ed were you with your accuracy in answering the case study? 2. On a scale of 1 (very unsatisfactory) to 7 (very satisfactory), how satis®ed were you with your speed in answering the case study? 3. On a scale of 1 (very dif®cult) to 7 (very easy), how dif®cult was it to do the case study? 4. On a scale of 1 (very boring) to 7 (very interesting), how

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291

279

Fig. 1. Diagram for checking the existence of proper authorization of sales orders, sales invoices, credit memo, and changes in payment conditions memos.

interesting was the task you performed?

3.5. Decision aid A prototype expert system to train management in internal control evaluation was constructed. The knowledge of the system was acquired from several sources including an experienced auditor, who served as the expert for the system, auditing texts, accounting texts, and the COSO framework. The expert was asked to describe, in detail and step-by-step, the techniques and processes he uses in evaluating an internal control system. The reasons for each decision were also acquired in an attempt to develop an expert system that would be able to emulate both the expert's knowledge and his reasoning behavior. Because the system was aimed at training managers in internal control evaluation, additional features, other than those included in previously reported expert systems, were integrated into the system for easy understanding and use by persons who are not familiar with internal control concepts. For instance, the system incorporated several diagrams to direct users to the next steps of detecting internal control weaknesses. Also, the system prompted users for entering data relevant to detecting the internal control weaknesses. A menu was also used to reduce the possibility of typing errors. The system reports each weakness identi®ed and why the weakness found is considered important, not just the adequacy of the overall internal control system within

the organization. Figs. 1±3 present examples of diagrams, input screens with menus, and system output. Once a prototype expert system was developed, the expert was asked to review the expertise that had been captured in the rule sets. Test cases were used during the validity review. 4. Results and discussion In order to assess the homogeneity of subjects' internal control knowledge between the experimental group and the control group, an ANOVA was performed on the ®rst session results to test for differences in Accuracy Score and in Time per Accuracy Score between the groups. At an alpha level of 0.05, no signi®cant difference was found between the groups' initial levels of internal control knowledge. The experimental data were analyzed as a completely randomized design with a one-way treatment structure using the Analysis of Covariance (ANCOVA) technique. The StatView w system was used to perform the test and each participant was an experimental unit. Hypotheses H1 and H2 were tested by calculating performance differences between Session IV and Session I, for each participant. Tests of hypotheses H3a, H3b, H4a, and H4b were based on participants' responses to the questionnaire items at the end of Session IV. The ANCOVA factors included experimental group and three covariates: initial score, percentage of management duties, and years of work experience.

280

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291

Fig. 2. A menu for selecting the person required to approve sales invoices.

Fig. 3. Report of weakness found for improper authorization of sales invoices of transactions and accounts.

Table 4 presents the ANCOVA results testing H1, regarding participants' accuracy in detecting potential weaknesses in internal control systems after being trained with the expert system. The results show that the improvement in

accuracy for participants in the ES group was signi®cantly higher than those in the NDA group (11.50 vs 3.67, respectively, with p , 0.008). Regarding the covariates, only the initial score was found to have an effect to the improvement

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291 Table 4 Improvement in accuracy score Source Panel A. Analysis of covariance Experimental group Initial score in Session I Percentage of management duties Years of works Error Total Panel B. Adjusted mean scores b Expert system (ES group) No decision aid (NDA group)

281

Table 5 Improvement in decision time DF

F-value

P-value a

1 1 1

6.205 4.109 0.007

0.008 0.02 0.47

1 59 63 b

0.951

0.17

11.50 3.67

Source Panel A. Analysis of covariance Experimental group Initial score in Session I Percentage of management duties Years of works Error Total

F-value

P-value a

1 1 1

6.205 4.109 0.007

, 0.0001 , 0.0001 0.35

1 59 63 b

0.951

0.43

DF

Panel B. Adjusted mean scores Expert system (ES group) No decision aid (NDA group)

3.67 1.18

a One-tailed directional test. No signi®cant interaction effects beyond level 1 were found. b One participant had an emergency call and left while performing case II. His data was, therefore, taken out.

a One-tailed directional test. No signi®cant interaction effects beyond level 1 were found. b One participant had an emergency call and left while performing case II. His data was, therefore, taken out.

of participants' accuracy (p , 0.02). Analysis of the beta coef®cients indicated that participants with lower initial scores had a greater improvement than participants with higher initial scores. In order to ensure the homogeneity of participant's accuracy score at the beginning of the experiment, an F-test was used to compare initial accuracy scores between the two groups. No signi®cant difference was found (p . 0.1). Hypothesis H2 examined whether there was an impact on participants' speed in accurately detecting potential weaknesses in internal control systems after being trained with the expert system. Because some participants obtained a 0 accuracy score, an ANOVA could not be run directly to test if there was a signi®cant difference in the mean of time per accuracy score between the sessions (i.e., the time cannot be divided by zero). In order to solve this problem, a positive theoretical minimum score (20) was added to each participant's accuracy score. The result is referred to as an adjusted time per accuracy score. The results are summarized in Table 5 and indicate that participants in the ES group had signi®cantly greater speed (i.e., the improvement in their performances in Session IV vs Session I) in accurately detecting internal control weaknesses than participants in the NDA group (3.67 vs 1.18, respectively, with p , 0.0001). This result suggests that using the expert system as a training tool did affect participants' ef®ciency in a positive direction. For the covariates, as with the accuracy score, only the initial decision time was found to affect the improvement of participants' decision time (p , 0.0001). Analysis of the beta coef®cients indicated that participants with lower initial decision time had a greater improvement than participants with higher initial decision time. In order to ensure the homogeneity of participant's decision time at the beginning of the experiment, an F-test was used to compare their initial speeds. No signi®cant difference was found (p . 0.2).

Hypotheses H3a and H3b examined the difference between the ES group and the NDA group regarding participants' satisfaction with performance. The results are summarized in Tables 6 and 7. On average, participants in the ES group were more satis®ed with their accuracy, but the difference between groups was not statistically signi®cant (3.94 vs 3.00, respectively, p . 0.1). However, the result shows that participants in the ES group were signi®cantly more satis®ed with their speed than participants in the NDA group (4.04 vs 2.87, respectively, p , 0.05). Hypotheses H4a and H4b examined the difference between the ES and the NDA group regarding participants' perceptions of the task. The results are summarized in Tables 8 and 9 and indicate that there was no signi®cant difference between the ES and the NDA group regarding attitude on the dif®culty of the task (3.65 vs 2.93, respectively, p . 0.17). These ®ndings might be attributed to participants in the ES group using the expert system in Table 6 Participants satisfaction with accuracy Source Panel A. Analysis of covariance Experimental group Initial score in Session I Percentage of management duties Years of works Error Total Panel B. Adjusted mean scores Expert system (ES group) No decision aid (NDA group)

F-value

P-value a

1 1 1

6.205 4.109 0.007

0.10 0.002 0.12

1 59 63 b

0.951

0.16

DF

3.94 3.00

a One-tailed directional test. No signi®cant interaction effects beyond level 1 were found. b One participant had an emergency call and left while performing case II. His data was, therefore, taken out.

282

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291

Table 7 Participants satisfaction with speed Source Panel A. Analysis of covariance Experimental group Initial score in Session I Percentage of management duties Years of works Error Total

Table 9 Participants perception of task attractiveness F-value

P-value a

1 1 1

6.205 4.109 0.007

0.05 0.02 0.08

1 59 63 b

0.951

0.10

DF

Panel B. Adjusted mean scores Expert system (ES group) No decision aid (NDA group)

4.04 2.87

Source Panel A. Analysis of covariance Experimental group Initial score in Session I Percentage of management duties Years of works Error Total Panel B. Adjusted mean scores Expert system (ES group) No decision aid (NDA group)

F-value

P-value a

1 1 1

6.205 4.109 0.007

0.05 0.005 0.10

1 59 63 b

0.951

0.06

DF

4.18 2.87

a One-tailed directional test. No signi®cant interaction effects beyond level 1 were found. b One participant had an emergency call and left while performing case II. His data was, therefore, taken out.

a One-tailed directional test. No signi®cant interaction effects beyond level 1 were found. b One participant had an emergency call and left while performing case II. His data was, therefore, taken out.

Sessions II and III and losing some con®dence when performing the task without an expert system. The results in Table 9 indicate that, on average, participants in the ES group perceived that the task as more interesting than participants in the NDA group (4.18 vs 2.87, respectively, p , 0.05).

of the use of expert systems to assist managers who are not auditing experts in detecting potential internal control weaknesses. The major ®nding of this study is that after being trained with the expert system, participants in the ES group signi®cantly outperformed participants in the NDA group on a variety of dimensions. This indicates that it is feasible for operations managers to acquire internal control evaluation knowledge via the use of expert systems. Such systems can help managers detect the weaknesses in their organizations' internal control systems more effectively and more ef®ciently. In addition, the results show that the system can help improve the users' satisfaction with their performances and perceptions of internal control evaluation tasks. On average, the users in the ES group reveal that they are more satis®ed with their accuracy than participants in the NDA group. The users in the ES group also perceive that the task was easier and more interesting than those in the NDA group. These ®ndings suggest that installing such a system may help managers maintain an effective internal control system, thus providing more reliable ®nancial data and better safeguarding assets. Such systems could enable organizations to save time and money by allowing weaknesses in internal control systems to be detected and resolved more quickly. The systems could also bene®t auditing ®rms by facilitating more reliable internal control in client ®rms, thereby reducing planned detection risk (i.e., less work and time). The scope of this research study may limit generalizability of the results in several respects. First, this research concentrates only on the evaluation of controls commonly found in the sales and collection cycle. Second, it investigates internal control systems commonly found in the merchandising industry. It is not expected to handle novel (uncommonly different) accounting systems. Third, the knowledge of the expert

5. Conclusions and direction for future research In the future, it is likely that operations managers will become more involved in performing reviews of internal control systems as operating processes are re-engineered and control systems for those processes are re-evaluated. Regarding operating processes, such operations managers are likely to have more expertise than student-novices and external auditors; however, because of their educational background, they are likely to have less fundamental knowledge of accounting. This research is the initial investigation Table 8 Participants perception of task dif®culty Source Panel A. Analysis of covariance Experimental group Initial score in Session I Percentage of management duties Years of works Error Total Panel B. Adjusted mean scores Expert system (ES group) No decision aid (NDA Group)

F-value

P-value a

1 1 1

6.205 4.109 0.007

0.17 0.17 0.10

1 59 63 b

0.951

0.23

DF

3.65 2.93

a One-tailed directional test. No signi®cant interaction effects beyond level 1 were found. b One participant had an emergency call and left while performing case II. His data was, therefore, taken out.

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291

system developed for this study is based primarily on one auditor who is a partner of an international accounting ®rm. The resulting system closely represents his reasoning about internal control evaluation. Thus, the system's knowledge may be ®rm-speci®c or expertspeci®c. These limitations point to directions in which the research presented here can be extended by future investigations. Future research might investigate the results of using expert systems over a long period of time (e.g., conducting additional sessions one week or month after the ®rst four sessions). Researchers might incorporate additional transaction cycles, industries, or other auditing functions into the expert system. They might develop additional expert systems by acquiring expertise from an internal auditor instead of an external auditor. Finally, researchers might investigate the feasibility of integrating this kind of expert system with the company's databases. Appendix A A.1. Basic internal control activities Internal control activities are those policies and procedures that management has established to meet its objectives for ®nancial reporting. There are potentially many such control activities in any entity. The extent of separation of duties depends heavily on the size of the organization. However, in general, the control activities may be classi®ed into ®ve categories as follows: A.1.1. Adequate segregation of duties When one person performs both functions, there is an excessive risk that person's disposing of the asset for personal gain and adjusting the records to relieve himself or herself of responsibility. For example, if the cashier receives cash and is responsible for data entry for cash receipts and sales, it is possible for the cashier to take the cash received from a customer and adjust the customer's account by failing to record a sale or by recording a ®ctitious credit to the account. A.1.2. Proper authorization of transactions and accounts To achieve effectiveness of internal control, appropriate individuals must authorize transactions. If there is any person in an organization who could acquire or expend assets at will, the complete chaos would result. For example, a merchandising business would require speci®c authorization to be obtained before incurring a long-term debt. A.1.3. Adequate documents and records Documents and records are the physical objects upon which transactions are entered and summarized. In order

283

to prevent fraud and irregularity, certain relevant principles should be adopted to dictate the proper design and use of documents and records. A.1.4. Physical control over assets and records It is essential to adequate internal control to protect assets and records. If assets and records are not adequately protected, they can be stolen, damaged, or lost. In the event of such an occurrence, the accounting process as well as normal operations could be seriously disrupted. The most important type of protective measure for safeguarding assets and records is the use of physical precautions. For example, a storeroom for inventory can be used to guard against pilferage. When the storeroom is under the control of a competent employee, there is also further assurance that obsolescence is minimized. A.1.5. Independent check on performance The need for independent checks arises because an internal control structures tends to change over time unless there is a mechanism for frequent review. Personnel are likely to forget or intentionally fail to follow procedures, or become careless unless someone observes and evaluates their performances. In addition, both fraudulent and unintentional misstatements are possible, regardless of the quality of the controls. Appendix B B.1. List of internal control weaknesses for sales and collection cycle of medium-size merchandising organizations grouped by basic control activities B.1.1. Adequate segregation of duties 1. Credit department is not independent from sales department. 2. Credit department is not independent from shipping department. 3. Credit department is not independent from accounting department. 4. Finance department is not independent from accounting department. 5. Collection department is not independent from shipping department. 6. The person who records sale invoices is the same or related to the persons who perform the following functions: 6.1. prepares sales invoices 6.2. prepares credit memo 6.3. records shipment 6.4. records goods returns 6.5. records customers' claims 6.6. receives cash/cheque 6.7. authorizes the voiding of invoices

284

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291

7. The person who records credit memos is the same or related to the persons who perform the following functions: 7.1. prepares sales invoices 7.2. prepares credit memo 7.3. records shipment 7.4. records goods returns 7.5. records customers' claims 7.6. receives cash/cheque 7.7. authorizes the voiding of invoices 8. The person records adjustments to customers' accounts is the same or related to the persons who perform the following functions: 8.1. prepares sales invoices 8.2. prepares credit memo 8.3. records shipment 8.4. records goods returns 8.5. records customers' claims 8.6. receives cash/cheque 8.7. authorizes the voiding of invoices 9. The person reconciles customers' account balances with control balances is the same or related to the persons who perform the following functions: 9.1. records accounts receivable 9.2. prepares customers' control accounts 9.3. receives cash/cheque 10. The person records general ledgers is the same or related to the persons who perform the following functions: 10.1. records accounts receivable 10.2. receives cash/cheque 11. The person who receives cash/cheque is the same or related to the persons who records cash/cheque. 12. The person who opens mails is the same or related to the persons who perform the following functions: 12.1. receives cash/cheque 12.2. records cash/cheque 12.3. records accounts receivable 12.4. records general ledger

B.1.2. Proper authorization of transactions and accounts 1. Sales orders are not approved by a sale manager. 2. Sales invoices are not approved by both a sales manager and a credit manager. 3. Credit memos are not approved by both a sales manager and a credit manager. 4. Changes in payment conditions memos are not approved by both a sales manager and a credit manager. 5. Debit notes are not approved by both a credit manager and an accounting manager/controller. 6. Adjustments to customers' account memos are not approved by both a credit manager and an accounting manager/controller.

7. Sales invoices with special terms and discounts are not approved by both a credit manager and a managing director. 8. Speci®c sales invoices (e.g., with substantial amount) are not approved by both a credit manager and a managing director. 9. Uncollectable account memos are not approved by both a credit manager and a managing director. 10. Write-off account memos are not approved by both a credit manager and a managing director. 11. Sales orders are not approved in writing. 12. Sales invoices are not approved in writing. 13. Credit memos are not approved in writing. 14. Changes in payment conditions memos are not approved in writing. 15. Debit notes are not approved in writing. 16. Adjustments to customers' accounts memos are not approved in writing. 17. Uncollectable accounts memos are not approved by in writing. 18. Write-off accounts memos are not approved in writing.

B.1.3. Adequate documents and records 1. There is no space for an authorized signature in a sales order. 2. There are no spaces for authorized signatures in a sales invoice. 3. There are no spaces for authorized signatures in a credit memo. 4. There are no spaces for authorized signatures in a changes in payment conditions memo. 5. There are no spaces for authorized signatures in a debit note. 6. There are no spaces for authorized signatures in a adjustments to customers' accounts memo. 7. There is no space for authorized signatures in an uncollectable accounts memo. 8. There is no space for authorized signatures in a write-off accounts memo. 9. There is no space for a customer's signature in a delivery order to acknowledge receipt of goods. 10. Sales orders are not prenumbered consecutively. 11. Sales invoices are not prenumbered consecutively. 12. Credit memos are not prenumbered consecutively. 13. Changes in payment conditions memos are not prenumbered consecutively. 14. Debit notes are not prenumbered consecutively. 15. Adjustments to customers' accounts memos are not prenumbered consecutively. 16. Uncollectable accounts memos are not prenumbered consecutively. 17. Write-off accounts memos are not prenumbered consecutively.

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291

18. Delivery orders are not prenumbered consecutively. 19. Cash receipts are not prenumbered consecutively. 20. There is no registration book for controlling customers' purchase orders. 21. Customer purchase order registration book does not contain receiving purchase order date, purchase order number, customer name, and contact person name. 22. There is no stock card for controlling inventory in and out. 23. A stock card does not contain date, reference number of instruction document (e.g., delivery order), item description, and quantity. 24. There is no policy that all customer purchase orders (both written and verbal) must be transcribed into a uniform internal sales order form. 25. There is no aging report showing the proportion of accounts receivable due in 30, 60, and 90 days. 26. A list of collections, which provides details of cash, cheque, and remittance advice received by mail, is not prepared when opening mails. 27. All cash/cheque received are not deposited on a daily basis. 28. All cash/cheque received are not posted to accounts receivable on a daily basis. 29. Account balance statements are not sent to all customers informing their current balances at least once a month. 30. The bank reconciliation is not made at least once a month. 31. Insuf®cient copies (seven copies should be prepared) of sales invoices to advise related departments of its action and retain in the sales department's ®le as the evidence of performance. 32. Original copies of sales invoices are not sent to customers for their references. 33. Copies of sales invoices are not distributed to appropriate departments as follows: ² the original to a customer ² one copy retained in sales department ² one copy to credit department ² one copy to accounting department ² one copy to storeroom department ² two copies to shipping department 34. Insuf®cient copies of credit memos for cash discount (four copies) to advise related departments of its action and retain in the ®nance department ®le as the evidence of performance. 35. Insuf®cient copies of credit memos for goods returns (six copies) to advise related departments of its action and retain in receiving department's ®le as the evidence of performance. 36. Original copies of credit memos are not sent to customers for their references. 37. Copies of credit memos for cash discount are not distributed to appropriate departments as follows: ² the original to a customer ² one copy retained in ®nance department

38.

39.

40. 41.

42.

43. 44.

45.

46. 47.

48.

49.

285

² one copy to credit department ² one copy to accounting department Copies of credit memos for goods returns are not distributed to appropriate departments as follows: ² the original to a customer ² one copy retained in receiving department ² one copy to credit department ² one copy to accounting department ² one copy to ®nance department ² one copy to storeroom department Insuf®cient copies (four copies should be prepared) of changes in payment conditions memos to advise related departments of its action and retain in the credit department's ®le as the evidence of performance. Original copies of changes in payment conditions memos are not sent to customers for their references. Copies of changes in payment conditions memos are not distributed to appropriate departments as follows: ² the original to a customer ² one copy retained in credit department ² one copy to ®nance department ² one copy to sales department Insuf®cient copies (®ve copies should be prepared) of debit notes to advise related departments of its action and retain in the credit department's ®le as the evidence of performance. Original copies of debit notes are not sent to customers for their references. Copies of debit notes are not distributed to appropriate departments as follows: ² the original to a customer ² one copy retained in credit department ² one copy to accounting department ² one copy to ®nance department ² one last copy to sales department Insuf®cient copies (®ve copies should be prepared) of adjustments to customers' accounts memos to advise related departments of its action and retain in the credit department's ®le as the evidence of performance. Original copies of adjustments to customers' accounts memos are not sent to customers for their references. Copies of adjustments to customers' accounts memos are not distributed to appropriate departments as follows: ² the original to a customer ² one copy retained in credit department ² one copy to accounting department ² one copy to ®nance department ² one copy to sales department Insuf®cient copies (four copies should be prepared) of uncollectable accounts memos to advise related departments of its action and retain in the credit department's ®le as the evidence of performance. Copies of uncollectable accounts memos are not distributed to appropriate departments as follows: ² the original retained in credit department

286

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291

² one copy to accounting department ² one copy to ®nance department ² one copy to sales department 50. Insuf®cient copies (four copies should be prepared) of write-off accounts memos to advise related departments of its action and retain in the credit department's ®le as the evidence of performance. 51. Copies of write-off accounts memos are not distributed to appropriate departments as follows: ² the original retained in credit department ² one copy to accounting department ² one copy to ®nance department ² one last copy to sales department B.1.4. Physical control over assets and records 1. The of®ce used to keep sales documents (e.g., sales orders, sales invoices, credit memos, debit memos, etc.) is unlocked after hours. 2. The cabinets used to keep sales documents (e.g., sales orders, sales invoices, credit memos, debit memos, etc.) are unlocked after hours. 3. The of®ce used to keep books of accounts (e.g., sales journal books, accounts receivable books, control account books, etc.) is unlocked after hours. 4. The cabinets used to keep books of accounts (e.g., sales journal books, accounts receivable books, control account books, etc.) are unlocked after hours. 5. Persons other than accounting manager/controller and accounts receivable clerks have key to open cabinets used to keep sales journal books and accounts receivable books. 6. Persons other than accounting manager/controller and general ledger clerks have key to open cabinets used to keep control account books. 7. Storeroom is not under supervision all time during the storeroom hours. 8. Storeroom is unlocked after hours. 9. Persons other than storeroom staff have key to open the storerooms. 10. Cashier's drawers are not locked all time. 11. Persons other than cashiers have key to open cashier's drawers. B.1.5. Independent check on performances 1. Credit-worthy customers' purchase orders are not reviewed at least once a month. 2. Overdue accounts are not reviewed at least once a month. 3. A list of collections by mail is not reviewed against a cash book by a person other than the person who open mails, the person who receives cash, and the person who

records cash at least once a month. 4. Cash/cheque on hand is not counted against a cash book by a person other than the person who receives cash/ cheque on a surprise basis. 5. Inventories in stock are not counted against the record in a book of accounts by a person other than the storeroom staff at least once a year. 6. Sales invoices are not checked by a person other than the person who prepares them or salesperson that prices and trade discounts are based on approved sales orders or price lists. 7. Sales invoices are not checked by a person other than the person who prepares them or the person who records shipments that quantities are based on actual records of goods shipped. 8. Customers' names in sales invoices are not checked by a persons other than the person who prepares them against the customers' master ®le or customers' purchase orders. 9. The extensions and additions in sales invoices are not recomputed by a person other than the person who prepares them. 10. Credit memos are not checked by a person other than the persons who prepares them that prices and cash discount conditions are agreed with original sales invoices. 11. Credit memos issued for goods returned are not checked by a person other than the person who prepares them or the person who records goods returns that quantities are based on actual records of goods returned. 12. Customers' names in credit memos are not checked by a persons other than the person who prepares them against the original sales invoices. 13. The extensions and additions in credit memos are not recomputed by a person other than the person who prepares them.

Appendix C C.1. Case a: PC mart company PC Mart Company (PCM) is a Kentucky merchandiser of personal computers and related equipment such as monitors, printers, modems, hard disk drives, diskettes, and business software package. The company is established in 1960. During its several years of existence, PCM has done very well. Not only have sales grown fairly rapidly, but the product lines have been expanded and the employees have increased in number. Currently, the company employs ®fty persons, ranging from the managing director to a janitor. C.1.1. Organization and functions PCM has nine departments, which are independent from

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291

each other. The company's managing director is Mr. Richard Gere. Reporting to him are nine managers as follows: 1. 2. 3. 4. 5. 6. 7. 8. 9.

Ms. Kim BarsingerÐThe Accounting Manager Mr. Jackie ChanÐThe Administrative Manager Mr. Tom CruiseÐThe Credit Manager Ms. Julia RobertÐThe Treasurer Mr. Kerk RusselÐThe Purchasing Manager Mr. Robert RedfordÐThe Receiving Manager Mr. Charlie SheanÐThe Sales Manager Mr. Harrison FordÐThe Shipping Manager Mr. Clint EastwoodÐThe Storeroom Manager

C.1.2. Accounting department Ms. Kim Barsinger oversees clerks and bookkeepers who process transactions and maintain the accounting records. Reporting to her are all accounting clerk and bookkeeper as follows: ² Ms. Barbara Bailey: Reconciles Account Balances and Control Accounts; Prepares Bank Reconciliation; Records Customers' Claims ² Mr. Richard Cosner: Prepares Sales Journal; Records Sales Invoices ² Mr. Darick Lykins: Prepares Purchase Journal; Records Purchase Orders ² Mr. Andrew Mabson: Records Credit Memos and Adjustments to Customers' Accounts ² Ms. Karen Stainley: Prepares Cash Receipts Journal, Cash Disbursement Journal, Accounts Receivable Subsidiary Ledger, and Accounts Payable Subsidiary Ledger; Records Cash, Cheque and Accounts Receivable ² Ms. Victoria Parks: Prepares General ledger, Trial Balances, and Financial Statements; Records General Ledgers ² Ms. Martha Kastner: Prepares General Journal and Payroll Register; Records General Expenses, Payrolls The accounting of®ce is always locked after hours to prevent unauthorized access to the books of accounts. In addition, there are two cabinets in the of®ce. The ®rst one is used to keep sales journal and accounts receivable books. The second is used to keep general ledger, trial balances, and ®nancial statements books. These two cabinets are kept locked after hours. Only Ms. Kim Barsinger (the accounting manager) and Ms. Karen Stainley (the accounts receivable clerk) have the keys to open these two cabinets. Ms. Victoria Parks (the general ledger clerk) only has the key to open the second cabinet. C.1.3. Administrative department Mr. Jackie Chan serves as an administrative manager, with responsibilities for personnel, insurance, budgeting, cost analysis, and systems and procedures.

287

C.1.4. Credit department Mr. Tom Cruise is in charge of the credit department. The department handles the preparations of debit notes, changes in payment conditions memos, adjustments to customers' account memos, uncollectable account memos, and writeoff accounts memos. Mr. Tom Cruise also reviews all overdue account at least once a month in order to keep track on the performances of collections and to ensure that all followup procedures are being done properly and ef®ciently. At the end of each month, the credit department's staff will send the account balance statements to all active customer informing their current balances with the company. They also prepare an aging report showing the proportion of accounts receivable due in 30, 60, and 90 days. Each month, this report is reviewed for setting up the uncollectable accounts. The uncollectable accounts memos must be approved by Mr. Richard Gere (the managing director) and Mr. Tom Cruise (the credit manager) in writing in order to ensure that only bad accounts with high uncollectable potential are transferred to the provision for uncollectable accounts. The memos also have spaces provided for authorized signatures and are prenumbered consecutively. Every two months, the uncollectable accounts are reviewed to check the performances of collections. If all follow-up procedure is performed but the accounts are still uncollectable, the write-off accounts memos will be issued. These memos must be approved by Mr. Richard Gere (the managing director) and Mr. Tom Cruise (the credit manager) in writing. The memos have spaces provided for authorized signatures and are prenumbered consecutively. Four copies of both uncollectable accounts memos and write-off accounts memos are prepared and distributed as follows: ² The original copy is retained in credit department's ®le for their own record, ² One copy is sent to accounting department, ² One copy is sent to ®nance department, and ² The last copy is sent to sales department C.1.5. Finance department Ms. Julia Robert performs the duties of treasurer. Reporting to her are cashiers and ®nance clerks. This department handles all collection from customers. C.1.6. Purchasing department Mr. Kerk Russel is in charge of purchasing activities and supervises several purchasing staffs who specialize in the various product lines. The department receives the purchase requisitions from other departments and further processes the acquisition of goods or any other assets. C.1.7. Receiving department Mr. Robert Redford handles all receiving function which

288

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291

includes receiving of goods from suppliers and also receiving the goods returned from customers. Ms. Julie Garwood, a receiving clerk, is in charge for the record of all goods received from suppliers and all goods returned from customers. C.1.8. Sales department Mr. Charlie Shean directs the operations of the sales functions. There are ten salespersons and two sales clerks under his supervision. Mr. Thomas Newston and Mr. Jimmy Carlton, the sales clerks, are responsible for preparing sales invoices. The of®ce is always locked after hours to prevent unauthorized access to sales documents. In addition, in order to reduce the opportunity of missing documents, the cabinets used to keep sales documents are also locked after hours. C.1.9. Shipping department Mr. Harrison Ford is in charge of the shipping department. This department maintains all shipping operation. Mr. Stephen Bendon, the shipping clerk, is in charge for the record of each shipment. C.1.10. Storeroom department Mr. Clint Eastwood maintains the storeroom operations. The storeroom is under the supervision of at least one storeroom staff all time during the store hours and is always locked after hours. Only storeroom staffs have the key to open the storeroom. At the end of each year, the inventory in stock will be counted against the record in the books of accounts by the person other than the storeroom staff in order to verify the actual amount of inventory in stock as well as to alert the storeroom staff to take a good care of inventory in and out. C.2. Sales and collections procedures The company sells its products for cash or on credit. It has about 200 credit customers. The company acquires products from 20 suppliers. The perpetual inventory method for recording purchases is employed by PCM. Most products are sold at established prices, but trade discounts are allowed on occasion to small businesses and professional ®rms. PCM maintains its cash in two bank accounts. There is one account for general funds and one for payrolls. The company also has petty cash and cashier drawer funds. Among the resources that it owns are the of®ce building, storeroom, the land on which these buildings are located, the furniture and ®xtures in each of the building, cash registers, other of®ce equipment, and two trucks for delivering ordered merchandise. Sales and collections procedures are divided into seven groups of functions. The following describes all the procedure performed by the company's employees pertaining to the sales and collections cycle.

C.2.1. Receiving customers' purchasing orders Customers' purchase orders are received by salespersons either in person, by phone, or by mail. Normally, ®fteen to twenty orders are received each day. If a customer's purchase order is received by mail, a salesperson will forward such order to a sales clerk for preparing a sales invoice. However, if an order is received in person or by phone, he/she must transcribe it into a uniform sales order before further processing. If cash in full is paid by the customer at the time of the sale, the sales order copy is marked paid, and the sale is rung up on a cash register. The customer also receives the cash receipt tape from a cashier. If no cash or if only a partial payment (i.e., a deposit) is received at the time of the sale, the sale is treated as a credit sale. The cash/cheque received each day via the cash registers is checked at the end of the day, then combined with the cash/cheque received by mail and deposited into the bank on a daily basis. The cashiers' drawers are locked all time. Only Ms. Kathy Mcdonald and Ms. Deborah Adams, the company's cashiers, have the key to open the drawers. The company also has a customers' purchase order registration book which contains purchase order receiving date, purchase order number, customer name, and the name of contact person. In addition, the credit-worthy customers' purchase orders are reviewed monthly by Mr. Charlie Shean (the sales manager) in order to ensure that such orders are handled properly and ef®ciently. C.2.2. Preparing sales order forms All sales order must be approved by Mr. Charlie Shean (the sales manager) in writing prior to further processing. The sales orders have a space provided for the sales manager's authorized signature and are also prenumbered consecutively. C.2.3. Authorizing credit terms and discounts and preparing sales invoices After approval is granted for a sale, seven copies of a sales invoice are prepared by the sales clerks, Mr. Thomas Newston and Mr. Jimmy Carlton. These sales invoices must also be approved by Mr. Charlie Shean (the sales manager) in writing in order to ensure that the credit sales are not approved to a poor-credit customer. Then, these copies are distributed as follows: ² The original copy is sent to a customer as a billing and also for his/her record. ² One copy is retained in sales department's ®le for their own record, ² One copy is sent to credit department, ² One copy is sent to ®nance department, ² One copy is sent to store department, and ² The last two copies are sent to shipping department Any voiding of sales invoices must be authorized by Mr.

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291

289

Richard Cosner. Occasionally, a special term and discount may be given to speci®c customers. This type of sales invoices including some speci®c invoices (e.g., an invoice with substantial amount) must be approved by Mr. Tom Cruise (the credit manager) and Mr. Richard Gere (the managing director) in order to ensure that such sales are in conform with the company's sales and credit policies. Sales invoices have spaces provided for authorized signatures and are prenumbered consecutively. Typically, Ms. Barbara Bailey (the chief accountant) will review sales invoices against sales orders and records of shipments in order to verify if the prices and trade discounts are based on approved sales orders and the quantities are agreed with the actual records of goods shipped. She also checks the customers' names against the customer master ®le and recomputes the extensions and additions in sales invoices.

of goods returned. She also checks the customers' names against the original sales invoices and recomputes the extensions and additions in credit memos. A change in payment conditions, which requires the issuance of debit notes, is sometimes allowed to a customer. Such change must be approved by Mr. Charlie Shean (the sales manager) and Mr. Tom Cruise (the credit manager) in writing. The changes in payment conditions memos are prenumbered consecutively and also have spaces provided for authorized signatures. Four copies of the memos are prepared and distributed as follows:

C.2.4. Preparing credit memos for cash discounts or goods returns, changes in payment conditions, debit notes, and adjustments to customers' accounts The company allows returns and allowances within a speci®ed period from the dates of sales. It also allows cash discounts on sales if paid in full within ten days, otherwise, it expects full payment within thirty days. In both cases, a credit memo will be prepared by Mr. Kevin Cosner or Mr. Dennis Justice. All credit memos must be approved by Mr. Charlie Shean (the sales manager) and Mr. Tom Cruise (the credit manager) in writing before further processing. Credit memos have spaces provided for authorized signatures and are prenumbered consecutively. In case of cash discounts, four copies of credit memo are prepared and distributed as follows:

An increase in sales amounts as well as other adjustments to customers' accounts must be approved by Mr. Tom Cruise (the credit manager) and Ms. Kim Barsinger (the accounting manager) in writing prior to further processing in order to prevent unauthorized changes to customers' accounts. Both debit notes and adjustments to customers' accounts memos have spaces provided for authorized signatures and are prenumbered consecutively. Five copies of them are prepared and distributed as follows:

² The original copy is sent to a customer for his/her record. ² One copy is retained in ®nance department's ®le for their own record, ² One copy is sent to credit department, and ² The last copy is sent to accounting department In case of goods returns, six copies of credit memo are prepared and distributed as follows: ² The original copy is sent to a customer for his/her record. ² One copy is retained in receiving department's ®le for their own record, ² One copy is sent to credit department, ² One copy is sent to accounting department, ² One copy is sent to ®nance department, and ² The last copy is sent to storeroom department Typically, Ms. Barbara Bailey (the chief accountant) will review credit memos against sales invoices and records of goods returns in order to verify if the prices and cash discount conditions are agreed with the original sales invoices and the quantities are based on the actual records

² The original copy is sent to a customer for his/her record. ² One copy is retained in credit department's ®le for their own record, ² One copy is sent to ®nance department, and ² The last copy is sent to sales department

² The original copy is sent to a customer for his/her record. ² One copy is retained in credit department's ®le for their own record, ² One copy is sent to accounting department, ² One copy is sent to ®nance department, and ² The last copy is sent to sales department

C.2.5. Preparing execution instruction, withdrawing goods from stock, and shipping goods After receiving a copy of sales invoices from sales department, a storeroom staff prepares goods for shipment. If an order cannot be completely ®lled because there is an insuf®ciently quantity of an item on hand, the storeroom staff uses the copy of the sales invoice to prepare a back order and sends it to the purchasing manager for further processing. After receiving two copies of sales invoices from sales department, a shipping staff will contact the storeroom department for the goods to be shipped and, then ships the goods to customers. One copy of sales invoices is retained in shipping department for their own record. Another copy, which may be called a delivery order, is accompanied with the goods shipped. The delivery order has a space for the customer's signature and is prenumbered consecutively. The shipping staff will have the customer signed the delivery order as the acknowledgment of goods received. Then,

290

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291

Table 10 1. On a scale of 1 (very dif®cult) to 7 (very easy), how dif®cult was it to do the case study? 1 2 3 4 5 very dif®cult dif®cult slightly dif®cult neutral slightly easy

6 easy

7 very easy

2. On a scale of 1 (very unsatisfactory) to 7 (very satisfactory), how satis®ed were you with your accuracy in answering the case study? 1 2 3 4 5 6 7 very unsatisfactory unsatisfactory slightly unsatisfactory neutral slightly satisfactory satisfactory very satisfactory 3. On a scale of 1 (very unsatisfactory) to 7 (very satisfactory), how satis®ed were you with your speed in answering the case study? 1 2 3 4 5 6 very unsatisfactory unsatisfactory slightly unsatisfactory neutral slightly satisfactory satisfactory

7 very satisfactory

4. On a scale of 1 (very boring) to 7 (very interesting), how interesting was the task you performed? 1 2 3 4 5 very boring boring slightly boring neutral slightly interesting

7 very interesting

such delivery order will be forwarded to ®nance department for further collection processes. C.2.6. Receiving cash, cheque, and remittance advice by mail Ms. Kathy Mcdonald and Ms. Deborah Adams perform the duty of cashiers. Besides the main duty of receiving cash, Ms. Kathy Mcdonald is also assigned to open the mail received each day. Each letter containing a cash remittance is set aside. While opening all mail, they will prepare a list of collections by mail, which provides the details of all cash, cheque, and remittance advice received by mail. At the end of each month, this list of collections will be reviewed against a cash book by Ms. Julia Robert (the treasurer) in order to ensure the correctness of cash/cheque recorded as well as to prevent the lapping on cash/cheque received. Ms. Kathy Mcdonald and Ms Deborah Adams (the cashiers) also prepare a deposit slip, then combine the cash/ cheque received in the mail with the cash/cheque from the cashiers' drawers and deposit into the bank on a daily basis. At the end of each day, Ms. Julia Robert (the treasurer) will count the cash/cheque on hand against the record in cash book in order to reduce the opportunity of lapping on cash/ cheque. Ms. Kathy Mcdonald and Ms. Deborah Adams will issue cash receipts to customers to acknowledge the receipts of their payments. They also send to pile of remittance noti®cations and advises (together with a copy of the deposit slip) to the accounting clerks for entry into the cash receipts journal, for posting to the accounts receivable and general ledgers, and for ®ling. C.2.7. Recording accounts receivable, sales and general ledgers The accounting records are entered and posted manually. Ms. Karen Stainley (the accounts receivable clerk) will post all cash/cheque received to an accounts receivable book on a daily basis. Ms. Victoria Parks (the general ledger clerk) supervises

6 interesting

the posting to the general ledgers. Then, she prepares trial balances and ®nancial statements with the aid of an electronic spreadsheet package to compete key ®nancial ratios and to print hard copy ®nancial reports for the other managers. Ms. Barbara Bailey (the chief accountant) is responsible for recording all customers' claim and reconciling customer's accounts with the control accounts. She also prepares a bank reconciliation at the end of each month in order to cross check the ªCash in Bankº account with the transactions provided in the bank statement. C.3. Relationships of the company's employees ² ² ² ²

Ms. Julia Robert is Ms. Julie Garwood's sister Mr. Thomas Newston is Mr. Dennis Justice's son Mr. Kevin Cosner is Mr. Richard Cosner's brother Mr. Jackie Chan and Ms. Karen Stainley are husband and wife

Appendix D D.1. Post-experiment questionnaires Please answer the following questions. Please keep in mind that the following questions refer ONLY to the last case study you have done (Table 10). References American Institute of Certi®ed Public Accountants (AICPA) (1996). Codi®cation of statements on auditing standards, New York: AICPA. Bailey Jr, A. D., Duke, G. L., Gerlach, J., Ko, C., Meservy, R., & Whinston, A. B. (1985). TICOM and the analysis of internal controls. Accounting Review, 60 (2), 186±211. Biggs, S. F., Messier Jr, W. F., & Hansen, J. V. (1987). A descriptive analysis of computer audit specialists' decision-making behavior in advanced computer environments. Auditing: A Journal of Practice & Theory, Spring, 1±21. Boritz, J. E. (1985). The effect of information presentation structures on

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291 audit planning and review judgments. Contemporary Accounting Research, 8 (1), 193±218. Borthick, A. F., & West, O. D. (1987). Expert systems-A new tool for the professional. Accounting Horizons, March, 9±16. Committee of Sponsoring Organizations of Treadway Commission (COSO) (1992). Internal control-integrated framework, New York: AICPA. Cormier, S., & Hagman, J. (1987). Transfer of learning, San Diego, CA: Academic Press. Cummings, B. K., & Apostolou, N. G. (1987). Expert systems in auditing: an emerging technology. Internal Auditing, 3 (2), 3±10. Cummings, B. K., Lauer, J., & Baker, R. (1988). Expert systems in internal auditing. Internal Auditing, 4 (1), 49±64. Curtis, M. B., & Viator, R. E. (2000). An investigation of multidimensional knowledge structure and computer auditor performance. Auditing: A Journal of Practice and Theory, 19 (2), 83±103. Davis, F. D., Bagozzi, R. D., & Warshaw, P. R. (1989). User acceptance of computer technology: a comparison of two theoretical models. Management Science, 35 (8), 982±1003. Eining, M. M., & Dorr, P. B. (1991). The impact of expert system usage on experiential learning in an auditing setting. Journal of Information Systems, 5 (1), 1±16. Fedorowicz, J., Oz, E., & Berger, P. D. (1992). A learning curve analysis of expert system use. Decision Sciences, July/August, 797±818. Gal, G. F. (1985). Using auditor knowledge to formulate data model constraints: an expert system for internal control evaluation. Ph.D. Dissertation, Michigan State University. Gal, G. F., & Steinbart, P. J. (1992). Interface style and training task dif®culty as determinants of effective computer-assisted knowledge transfer. Decision Sciences, 23, 128±143. Graham, L. E., Damens, J., & Ness, G. V. (1991). Developing risk advisor SM: An expert system for risk identi®cation. Auditing: A Journal of Practice & Theory, 10 (1), 69±96. Holsapple, C. W., & Whinston, A. B. (1986). Managers' guide to expert systems, Homewood, IL: Dow Jones-Irwin. Libby, R. (1995). The role of knowledge and memory in audit judgment. In

291

R. Ashton & A. Ashton, Judgment and decision making research in accounting and auditing. Cambridge, MA: Cambridge University Press. Lieb, E. B., & Gillease, J. K. (1996). Du Pont uses a decision support system to select its audit portfolio. Interfaces, 26 (3), 10±21. Miller, G. A. (1956). The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychological Review, 63, 81±97. Murphy, D. S. (1990). Expert system use and the development of expertise in auditing: a preliminary investigation. Journal of Information Systems, Fall, 18±35. Nelson, M. W., Libby, R., & Bonner, S. E. (1995). Knowledge structure and the estimation of conditional probabilities in audit planning. Accounting Review, January, 27±47. Odem, M. D., & Dorr, P. B. (1995). The impact of elaboration-based expert system interfaces on de-skilling: An epistemological issue. Journal of Information Systems, 9 (1), 1±17. O'Leary, D. E., & Watkins, P. R. (1989). Review of expert systems in auditing. Expert Systems Review for Business & Accounting, 9 (2), 3±22. Oz, E. (1989). A study of improvement in decision-making skills through the use of expert systems. Unpublished Ph.D. Dissertation. Boston University. Sharda, R., Barr, S. H., & McDonnell, J. C. (1988). Decision support system effectiveness: a review and an empirical test. Management Science, 34 (2), 139. Smith, V. L. (1982). Microeconomic systems as experimental science. American Economic Review, 72 (5), 923±955. Viator, R. E., & Curtis, M. B. (1998). Computer auditor reliance on automated and non-automated controls as a function of training and experience. Journal of Information Systems, Spring, 19±30. Vinze, A. S., Karan, V., & Murthy, U. S. (1991). A generalizable knowledge-based framework for audit planning expert systems. Journal of Information Systems, Fall, 78±91. Weber, R. (1999). Information systems control and audit, Upper Saddle River, NJ: Prentice-Hall.